First, several sources said that you should not adjust for intermediate variables or variables in the causal pathway.

“Confounders should not be confused with intermediate variables, which are part of the causal pathway (Figure 1b), because adjusting for an intermediate variable would attenuate the measure of association.” Wakkee (2014)

“An intermediate variable is generally thought of as a post-treatment variable influenced by treatment that may or may not lie on the causal pathway between the treatment and the outcome. When the goal of CER is to estimate the total causal effect of the treatment on the outcome, adjustment for variables on the causal pathway between treatment and outcome is unnecessary and likely to induce bias2 toward a relative risk of 1.0, though the direction can sometimes be in the opposite direction.” Sauer (2013)

In general, it is better to include a variable in a risk adjusted model than to exclude it, provided that you have enough data.

“When unmeasured confounding remains, strong arguments exist for error on the side of over-adjustment (adjusting for instruments and colliders) rather than failing to adjust for measured confounders (under-adjustment).” Sauer (2013)

Of course, you need a lot of data for this, sometimes.

“For multiple logistic regression and Cox proportional hazards analysis, it is recommended that for every independent variable screened for association there are at least 10 events (Harrell, 2001). In multivariable linear regression it is recommended that for every independent variable approximately 10 subjects are included (Harrell, 2001).” Wakkee (2014)

If you try to avoid this problem by a preliminary screening step or by using some type of stepwise selection of variables, you may be jumping from the frying pan into the fire.

“One option is bivariable analysis, where confounders that change the studied association by 10% or more are included in the final model (Vandenbroucke et al., 2007). Variables can also be selected based on a certain P value (e.g., <0.05) in univariable analysis, although a disadvantage of this technique is that variables that are not important in the univariable association, and are therefore excluded, can be important in the full model. Another option is to allow the statistical program to choose the variables by forward or backward selection. In this technique the role of each independent variable is evaluated stepwise based on statistical significance. The most significant variables are added one by one, starting with an empty model (forward selection), or nonsignificant variables are removed stepwise (backward selection), starting with a full model. This can lead to unpredictable effects because the significance may depend on the order of adding or removing covariates. In addition, this technique may also lead to the exclusion of variables you might have preferred to keep in your model based on clinical reasons.” Wakkee (2014)

“Variable selection procedures also suffer from the fact that estimates about treatment effects are made after having already used the data to decide on covariates.” Sauer (2013)

“In practice, variable selection is dogged by the fact that the final variance estimates tend to be downwardly biased if they do not account for the selection, while the point estimates may suffer related distortions” Greenland (2008)

Greenland suggests various shrinkage approaches here, which are difficult to summarize with a few brief quotes. Also it is worth looking at some of the approaches in Sauer (2013).

References

Marlies Wakkee, Loes M Hollestein and Tamar Nijsten. “Multivariable Analysis” Journal of Investigative Dermatology (2014) 134, e20. doi:10.1038/jid.2014.132. Available at http://www.nature.com/jid/journal/v134/n5/full/jid2014132a.html.

Sander Greenland (2007) “Invited Commentary: Variable Selection versus Shrinkage in the Control of Multiple Confounders”  American Journal of Epidemiology. Volume 167, Issue 5, Pp. 523-529. Available at http://aje.oxfordjournals.org/content/167/5/523.full.

Brian C. Sauer, Alan Brookhart, Jason Roy, and Tyler Vanderweele (2013) “A Review of Covariate Selection for Nonexperimental Comparative Effectiveness Research” Pharmacoepidemiol Drug Saf. 2013 Nov; 22(11): 1139–1145. doi: 10.1002/pds.3506. Available at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4190055/.

A covariate is a variable that is not of direct interest in your study, but one that you need to account for because it might potentially influence your outcome variable. If a covariate is imbalanced between the treatment and control group, then there is potential for bias and you need to look at an adjusted model (sometimes called risk adjustment).

For example, I was looking at a study of soldiers who were part of the artillery. They would fire these big guns and it would shoot forward a large projectile. But at the same time, the big guns would send a puff of smoke backwards. This smoke has a fair amount of lead in it. The scientist wanted to know if the lead exposure caused problems with the soldiers’ sperm counts.

This was a study that you could not randomize because (among other things) if you randomly assigned half of a group of soldiers who had never fired artillery before to go out and fire the big guns for an afternoon, their lack of training would probably cause them to kill themselves.

So you had to take a cohort of soldiers with artillery experience and compare them to a cohort of solders with a totally different and lead free experience.

So that’s what this scientist did and when he collected the data he noticed an imbalance in a key covariate: smoking.

Anyone who works around artillery is strongly discouraged from smoking anywhere near the ammunition. Maybe they could smoke while off duty, but it turns out that smokers were almost non-existent in the artillery cohort. In the controls, smoking was not very high but certainly a lot higher than in the artillery group.

Does smoking influence sperm counts? The data is mixed on this, but we decided that we had to adjust for smoking in any comparison of artillery soldiers versus the controls.

So that’s an example of covariate imbalance. I’m sure your application doesn’t require your subjects to fire Howitzers, but I hope you can still draw a rough analogy to your work.

Now covariate imbalance is not quite the same thing as confounding, but the distinction between the two is often not that important.