Some asked a question about a retrospective study where you have a control cohort matched to a case cohort so the cohorts are similar on important (potentially confounding) variables. I pointed out that the two consecutive words “case cohort” are ambiguous and tried to explain  how I define a retrospective cohort design versus a (retrospective) case-control design.

As I understand it, a cohort is a group of patients defined by an exposure. A case is a patient who has a particular outcome. So the phrase “case cohort” is ambiguous. If you are mining the electronic health records and select every child who received aspirin to control a fever and compare them to a control group of children who received acetaminophen to control a fever with the hypothesis that Reye’s syndrome is more likely in the aspirin cohort, that’s a retrospective cohort design. If you are mining the same electronic health records and select every child who came down with Reye’s syndrome and compare them to a control group of children who did not experience Reye’s syndrome with the hypothesis that aspirin was taken more often among the Reye’s syndrome group, that’s a case control design.

It gets a little bit tricky because context is important. Low birth weight is an outcome to an obstetrician, but it is an exposure to a neonatologist. But if there is a temporal order where A precedes B in time and A is thought to be a cause of B, then defining your controls as not A makes it a cohort study and defining your controls as not B makes is a case control study.

I suspect you are using the word “cohort” in its generic sense meaning a group of people who share something in common. Or maybe you are using the word “case” in its generic sense meaning an instance. But if you are defining a case cohort from an Epidemiologic perspective, I would argue that it represents the intersection of A and B (all children who take aspirin AND develop Reye’s syndrome). If you think about this long enough, you realize that a “case cohort” makes no sense from a statistical perspective because no matter how you choose your controls, you’re going to end up missing one of the four cells in the two by two classification of A and B.

Another way of thinking about it is the direction of travel. If you retrospectively select on B and not B and go further back in time to see who does or does not have A, you have a case control design. If you retrospectively select on A and not A and then move forward in time (but not all the way back to the present) to see who does or does not have B, you have a retrospective cohort design.

Now there’s such a thing as a nested case control design. So you study a cohort versus a control (usually prospectively) and you find that some of your cohort develop your outcome and some do not. You’d like to run some additional tests (e.g., genetics) on all of the cases within your cohort, but it’s too expensive to run the tests on all the non-cases within your cohort. So you get a subset of non-cases from your cohort and because your sample size is smaller, you do some matching. So you have a case control design nested within a cohort design. Your hypothesis is that a positive result on the expensive test is more likely among cases than controls. Note that both cases and controls are drawn from the exposed cohort–the unexposed control cohort patients do not participate in a nested case-control study.

It’s also possible to nest a case control design within a randomized trial. Select all of the patients randomized to the active arm AND who experience the outcome of interest. Run a super expensive test on these patients. Now pick a small but carefully matched set of patients in the active arm who did not experience the outcome of interest and run those same tests.

To make things even more complicated, there is a case-cohort design and a case-crossover design. See 7.2 – Advanced Case-Control Designs | STAT 507 for details. I think I’ll stop here before my head explodes.

I can only offer a subjective impression, but in my experience, the projects that take a lot longer are secondary data analyses. There are two reasons for this. First, the data management in a secondary data analysis is often a lot harder than you might expect. You find a lot of unexpected inconsistencies that you have to resolve before you can get confident that you have the right results.

Second, journal reviewers are often pretty harsh. They’ll tell you that your data set is the wrong data set for your hypothesis. When this happens, it’s really hard to argue back and you can’t re-analyze the data a different way if they are convinced you have the wrong data. Your only solution is to go to a different journal. Another problem with reviewers is that they sometimes tend to be nitpickers and secondary data analysis has lots of pickable nits if you really want to go down that road.

For what it’s worth, a small retrospective chart review of your own patients, which is a type of secondary data analysis, doesn’t seem to have this problem. It’s when you are using someone else’s data and not your own that the problems occur.

Wacholder S et al. Selection of Controls in Case-Control Studies. I. Principes. American Journal of Epidemiology 1992: 135(9); 1019-1028. Available at http://aje.oxfordjournals.org/content/135/9/1019.abstract.

You need to be a bit careful here. An observational study means (more or less) that the patients get to select the treatment they are getting rather than having their choice be dictated by the flip of a coin. Any time the patients get to choose, you have the risk of selection bias, even if 100% percent of the patients participate in the observational study.

Now I know there are some observational studies where patients don’t get to choose, such as a study comparing BRCA1 to BRCA2 breast cancer patients, but the overall point is still valid.

Problems when patients fail to respond to a call to participate in a survey, for example, might be better characterized as nonresponse bias. There is no consensus in the research community as to how low a participation rate would cause concerns about nonresponse bias. In some settings, you have basic demographic information about the nonresponders, and you can mitigate concerns about nonresponse if you can show that the demographic profile of the nonresponders is comparable to the demographic profile of the responders.

I personally do not worry about nonreponse if the response rate is greater than 90%, I start to worry if the response rate drops below 70% and I get very nervous if it drops below 50%. But this is a very arbitrary choice on my part. You can argue some of these numbers based on a sensitivity analysis, perhaps.

I have not seen any published cutpoints, arbitrary or not, in the peer-reviewed literature. If there are any, I’d love to see them.

Not all scientific studies are created equal – David H. Schwartz.; 2014. Available at: https://www.youtube.com/watch?v=GUpd2HJHUt8. Accessed May 1, 2014.