Tag Archives: Sample size

PMean: Sample size for a study of reproducibility

Dear Professor Mean: I am using a risk stratification tool for patients presenting to the ED with chest pain. This has been a well validated tool in the ED, but I want to show that the scores are reproducible irrespective of the grade of doctor or assessment nurse calculating the score. I’m going to collect a convenience sample of patients presenting to the ED, and after I get informed consent, I will have those patients assessed separately by a triage-trained nurse, an intern doctor, a registrar and a consultant. I will calculation agreement using the intraclass correlation coefficient (ICC). My question is: How do I calculate the sample size in this context? Continue reading →

Pmean: The IRB questions my sample size calculation

I got a question today from someone submitting a research protocol to the Institutional Review Board (IRB). The IRB had some concerns about the power calculation. In particular, they said “The IRB would like to know, how you set the parameters for the power calculation, such as effect size, alpha level. For effect size, you need to have some data to justify or should choose a conservative one.”

Part of this was due to an error in the submission of the protocol. We had specified a paired t-test rather than an independent samples t-test, which is a major gaffe on my part. But they were pushing into some tricky territory and I wanted to clear things up. Here is the response that I suggested that we share with the IRB. Continue reading →

PMean: Stretching an already borderline sample size

I was working with a client who had a limited population of medical residents to choose from, and it would be a struggle to get even 60 of them. The primary outcome was binary: whether a certain medical procedure was run properly in a test setting. The intervention was special training on a model; the control was normal training without the model. I got a phone call back that said, what would the power be if I used three groups rather than two?  I thought to myself “Good grief!” You can’t say that to a client, of course, so here’s what I said. Continue reading →

Recommended: Why randomized controlled trials fail but needn’t: 2. Failure to employ physiological statistics, or the only formula a clinician-trialist is ever likely to need (or understand!)

This is a classic article about the relationship between signal, noise, sample size, and confidence. It provided pragmatic guidance on designing a trial to make the best use of limited resources. It should be the first article that you read before you sit down and design a clinical trial. Continue reading →

PMean: No power calculation for a Phase II trial

There was an discussion on the message board for the Statistical Consulting Section of the American Statistical Association started by a question about a Phase II trial. The questioner was part of an Institutional Review Board and was reviewing a proposal for a Phase II clinical trial. This particular trial had a fairly small sample size with no justification of the choice of sample size. The questioner wanted to know if this was the norm for Phase II trials. Here are some of my thoughts combined with a synthesis of other comments. Continue reading →