Someone wanted to know how to run a Bayesian data analysis for a two group longitudinal study. There are several ways you can do this, but I had to confess I did not have an immediate answer. So I took some time to figure out how to do this using jags software inside of R. I’ve done a fair amount of stuff in jags, but not anything close to a longitudinal design. The general principle is to start with something easy and work your way slowly up to the final analysis.

You can find the code and some supporting text files at

https://github.com/pmean/bayesian-longitudinal

I usually start with a very simple Bayesian analysis because if something is awry, I’ll notice it faster with a simple example. The code below implements a simple beta-binomial model.

Make sure that you have jags installed on your computer

http://sourceforge.net/projects/mcmc-jags/files/

as well as the rjags package

https://cran.r-project.org/web/packages/rjags/index.html

Assume you have a beta prior distribution with alpha=4 and beta=16. You collect 60 observations and observe 18 successes and 42 failures.

You don’t really need jags for this. The posterior distribution is beta with alpha=4+18=22 and beta=16+42=58. The posterior mean is 0.275 and the posterior standard deviation is 0.05.

library("rjags")

## Loading required package: coda

## Linked to JAGS 4.2.0

## Loaded modules: basemod,bugs

fnm <- "jags_beta_binomial.txt"
mon <- c("pi")

out <- list(description="jags output")

dat <- list(a=4, b=16, x=18, n=60)
mod <- jags.model(fnm, data=dat, quiet=TRUE)
out$bb1 <- 
  coda.samples(mod, variable.names=mon, n.iter=1000, by=1000, progress.bar=NULL)
summary(out$bb1)

## 
## Iterations = 1:1000
## Thinning interval = 1 
## Number of chains = 1 
## Sample size per chain = 1000 
## 
## 1. Empirical mean and standard deviation for each variable,
##    plus standard error of the mean:
## 
##           Mean             SD       Naive SE Time-series SE 
##       0.273157       0.049814       0.001575       0.001575 
## 
## 2. Quantiles for each variable:
## 
##   2.5%    25%    50%    75%  97.5% 
## 0.1811 0.2367 0.2718 0.3086 0.3698

Now, try to run one of the examples in the BUGS manuals that is reasonably close to your problem. For our problem, the very first example in volume 1 works well.

http://www.openbugs.net/Examples/Rats.html

I had to fix two things to get this example to run properly in jags. First, I had to remove some lines of code that used functions like post.p.value. Second, I have to transpose the matrix of data. But with these changes it ran just fine.

dat <- list(x = c(8.0, 15.0, 22.0, 29.0, 36.0), xbar = 22, N = 30, T = 5,   
      Y = structure(
         .Data = c(151, 199, 246, 283, 320,
                     145, 199, 249, 293, 354,
                     147, 214, 263, 312, 328,
                     155, 200, 237, 272, 297,
                     135, 188, 230, 280, 323,
                     159, 210, 252, 298, 331,
                     141, 189, 231, 275, 305,
                     159, 201, 248, 297, 338,
                     177, 236, 285, 350, 376,
                     134, 182, 220, 260, 296,
                     160, 208, 261, 313, 352,
                     143, 188, 220, 273, 314,
                     154, 200, 244, 289, 325,
                     171, 221, 270, 326, 358,
                     163, 216, 242, 281, 312,
                     160, 207, 248, 288, 324,
                     142, 187, 234, 280, 316,
                     156, 203, 243, 283, 317,
                     157, 212, 259, 307, 336,
                     152, 203, 246, 286, 321,
                     154, 205, 253, 298, 334,
                     139, 190, 225, 267, 302,
                     146, 191, 229, 272, 302,
                     157, 211, 250, 285, 323,
                     132, 185, 237, 286, 331,
                     160, 207, 257, 303, 345,
                     169, 216, 261, 295, 333,
                     157, 205, 248, 289, 316,
                     137, 180, 219, 258, 291,
                     153, 200, 244, 286, 324),
                  .Dim = c(5,30))) 

dat$Y <- t(dat$Y)
init <- list(alpha = c(250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250,
                  250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250),
      beta = c(6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
                  6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6),         
      alpha.c = 150, beta.c = 10,
      tau.c = 1, alpha.tau = 1, beta.tau = 1) 

fnm <- "jags_rats.txt"
mon <- c("alpha0", "beta.c", "sigma")

mod <- jags.model(fnm, dat, init, quiet=TRUE)
out$rats <- 
  coda.samples(mod, variable.names=mon, n.iter=1000, by=1000, progress.bar=NULL)
summary(out$rats)

## 
## Iterations = 1:1000
## Thinning interval = 1 
## Number of chains = 1 
## Sample size per chain = 1000 
## 
## 1. Empirical mean and standard deviation for each variable,
##    plus standard error of the mean:
## 
##           Mean     SD Naive SE Time-series SE
## alpha0 106.283 3.5884 0.113476       0.125482
## beta.c   6.196 0.1072 0.003391       0.004276
## sigma    6.108 0.6291 0.019893       0.033978
## 
## 2. Quantiles for each variable:
## 
##          2.5%     25%     50%     75%   97.5%
## alpha0 99.495 104.008 106.248 108.670 113.344
## beta.c  5.996   6.128   6.193   6.268   6.399
## sigma   5.205   5.735   6.060   6.402   7.227

Now let’s see if we can modify this example to fit our particular setting.

Our longitudinal setting has two measurements: pre and post. So strip the last three columns from the matrix Y. There is one within subject factor, time, which is represents by X1. There is a between subjects factor, treatment group, which is represented by X2. Let’s pretend for now that that the first twenty observations represent your treatment group and the last ten observatitons represent your control group.

dat <- list(X1 = c(0, 1), X2 = rep(0:1, c(20, 10)), N = 30, T = 2, 
      Y = structure(
         .Data = c(151, 199,
                     145, 199,
                     147, 214,
                     155, 200,
                     135, 188,
                     159, 210,
                     141, 189,
                     159, 201,
                     177, 236,
                     134, 182,
                     160, 208,
                     143, 188,
                     154, 200,
                     171, 221,
                     163, 216,
                     160, 207,
                     142, 187,
                     156, 203,
                     157, 212,
                     152, 203,
                     154, 205,
                     139, 190,
                     146, 191,
                     157, 211,
                     132, 185,
                     160, 207,
                     169, 216,
                     157, 205,
                     137, 180,
                     153, 200),
                  .Dim = c(2,30))) 

dat$Y <- t(dat$Y)
init <- list(alpha = rep(150, 30), beta1 = rep(50, 30),
      beta1.c = 50, beta1.tau=1,
      beta2 = 20, beta3 = 5,
      alpha.c = 150,
      tau.c = 1, alpha.tau = 1) 

fnm <- "jags_lon.txt"
mon <- c("alpha.c", "beta1.c", "beta2", "beta3")

mod <- jags.model(fnm, dat, init, quiet=TRUE)
out$lon <- 
  coda.samples(mod, variable.names=mon, n.iter=1000, by=1000, progress.bar=NULL)
summary(out$lon)

## 
## Iterations = 1:1000
## Thinning interval = 1 
## Number of chains = 1 
## Sample size per chain = 1000 
## 
## 1. Empirical mean and standard deviation for each variable,
##    plus standard error of the mean:
## 
##            Mean    SD Naive SE Time-series SE
## alpha.c 154.095 2.576  0.08145         0.4494
## beta1.c  50.186 1.136  0.03593         0.1414
## beta2    -5.756 4.172  0.13193         1.4509
## beta3    -1.593 1.723  0.05447         0.5767
## 
## 2. Quantiles for each variable:
## 
##            2.5%     25%      50%      75%    97.5%
## alpha.c 148.583 152.538 154.2840 155.7871 158.7248
## beta1.c  47.995  49.452  50.1588  50.9269  52.4221
## beta2    -8.877  -7.483  -7.1730  -6.8746   9.7413
## beta3    -5.556  -2.352  -0.9216  -0.6312   0.9299

Okay, so far, so good. Your next step would be to put your own data in place of the data from the example. I won’t show that here because I don’t have your data. But I hope you can try the general outline suggested here.

1. Fit the simplest model possible, a beta-binomial.
2. Fit an example taken from the BUGS software that is reasonably close to your example.
3. Modify the example to match some of the structure of your data set, but also keep some of the original data.
4. Substitute your data for the example data.

You might want to iterate step 3, making a small change to match your data a bit more closely, then another small change, and so forth.

Most of the books are very mathematical. Your best bet might be Doing Bayesian Data Analysis, Second Edition: A Tutorial with R, JAGS, and Stan by John Kruschke. It starts at a fairly basic level and tries to illustrate many of the Bayesian approaches using a grid approximation. That’s explained in the book, but a grid approximation is an easy way to get comfortable with Bayesian statistics without having to rely on a lot of Calculus. That’s not to say that the book isn’t a challenge to read. It’s just that most of the other books that I’m aware of are far harder to read.

You might also read “The Signal and the Noise” by Nate Silver. You won’t understand Bayesian statistics when you’re done reading the book, but you’ll understand why we need Bayesian statistics.

If you run a google search on the hierarchical beta binomial model, you see a lot of helpful choices.  Many of these are PDFs and my system does not let me track the URLs of PDF files that easily. But you should be able to find these easily. Also, many of these PDFs are Powerpoint slides. Powerpoint slides are usually too sketchy to be useful, but these slides are pretty nice.

There are at least three general approaches and within each approach, there are differing recommendations.

Approach #1. Assume a beta prior on the binomial proportions, model the hyperparameters of the beta distribution directly.

Conjugate Priors, Hierarchical Models, Comparing Ranks through Simulation (no author given) starts with a binomial distribution and places a beta prior on that distribution. It then suggests two gamma distribution for the hyperparameters of the beta distribution, though this reference does not say which particular gamma distribution.

Brice D. L. Acree & Thomas Carsey, Bayesian Hierarchical Models, June 4, 2015 proposes that the binomial proportions come from a beta distribution and that the two hyperparameters from the beta distribution are both uniform on the interval 0 to 10.

Approach #2. Assume a beta prior distribution on the binomial proportions, but transform the hyperparameters of the beta distribution.

Kathryn Blackmond Laskey, Bayesian Inference and Decision Theory, Unit 7: Hierarchical Bayesian Models recommends that you transform the hyperparameters of the beta distribution. Dr. Laskey proposes this transformation because the two hyperparameters of the beta distribution are so closely correlated that the Gibbs sampling is inefficient. Her transformation, and the only transformation that others consider is U=alpha/(alpha+beta) and V=alpha+beta. U is the mean of the beta distribution and V is rough measure of the precision of the beta distribution with larger values implying more precision. She suggests a uniform distribution for U and a gamma(1,20) distribution for V. She uses the rat tumor data from Gelman et al.

Brian Junker, 36-724 Spring 2006: Metropolis-Hastings Example, February 22, 2006 also uses a beta distribution for alpha/(alpha+beta) but suggests a Pareto distribution for (alpha+beta).  The Pareto is a power law distribution and the power that Dr. Junker recommends 1.5. You have to read his arguments, but it involves a transformation of a fairly standard normal hierarchical model.

Manojit Nandi, A/B Testing with Hierarchical Models in Python – Domino Blog, also recommends the same Pareto distribution, and cites Andrew Gelman’s book (Bayesian Data Analysis, with John Carlin, Hal Stern, David Dunson, Aki Veharti, and Donald Rubin) as the rationale.

Bob Carpenter, Hierarchical Bayesian Batting Ability, with Multiple Comparisons, also uses the Pareto distribution, but with no references or other justification.

Andrew Gelman, Some practical questions about prior distributions, October 21, 2009 reviews his earlier book recommendations and suggests that perhaps a half Cauchy distribution for 1/sqrt(alpha+beta) would be better than his earlier recommendation of an improper uniform(0,infinity) prior on that same parameter.

Approach #3. Transform the binomial proportions to something that can be easily modeled using normal distributions.

Pablo Verde, Lecture 12: Introduction to Hierarchical Models recommends a logit transformation of the binomial proportion. You attach a normal prior to these logits. The mean of the normal prior is assumed to be normal(0, 100) and the standard deviation is assumed to be uniform(0.01, 10).

Nicky Best, Alexina Mason and Philip Li, Bayesian Hierarchical Modelling using WinBUGS, February 17-18, 2011 recommends a logit transformation of the binomial proportion. You place a wide uniform distribution on the mean of the normal distribution (e.g., U(-100, 100)) or a normal distribution with very large variance. The distribution that you place on the variance or standard deviation of the normal distribution is tricky. They consider gamma priors for the reciprocal of the variance, a uniform(0, 100) distribution for the variance, a uniform(0,100) distribution for the standard deviation (not recommended, they say), and a half normal distribution with mean zero and variance 0.4^2 for the standard deviation.

Are you confused yet? The good news is that you have some latitude in making choices that fit your individual needs. The bad news is that when you try to publish, the peer reviewers are going to make you change your prior choice to the one they like best, and there’s no way of predicting what their choice will be.

I should read some of the books in my library and check out some peer-reviewed references.

Havard Rue. Bayesian computing with INLA. Available at http://www.r-inla.org/home.

This page has moved to a new website.

Trafimow D and Marks M (2015) Editorial. Basic and Applied Psychology, 37(1). DOI: 10.1080/01973533.2015.1012991. Available at: http://www.tandfonline.com/doi/full/10.1080/01973533.2015.1012991

This is a question that is Bayesian. If you believe in classical statistics, the probability that the population contains only Gs is either zero or one. There is nothing random in a population, so you can’t assign any meaningful probability statements to the population.

If you are a Bayesian, then this sort of thing doesn’t bother you. You have a parameter, pi, and you place a prior distribution on pi. Collect some data and then combine it with the prior using Bayes rule to get a posterior distribution. How would this work in practice?

Well, the classic example is to put a beta prior on the binomial distribution. Often this is a flat or uniform prior, which is a beta(1,1). Here’s what the prior looks like.

And now suppose that you observe 10 values from the sample and they are all Gs. The posterior distribution is now beta(11,1), which looks like this.

Most of the probability is concentrated above 0.7, which makes sense.

The problem with the beta prior distribution, though, is that it does not assign a non-zero probability to the probability of 0 or 1 or any other value. Let’s change the prior slightly. There are N items in the population and let’s assign a discrete uniform probability to the values 0, 1/n, 2/n, …, 1. Here’s what this looks like for N=100.

Now, suppose that you observe 10 patients in this sample and they are all Gs. The posterior distribution would look something like this.

The probability that pi equals one is roughly 10%. When I get a chance, I will show the math for this example, and maybe the R code.

First, you need to remember the basics of Bayesian statistics. Bayesian statistics takes a different philosophical approach than traditional hypothesis testing. It views the parameters of a hypothesis as random variables. These random variables are described using a prior distribution. You collect some data and examine the likelihood of the data given the prior distribution. Using Bayes Theorem, you combine the prior distribution and the likelihood to get a posterior distribution.

Here’s a simple practical example. You are running a randomized clinical trial that needs to recruit 350 patients. You’re not sure how long it will take to recruit this many patients. You’re thinking it will take three years to recruit that many patients. This implies that you will be able to find a new patient every 3.1 days on average. There is substantial uncertainty about this estimate, though and we characterize the waiting time distribution using an inverse gamma distribution with parameters k=175 and V=182.5. Here’s what that distribution looks like:

Now, you start running your clinical trial, and you notice that things are going slower than planned. After 239 days, you’ve recruited only 41 subjects. The average waiting time is 239 /41 = 5.8 days per patient. What is your expected completion time now?

You combine your prior beliefs with the existing data to get a posterior distribution of the waiting time, which turns out to be a weighted average of the observed accrual data and your prior beliefs. This leads to an updated prediction of the completion time for the clinical trial.

This use of prior distributions is extremely controversial, because it incorporates subjective opinions into the data analysis process. Often a Bayesian data analyst will choose a diffuse or non-informative prior distribution. This represents a prior distribution that is spread so thinly across the possible parameters that it does not influence the posterior distribution. Here’s what such a distribution would produce in the patient accrual example.

Notice that this represents a simple linear extrapolation of the waiting times. These graphs come from a webpage I wrote a while back.

The mathematics of Bayesian data analysis is quite complex. Here’s a description on the Wikipedia page on posterior distributions.

The denominator, the normalizing constant, is a killer. Often the integral is impossible to compute. It is often difficult to come up with a numerical estimate. You might recall the trapezoidal rule and Simpson’s rule from Calculus, but this integral is often mutli-dimensional, making these approaches impractical.

Back in the early 1990s statisticians discovered a new computer simulation approach that avoids the complexities of the normalizing constant. It was actually a rediscovery of methods developed much earlier, but which were largely unknown in the Statistics community. This approach, which goes under a variety of names, such as the Metropolis algorithm, the Gibbs sampler, and Markov Chain Monte Carlo requires only that you can specify the numerator of the equation above. It uses an acceptance/rejection approach where the estimate of the posterior distribution either stays at its current location or jumps to a different location. I have a nice example of how this works in a very simple unidimensional case, but this process works very efficiently even in the multidimensional case.

The simulation approach has two issues that you must account for at all times. First, the consecutive values in the simulation are positively correlated to a more than trivial amount. If you don’t get a huge number of simulations, the results will look “patchy” and will be a poor representation of the posterior distribution. Often you will need thousands or tens of thousands of values.

Second, the simulation has to be given a starting value and it has to be given time to converge. You will often need to run the simulation a thousand times or more before the results become reliable. Diagnosing convergence is tricky. One possibility is to run the same problem with two different starting points and see when the two distributions mix properly.

This example from a webpage discussing segmented linear regression models shows that it is naive to expect convergence after a thousand simulations and that perhaps as many as 3,000 simulations should be run before convergence.

The software to run these simulations was developed by a group headed by David Spiegelhalter. Their program, WinBUGS, is with a difficult syntax (which is an unavoidable manifestation of the complexity of the simulation models) and a menu driven interface. WinBUGS, as the name implies, is Windows specific. A newer version, OpenBUGS, was written to run under a wider range of operating systems. OpenBUGS was written in the Component Pascal programming language, and that choice has led to other difficulties with portability, so a new program, JAGS, was developed in C++.

The acronym BUGS stands for Bayes Using Gibbs Sampler and the acronym JAGS stands for Just Another Gibbs Sampler.

If you want to run WinBUGS, OpenBUGS, or JAGS, you should do it within R, as it bypasses the tedious menu driven interface and it allows you to use all the data management capabilities within R. There are a variety of interfaces, but you need to be aware of issues associated with the Windows specific operating systems, including the use of 32 bit versus 64 bit Windows. I wrote up a webpage covering some of these complexities.

I have recently been using JAGS and the R interface to JAGS (rjags). It seems to have fewer compatibility issues and the error messages are a bit less cryptic. Here’s an example of how to run JAGS from within R.

Why would you want to learn Bayesian data analysis software? Bayesian data analysis is moderately complex, but it turns out that this level of complexity does not get worse as the problems themselves get more complex. This is in contrast to classical methods that are easy for easy problems, but which escalate rapidly into the incomprehensible.

Bayesian methods are especially useful for multi-level models (also known as mixed models), which are difficult to the point of being almost intractactable using classical methods.