I got an email asking for a recommendation for an introductory book on Bayesian Statistics from someone who recently graduated from our program. It’s kind of a difficult request because the mathematical demands needed to understand Bayesian statistics are not trivial. Here’s what I recommended.
Most of the books are very mathematical. Your best bet might be Doing Bayesian Data Analysis, Second Edition: A Tutorial with R, JAGS, and Stan by John Kruschke. It starts at a fairly basic level and tries to illustrate many of the Bayesian approaches using a grid approximation. That’s explained in the book, but a grid approximation is an easy way to get comfortable with Bayesian statistics without having to rely on a lot of Calculus. That’s not to say that the book isn’t a challenge to read. It’s just that most of the other books that I’m aware of are far harder to read.
You might also read “The Signal and the Noise” by Nate Silver. You won’t understand Bayesian statistics when you’re done reading the book, but you’ll understand why we need Bayesian statistics.