Bayesian Statistics

The two most prominent schools of thought in statistics are called Bayesian (named after Reverend Thomas Bayes) and Frequentist or Classical statistics.

The photograph on the right is Sir Ronald Fisher, one of the key architects of the Frequentist approach and a staunch critic of the Bayesian approach. Fisher's stature and strident opposition to the Bayesian approach ("It must be wholly rejected.") was largely responsible for it being very much out of favor for a good part of the 20th century. When I was a graduate student in statistics in the 1970s, it was considered heresy.

The drawing on the left has been described in many references as a portrait of Bayes. What we can say is that it is almost certainly not him. Experts on the garments worn in the period by various denominations of clergy have confirmed that those are not the garments that Bayes would have worn.

Use of the alternative term Classical to describe the Frequentist approach is somewhat misleading, because the Bayesian approach predates the Frequentist by a century.

While a few prominent statisticians continued to argue that the Bayesian approach had value and deserved to be considered, they were hampered by a very inconvenient truth: most of the interesting real-world Bayesian models are analytically intractable. That is to say, you can write down the equations involved in a Bayesian data analysis, but no one was able to solve them algebraically.

In 1953, physicists working on the Manhattan Project developed an algorithm to assist with modeling neutron diffusion. The team was led by Nicholas Metropolis and included Edward Teller and his wife. In 1990, Alan Gelfand, a statistics professor at the University of Connecticut on sabbatical in England published a paper with British statistician Adrian Smith that showed how the Metropolis Algorithm could be used to generate observations from the intractable probability distributions of Bayesian statistics.

The repurposing of the Metropolis Algorithm coincided in time with the advent of personal computers powerful enough to perform the intensive computations required, and the availability in the late 1990s of freely available software (Winbugs) that implemented the algorithms generated an explosion of interest in the Bayesian approach. Today it can be said to be on an equal footing with Frequentist statistics and is very widely used in science and engineering.

That said, aside from Economics and Actuarial Science programs, Bayesian statistics has not yet found its way into the standard business school curriculum. In 2017 I audited the first graduate Bayesian statistics course at URI, and in 2018 I taught the first Bayesian statistics course at Stonehill College. The professor in the URI course was Gavino Puggioni. Alan Gelfand, now at Duke, was Dr. Puggioni's thesis advisor.

Nearly 35 years have passed since Gelfand and Smith's paper was published, and a great deal of effort has gone into developing and improving the methods it pioneered, known as Markov Chain Monte Carlo methods.

One of the best tools available for Bayesian statistical modeling is Stan. First released in 2012, Stan implements a number of important algorithms for Bayesian data analysis, including a third generation Markov Chain Monte Carlo algorithm known as Hamiltonian Monte Carlo.

For a sense of the variety of problems people are using Stan to solve, see the Modeling Forum

The following articles discuss the Bayesian statistics process, which is different from the classical or frequentist process in many respects:
• Gabry et al,Visualization in Bayesian Workflow, Journal of the Royal Statistical Society Series A: Statistics in Society, Volume 182, Issue 2, February 2019, Pages 389–402, https://doi.org/10.1111/rssa.12378
• Gelman et al,Bayesian Workflow, arXiv:2011.01808 [stat.ME]