Discussion

Bayesian statistics for confused data scientists

statskier: I went through grad school in a very frequentist environment. We “learned” Bayesian methods but we never used them much.In my professional life I’ve never personally worked on a problem that I felt wasn’t adequately approached with frequentist methods. I’m sure other people’s experiences are different depending on the problems you gravitate towards.In fact, I tend to get pretty frustrated with Bayesian approaches because when I do turn to them it tends to be in situations that already quite complex and large. In basically every instance of that I’ve never been able to make the Bayesian approach work. Won’t converge or the sampler says it will take days and days to run. I can almost always just resort to some resampling method that might take a few hours but it runs and gives me sensible results.I realize this is heavily biased by basically only attempting on super-complex problems, but it has sort of soured me on even trying anymore.To be clear I have no issue with Bayesian methods. Clearly they work well and many people use them with great success. But I just haven’t encountered anything in several decades of statistical work that I found really required Bayesian approaches, so I’ve really lost any motivation I had to experiment with it more.

storus: A large portion of generative AI is based on Bayesian statistics, like stable diffusion, regularization, LLM as a learned prior (though trained with frequentist MLE), variational autoencoders etc. Chain-of-thought and self-consistency can be viewed as Bayesian as well.