When Osvaldo asked me to write the foreword to his new book I felt honored, excited, and a bit scared, so naturally I accepted. What follows is my best attempt to convey what makes probabilistic programming so exciting to me. Osvaldo did a great job with the book, it is the most up-do-date resource you will find and great introduction to get into probabilistic programming, so make sure to grab a copy of Bayesian Analysis with Python: Introduction to statistical modeling and probabilistic programming using PyMC3 and ArviZ, 2nd Edition.
If you need more motivation than that, here is the foreword:
“Probabilistic programming is a framework which allows you to flexibly build Bayesian statistical models in computer code. Once built, powerful inference algorithms, which work independently of the model you formulated, can be used to fit your model to data. This combination of flexible model specification and automatic inference provides a powerful tool for the researcher to quickly build, analyze and iteratively improve novel statistical models. This iterative approach is in stark contrast to the way Bayesian models were fit to data before: previous inference algorithms usually only worked for one specific model. Not only did this require strong mathematical skills to formulate the model and devise an inference scheme, it also considerably slows down the iterative cycle: change the model, rederive your inference. Probabilistic programming thus democratizes statistical modeling by considerably lowering the mathematical understanding and time required to successfully build novel models and gain unique insights into your data.
The idea behind probabilistic programming is not new: BUGS, the first of its kind, was first released in 1989. The kind of models that could be fit successfully was extremely limited and inference was slow, rendering these first-generation languages not very practical. Today, there are a multitude of probabilistic programming languages which are being widely used in academia and at companies like Google, Microsoft, Amazon, Facebook and Uber to solve large and complex problems. What has changed? The key factor in lifting probabilistic programming from being a cute toy to the powerful engine that can solve complex large-scale problems is the advent of Hamiltonian Monte Carlo samplers which are several orders of magnitude more powerful than previous sampling algorithms. While originally devised in 1987, only the more recent probabilistic programming systems named Stan and PyMC3 made these samplers widely available and usable.
This book will give you a practical introduction to this extremely powerful and flexible tool. It will have a deep impact in how you think about and solve complex analytical problems. There are few people better suited to have written it than PyMC3 core developer Osvaldo Martin. Osvaldo has the rare talent of breaking complex topics down to make them easily digestible. His deep practical understanding, gained through hard-won experience allow him to take you, the reader, on the most efficient route through this terrain that could otherwise easily seem impenetrable. The visualizations and code examples make this book an imminently practicable resource through which you will gain an intuitive understanding of their theoretical underpinnings.
I also would like to commend you, dear reader, for having picked up this book. It is not the fast and easy route. In a time where headlines advertise deep learning as the technique to solve all current and future analytical problems, the more careful and deliberate approach of building a custom model for a specific purpose might not seem quite as attractive. However, you will be able to solve problems that can hardly be solved any other way.
This is not to say that deep learning is not an extremely exciting technique. In fact, probabilistic programming itself is not constrained to classic statistical models. Reading the current machine learning literature, you will find that Bayesian statistics is emerging as a powerful framework to express and understand next-generation deep neural networks. This book will thus equip you not only with the skills to solve hard analytical problems, but also to have a front-row seat in humanities perhaps greatest endeavour: the development of artificial intelligence. Enjoy!
Thomas Wiecki, PhD”