Bayesian Analysis with Python: Introduction to statistical modeling and probabilistic programming using PyMC3 and ArviZ

Description

Bayesian modeling with PyMC3 and exploratory analysis of Bayesian models with ArviZ Key Features • A step-by-step guide to conduct Bayesian data analyses using PyMC3 and ArviZ • A modern, practical and computational approach to Bayesian statistical modeling • A tutorial for Bayesian analysis and best practices with the help of sample problems and practice exercises. Book Description The second edition of Bayesian Analysis with Python is an introduction to the main concepts of applied Bayesian inference and its practical implementation in Python using PyMC3, a state-of-the-art probabilistic programming library, and ArviZ, a new library for exploratory analysis of Bayesian models. The main concepts of Bayesian statistics are covered using a practical and computational approach. Synthetic and real data sets are used to introduce several types of models, such as generalized linear models for regression and classification, mixture models, hierarchical models, and Gaussian processes, among others. By the end of the book, you will have a working knowledge of probabilistic modeling and you will be able to design and implement Bayesian models for your own data science problems. After reading the book you will be better prepared to delve into more advanced material or specialized statistical modeling if you need to. What you will learn • Build probabilistic models using the Python library PyMC3 • Analyze probabilistic models with the help of ArviZ • Acquire the skills required to sanity check models and modify them if necessary • Understand the advantages and caveats of hierarchical models • Find out how different models can be used to answer different data analysis questions • Compare models and choose between alternative ones • Discover how different models are unified from a probabilistic perspective • Think probabilistically and benefit from the flexibility of the Bayesian framework Who this book is for If you are a student, data scientist, researcher, or a developer looking to get started with Bayesian data analysis and probabilistic programming, this book is for you. The book is introductory so no previous statistical knowledge is required, although some experience in using Python and NumPy is expected. Chapter 1: Thinking Probabilistically......Page 20 Statistics, models, and this book's approach......Page 21 Working with data......Page 22 Bayesian modeling......Page 23 Interpreting probabilities......Page 24 Defining probabilities......Page 25 Probability distributions......Page 26 Independently and identically distributed variables......Page 30 Bayes' theorem......Page 32 The coin-flipping problem......Page 35 Choosing the likelihood......Page 36 Choosing the prior......Page 38 Getting the posterior......Page 40 Computing and plotting the posterior......Page 41 The influence of the prior and how to choose one......Page 44 Model notation and visualization......Page 46 Highest-posterior density......Page 47 Posterior predictive checks......Page 48 Summary......Page 50 Exercises......Page 52 Chapter 2: Programming Probabilistically......Page 54 Probabilistic programming......Page 55 PyMC3 primer......Page 56 Model specification......Page 57 Pushing the inference button......Page 58 Summarizing the posterior......Page 59 ROPE......Page 61 Loss functions......Page 63 Gaussian inferences......Page 66 Robust inferences......Page 72 Student's t-distribution......Page 73 Groups comparison......Page 78 Cohen's d......Page 80 Probability of superiority......Page 81 The tips dataset......Page 82 Hierarchical models......Page 86 Shrinkage......Page 90 One more example......Page 94 Summary......Page 98 Exercises......Page 99 Chapter 3: Modeling with Linear Regression......Page 101 The machine learning connection......Page 102 The core of the linear regression models......Page 103 Linear models and high autocorrelation......Page 108 Modifying the data before running......Page 109 Interpreting and visualizing the posterior......Page 111 Pearson correlation coefficient......Page 114 Pearson coefficient from a multivariate Gaussian......Page 115 Robust linear regression......Page 118 Hierarchical linear regression......Page 123 Correlation, causation, and the messiness of life......Page 129 Polynomial regression......Page 130 Polynomial regression – the ultimate model?......Page 133 Multiple linear regression......Page 134 Confounding variables and redundant variables......Page 139 Multicollinearity or when the correlation is too high......Page 142 Masking effect variables......Page 147 Adding interactions......Page 149 Variable variance......Page 150 Exercises......Page 154 Chapter 4: Generalizing Linear Models......Page 157 Generalized linear models......Page 158 Logistic regression......Page 159 The logistic model......Page 160 The Iris dataset......Page 161 The logistic model applied to the iris dataset......Page 164 The boundary decision......Page 167 Implementing the model......Page 168 Interpreting the coefficients of a logistic regression......Page 169 Dealing with correlated variables......Page 172 Dealing with unbalanced classes......Page 174 Softmax regression......Page 176 Discriminative and generative models......Page 178 Poisson distribution......Page 181 The zero-inflated Poisson model......Page 183 Poisson regression and ZIP regression......Page 185 Robust logistic regression......Page 188 The GLM module......Page 189 Summary......Page 190 Exercises......Page 191 Chapter 5: Model Comparison......Page 193 Posterior predictive checks......Page 194 Occam's razor – simplicity and accuracy......Page 199 Too many parameters leads to overfitting......Page 201 The balance between simplicity and accuracy......Page 203 Predictive accuracy measures......Page 204 Cross-validation......Page 205 Log-likelihood and deviance......Page 206 Akaike information criterion......Page 207 Pareto smoothed importance sampling leave-one-out cross-validation......Page 208 Model comparison with PyMC3......Page 209 Model averaging......Page 212 Bayes factors......Page 216 Some remarks......Page 217 Computing Bayes factors......Page 218 Common problems when computing Bayes factors......Page 221 Using Sequential Monte Carlo to compute Bayes factors......Page 222 Bayes factors and Information Criteria......Page 223 Regularizing priors......Page 226 WAIC in depth......Page 227 Entropy......Page 228 Kullback-Leibler divergence......Page 230 Summary......Page 233 Exercises......Page 234 Chapter 6: Mixture Models......Page 235 Mixture models......Page 236 Finite mixture models......Page 237 The categorical distribution......Page 239 The Dirichlet distribution......Page 240 Non-identifiability of mixture models......Page 244 How to choose K......Page 246 Non-finite mixture model......Page 251 Dirichlet process......Page 252 Beta-binomial and negative binomial......Page 260 The Student's t-distribution......Page 261 Summary......Page 262 Exercises......Page 263 Chapter 7: Gaussian Processes......Page 264 Linear models and non-linear data......Page 265 Modeling functions......Page 266 Covariance functions and kernels......Page 268 Gaussian processes......Page 271 Gaussian process regression......Page 272 Regression with spatial autocorrelation......Page 279 Gaussian process classification......Page 285 The coal-mining disasters......Page 293 The redwood dataset......Page 296 Exercises......Page 300 Chapter 8: Inference Engines......Page 302 Inference engines......Page 303 Grid computing......Page 304 Quadratic method......Page 307 Variational methods......Page 309 Markovian methods......Page 312 Monte Carlo......Page 314 Metropolis-Hastings......Page 316 Hamiltonian Monte Carlo......Page 321 Sequential Monte Carlo......Page 322 Diagnosing the samples......Page 325 Convergence......Page 326 Autocorrelation......Page 331 Effective sample sizes......Page 332 Divergences......Page 333 Non-centered parameterization......Page 336 Exercises......Page 337 Chapter 9: Where To Go Next?......Page 339