Bayesian Scientific Computing

Description

Preface Preface to the 2007 Book Introduction to Bayesian Scientific Computing Contents 1 Bayesian Scientific Computing and Inverse Problems 1.1 What Do We Talk About When We Talk About Random Variables? 1.2 Through the Formal Theory, Lightly 1.2.1 Elementary Probabilities 1.2.2 Probability Distributions and Densities 1.2.3 Expectation and Covariance 1.2.4 Change of Variables in Probability Densities 2 Linear Algebra 2.1 Vectors and Matrices 2.1.1 The Singular Value Decomposition 2.1.2 The Four Fundamental Subspaces 2.2 Solving Linear Systems 2.2.1 What Is a Solution? 2.2.2 Direct Linear System Solvers 3 Continuous and Discrete Multivariate Distributions 3.1 Covariance Matrices 3.2 Normal Distributions 3.3 How Normal is it to be Normal? 3.4 Discrete Distributions 3.4.1 Normal Approximation to the Poisson Distribution 4 Introduction to Sampling 4.1 On Averaging 4.2 Whitening and P–P Plots 4.3 Quadratures and Law of Large Numbers 4.4 Drawing from Discrete Densities 4.5 Sampling from a One-Dimensional Continuous Density 4.6 Sampling from Gaussian Distributions 4.7 Some Useful Sampling Algorithms 4.7.1 Importance Sampling 4.7.2 Drawing from Mixtures: SIR and Weighted Bootstrap 4.8 Rejection Sampling: Prelude to Metropolis–Hastings 5 The Praise of Ignorance: Randomness as Lack of Certainty 5.1 Construction of Likelihood 5.2 Noise Models 5.2.1 Additive Noise 5.2.2 Multiplicative Noise 5.2.3 Poisson Noise 5.2.4 Composite Noise Models 6 Enter Subject: Construction of Priors 6.1 Smoothness Priors 6.1.1 Freeing the Boundary Values 6.2 Generalization to Higher Dimensions 6.3 Whittle–Matérn Priors 6.4 Smoothness Priors with Structure 6.5 Conditionally Gaussian Priors and Hierarchical Models 6.6 Sparsity-Promoting Priors 6.7 Kernel-Based Priors 6.8 Data-Driven Priors 7 Posterior Densities, Ill-Conditioning, and Classical Regularization 7.1 Likelihood Densities and Ill-Posedness of Inverse Problems 7.2 Maximum a Posteriori Estimate and Regularization 8 Conditional Gaussian Densities 8.1 Gaussian Conditional Densities 8.2 Linear Inverse Problems 8.3 Interpolation and Conditional Densities 8.4 Covariance or Precision? 8.5 Some Computed Examples 8.5.1 Bayesian Interpolation of Multi-Dimensional Data 8.5.2 Posterior Density by Low-Rank Updating 9 Iterative Linear Solvers and Priorconditioners 9.1 Iterative Methods in Linear Algebra 9.2 Krylov Subspace Iterative Methods 9.2.1 Conjugate Gradient Algorithm 9.2.2 Conjugate Gradient Method for Least Squares 9.3 Ill-Conditioning and Errors in the Data 9.4 Iterative Solvers in the Bayesian Framework 9.4.1 Preconditioning and Tikhonov Regularization 9.4.2 Priorconditioners: Specially Chosen Preconditioners 9.4.3 Stopping Rule Revisited 10 Hierarchical Models and Bayesian Sparsity 10.1 Posterior Densities with Conditionally Gaussian Priors 10.1.1 IAS, Sparsity, Sensitivity Weighting and Exchangeability 10.1.2 IAS with Priorconditioned CGLS 10.2 More General Sparse Representations 10.3 Some Examples 11 Sampling: The Real Thing 11.1 Preliminaries: Markov Chains and Random Walks 11.1.1 An Introductory Example 11.1.2 Random Walks in mathbbRn 11.2 Metropolis–Hastings Algorithm 11.2.1 Balance and Detailed Balance Equations 11.2.2 Construction of the MH Transition 11.2.3 Metropolis–Hastings in Action 11.3 Gibbs Sampler 11.4 Preconditioned Crank–Nicholson 12 Dynamic Methods and Learning from the Past 12.1 The Dog and the Hunter 12.2 Sampling Importance Resampling (SIR) 12.2.1 Survival of the Fittest 12.2.2 Estimation of Static Parameters 13 Bayesian Filtering for Gaussian Densities 13.1 Kalman Filtering 13.2 The Best of Two Worlds: Ensemble Kalman Filtering 13.2.1 Adding Unknown Parameters Appendix References Index