Flexible Bayesian Regression Modelling

Description

Flexible Bayesian Regression Modeling is a step-by-step guide to the Bayesian revolution in regression modeling, for use in advanced econometric and statistical analysis where datasets are characterized by complexity, multiplicity, and large sample sizes, necessitating the need for considerable flexibility in modeling techniques. It reviews three forms of flexibility: methods which provide flexibility in their error distribution; methods which model non-central parts of the distribution (such as quantile regression); and finally models that allow the mean function to be flexible (such as spline models). Each chapter discusses the key aspects of fitting a regression model. R programs accompany the methods. This book is particularly relevant to non-specialist practitioners with intermediate mathematical training seeking to apply Bayesian approaches in economics, biology, finance, engineering and medicine. Contents Contributors Preface 1 Bayesian quantile regression with the asymmetric Laplace distribution 1.1 Introduction 1.2 The asymmetric Laplace distribution for quantile regression 1.2.1 A simple and efficient sampler 1.2.2 Quantile curve fitting 1.2.3 Additive models 1.3 On coverage probabilities 1.4 Postprocessing for multiple fittings 1.5 Final remarks and conclusion References 2 A vignette on model-based quantile regression: analysing excess zero response 2.1 Introduction 2.2 Excess zero regression analysis 2.3 Case study data and objective 2.4 Fitting single covariate basal area models 2.4.1 Joint quantile regression call 2.4.2 MCMC progress and convergence assessment 2.5 Interpreting quantile regressions 2.5.1 Coefficient plots 2.5.2 Quantile line plots 2.6 Assessing model assumptions and making improvements 2.6.1 Obtaining estimated quantile levels 2.6.2 Assessing overall fit 2.6.3 Assessing linearity 2.6.4 Model improvement 2.7 Prediction and interpreting predicted responses 2.7.1 Quantiles for positive reals 2.7.2 Probability of zero 2.8 Fitting multiple regression basal area models 2.8.1 Model terms, transformations and interactions 2.8.2 Assessing model assumptions 2.8.3 Interpreting coefficients 2.8.4 Understanding marginal and interaction effects 2.8.5 Understanding effects on probability of zero 2.8.6 Further model refinement and comparison 2.9 Conclusions and final remarks Acknowledgement References 3 Bayesian nonparametric density regression for ordinal responses 3.1 Introduction 3.2 Bayesian nonparametric density regression 3.2.1 Dirichlet process priors and Dirichlet process mixtures 3.2.2 Dirichlet process mixture modelling for density regression 3.3 Mixture modelling for ordinal responses 3.3.1 Modelling approach 3.3.2 Implementation details 3.3.2.1 Prior specification 3.3.2.2 Posterior inference 3.3.3 Illustration: ozone concentration 3.3.3.1 Density regression 3.3.3.2 Ordinal regression 3.3.4 Related work and extensions 3.4 Summary Acknowledgements References 4 Bayesian nonparametric methods for financial and macroeconomic time series analysis 4.1 Introduction 4.2 Bayesian nonparametric methods for the innovation distribution in volatility models 4.3 Bayesian nonparametric methods for long-range dependence in SV models 4.4 Bayesian nonparametric methods for the analysis of macroeconomic time series 4.5 Conclusion References 5 Bayesian mixed binary-continuous copula regression with an application to childhood undernutrition 5.1 Introduction 5.2 Bivariate copula models with mixed binary-continuous marginals 5.2.1 Copula-based construction of bivariate binary-continuous distributions 5.2.2 Specifying the marginal distributions 5.2.3 Specifying the copula 5.2.4 Embedding copula regression in the distributional regression framework 5.2.5 Predictor specification Linear effects Nonlinear effects Spatial effects Other effect types 5.3 Bayesian inference Data augmentation for the binary response Iteratively weighted least squares proposals Updating parameters of the marginal distributions Updating parameters of the copula Updating the smoothing variances Implementation 5.4 Model selection and model evaluation Quantile residuals for the marginal fits Model choice via information criteria Predictive ability 5.5 Results Effects of covariates Joint probabilities 5.6 Summary and discussion Acknowledgements References 6 Nonstandard flexible regression via variational Bayes 6.1 Introduction 6.2 Preparatory modelling components 6.2.1 Mixed model-based penalised splines 6.2.2 Semiparametric regression 6.2.3 Priors 6.2.4 Mean field variational Bayes 6.2.5 Tricks when VB is not easy to apply 6.2.6 Comparisons 6.3 A standard semiparametric regression model 6.3.1 Generalised additive model for air quality data 6.4 Robust nonparametric regression 6.4.1 Hauser respiratory experiment 6.5 Generalised additive model with heteroscedastic variance 6.5.1 Milan air pollution example 6.6 Generalised additive negative binomial model 6.6.1 Lung cancer data 6.7 Logistic regression with missing covariates 6.7.1 Pima-Indians diabetes 6.8 Conclusion Acknowledgements References 7 Scalable Bayesian variable selection regression models for count data 7.1 Introduction 7.2 Bayesian variable selection via spike-and-slab priors 7.3 Negative binomial regression models 7.3.1 Data augmentation 7.3.2 MCMC algorithm 7.3.3 Variational inference algorithm 7.4 Dirichlet-multinomial regression models 7.4.1 MCMC algorithm 7.4.2 Variational inference with reparameterisation 7.4.2.1 Reparameterisation of the gradient 7.4.2.2 Concrete relaxation 7.4.2.3 Hard concrete distribution 7.4.2.4 Variational inference approximation 7.4.2.5 Posterior inference using tensorflow 7.5 Simulation study Negative binomial - small p large n example Negative binomial - large p small n example Dirichlet-multinomial example 7.6 Benchmark applications Boston housing data Microbiome data 7.7 Conclusion References 8 Bayesian spectral analysis regression 8.1 Introduction 8.2 Smooth operators 8.3 Bayesian spectral analysis regression 8.4 Shape constraints 8.4.1 Monotonic functions 8.4.2 S- and U-shaped functions 8.5 Nonnormal distributions 8.6 R library bsamGP 8.7 Conclusion Acknowledgements References 9 Flexible regression modelling under shape constraints 9.1 Introduction 9.2 Orthonormal design matrices 9.3 Monotonic polynomial model 9.3.1 Model specification and estimation 9.3.2 Posterior simulation 9.3.2.1 Proposal mechanisms and acceptance probabilities Proposal for γ Proposal for σ2y Acceptance probability 9.3.2.2 Summary 9.3.3 Implementation details Implementing the monotonicity constraint and calculating the log posterior General MCMC scheme 9.3.4 A simulated data example 9.4 Covariate selection 9.4.1 Model specification 9.4.1.1 Likelihood expression 9.4.1.2 Prior specification Regression coefficient prior Variance prior Polynomial degree prior 9.4.1.3 Posterior expression 9.4.2 Sampling from the posterior distribution 9.4.2.1 Dimension proposal 9.4.2.2 Regression coefficient proposal 9.4.2.3 Variance proposal 9.4.3 Acceptance probability 9.4.3.1 Summary 9.4.4 Simulated data examples 9.4.4.1 The relationship between noise and estimated degree 9.4.4.2 The relationship between the number of data points and estimated degree Repeated X-values Unique X-values 9.4.5 Child growth data 9.5 Conclusion References Index