Large Sample Techniques for Statistics

Description

Preface Contents 1 The ε-δ Arguments 1.1 Introduction 1.2 Getting used to the ε-δ arguments 1.3 More examples 1.4 Case study: Consistency of MLE in the i.i.d. case 1.5 Some useful results 1.5.1 Infinite sequence 1.5.2 Infinite series 1.5.3 Topology 1.5.4 Continuity, differentiation, and integration 1.6 Exercises 2 Modes of Convergence 2.1 Introduction 2.2 Convergence in probability 2.3 Almost sure convergence 2.4 Convergence in distribution 2.5 Lp convergence and related topics 2.6 Case study: χ2-test 2.7 Summary and additional results 2.8 Exercises 3 Big O, Small o, and the Unspecified c 3.1 Introduction 3.2 Big O and small o for sequences and functions 3.3 Big O and small o for vectors and matrices 3.4 Big O and small o for random quantities 3.5 The unspecified c and other similar methods 3.6 Case study: The baseball problem 3.7 Case study: Likelihood ratio for a clustering problem 3.8 Exercises 4 Asymptotic Expansions 4.1 Introduction 4.2 Taylor expansion 4.3 Edgeworth expansion; method of formal derivation 4.4 Other related expansions 4.4.1 Fourier series expansion 4.4.2 Cornish–Fisher expansion 4.4.3 Two time series expansions 4.5 Some elementary expansions 4.6 Laplace approximation 4.7 Case study: Asymptotic distribution of the MLE 4.8 Case study: The Prasad–Rao method 4.9 Exercises 5 Inequalities 5.1 Introduction 5.2 Numerical inequalities 5.2.1 The convex function inequality 5.2.2 Hölder's and related inequalities 5.2.3 Monotone functions and related inequalities 5.3 Matrix inequalities 5.3.1 Nonnegative definite matrices 5.3.2 Characteristics of matrices 5.4 Integral/moment inequalities 5.5 Probability inequalities 5.6 Case study: Some problems on existence of moments 5.7 Case study: A variance inequality 5.8 Exercises 6 Sums of Independent Random Variables 6.1 Introduction 6.2 The weak law of large numbers 6.3 The strong law of large numbers 6.4 The central limit theorem 6.5 The law of the iterated logarithm 6.6 Further results 6.6.1 Invariance principles in CLT and LIL 6.6.2 Large deviations 6.7 Case study: The least squares estimators 6.8 Exercises 7 Empirical Processes 7.1 Introduction 7.2 Glivenko–Cantelli theorem and statistical functionals 7.3 Weak convergence of empirical processes 7.4 LIL and strong approximation 7.5 Bounds and large deviations 7.6 Non-i.i.d. observations 7.7 Empirical processes indexed by functions 7.8 Case study: Estimation of ROC curve and ODC 7.9 Exercises 8 Martingales 8.1 Introduction 8.2 Examples and simple properties 8.3 Two important theorems of martingales 8.3.1 The optional stopping theorem 8.3.2 The martingale convergence theorem 8.4 Martingale laws of large numbers 8.4.1 A weak law of large numbers 8.4.2 Some strong laws of large numbers 8.5 A martingale central limit theorem and related topic 8.6 Convergence rate in SLLN and LIL 8.7 Invariance principles for martingales 8.8 Case study: CLTs for quadratic forms 8.9 Case study: Martingale approximation 8.10 Exercises 9 Time and Spatial Series 9.1 Introduction 9.2 Autocovariances and autocorrelations 9.3 The information criteria 9.4 ARMA model identification 9.5 Strong limit theorems for i.i.d. spatial series 9.6 Two-parameter martingale differences 9.7 Sample ACV and ACR for spatial series 9.8 Case study: Spatial AR models 9.9 Exercises 10 Stochastic Processes 10.1 Introduction 10.2 Markov chains 10.3 Poisson processes 10.4 Renewal theory 10.5 Brownian motion 10.6 Stochastic integrals and diffusions 10.7 Case study: GARCH models and financial SDE 10.8 Exercises 11 Nonparametric Statistics 11.1 Introduction 11.2 Some classical nonparametric tests 11.3 Asymptotic relative efficiency 11.4 Goodness-of-fit tests 11.5 U-statistics 11.6 Density estimation 11.7 Exercises 12 Mixed Effects Models 12.1 Introduction 12.2 REML: Restricted maximum likelihood 12.3 Linear mixed model diagnostics 12.4 Inference about GLMM 12.5 Mixed model selection 12.6 Exercises 13 Small-Area Estimation 13.1 Introduction 13.2 Empirical best prediction with binary data 13.3 The Fay–Herriot model 13.4 Nonparametric small-area estimation 13.5 Model selection for small-area estimation 13.6 Exercises 14 Jackknife and Bootstrap 14.1 Introduction 14.2 The jackknife 14.3 Jackknifing the MSPE of EBP 14.4 The bootstrap 14.5 Bootstrapping time series 14.6 Bootstrapping mixed models 14.7 Exercises 15 Markov-Chain Monte Carlo 15.1 Introduction 15.2 The Gibbs sampler 15.3 The Metropolis–Hastings algorithm 15.4 Monte Carlo EM algorithm 15.5 Convergence rates of Gibbs samplers 15.6 Exercises 16 Random Matrix Theory 16.1 Introduction 16.2 Fundamental theorems of RMT 16.3 Large covariance matrices 16.4 High-dimensional linear models 16.5 Genome-wide association study 16.6 Application to time series 16.7 Exercises Appendix A A.1 Matrix algebra A.1.1 Numbers associated with a matrix A.1.2 Inverse of a matrix A.1.3 Kronecker products A.1.4 Matrix differentiation A.1.5 Projection A.1.6 Decompositions of matrices and eigenvalues A.2 Measure and probability A.2.1 Measures A.2.2 Measurable functions A.2.3 Integration A.2.4 Distributions and random variables A.2.5 Conditional expectations A.2.6 Conditional distributions A.3 Some results in statistics A.3.1 The multivariate normal distribution A.3.2 Maximum likelihood A.3.3 Exponential family and generalized linear models A.3.4 Bayesian inference A.3.5 Stationary processes A.4 List of notation and abbreviations References Index