Fundamentals of High-Dimensional Statistics. With Exercises and R Labs

Description

Preface Exercises, Labs, and Literature Notation Contents 1 Introduction Contents 1.1 Embracing High-Dimensionality 1.2 Statistical Limitations of Classical Estimators 1.3 Incorporating Prior Information 1.4 Regularization for Increasing the Numerical Stability* 1.5 Outlook 1.6 Exercises 1.6.1 Exercises for ▶Sect. 1.1 1.6.2 Exercises for ▶Sect. 1.2 1.6.3 Exercises for ▶Sect. 1.3 1.6.4 Exercises for ▶Sect. 1.4 1.7 R Lab: Least-Squares vs. Ridge Estimation 1.7.1 Generating Toy Data 1.7.2 Implementing the Estimators 1.7.3 Showing that the Ridge Estimator Approximates a Least-Squares Solution 1.7.4 Comparing Estimation Errors 1.7.5 Showing That the Ridge Estimator Is Continuous in the Data 1.7.6 Showing That the Coordinates of the Ridge Estimator Are Not Necessarily Monotone in the Tuning Parameter 1.7.7 Comparing Ridge and Least-Squares on Economic Data 1.7.8 Bonus: Checking the Theoretical Formulae of ▶Sect. 1.4 1.7.9 Bonus: A Technical Note 1.8 Notes and References 2 Linear Regression Contents 2.1 Overview 2.2 Sparsity-Inducing Prior Functions 2.3 Post-Processing Methods 2.4 Hölder Inequality* 2.5 Optimality Conditions* 2.6 Exercises 2.6.1 Exercises for ▶Sect. 2.2 2.6.2 Exercises for ▶Sect. 2.4 2.6.3 Exercises for ▶Sect. 2.5 2.7 R Lab: Overfitting 2.7.1 Generating Data 2.7.2 Implementing the Estimators 2.7.3 Computing and Visualizing the Results 2.7.4 Further Illustrations 2.8 Notes and References 3 Graphical Models Contents 3.1 Overview 3.2 Gaussian Graphical Models 3.3 Maximum Regularized Likelihood Estimation 3.4 Neighborhood Selection 3.5 Exercises 3.5.1 Exercises for ▶Sect. 3.1 3.5.2 Exercises for ▶Sect. 3.3 3.5.3 Exercises for ▶Sect. 3.4 3.6 R Lab: Estimating a Gene–Gene Coactivation Network 3.6.1 Tests on Synthetic Data 3.6.1.1 Generating Data 3.6.1.2 Parameter Estimation via Maximum Likelihood 3.6.1.3 Parameter Estimation via Neighborhood Selection 3.6.2 A Low-Dimensional Gene Network 3.6.2.1 Loading and Pre-processing the Data 3.6.2.2 Estimating the Inverse Covariance Matrix 3.6.2.3 Estimating the Graph 3.6.3 A High-Dimensional Gene Network 3.6.3.1 Loading and Pre-processing the Data 3.6.3.2 Estimating the Inverse Covariance Matrix 3.6.3.3 Visualizing the Inverse Covariance Matrix 3.7 Notes and References 4 Tuning-Parameter Calibration Contents 4.1 Overview 4.2 Bounds on the Lasso's Effective Noise 4.3 Cross-Validation 4.4 Adaptive Validation 4.5 Exercises 4.5.1 Exercises for ▶Sect. 4.2 4.5.2 Exercises for ▶Sect. 4.3 4.5.3 Exercises for ▶Sect. 4.4 4.6 R Lab: Cross-Validation 4.6.1 Data Generation 4.6.2 Computing a Set of Estimators 4.6.3 Implementing a Monte Carlo Cross-Validation 4.6.4 The Double Role of the Training Sample Size 4.6.5 The Role of the Number of Splits 4.7 Notes and References 5 Inference Contents 5.1 One-Step Estimators 5.2 Confidence Intervals 5.3 Exercises 5.3.1 Exercises for ▶Sect. 5.1 5.3.2 Exercises for ▶Sect. 5.2 5.4 R Lab: Confidence Intervals in Low and High Dimensions 5.4.1 Generating Synthetic Data 5.4.2 Defining Initial Estimators 5.4.3 Constructing Approximate Inverses 5.4.4 Testing the Inversion Algorithm 5.4.5 Bonus: A Scalable Optimization Algorithm 5.4.6 Debiasing 5.4.7 Constructing Confidence Intervals 5.4.8 Validating the Pipeline 5.4.9 Real Data Analysis 5.4.9.1 Data Loading 5.4.9.2 Estimating Confidence Intervals 5.4.9.3 Plotting the Intervals 5.4.9.4 Testing Against a Standard Function 5.5 Notes and References 6 Theory I: Prediction Contents 6.1 Overview 6.2 Basic Inequalities 6.3 Prediction Guarantees 6.4 Prediction Guarantees for Sparse and Weakly Correlated Models 6.5 Exercises 6.5.1 Exercises for ▶Sect. 6.2 6.5.2 Exercises for ▶Sect. 6.3 6.5.3 Exercises for ▶Sect. 6.4 6.6 Notes and References 7 Theory II: Estimation and Support Recovery Contents 7.1 Overview 7.2 Estimation Guarantees in Prior Loss 7.3 Estimation Guarantees in Dual Loss 7.3.1 Step 1: Construction of Primal-Dual Pairs 7.3.2 Step 2: Establishing Conditions for a Successful Construction 7.3.3 Step 3: Deriving Dual Bounds 7.4 Support Recovery Guarantees 7.5 Exercises 7.5.1 Exercises for ▶Sect. 7.2 7.5.2 Exercises for ▶Sect. 7.3 7.6 Notes and References Supplementary Information A Solutions A.1 Solutions for ▶Chap. 1 Solutions for ▶Sect. 1.1 Solutions for ▶Sect. 1.2 Solutions for ▶Sect. 1.3 Solutions for ▶Sect. 1.4 A.2 Solutions for ▶Chap. 2 Solutions for ▶Sect. 2.2 Solutions for ▶Sect. 2.4 Solutions for ▶Sect. 2.5 A.3 Solutions for ▶Chap. 3 Solutions for ▶Sect. 3.1 Solutions for ▶Sect. 3.3 Solutions for ▶Sect. 3.4 A.4 Solutions for ▶Chap. 4 Solutions for ▶Sect. 4.2 Solutions for ▶Sect. 4.3 A.5 Solutions for ▶Chap. 5 Solutions for ▶Sect. 5.1 Solutions for ▶Sect. 5.2 A.6 Solutions for ▶Chap. 6 Solutions for ▶Sect. 6.2 Solutions for ▶Sect. 6.4 A.7 Solutions for ▶Chap. 7 Solutions for ▶Sect. 7.2 Solutions for ▶Sect. 7.3 B Mathematical Background B.1 Analysis B.2 Matrix Algebra Basic Definitions Transpose of Matrices and Symmetric Matrices Positive (Semi-)Definite Matrices Determinant of Matrices Inverse of Matrices Trace of Matrices Eigenvectors and Eigenvalues of Matrices Singular Value Decomposition Specific Results for Gaussian Graphical Models Bibliography Index