Introduction to Linear Regression Analysis

Description

Cover Title Page Copyright CONTENTS PREFACE CHANGES IN THE FIFTH EDITION USING THE BOOK AS A TEXT ACKNOWLEDGMENTS CHAPTER 1: INTRODUCTION 1.1 REGRESSION AND MODEL BUILDING 1.2 DATA COLLECTION 1.3 USES OF REGRESSION 1.4 ROLE OF THE COMPUTER CHAPTER 2: SIMPLE LINEAR REGRESSION 2.1 SIMPLE LINEAR REGRESSION MODEL 2.2 LEAST - SQUARES ESTIMATION OF THE PARAMETERS 2.2.1 Estimation of β0 and β1 2.2.2 Properties of the Least - Squares Estimators and the Fitted Regression Model 2.2.3 Estimation of σ2 2.2.4 Alternate Form of the Model 2.3 HYPOTHESIS TESTING ON THE SLOPE AND INTERCEPT 2.3.1 Use of t Tests 2.3.2 Testing Significance of Regression 2.3.3 Analysis of Variance 2.4 INTERVAL ESTIMATION IN SIMPLE LINEAR REGRESSION 2.4.1 Confidence Intervals on β0, β1, and σ2 2.4.2 Interval Estimation of the Mean Response 2.5 PREDICTION OF NEW OBSERVATIONS 2.6 COEFFICIENT OF DETERMINATION 2.7 A SERVICE INDUSTRY APPLICATION OF REGRESSION 2.8 USING SAS ® AND R FOR SIMPLE LINEAR REGRESSION 2.9 SOME CONSIDERATIONS IN THE USE OF REGRESSION 2.10 REGRESSION THROUGH THE ORIGIN 2.11 ESTIMATION BY MAXIMUM LIKELIHOOD 2.12 CASE WHERE THE REGRESSOR x IS RANDOM 2.12.1 x and y Jointly Distributed 2.12.2 x and y Jointly Normally Distributed: Correlation Model CHAPTER 3: MULTIPLE LINEAR REGRESSION 3.1 MULTIPLE REGRESSION MODELS 3.2 ESTIMATION OF THE MODEL PARAMETERS 3.2.1 Least - Squares Estimation of the Regression Coefficients 3.2.2 A Geometrical Interpretation of Least Squares 3.2.3 Properties of the Least - Squares Estimators 3.2.4 Estimation of σ2 3.2.5 Inadequacy of Scatter Diagrams in Multiple Regression 3.2.6 Maximum - Likelihood Estimation 3.3 HYPOTHESIS TESTING IN MULTIPLE LINEAR REGRESSION 3.3.1 Test for Significance of Regression 3.3.2 Tests on Individual Regression Coefficients and Subsets of Coefficients 3.3.3 Special Case of Orthogonal Columns in X 3.3.4 Testing the General Linear Hypothesis 3.4 CONFIDENCE INTERVALS IN MULTIPLE REGRESSION 3.4.1 Confidence Intervals on the Regression Coefficients 3.4.2 CI Estimation of the Mean Response 3.4.3 Simultaneous Confidence Intervals on Regression Coefficients 3.5 PREDICTION OF NEW OBSERVATIONS 3.6 A MULTIPLE REGRESSION MODEL FOR THE PATIENT SATISFACTION DATA 3.7 USING SAS AND R FOR BASIC MULTIPLE LINEAR REGRESSION 3.8 HIDDEN EXTRAPOLATION IN MULTIPLE REGRESSION 3.9 STANDARDIZED REGRESSION COEFFLCIENTS 3.10 MULTICOLLINEARITY 3.11 WHY DO REGRESSION COEFFICIENTS HAVE THE WRONG SIGN? CHAPTER 4: MODEL ADEQUACY CHECKING 4.1 INTRODUCTION 4.2 RESIDUAL ANALYSIS 4.2.1 Definition of Residuals 4.2.2 Methods for Scaling Residuals 4.2.3 Residual Plots 4.2.4 Partial Regression and Partial Residual Plots 4.2.5 Using Minitab ®, SAS, and R for Residual Analysis 4.2.6 Other Residual Plotting and Analysis Methods 4.3 PRESS STATISTIC 4.4 DETECTION AND TREATMENT OF OUTLIERS 4.5 LACK OF FIT OF THE REGRESSION MODEL 4.5.1 A Formal Test for Lack of Fit 4.5.2 Estimation of Pure Error from Near Neighbors CHAPTER 5: TRANSFORMATIONS AND WEIGHTING TO CORRECT MODEL INADEQUACIES 5.1 INTRODUCTION 5.2 VARIANCE-STABILIZING TRANSFORMATIONS 5.3 TRANSFORMATIONS TO LINEARIZE THE MODEL 5.4 ANALYTICAL METHODS FOR SELECTING A TRANSFORMATION 5.4.1 Transformations on y: The Box-Cox Method 5.4.2 Transformations on the Regressor Variables 5.5 GENERALIZED AND WEIGHTED LEAST SQUARES 5.5.1 Generalized Least Squares 5.5.2 Weighted Least Squares 5.5.3 Some Practical Issues 5.6 REGRESSION MODELS WITH RANDOM EFFECTS 5.6.1 Subsampling 5.6.2 The General Situation for a Regression Model with a Single Random Effect 5.6.3 The Importance of the Mixed Model in Regression CHAPTER 6: DIAGNOSTICS FOR LEVERAGE AND INFLUENCE 6.1 IMPORTANCE OF DETECTING INFLUENTIAL OBSERVATIONS 6.2 LEVERAGE 6.3 MEASURES OF INFLUENCE: COOK’S D 6.4 MEASURES OF INFLUENCE: DFFITS AND DFBETAS 6.5 A MEASURE OF MODEL PERFORMANCE 6.6 DETECTING GROUPS OF INFLUENTIAL OBSERVATIONS 6.7 TREATMENT OF INFLUENTIAL OBSERVATIONS CHAPTER 7: POLYNOMIAL REGRESSION MODELS 7.1 INTRODUCTION 7.2 POLYNOMIAL MODELS IN ONE VARIABLE 7.2.1 Basic Principles 7.2.2 Piecewise Polynomial Fitting (Splines) 7.2.3 Polynomial and Trigonometric Terms 7.3 NONPARAMETRIC REGRESSION 7.3.1 Kernel Regres 7.3.2 Locally Weighted Regression (Loess) 7.3.3 Final Cautions 7.4 POLYNOMIAL MODELS IN TWO OR MORE VARIABLES 7.5 ORTHOGONAL POLYNOMIALS CHAPTER 8: INDICATOR VARIABLES 8.1 GENERAL CONCEPT OF INDICATOR VARIABLES 8.2 COMMENTS ON THE USE OF INDICATOR VARIABLES 8.2.1 Indicator Variables versus Regression on Allocated Codes 8.2.2 Indicator Variables as a Substitute for a Quantitative Regressor 8.3 REGRESSION APPROACH TO ANALYSIS OF VARIANCE CHAPTER 9: MULTICOLLINEARITY 9.1 INTRODUCTION 9.2 SOURCES OF MULTICOLLINEARITY 9.3 EFFECTS OF MULTICOLLINEARITY 9.4 MULTICOLLINEARITY DIAGNOSTICS 9.4.1 Examination of the Correlation Matrix 9.4.2 Variance Inflation Factors 9.4.3 Eigensystem Analysis of X′X 9.4.4 Other Diagnostics 9.4.5 SAS and R Code for Generating Multicollinearity Diagnostics 9.5 METHODS FOR DEALING WITH MULTICOLLINEARITY 9.5.1 Collecting Additional Data 9.5.2 Model Respecifi cation 9.5.3 Ridge Regression 9.5.4 Principal - Component Regression 9.5.5 Comparison and Evaluation of Biased Estimators 9.6 USING SAS TO PERFORM RIDGE AND PRINCIPAL - COMPONENT REGRESSION CHAPTER 10: VARIABLE SELECTION AND MODEL BUILDING 10.1 INTRODUCTION 10.1.1 Model - Building Problem 10.1.2 Consequences of Model Misspecification 10.1.3 Criteria for Evaluating Subset Regression Models 10.2 COMPUTATIONAL TECHNIQUES FOR VARIABLE SELECTION 10.2.1 All Possible Regressions 10.2.2 Stepwise Regression Methods 10.3 STRATEGY FOR VARIABLE SELECTION AND MODEL BUILDING 10.4 CASE STUDY: GORMAN AND TOMAN ASPHALT DATA USING SAS CHAPTER 11: VALIDATION OF REGRESSION MODELS 11.1 INTRODUCTION 11.2 VALIDATION TECHNIQUES 11.2.1 Analysis of Model Coefficients and Predicted Values 11.2.2 Collecting Fresh Data — Confirmation Runs 11.2.3 Data Splitting 11.3 DATA FROM PLANNED EXPERIMENTS CHAPTER 12: INTRODUCTION TO NONLINEAR REGRESSION 12.1 LINEAR AND NONLINEAR REGRESSION MODELS 12.1.1 Linear Regression Models 12.1.2 Nonlinear Regression Models 12.2 ORIGINS OF NONLINEAR MODELS 12.3 NONLINEAR LEAST SQUARES 12.4 TRANFORMATION TO A LINEAR MODEL 12.5 PARAMETER ESTIMATION IN A NONLINEAR SYSTEM 12.5.1 Linearization 12.5.2 Other Parameter Estimation Methods 12.5.3 Starting Values 12.6 STATISTICAL INFERENCE IN NONLINEAR REGRESSION 12.7 EXAMPLES OF NONLINEAR REGRESSION MODELS 12.8 USING SAS AND R CHAPTER 13: GENERALIZED LINEAR MODELS 13.1 INTRODUCTION 13.2 LOGISTIC REGRESSION MODELS 13.2.1 Models with a Binary Response Variable 13.2.2 Estimating the Parameters in a Logistic Regression Model 13.2.3 Interpretation of the Parameters in a Logistic Regression Model 13.2.4 Statistical Inference on Model Parameters 13.2.5 Diagnostic Checking in Logistic Regression 13.2.6 Other Models for Binary Response Data 13.2.7 More Than Two Categorical Outcomes 13.3 POISSON REGRESSION 13.4 THE GENERALIZED LINEAR MODEL 13.4.1 Link Functions and Linear Predictors 13.4.2 Parameter Estimation and Inference in the GLM 13.4.3 Prediction and Estimation with the GLM 13.4.4 Residual Analysis in the GLM 13.4.5 Using R to Perform GLM Analysis 13.4.6 Overdispersion CHAPTER 14: REGRESSION ANALYSIS OF TIME SERIES DATA 14.1 INTRODUCTION TO REGRESSION MODELS FOR TIME SERIES DATA 14.2 DETECTING AUTOCORRELATION: THE DURBIN – WATSON TEST 14.3 ESTIMATING THE PARAMETERS IN TIME SERIES REGRESSION MODELS CHAPTER 15: OTHER TOPICS IN THE USE OF REGRESSION ANALYSIS 15.1 ROBUST REGRESSION 15.1.1 Need for Robust Regression 15.1.2 M-Estimators 15.1.3 Properties of Robust Estimators 15.2 EFFECT OF MEASUREMENT ERRORS IN THE REGRESSORS 15.2.1 Simple Linear Regression 15.2.2 The Berkson Model 15.3 INVERSE ESTIMATION — THE CALIBRATION PROBLEM 15.4 BOOTSTRAPPING IN REGRESSION 15.4.1 Bootstrap Sampling in Regression 15.4.2 Bootstrap Confidence Intervals 15.5 CLASSIFICATION AND REGRESSION TREES ( CART ) 15.6 NEURAL NETWORKS 15.7 DESIGNED EXPERIMENTS FOR REGRESSION APPENDIX A: STATISTICAL TABLES APPENDIX B: DATA SETS FOR EXERCISES APPENDIX C: SUPPLEMENTAL TECHNICAL MATERIAL C.1 BACKGROUND ON BASIC TEST STATISTICS C.1.1 Central Distributions C.1.2 Noncentral Distributions C.2 BACKGROUND FROM THE THEORY OF LINEAR MODELS C.2.1 Basic Definitions C.2.2 Matrix Derivatives C.2.3 Expectations C.2.4 Distribution Theory C.3 IMPORTANT RESULTS ON SS R AND SS RES C.3.1 SS R C.3.2 SS Res C.3.3 Global or Overall F Test C.3.4 Extra-Sum-of-Squares Principle C.3.5 Relationship of the t Test for an Individual Coefficient and the Extra-Sum-of-Squares Principle C.4 GAUSS–MARKOV THEOREM, VAR(ε) = σ2I C.5 COMPUTATIONAL ASPECTS OF MULTIPLE REGRESSION C.6 RESULT ON THE INVERSE OF A MATRIX C.7 DEVELOPMENT OF THE PRESS STATISTIC C.8 DEVELOPMENT OF S2(i) C.9 OUTLIER TEST BASED ON R - STUDENT C.10 INDEPENDENCE OF RESIDUALS AND FITTED VALUES C.11 GAUSS - MARKOV THEOREM, VAR( ε ) = V C.12 BIAS IN MS RES WHEN THE MODEL IS UNDERSPECIFIED C.13 COMPUTATION OF INFLUENCE DIAGNOSTICS C.13.1 DFFITSi C.13.2 Cook’s Di C.13.3 DFBETAS j,i C.14 GENERALIZED LINEAR MODELS C.14.1 Parameter Estimation in Logistic Regression C.14.2 Exponential Family C.14.3 Parameter Estimation in the Generalized Linear Model APPENDIX D: INTRODUCTION TO SAS D.1 BASIC DATA ENTRY A. Using the SAS Editor Window B. Entering Data from a Text File D.2 CREATING PERMANENT SAS DATA SETS D.3 IMPORTING DATA FROM AN EXCEL FILE D.4 OUTPUT COMMAND D.5 LOG FILE D.6 ADDING VARIABLES TO AN EXISTING SAS DATA SET APPENDIX E: INTRODUCTION TO R TO PERFORM LINEAR REGRESSION ANALYSIS E.1 BASIC BACKGROUND ON R E.2 BASIC DATA ENTRY E.3 BRIEF COMMENTS ON OTHER FUNCTIONALITY IN R E.4 R COMMANDER REFERENCES INDEX