Time Series for Data Science: Analysis and Forecasting

Description

Data Science students and practitioners want to find a forecast that “works” and don’t want to be constrained to a single forecasting strategy, Time Series for Data Science: Analysis and Forecasting discusses techniques of ensemble modelling for combining information from several strategies. Covering time series regression models, exponential smoothing, Holt-Winters forecasting, and Neural Networks. It places a particular emphasis on classical ARMA and ARIMA models that is often lacking from other textbooks on the subject. This book is an accessible guide that doesn’t require a background in calculus to be engaging but does not shy away from deeper explanations of the techniques discussed. Features: Provides a thorough coverage and comparison of a wide array of time series models and methods: Exponential Smoothing, Holt Winters, ARMA and ARIMA, deep learning models including RNNs, LSTMs, GRUs, and ensemble models composed of combinations of these models.Introduces the factor table representation of ARMA and ARIMA models. This representation is not available in any other book at this level and is extremely useful in both practice and pedagogy.Uses real world examples that can be readily found via web links from sources such as the US Bureau of Statistics, Department of Transportation and the World Bank.There is an accompanying R package that is easy to use and requires little or no previous R experience. The package implements the wide variety of models and methods presented in the book and has tremendous pedagogical use. Cover Half Title Series Information Title Page Copyright Page Dedication Table of Contents Preface Acknowledgments Authors 1 Working With Data Collected Over Time 1.1 Introduction 1.2 Time Series Datasets 1.2.1 Cyclic Data 1.2.1.1 Sunspot Data 1.2.1.2 DFW Temperature Data 1.2.1.3 Air Passengers Data 1.2.2 Trends 1.2.2.1 Real Datasets That Have Trending Behavior 1.2.2.2 The Problem With Trends 1.3 The Programming Language R 1.3.1 The Tswge Time Series Package 1.3.2 Base R 1.3.3 Plotting Time Series Data in R 1.3.4 The Ts Object 1.3.4.1 Creating a Ts Object 1.3.4.2 More About Ts Objects 1.3.5 The Plotts.wge Function in Tswge 1.3.5.1 Modifying the Appearance of Plots Using the Tswge Plotts.wge Function 1.3.6 Loading Time Series Data Into R 1.3.6.1 The .csv File 1.3.6.2 The .txt File 1.3.6.3 Other File Formats 1.3.7 Accessing Time Series Data 1.3.7.1 Accessing Data From the Internet 1.3.7.2 Business / Proprietary Data: Ozona Bar and Grill 1.4 Dealing With Messy Data 1.4.1 Preparing Time Series Data for Analysis: Cleaning, Wrangling, and Imputation 1.4.1.1 Missing Data 1.4.1.2 Downloading When No .csv Download Option Is Available 1.4.1.3 Data That Require Cleaning and Wrangling 1.4.1.4 Programatic Method of Ingestion and Wrangling Data From Tables On Web Pages 1.5 Concluding Remarks Appendix 1A tswge Datasets INTRODUCED IN this Chapter Problems 2 Exploring Time Series Data 2.1 Understanding and Visualizing Data 2.1.1 Smoothing Time Series Data 2.1.1.1 Smoothing Data Using a Centered Moving Average Smoother 2.1.1.2 Other Methods Available for Smoothing Data 2.1.1.3 Moving Average Smoothing Versus Aggregating 2.1.1.4 Using Moving Average Smoothing for Estimating Trend in Data With Fixed Cycle Lengths 2.1.2 Decomposing Seasonal Data 2.1.2.1 Additive Decompositions 2.1.2.2 Multiplicative Decompositions 2.1.3 Seasonal Adjustment 2.1.3.1 Additive Seasonal Adjustment 2.1.3.2 Multiplicative Seasonal Adjustment 2.2 Forecasting 2.2.1 Predictive Moving Average Smoother 2.2.2 Exponential Smoothing 2.2.2.1 Forecasting With Exponential Smoothing Beyond the Observed Dataset 2.2.3 Holt-Winters Forecasting 2.2.3.1 Additive Holt-Winters Equations 2.2.3.2 Multiplicative Holt-Winters Equations 2.2.4 Assessing the Accuracy of Forecasts 2.3 Concluding Remarks Appendix 2A tswge Functions Tswge Datasets Introduced In Chapter 2 3 Statistical Basics for Time Series Analysis 3.1 Statistics Basics 3.1.1 Univariate Data 3.1.2 Multivariate Data 3.1.2.1 Measuring Relationships Between Two Random Variables in a Bivariate Random Sample 3.1.2.2 Assessing Association From a Bivariate Random Sample 3.1.3 Independent Vs Dependent Data 3.2 Time Series and Realizations 3.2.1 Multiple Realizations 3.2.1.1 Time Series 1: Xt 3.2.1.2 Time Series 2: Yt (Example 3.3 Continued) 3.2.2 The Effect of Realization Length 3.3 Stationary Time Series 3.3.1 Plotting the Autocorrelations of a Stationary Process 3.3.2 Estimating the Parameters of a Stationary Process 3.3.2.1 Estimating 3.3.2.2 Estimating the Variance 3.3.2.3 Estimating the Autocovariance and Autocorrelation 3.3.2.4 Plotting Sample Autocorrelations 3.4 Concluding Remarks Appendix 3A Appendix 3B TSWGE FUNCTION BASE R COMMANDS TSWGE DATASETS RELATED TO THIS CHAPTER Problems 4 The Frequency Domain 4.1 Trigonometric Review and Terminology 4.2 The Spectral Density 4.2.1 Euler’s Formula 4.2.2 Definition and Properties of the Spectrum and Spectral Density 4.2.2.1 The Nyquist Frequency 4.2.2.2 Frequency F.=.0 4.2.2.3 The Spectral Density and the Autocorrelation Function 4.2.3 Estimating the Spectral Density 4.2.3.1 The Sample Spectral Density 4.2.3.2 Smoothing the Sample Spectral Density 4.2.3.3 Parzen Spectral Density Estimate Vs Sample Autocorrelations 4.2.3.4 Why We Plot Spectral Densities in Log Scale 4.3 Smoothing and Filtering 4.3.1 Types of Filters 4.3.2 The Butterworth Filter 4.4 Concluding Remarks Appendix 4A tswge Functions Problems 5 ARMA Models 5.1 The Autoregressive Model 5.1.1 The AR(1) Model 5.1.1.1 The AR(1) in Backshift Operator Notation 5.1.1.2 The AR(1) Characteristic Polynomial and Characteristic Equation 5.1.1.3 Properties of a Stationary AR(1) Model 5.1.1.4 Spectral Density of an AR(1) 5.1.1.5 AR(1) Models With Positive Roots of the Characteristic Equation 5.1.1.6 AR(1) Models With Roots Close to +1 5.1.1.7 AR(1) Models With Negative Roots of the Characteristic Equation 5.1.1.8 Nonstationary 1st-Order Models 5.1.1.9 Final Comments Regarding AR(1) Models 5.1.2 The AR(2) Model 5.1.2.1 Facts About the AR(2) Model 5.1.2.2 Operator Notation and Characteristic Equation for an AR(2) 5.1.2.3 Stationary AR(2) With Two Real Roots 5.1.2.4 Stationary AR(2) With Complex Conjugate Roots 5.1.2.5 Summary of AR(1) and AR(2) Behavior 5.1.3 The AR(p) Models 5.1.3.1 Facts About the AR(p) Model 5.1.3.2 Operator Notation and Characteristic Equation for an AR(p) 5.1.3.3 Factoring the AR(p) Characteristic Polynomial 5.1.3.4 Factor Tables for AR(p) Models 5.1.3.5 Dominance of Roots Close to the Unit Circle 5.1.4 Linear Filters, the General Linear Process, and AR(p) Models 5.1.4.1 AR(1) in GLP Form 5.1.4.2 AR(p) in GLP Form 5.2 Autoregressive-Moving Average (ARMA) Models 5.2.1 Moving Average Models 5.2.1.1 The MA(1) Model 5.2.1.2 The MA(2) Model 5.2.1.3 The General MA(q) Model 5.2.1.4 Invertibility 5.2.2 ARMA(p,q) Models 5.2.2.1 Stationarity and Invertibility of an ARMA(p,q) Process 5.2.2.2 AR Versus ARMA Models 5.3 Concluding Remarks Appendix 5A tswge Functions Appendix 5B Stationarity Conditions of an AR(1) Problem Set 6 ARMA Fitting and Forecasting 6.1 Fitting ARMA Models to Data 6.1.1 Estimating the Parameters of an ARMA(p,q) Model 6.1.1.1 Maximum Likelihood Estimation of the and Coefficients of an ARMA Model 6.1.1.2 Estimating . 6.1.1.3 Estimating 6.1.1.4 Alternative Estimates for AR(p) Models 6.1.2 ARMA Model Identification 6.1.2.1 Plotting the Data and Checking for White Noise 6.1.2.2 Model Identification Types 6.1.2.3 AIC-Type Measures for ARMA Model Fitting 6.1.2.4 The Special Case of AR Model Identification 6.2 Forecasting Using an ARMA(p,q) Model 6.2.1 ARMA Forecasting Setting, Notation, and Strategy 6.2.1.1 Strategy and Notation 6.2.1.2 Forecasting for 6.2.1.3 Forecasting At0+l for l > 0 6.2.2 Forecasting Using an AR Model 6.2.2.1 Forecasting Using an AR(1) Model 6.2.3 Basic Formula for Forecasting Using an ARMA() Model 6.2.4 Eventual Forecast Functions 6.2.5 Probability Limits for ARMA Forecasts 6.2.5.1 Facts About Forecast Errors 6.2.5.2 Lack of Symmetry 6.2.6 Assessing Forecast Performance 6.2.6.1 How “Good” Are the Forecasts? 6.2.6.2 Some Strategies for Using RMSE to Measure Forecast Performance 6.3 Concluding Remarks Appendix 6A tswge Functions Problems 7 ARIMA and Seasonal Models 7.1 ARIMA(p, D, Q) Models 7.1.1 Properties of the ARIMA(p,d,q) Model 7.1.1.1 Some ARIMA() Models 7.1.1.2 Characteristic Equations for Models (A) . (C) 7.1.1.3 Limiting Autocorrelations 7.1.1.4 Lack of Attraction to a Mean 7.1.1.5 Random Trends 7.1.1.6 Differencing an ARIMA(0,1,0) Model 7.1.1.7 ARIMA Models With Stationary and Nonstationary Components 7.1.1.8 The Stationary AR(2) Model: (1-1.4B+.65B2)Xt = at 7.1.2 Model Identification and Parameter Estimation of ARIMA(p,d,q) Models 7.1.2.1 Deciding Whether to Include One Or More 1–.B Factors (That Is, Unit Roots) in the Model 7.1.2.2 General Procedure for Fitting an ARIMA(p,d,q) Model to a Set of Time Series Data 7.1.3 Forecasting With ARIMA Models 7.1.3.1 ARMA Forecast Formula 7.2 Seasonal Models 7.2.1 Properties of Seasonal Models 7.2.1.1 Some Seasonal Models 7.2.2 Fitting Seasonal Models to Data 7.2.2.1 Overfitting 7.2.3 Forecasting Using Seasonal Models 7.3 ARCH and GARCH Models 7.3.1 ARCH(1) Model 7.3.2 The ARCH(p) and GARCH(p,q) Processes 7.3.3 Assessing the Appropriateness of an ARCH/GARCH Fit to a Set of Data 7.3.4 Fitting ARCH/GARCH Models to Simulated Data 7.3.5 Modeling Daily Rates of Return Data 7.4 Concluding Remarks Appendix 7A tswge Functions Appendix 7B Problems 8 Time Series Regression 8.1 Line+Noise Models 8.1.1 Testing for Linear Trend 8.1.1.1 Testing for Trend Using Simple Linear Regression 8.1.1.2 A T-Test Simulation 8.1.1.3 Cochrane-Orcutt Test for Trend 8.1.1.4 Bootstrap-Based Test for Trend 8.1.1.5 Other Methods for Testing for Trend in Time Series Data 8.1.2 Fitting Line+Noise Models to Data 8.1.3 Forecasting Using Line+Noise Models 8.2 Cosine Signal+Noise Models 8.2.1 Fitting a Cosine Signal+Noise Model to Data 8.2.2 Forecasting Using Cosine Signal+Noise Models 8.2.2.1 Using Fore.sigplusnoise.wge 8.2.3 Deciding Whether to Fit a Cosine Signal+Noise Model to a Set of Data 8.2.3.1 A Closer Look at the Cyclic Behavior 8.3 Concluding Remarks Appendix 8A tswge Functions Exercises 9 Model Assessment 9.1 Residual Analysis 9.1.1 Checking Residuals for White Noise 9.1.1.1 Check Residual Sample Autocorrelations Against 95% Limit Lines 9.1.1.2 Ljung-Box Test 9.1.2 Checking the Residuals for Normality 9.2 CASE STUDY 1: Modeling the Global Temperature Data 9.2.1 A Stationary Model 9.2.1.1 Checking the Residuals 9.2.1.2 Realizations and Their Characteristics 9.2.1.3 Forecasting Based On the ARMA(4,1) Model 9.2.2 A Correlation-Based Model With a Unit Root 9.2.2.1 Checking the Residuals 9.2.2.2 Realizations and Their Characteristics 9.2.2.3 Forecasting Based On ARIMA(0,1,1) Model 9.2.3 Line+Noise Models for the Global Temperature Data 9.2.3.1 Checking the Residuals, for White Noise 9.2.3.2 Realizations and Their Characteristics 9.2.3.3 Forecasting Based On the Signal-Plus-Noise Model 9.2.3.4 Other Forecasts 9.3 CASE STUDY 2: Comparing Models for the Sunspot Data 9.3.1 Selecting the Models for Comparison 9.3.2 Do the Models Whiten the Residuals? 9.3.3 Do Realizations and Their Characteristics Behave Like the Data? 9.3.4 Do Forecasts Reflect What Is Known About the Physical Setting? 9.3.4.1 Final Comments About the Models Fit to the Sunspot Data 9.4 Comprehensive Analysis of Time Series Data: A Summary 9.5 Concluding Remarks Appendix 9A tswge Function Base R Function Problems 10 Multivariate Time Series 10.1 Introduction 10.2 Multiple Regression With Correlated Errors 10.2.1 Notation for Multiple Regression With Correlated Errors 10.2.2 Fitting Multiple Regression Models to Time Series Data 10.2.2.1 Including a Trend Term in the Multiple Regression Model 10.2.2.2 Adding Lagged Variables 10.2.2.3 Using Lagged Variables and a Trend Variable 10.2.3 Cross Correlation 10.3 Vector Autoregressive (VAR) Models 10.3.1 Forecasting With VAR(p) Models 10.3.1.1 Univariate Forecasts 10.3.1.2 VAR Analysis 10.3.1.3 Comparing RMSEs 10.3.1.4 Final Comments 10.4 Relationship Between MLR and VAR Models 10.5 A Comprehensive and Final Example: Los Angeles Cardiac Mortality 10.5.1 Applying the VAR(p) to the Cardiac Mortality Data (1) Use VARselect to Identify Candidate Model Orders (2) Use VAR to Fit the VAR Models to the Training Set (3) Use Predict to Forecast Data Values in the Test Set 10.5.2 The Seasonal VAR(p) Model (1) Use VARselect to Identify Candidate Seasonal Models (2) Use VAR to Fit the VAR Models to the Training Set (3) Use Predict to Forecast Data Values in the Test Set 10.5.3 Forecasting the Future 10.5.3.1 Short Vs. Long Term Forecasts 10.6 Conclusion Appendix 10A BASE R FUNCTIONS CRAN Package Dpylr CRAN Package Vars tswge Datasets Introduced in this Chapter Appendix 10B Relationship Between MLR With Correlated Errors and VAR Three Important Points Should Be Considered Problems 11 Deep Neural Network-Based Time Series Models 11.1 Introduction 11.2 The Perceptron 11.3 The Extended Perceptron for Univariate Time Series Data 11.3.1 A Neural Network Similar to the AR(1) 11.3.1.1 The Architecture 11.3.1.2 Fitting the MLP 11.3.1.3 Forecasting 11.3.1.4 Cross Validation Using the Rolling Window RMSE 11.3.2 A Neural Network Similar to AR(p): Adding More Lags 11.3.3 A Deeper Neural Network: Adding a Hidden Layer 11.3.3.1 Differences and Seasonal “Dummies” 11.4 The Extended Perceptron for Multivariate Time Series Data 11.4.1 Forecasting Melanoma Using Sunspots 11.4.1.1 Architecture 11.4.1.2 Fitting the Baseline Model 11.4.1.3 Forecasting Future Sunspot Data for Predicting Future Melanoma 11.4.1.4 Forecasting the Last Eight Years of Melanoma 11.4.1.5 Fitting a Competing Model 11.4.1.6 Assessing the Competing Model On the Last Eight Years of Melanoma Data 11.4.1.7 Forecasting the Next Eight Years of Melanoma 11.4.2 Forecasting Cardiac Mortality Using Temperature and Particulates 11.4.2.1 General Architecture 11.4.2.2 Train / Test Split 11.4.2.3 Forecasting Covariates: Temperature and Particulates 11.4.2.4 Model Without Seasonal Indicator Variables 11.4.2.5 Model With Seasonal Indicator Variables 11.5 An “Ensemble” Model 11.5.1 Final Forecasts for the Next Fifty-Two Weeks 11.5.2 Final Forecasts for the Next Three Years (Longer Term Forecasts) 11.6 Concluding Remarks Appendix 11A tswge Function Appendix 11B Chapter 11 Problems Mini Research Project References Index