Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control

Description

Contents Preface Common Optimization Techniques, Equations, Symbols, and Acronyms Part I Dimensionality Reduction and Transforms 1 Singular Value Decomposition (SVD) 1.1 Overview 1.2 Matrix Approximation 1.3 Mathematical Properties and Manipulations 1.4 Pseudo-Inverse, Least-Squares, and Regression 1.5 Principal Component Analysis (PCA) 1.6 Eigenfaces Example 1.7 Truncation and Alignment 1.8 Randomized Singular Value Decomposition 1.9 Tensor Decompositions and N-Way Data Arrays 2 Fourier and Wavelet Transforms 2.1 Fourier Series and Fourier Transforms 2.2 Discrete Fourier Transform (DFT) and Fast Fourier Transform (FFT) 2.3 Transforming Partial Differential Equations 2.4 Gabor Transform and the Spectrogram 2.5 Wavelets and Multi-Resolution Analysis 2.6 2D Transforms and Image Processing 3 Sparsity and Compressed Sensing 3.1 Sparsity and Compression 3.2 Compressed Sensing 3.3 Compressed Sensing Examples 3.4 The Geometry of Compression 3.5 Sparse Regression 3.6 Sparse Representation 3.7 Robust Principal Component Analysis (RPCA) 3.8 Sparse Sensor Placement Part II Machine Learning and Data Analysis 4 Regression and Model Selection 4.1 Classic Curve Fitting 4.2 Nonlinear Regression and Gradient Descent 4.3 Regression and Ax = b: Over- and Under-Determined Systems 4.4 Optimization as the Cornerstone of Regression 4.5 The Pareto Front and Lex Parsimoniae 4.6 Model Selection: Cross-Validation 4.7 Model Selection: Information Criteria 5 Clustering and Classification 5.1 Feature Selection and Data Mining 5.2 Supervised versus Unsupervised Learning 5.3 Unsupervised Learning: k-means Clustering 5.4 Unsupervised Hierarchical Clustering: Dendrogram 5.5 Mixture Models and the Expectation-Maximization Algorithm 5.6 Supervised Learning and Linear Discriminants 5.7 Support Vector Machines (SVM) 5.8 Classification Trees and Random Forest 5.9 Top 10 Algorithms in Data Mining 2008 6 Neural Networks and Deep Learning 6.1 Neural Networks: 1-Layer Networks 6.2 Multi-Layer Networks and Activation Functions 6.3 The Backpropagation Algorithm 6.4 The Stochastic Gradient Descent Algorithm 6.5 Deep Convolutional Neural Networks 6.6 Neural Networks for Dynamical Systems 6.7 The Diversity of Neural Networks Part III Dynamics and Control 7 Data-Driven Dynamical Systems 7.1 Overview, Motivations, and Challenges 7.2 Dynamic Mode Decomposition (DMD) 7.3 Sparse Identification of Nonlinear Dynamics (SINDy) 7.4 Koopman Operator Theory 7.5 Data-Driven Koopman Analysis 8 Linear Control Theory 8.1 Closed-Loop Feedback Control 8.2 Linear Time-Invariant Systems 8.3 Controllability and Observability 8.4 Optimal Full-State Control: Linear Quadratic Regulator (LQR) 8.5 Optimal Full-State Estimation: The Kalman Filter 8.6 Optimal Sensor-Based Control: Linear Quadratic Gaussian (LQG) 8.7 Case Study: Inverted Pendulum on a Cart 8.8 Robust Control and Frequency Domain Techniques 9 Balanced Models for Control 9.1 Model Reduction and System Identification 9.2 Balanced Model Reduction 9.3 System identification 10 Data-Driven Control 10.1 Nonlinear System Identification for Control 10.2 Machine Learning Control 10.3 Adaptive Extremum-Seeking Control Part IV Reduced Order Models 11 Reduced Order Models (ROMs) 11.1 POD for Partial Differential Equations 11.2 Optimal Basis Elements: The POD Expansion 11.3 POD and Soliton Dynamics 11.4 Continuous Formulation of POD 11.5 POD with Symmetries: Rotations and Translations 12 Interpolation for Parametric ROMs 12.1 Gappy POD 12.2 Error and Convergence of Gappy POD 12.3 Gappy Measurements: Minimize Condition Number 12.4 Gappy Measurements: Maximal Variance 12.5 POD and the Discrete Empirical Interpolation Method (DEIM) 12.6 DEIM Algorithm Implementation 12.7 Machine Learning ROMs Glossary Bibliography Index