Control and System Theory of Discrete-Time Stochastic Systems

Description

This book helps students, researchers, and practicing engineers to understand the theoretical framework of control and system theory for discrete-time stochastic systems so that they can then apply its principles to their own stochastic control systems and to the solution of control, filtering, and realization problems for such systems. Applications of the theory in the book include the control of ships, shock absorbers, traffic and communications networks, and power systems with fluctuating power flows. The focus of the book is a stochastic control system defined for a spectrum of probability distributions including Bernoulli, finite, Poisson, beta, gamma, and Gaussian distributions. The concepts of observability and controllability of a stochastic control system are defined and characterized. Each output process considered is, with respect to conditions, represented by a stochastic system called a stochastic realization. The existence of a control law is related to stochastic controllability while the existence of a filter system is related to stochastic observability. Stochastic control with partial observations is based on the existence of a stochastic realization of the filtration of the observed process.​ Preface Motivation of Control and Filtering of Stochastic Systems About the Book Chapter Relations To the Reader To the Teacher Concluding Remarks Acknowledgements Contents Abbreviations and Symbols Abbreviations Symbols 1 Control Problems 1.1 Control of a Mooring Tanker 1.2 Control of Freeway Traffic Flow 1.3 Control of a Shock Absorber 1.4 Further Reading References 2 Probability 2.1 Probability Distribution Functions 2.2 Motivation of the Concept of a Probability Measure 2.3 Sets and σ-Algebras 2.3.1 Operations on Sets 2.3.2 σ-Algebras 2.3.3 The Borel σ-Algebra of the Real Numbers 2.4 Probability Measures 2.4.1 Probability Measures on the Real Numbers 2.4.2 Finite Probability Spaces 2.4.3 Independence 2.5 Random Variables 2.5.1 Real-Valued Measurable Functions 2.5.2 Transformation of a Probability Space by a Random Variable 2.5.3 Support of a Measure of a Random Variable 2.5.4 Finite-Valued Random Variables 2.5.5 Random Variables and σ-Algebras 2.6 Expectation and the Characteristic Function 2.6.1 The Characteristic Function 2.6.2 Expectations of Several Probability Distributions 2.7 Gaussian Random Variables 2.8 Conditional Expectation 2.8.1 Properties of Conditional Expectation 2.8.2 Special Cases of Conditional Expectation 2.9 Conditional Independence 2.10 Computations 2.11 Exercises 2.12 Further Reading References 3 Stochastic Processes 3.1 Concepts 3.1.1 Construction of a Stochastic Process 3.1.2 Equivalent Processes 3.2 Special Subsets of Stochastic Processes 3.3 Properties of Stochastic Processes 3.3.1 Integrability of Stochastic Processes 3.3.2 Stationarity and Time-Reversibility 3.3.3 Markov Processes 3.4 Gaussian Processes 3.4.1 Covariance Functions 3.4.2 Stationarity and Time-Reversibility of Gaussian Processes 3.4.3 Gauss–Markov Processes 3.5 Finite-Valued Stochastic Processes 3.6 Exercises 3.7 Further Reading References 4 Gaussian Stochastic Systems 4.1 Modeling of Phenomena as a Stochastic System 4.2 The Concept of a Stochastic System 4.3 Time-Varying Gaussian Systems 4.4 Time-Invariant Gaussian Systems 4.4.1 The Invariant Measure of a Forward Gaussian System Representation 4.4.2 The Invariant Measure of a Backward Gaussian System 4.4.3 Transformations of a Gaussian System 4.5 Relation of Forward and Backward Gaussian System Representations 4.6 Stochastic Observability and Stochastic Co-Observability 4.6.1 Observability and Co-Observability 4.6.2 Stochastic Observability and Stochastic Co-Observability 4.6.3 Stochastic Observability and Stochastic Co-Observability of Gaussian Systems 4.7 Interconnections of Gaussian Systems 4.8 Stochastic Stability 4.9 Gaussian Factor Models and Gaussian Factor Systems 4.10 Computations 4.11 Exercises 4.12 Further Reading References 5 Stochastic Systems 5.1 Stochastic Systems and Probability Distributions 5.1.1 Output-State Conditional Probability Distribution 5.1.2 Next-State–Current-State Conditional Probability Distributions 5.2 Output in Binary Set 5.3 Output in the Natural Numbers 5.4 Output in a Bounded Interval 5.5 Output in the Positive Real Numbers 5.6 Output in the Real Numbers 5.7 Output-Finite–State-Polytopic Stochastic Systems 5.7.1 Time-Varying Stochastic Systems 5.7.2 Time-Invariant Stochastic Systems 5.7.3 State Set a Polytope 5.7.4 Decompositions of the State Set 5.7.5 Forward and Backward System Representations 5.7.6 Stochastic Observability 5.8 σ-Algebraic Stochastic System 5.9 The Multiple Conditional Independence Relation 5.10 Technicalities 5.11 Further Reading References 6 Stochastic Realization of Gaussian Systems 6.1 Introduction to Realization Theory 6.2 Motivation 6.3 Weak Gaussian Stochastic Realization Problem 6.4 The Theorem 6.5 Explanation 6.6 The Proof 6.7 Realization Procedures 6.8 State-Space Reduction of a Gaussian System 6.9 Special Stochastic Realizations-1 6.10 Special Stochastic Realizations-2 6.11 A Canonical Form 6.12 Exercises 6.13 Further Reading References 7 Stochastic Realization 7.1 The Conceptual Framework of Stochastic Realization 7.2 Stochastic Realization of a Tuple of Gaussian Random Variables 7.2.1 Concepts 7.2.2 The Problem 7.2.3 Characterization of Minimality 7.2.4 Classification 7.2.5 Strong Stochastic Realization of a Tuple of Gaussian Random Variables 7.2.6 States in the Frame σ-algebra 7.2.7 Classification 7.3 Stochastic Realization of a Tuple of σ-Algebras 7.3.1 Problem of Stochastic Realization 7.3.2 Concepts 7.3.3 Existence of Minimal State σ-Algebras 7.3.4 State σ-Algebras in the Frame σ-Algebra 7.3.5 Introduction to Characterization 7.3.6 Characterization of Minimal State σ-Algebras-1 7.3.7 Characterization of Minimal State σ-Algebras-2 7.3.8 Relations of Tuples of Minimal State σ-Algebras 7.4 Stochastic Realization of a σ-Algebra Family 7.4.1 Problem 7.4.2 Concepts 7.4.3 Characterization of a Stochastic Realization 7.4.4 Stochastic Realization as a Filter System 7.4.5 Minimality of a Stochastic Realization 7.5 Stochastic Realization of Output-Finite Stochastic Systems 7.6 Further Reading References 8 Filtering of Gaussian Systems 8.1 Problems of Filtering, Prediction, Smoothing, and Interpolation 8.2 Problem of Filtering 8.3 Time-Varying Kalman Filter 8.4 Time-Varying Kalman Filter and Stochastic Realization 8.4.1 The Kalman Filter and the Wiener Filter 8.4.2 The Kalman Filter and Stochastic Realization 8.4.3 Derivation of the Kalman Filter via Stochastic Realization 8.4.4 Levinson Filter 8.5 Time-Invariant Kalman Filter 8.6 Approximations of a Time-Invariant Kalman Filter 8.7 Sensor Allocation 8.8 Prediction 8.9 Interpolation 8.10 Conditional Kalman Filter 8.11 Exercises 8.12 Further Reading References 9 Filtering of Stochastic Systems 9.1 Problems of Estimation, Sequential Estimation, and of Filtering 9.2 Finite-Dimensional Filter Systems 9.3 Estimation Theory 9.3.1 Estimator Binomial–Beta 9.3.2 Estimator Poisson–Gamma 9.3.3 Estimator Gamma–Gamma 9.3.4 Estimator Gauss–Gauss 9.3.5 Estimation of a Finite-Valued Random Variable 9.4 Sequential Estimation 9.4.1 Sequential Estimator Binomial–Beta 9.4.2 Sequential Estimator Gamma–Gamma 9.5 Filtering Theory 9.6 Filter of a Poisson–Gamma System 9.7 Filter of an Output-Finite–State-Finite Stochastic System 9.8 Further Reading References 10 Stochastic Control Systems 10.1 Stochastic Control System 10.2 Gaussian Stochastic Control Systems 10.3 Stochastic Controllability and Stochastic Co-Controllability 10.3.1 Controllability of a Deterministic System 10.3.2 The Concept of Stochastic Controllability 10.3.3 Stochastic Controllability of a Gaussian Control System 10.4 State-Finite Stochastic Control Systems 10.4.1 Definition 10.4.2 Stochastic Controllability 10.5 Further Reading References 11 Stochastic Control Problems 11.1 Control Problems of Stochastic Control 11.2 Control Laws 11.3 Closed-Loop Stochastic Control Systems 11.4 Stochastic Control Problems 11.5 Control Synthesis and Control Design 11.6 Statistical Decision Problems 11.7 Exercises 11.8 Further Reading References 12 Stochastic Control with Complete Observations on a Finite Horizon 12.1 Control Problems 12.2 Problem Formulation 12.3 Explanation of Dynamic Programming 12.4 Digression on Optimization 12.5 Digression on Measurable Control Laws 12.6 Dynamic Programming for Additive Cost Functions 12.7 Control of a Gaussian Control System 12.8 Control of a State-Finite Stochastic Control System 12.9 Invariance of a Subset of Value Functions 12.10 Relation of Optimal Control Law and State 12.11 Dynamic Programming for Multiplicative Cost Functions 12.12 Stochastic Control Problems of Economics and of Finance 12.13 Control via System Approximation 12.14 Exercises 12.15 Further Reading References 13 Stochastic Control with Complete Observations on an Infinite Horizon 13.1 Introduction to Control on an Infinite Horizon 13.2 Average Cost 13.2.1 Problem Formulation 13.2.2 Positive Cost 13.2.3 Control of a Gaussian Control System 13.2.4 Control of a State-Finite Stochastic Control System 13.2.5 Derivation of Dynamic Programming Equation 13.2.6 Dynamic Programming 13.2.7 Computation of an Optimal Control Law 13.3 Discounted Cost 13.3.1 Positive Cost 13.3.2 Control of a Gaussian Stochastic Control System 13.3.3 Control of a State-Finite Stochastic Control System 13.3.4 Procedures 13.4 Minimum-Variance Control with Complete Observations 13.5 Exercises 13.6 Further Reading References 14 Stochastic Control with Partial Observations on a Finite Horizon 14.1 Motivation 14.2 Problem Formulation 14.3 Stochastic Realization of a Stochastic Control System 14.4 Stochastic Control of a Gaussian Stochastic Control System 14.4.1 Stochastic Realization by Filtering 14.4.2 Stochastic Realization by the Conditional Kalman Filter 14.4.3 Dynamic Programming 14.4.4 Quadratic Cost Rate 14.4.5 Examples 14.4.6 Cost Function Is Exponential–Quadratic 14.4.7 A Tracking Problem 14.5 Control of a State-Finite Stochastic Control System 14.5.1 Problem Formulation 14.5.2 Filtering 14.5.3 Dynamic Programming 14.5.4 Example 14.5.5 Alternative Control Problem 14.6 Exercises 14.7 Further Reading References 15 Stochastic Control with Partial Observations on an Infinite Horizon 15.1 Problem Issues 15.2 Control of a Gaussian Stochastic Control System 15.2.1 Problem Formulation 15.2.2 Stochastic Realization 15.2.3 Dynamic Programming 15.2.4 Quadratic Cost Rate 15.3 Minimum-Variance Control with Partial Observations 15.4 Further Reading References 16 Stochastic Control Theory 16.1 Research Problems of Control of Stochastic Systems 16.1.1 Control for the Effective Interaction of Control and Observation 16.1.2 Control of Partially Observed Stochastic Control Systems 16.1.3 Control of Communication Systems 16.1.4 Control of a Networked Stochastic Control System 16.1.5 Multiple Conditional Independence 16.2 General Optimality Conditions 16.3 Stochastic Control Via a Measure Transformation 16.4 Further Reading References 17 Appendix: Mathematics 17.1 Algebra of Sets 17.1.1 Relations and Canonical Forms 17.1.2 Order Relation 17.1.3 Functions 17.2 Algebraic Structures 17.3 Linear Algebra and Linear Dependence 17.4 Matrices 17.4.1 Linear Transformation and Their Matrix Representations 17.4.2 Multiplicative Factorization of a Matrix 17.4.3 Square Matrices 17.4.4 Determinant, Trace, and Norm 17.4.5 Spectral Theory 17.4.6 Inverses of Nonsingular Square Matrices 17.4.7 Symmetric Square Matrices 17.4.8 Contragradient Transform 17.5 Analysis 17.6 Geometry 17.7 Optimization 17.8 Further Reading References 18 Appendix: Positive Matrices 18.1 Problems 18.2 The Positive Real Numbers and a Positive Vector Space 18.3 Definitions of Positive Matrices 18.4 Geometry and Cones 18.5 Units 18.6 Similarity 18.7 Eigenvalues and Eigenvectors of Positive Matrices 18.8 Eigenvalues and Eigenvectors of Stochastic Matrices 18.8.1 A Partition of the Set of Stochastic Matrices 18.8.2 Overview of Convergence Results 18.8.3 Irreducible Matrices 18.8.4 Reducible Matrices 18.8.5 Weighted Average of Powers 18.9 Multiplicative Factorization 18.9.1 Problem 18.9.2 Equivalence 18.9.3 Permutation Matrices 18.9.4 Circulant Doubly Stochastic Matrices 18.9.5 Doubly Stochastic Matrices 18.9.6 Multiplicative Factorizations 18.9.7 Positive Matrices 18.9.8 Positive Matrices—Extremal Cones 18.9.9 Embedding in an Extremal Cone 18.10 Computations 18.11 Further Reading References 19 Appendix: Probability 19.1 Sets and the Monotone Class Theorems 19.2 Probability Measures 19.3 Stable Subsets of Probability Distribution Functions 19.4 Gaussian Random Variables 19.5 Spaces and Sequences of Random Variables 19.5.1 Sequences of Random Variables 19.5.2 Convergence of Sequences of Random Variables 19.6 Conditional Expectation and Conditional Probability 19.7 Conditionally Gaussian Random Variables 19.8 Conditional Independence Continued 19.8.1 Conditional Independence and Finite Probability Spaces 19.9 Measure Transformations 19.10 The Family of Exponential Probability Distributions 19.11 Pseudo-distances on the Set of Probability Measures 19.12 P-Essential Infima 19.13 Further Reading References 20 Appendix: Stochastic Processes 20.1 Stochastic Processes and Filtrations 20.2 Martingale Theory 20.3 Stochastic Processes and Stopping Times 20.4 Supermartingale Convergence 20.5 Ergodicity 20.6 Further Reading References 21 Appendix: Control and System Theory of Deterministic Systems 21.1 Deterministic Control Systems 21.2 Controllability 21.2.1 Stabilizability 21.2.2 Controllability After Feedback 21.2.3 Controllability of a Time-Varying Linear Control System 21.3 Observability 21.4 Geometric Approach to Linear Systems 21.5 Zero-Output Dynamics 21.6 Inverse of a Linear System 21.7 Canonical Factorization of a Deterministic Map 21.8 Realization Theory for Linear Systems 21.8.1 Realization Theorem and Its Proof 21.8.2 Reduction of a Nonminimal Realization 21.9 Stability 21.10 Further Reading References 22 Appendix: Matrix Equations 22.1 Lyapunov Equation 22.2 Algebraic Riccati Equations of Filtering and of Control 22.2.1 Existence and Characterization of Solution 22.2.2 Proof for the Filter Algebraic Riccati Equation 22.2.3 Dependence of the Solution on Matrices 22.2.4 Computation of the Solution 22.2.5 The Hamiltonian Approach 22.3 Algebraic Riccati Equation of Gaussian Stochastic Realization 22.3.1 Existence and Characterization of the Solution 22.3.2 An Iterative Algorithm for the Algebraic Riccati Equation of Stochastic Realization 22.3.3 The Hamiltonian Approach to the Algebraic Riccati Equation of Stochastic Realization 22.4 Further Reading References 23 Appendix: Covariance Functions and Dissipative Systems 23.1 Definitions 23.2 Storage Functions 23.3 Relations 23.4 Algebraic Characterization of Dissipative Linear Systems 23.5 Further Reading References 24 Appendix: State-Variance Matrices 24.1 Definition and Problem Formulation 24.2 Transformations 24.3 The Geometric Structure 24.4 Regularity 24.5 The Boundary of the Set of State-Variance Matrices 24.6 Singular Boundary Matrices 24.7 The Classification of State-Variance Matrices 24.8 Further Reading References Index