Distributed Optimization, Game and Learning Algorithms: Theory and Applications in Smart Grid Systems

Description

This book provides the fundamental theory of distributed optimization, game and learning. It includes those working directly in optimization,-and also many other issues like time-varying topology, communication delay, equality or inequality constraints,-and random projections. This book is meant for the researcher and engineer who uses distributed optimization, game and learning theory in fields like dynamic economic dispatch, demand response management and PHEV routing of smart grids. Preface Acknowledgements Contents List of Figures 1 Cooperative Distributed Optimization in Multiagent Networks with Delays 1.1 Introduction 1.2 Related Work 1.3 Preliminary 1.4 Main Results 1.5 Convergence Analysis 1.6 Numerical Examples 1.7 Conclusions References 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology 2.1 Introduction 2.2 Preliminaries 2.3 Main Results 2.3.1 Model Transformation 2.3.2 State Transition Matrix 2.3.3 Constrained Consensus 2.4 Numerical Examples 2.4.1 Constrained Consensus of a Simple Directed Multi-agent System 2.4.2 Constrained Consensus of a Large-Scale Directed Multi-agent System 2.5 Conclusion References 3 Distributed Optimization Under Inequality Constraints and Random Projections 3.1 Introduction 3.2 Problem Formulation and Assumptions 3.2.1 Network Model 3.2.2 Problem Formulation 3.2.3 Lagrangian Duality 3.2.4 Distributed Primal-Dual Random Projection Subgradient Algorithm 3.3 Basic Relations 3.4 Convergence Results for Diminishing Stepsize 3.5 Illustrative Examples 3.6 Conclusion References 4 Accelerated Distributed Optimization over Digraphs with Stochastic Matrices 4.1 Introduction 4.2 Preliminaries and Problem Formulation 4.3 Algorithm Development 4.4 Convergence Analysis 4.4.1 Auxiliary Relations 4.4.2 Main Results 4.5 Illustrative Examples 4.6 Conclusion References 5 Linear Convergence for Constrained Optimization Over Time-Varying Digraphs 5.1 Introduction 5.2 Preliminaries 5.2.1 Problem Formulation 5.2.2 Communication Network 5.2.3 Necessary Assumptions 5.3 Main Algorithm 5.3.1 Centralized Primal-Dual Method 5.3.2 Motivation 5.3.3 Distributed Primal-Dual Push-DIGing Algorithm 5.3.4 Supporting Lemmas 5.4 Convergence Analysis 5.5 Numerical Experiments 5.5.1 Case Study 1: Simulation for Five-Agents Network 5.5.2 Case Study 2: Simulation for Large-Scale Network 5.5.3 Case Study 3: Performance Comparison 5.6 Conclusion References 6 Stochastic Gradient-Push for Economic Dispatch on Time-Varying Digraphs 6.1 Introduction 6.2 Problem Formulation 6.2.1 Graph Theory 6.2.2 Economic Dispatch Problem 6.2.3 Centralized Lagrangian-Based Method 6.3 Algorithm and Main Results 6.3.1 State-of-the-Art Method for EDP 6.3.2 Distributed Stochastic Gradient-Push Algorithm 6.3.3 Necessary Assumptions 6.3.4 Main Results 6.4 Convergence Analysis 6.4.1 Supporting Lemmas 6.4.2 Proof of Theorem 6.6 6.4.3 Proof of Theorem 6.7 6.4.4 Proof of Theorem 6.8 6.5 Simulations 6.5.1 Without Time Delay 6.5.2 With Time Delays 6.6 Conclusion References 7 Reinforcement Learning in Energy Trading Game Among Smart Microgrids 7.1 Introduction 7.2 Problem Formulation 7.2.1 Smart Grid Model 7.2.2 Specifical Utility Functions 7.2.3 Stackelberg Game Scheme 7.3 Utility Maximization and Best Strategy 7.3.1 Utility Maximization and the Best Action 7.3.2 Learning the Best Response 7.3.3 Properties of Stackelberg Equilibrium 7.4 Learning Algorithms Design 7.4.1 Finite Action Learning Automaton (FALA) 7.4.2 Continuous Action Learning Automaton (CALA) 7.5 Numerical Case Studies and Discussion 7.5.1 Case Configuration 7.5.2 Buyer Side 7.5.3 Seller Side 7.5.4 Stackelberg Game 7.6 Conclusion References 8 Reinforcement Learning for Constrained Games with Incomplete Information 8.1 Introduction 8.2 Problem Formulation 8.2.1 Smart Grid Model 8.2.2 Double Auction Mechanism 8.3 Learning-Based Game to Energy Trading 8.3.1 Strategy Updating as Learning Automaton 8.3.2 Repeated Game with Incomplete Information 8.4 Mixed-Strategy NE and Learning Games 8.4.1 Mixed Strategy and Nash Equilibrium 8.4.2 Stationary Strategies and Convergence Analysis 8.5 Properties of Nash Equilibrium 8.5.1 Existence of NE 8.5.2 Diagonal Concavity 8.5.3 Lagrange Multiplier and Uniqueness of NE 8.6 Learning Algorithm Design and Analysis 8.7 Numerical Case Studies and Discussion 8.8 Conclusions References 9 Reinforcement Learning for PHEV Route Choice Based on Congestion Game 9.1 Introduction 9.2 Problem Formulation 9.2.1 Route Choice Problem in Traffic Networks 9.2.2 Learning Automaton 9.3 Main Results 9.3.1 Existence and Uniqueness of Optimal Route Choice Strategy 9.3.2 Reinforcement Learning Scheme 9.4 Numerical Experiments 9.4.1 Experiment Configuration 9.4.2 Experimental Results 9.5 Conclusion References