Deep Reinforcement Learning for Wireless Communications and Networking: Theory, Applications and Implementation

Description

Deep Reinforcement Learning for Wireless Communications and Networking Comprehensive guide to Deep Reinforcement Learning (DRL) as applied to wireless communication systems Deep Reinforcement Learning for Wireless Communications and Networking presents an overview of the development of DRL while providing fundamental knowledge about theories, formulation, design, learning models, algorithms and implementation of DRL together with a particular case study to practice. The book also covers diverse applications of DRL to address various problems in wireless networks, such as caching, offloading, resource sharing, and security. The authors discuss open issues by introducing some advanced DRL approaches to address emerging issues in wireless communications and networking. Covering new advanced models of DRL, e.g., deep dueling architecture and generative adversarial networks, as well as emerging problems considered in wireless networks, e.g., ambient backscatter communication, intelligent reflecting surfaces and edge intelligence, this is the first comprehensive book studying applications of DRL for wireless networks that presents the state-of-the-art research in architecture, protocol, and application design. Deep Reinforcement Learning for Wireless Communications and Networking covers specific topics such as Deep reinforcement learning models, covering deep learning, deep reinforcement learning, and models of deep reinforcement learning Physical layer applications covering signal detection, decoding, and beamforming, power and rate control, and physical-layer security Medium access control (MAC) layer applications, covering resource allocation, channel access, and user/cell association Network layer applications, covering traffic routing, network classification, and network slicing With comprehensive coverage of an exciting and noteworthy new technology, Deep Reinforcement Learning for Wireless Communications and Networking is an essential learning resource for researchers and communications engineers, along with developers and entrepreneurs in autonomous systems, who wish to harness this technology in practical applications. Cover Table of Contents Title Page Copyright Dedication Notes on Contributors Foreword Preface Acknowledgments Acronyms Introduction Part I: Fundamentals of Deep Reinforcement Learning 1 Deep Reinforcement Learning and Its Applications 1.1 Wireless Networks and Emerging Challenges 1.2 Machine Learning Techniques and Development of DRL 1.3 Potentials and Applications of DRL 1.4 Structure of this Book and Target Readership 1.5 Chapter Summary References 2 Markov Decision Process and Reinforcement Learning 2.1 Markov Decision Process 2.2 Partially Observable Markov Decision Process 2.3 Policy and Value Functions 2.4 Bellman Equations 2.5 Solutions of MDP Problems 2.6 Reinforcement Learning 2.7 Chapter Summary References 3 Deep Reinforcement Learning Models and Techniques 3.1 Value-Based DRL Methods 3.2 Policy-Gradient Methods 3.3 Deterministic Policy Gradient (DPG) 3.4 Natural Gradients 3.5 Model-Based RL 3.6 Chapter Summary References 4 A Case Study and Detailed Implementation 4.1 System Model and Problem Formulation 4.2 Implementation and Environment Settings 4.3 Simulation Results and Performance Analysis 4.4 Chapter Summary References Note Part II: Applications of DRL in Wireless Communications and Networking 5 DRL at the Physical Layer 5.1 Beamforming, Signal Detection, and Decoding 5.2 Power and Rate Control 5.3 Physical-Layer Security 5.4 Chapter Summary References 6 DRL at the MAC Layer 6.1 Resource Management and Optimization 6.2 Channel Access Control 6.3 Heterogeneous MAC Protocols 6.4 Chapter Summary References 7 DRL at the Network Layer 7.1 Traffic Routing 7.2 Network Slicing 7.3 Network Intrusion Detection 7.4 Chapter Summary References 8 DRL at the Application and Service Layer 8.1 Content Caching 8.2 Data and Computation Offloading 8.3 Data Processing and Analytics 8.4 Chapter Summary References Part III: Challenges, Approaches, Open Issues, and Emerging Research Topics 9 DRL Challenges in Wireless Networks 9.1 Adversarial Attacks on DRL 9.2 Multiagent DRL in Dynamic Environments 9.3 Other Challenges 9.4 Chapter Summary References 10 DRL and Emerging Topics in Wireless Networks 10.1 DRL for Emerging Problems in Future Wireless Networks 10.2 Advanced DRL Models 10.3 Chapter Summary References Note Index End User License Agreement