Federated Learning

Description

Preface Contents Contributors Acronyms 1 Federated Learning Research: Trends and Bibliometric Analysis 1.1 Introduction 1.2 Material and Method 1.2.1 Data Collection 1.2.2 Data Analysis 1.3 Results and Discussion 1.3.1 Growth Pattern Over the Years 1.3.2 Top Cited Papers 1.3.3 Productivity Measures 1.3.4 Domain Profile 1.4 Related Work 1.5 Conclusion and Future Research Directions References 2 A Review of Privacy-Preserving Federated Learning for the Internet-of-Things 2.1 Introduction 2.2 Distributed Machine Learning 2.2.1 Concurrency 2.2.2 Model Consistency 2.2.3 Centralized Versus Decentralized Learning 2.3 Federated Learning 2.3.1 Overview 2.3.2 Specific Challenges for FL in IoT Context 2.3.3 Applied FL Research 2.4 Privacy Preservation 2.4.1 Privacy Preserving Methods 2.5 Privacy Preservation in FL 2.6 Challenges in Applying Privacy-Preserving FL to the IoT 2.6.1 Optimal Model Architecture/Hyperparameters 2.6.2 Continual Learning 2.6.3 Better Privacy-Preserving Methods 2.6.4 FL Combined with Fog Computing 2.6.5 FL on Low Power Devices 2.7 Conclusion References 3 Differentially Private Federated Learning: Algorithm, Analysis and Optimization 3.1 Introduction 3.2 Preliminaries 3.2.1 Federated Learning 3.2.2 Differential Privacy 3.2.3 Threat Model 3.3 Federated Learning with Differential Privacy 3.3.1 Global Differential Privacy 3.3.2 Proposed NbAFL 3.4 Convergence Analysis on NbAFL 3.5 K-Client Random Scheduling Policy 3.6 Differentially Private FL Based Client Selection 3.6.1 Algorithm Description 3.6.2 Noise Recalculation for Varying K 3.7 Experimental Results 3.7.1 Performance Evaluation on Protection Levels 3.7.2 Impact of the Number of Chosen Clients K 3.7.3 Impact of the Clipping Threshold 3.7.4 Impact of the Number of Clients N 3.7.5 Impact of the Number of Maximum Aggregation Times T 3.7.6 Impact of the Number of Chosen Clients K 3.7.7 Performance of DP-FedCS Algorithm 3.8 Conclusion References 4 Advancements of Federated Learning Towards Privacy Preservation: From Federated Learning to Split Learning 4.1 Introduction 4.2 Federated Learning to Split Learning 4.2.1 Overview of Federated Learning and Key Results 4.2.2 Split Learning and Key Results 4.2.3 Splitfed Learning and Key Results 4.3 Data Privacy and Privacy-Enhancing Techniques 4.3.1 Differential Privacy and Its Application 4.3.2 Differential Privacy in Federated Learning 4.3.3 Privacy in Split Learning 4.3.4 Privacy in Splitfed Learning 4.4 Applications and Implementation 4.4.1 Applications of Split Learning 4.4.2 Implementation of Split Learning 4.4.3 Implementation of Splitfed Learning with a Code Example 4.5 Challenges and Open Problems 4.5.1 Challenges and Open Problems in Federated Learning 4.5.2 Challenges and Open Problems in Split Learning 4.5.3 Challenges and Open Problems in Splitfed Learning 4.6 Conclusion References 5 PySyft: A Library for Easy Federated Learning 5.1 Introduction to PySyft 5.1.1 The PySyft Library 5.1.2 Privacy and FL 5.1.3 Differential Privacy 5.1.4 Secure Multi-party Computation 5.1.5 Homomorphic Encryption 5.2 PySyft Implementation 5.2.1 A Standardized Framework to Abstract Operations on Tensors 5.2.2 Building MPC-Aware Tensors 5.2.3 Actions, Plans and Protocols 5.3 The Road-Map for PySyft 5.3.1 Communications Across Nodes and Heterogeneous Languages 5.3.2 Improved Secure Multi-party Computation Implementations 5.4 Introducing `Duet': Easier FL for Scientists and Data Owners 5.5 PySyft Demonstration—Federated Learning on MNIST Using a CNN 5.5.1 Imports and Model Specifications 5.5.2 Data Loading and Sending to Workers 5.5.3 Convolutional Neural Network Specification 5.5.4 Define the Training and Test Functions 5.5.5 Launch the Training 5.6 PySyft Literature Review and Future Use Cases 5.6.1 Comparisons with Other Frameworks 5.6.2 Use Case: Benchmarking and Standardizing FL Systems 5.6.3 Use Case: FL on Edge Devices 5.6.4 Use Case: Healthcare and Medical Research 5.6.5 Use Case: Text and Language Processing 5.6.6 Use Case: Finance, Business or Industry 5.6.7 Use Case: Anomaly Detection 5.6.8 Other Use Cases 5.6.9 Comparisons and Adaptations to PySyft 5.7 Conclusion References 6 Federated Learning Systems for Healthcare: Perspective and Recent Progress 6.1 Introduction 6.2 Background Study 6.2.1 How FL Works in Healthcare 6.2.2 Importance of FL in Healthcare 6.2.3 Comparison Between Centralized Learning Versus Edge Computing Versus Decentralized Learning 6.2.4 Platforms for FL 6.2.5 Applications of FL 6.3 Reported Work 6.3.1 Electronic Health Record System Using FL 6.3.2 FL for Drug Discovery and Disease Detection 6.4 Analysis of FL in Healthcare System 6.5 Conclusion and Future Directions References 7 Towards Blockchain-Based Fair and Trustworthy Federated Learning Systems 7.1 Introduction 7.1.1 Cloud-Based Distributed Machine Learning 7.1.2 Federated Learning System 7.1.3 Decentralized Federated Learning System 7.2 Background and Related Work 7.2.1 Blockchain Technology 7.2.2 Federated Learning 7.2.3 Blockchain-Based Decentralized Federated Learning 7.3 Trust Requirements 7.3.1 Importance of the Fairness 7.3.2 Incentive Mechanism 7.3.3 Security and Privacy Requirements 7.4 Decentralized Federated Learning Systems 7.4.1 Proposed Systems 7.4.2 Discussion on Fairness for Decentralized FL Systems 7.4.3 Discussion on Security and Privacy for Decentralized FL Systems 7.5 Summary of the Implemented Features in Fair FL Systems 7.6 Conclusion References 8 An Overview of Federated Deep Learning Privacy Attacks and Defensive Strategies 8.1 Introduction 8.2 Threat Model 8.2.1 Attack Surface 8.2.2 Client or Server-Side Attacks 8.2.3 Black-Box or White-Box Attacks 8.2.4 Active or Passive Attacks 8.2.5 Attacker Goal 8.3 Attack Methods 8.3.1 Attacks Targeting Reconstruction 8.3.2 Attacks Targeting Inference 8.3.3 Attacks Targeting Misclassification 8.3.4 Attacks Targeting Model Corruption 8.4 Defensive Measures 8.4.1 Sharing Gradient Subset 8.4.2 Gradient Compression 8.4.3 Dropout 8.4.4 Differential Privacy (DP) 8.4.5 Secure Multiparty Computation (SMC) 8.4.6 Homomorphic Encryption 8.4.7 Robust Aggregation 8.4.8 Keep the Server in the Dark 8.5 Related Work 8.6 Conclusion References