Data Center Networking: Network Topologies and Traffic Management in Large-Scale Data Centers

Description

This book provides a comprehensive reference in large data center networking. It first summarizes the developing trend of DCNs, and reports four novel DCNs, including a switch-centric DCN, a modular DCN, a wireless DCN, and a hybrid DCN.  Furthermore another important factor in DCN targets at managing and optimizing the network activity at the level of transfers to aggregate correlated data flows and thus directly to lower down the network traffic resulting from such data transfers. In particular, the book reports the in-network aggregation of incast transfer, shuffle transfer, uncertain incast transfer, and the cooperative scheduling of uncertain multicast transfer. Preface Background Content Organization Contents Part I Basic Knowledge 1 Introduction of Data Center 1.1 Evolution of Data Centers 1.1.1 Basic Concept and Intuitive Classification 1.1.2 Demands for Data Centers from Cloud Computing 1.1.3 Demands for Data Centers from Big Data Applications 1.1.4 The Development of New Generation Data Centers 1.2 Fundamental Services Offered by Data Centers 1.2.1 Storage Service 1.2.2 Computing Services 1.2.3 Big Data Applications 1.3 Challenges for Data Center Networks 1.3.1 Customization of Network Functionality 1.3.2 High Scalability of Data Center Networks 1.3.3 Efficient Multiplexing of Network Resources 1.3.4 Network Virtualization of Data Centers 1.3.5 Cooperative Transmission of Correlated Traffic References 2 State-of-the-Art DCN Topologies 2.1 Introduction 2.2 Switch-Centric DCN Topologies 2.2.1 Tree-Like Topologies 2.2.2 Flat Topologies 2.2.3 Optical Switching Topologies 2.3 Server-Centric DCN Topologies 2.3.1 Compound Graph-Based DCN Topologies 2.3.2 Non-Compound Graph-Based DCN Topologies 2.4 Modular DCN Topologies 2.4.1 Intra-module Network Topology 2.4.2 Inter-module Network Topology 2.5 Random DCN Topologies 2.5.1 Small-World Network-Based DCN Topology 2.5.2 Random Regular Graph-Based DCN Topology 2.5.3 Scale-Free Network-Based DCN Topology 2.6 Wireless DCN Topologies 2.6.1 60 GHz-Based Hybrid DCN Topology 2.6.2 60 GHz-Based Wireless DCN Topology 2.6.3 FSO-Based DCN Topology 2.6.4 VLC-Based DCN Topology 2.7 Evolution and Future Trends of DCN Topology Design 2.7.1 Evolution of DCN Topology Design 2.7.2 Future Trends of DCN Topology Design References Part II Novel Data Center Network Structures 3 HCN: A Server-Centric Network Topology for Data Centers 3.1 Introduction 3.2 The Design of HCN Topology 3.2.1 Basic Theories of the Compound Graph 3.2.2 The Construction Methodology of HCN 3.3 The Design of BCN Topology 3.3.1 Description of BCN Topology 3.3.2 The Construction Methodology of BCN 3.4 Routing Mechanism of BCN 3.4.1 Single-path for the Unicast Traffic 3.4.2 Multi-paths for Unicast Traffic 3.4.3 Fault-Tolerant Routing in BCN 3.5 Performance Evaluation 3.5.1 Network Order 3.5.2 Low Network Diameter and Server Degree 3.5.3 Connectivity and Path Diversity 3.5.4 Evaluation of the Path Length 3.6 Discussion 3.6.1 Extension to More Server Ports 3.6.2 Locality-Aware Task Placement 3.6.3 Impact of the Server Routing References 4 DCube: A Family of Network Topologies for Containerized Data Centers 4.1 Introduction 4.2 The DCube Topology 4.2.1 Design Idea of DCube 4.2.2 H-DCube 4.2.3 M-DCube 4.3 Single-Path Routing for the Unicast Traffic 4.3.1 Single-Path Routing in H-DCube 4.3.2 Single-Path Routing in M-DCube 4.4 Multi-paths Routing for the Unicast and Multicast Traffic 4.4.1 Multi-path Routing for the Unicast Traffic in H-DCube 4.4.2 Multi-path Routing for the Unicast Traffic in M-DCube 4.4.3 Speedup for the Multicast Traffic 4.5 Analysis and Evaluation 4.5.1 Speedup for the Unicast and Multicast Traffic 4.5.2 Aggregate Bottleneck Throughput 4.5.3 Qualification of Cost and Cabling Complexity 4.5.4 Summary 4.6 Discussions 4.6.1 Locality-Aware Task Placement 4.6.2 Extension to More Server Interfaces 4.6.3 Impact of Server Routing References 5 R3: A Hybrid Network Topology for Data Centers 5.1 Introduction 5.2 The Design Methodology of Hybrid Topologies 5.2.1 Overview of Network Topologies 5.2.2 R3: A Hybrid Topology Based on the Compound Graph Theory 5.2.3 Deployment Strategy for Data Centers with a Hybrid Topology 5.3 Efficient Routing Methods of R3 5.3.1 Edge Coloring Based Identifier Allocation 5.3.2 Identifier-Based Routing Algorithm 5.4 Topology Optimization 5.4.1 Impact Factors of the R3 Topology 5.4.2 Optimization Strategy of Topologies 5.5 Incremental Expansion of R3 5.5.1 Expansion within an Existing Random Cluster 5.5.2 Expansion by Adding an Extra Random Cluster 5.6 Performance Evaluation 5.6.1 The Routing Flexibility 5.6.2 The Cabling Cost 5.6.3 The Network Performance 5.7 Discussion References 6 VLCcube: A Network Topology for VLC Enabled Wireless Data Centers 6.1 Introduction 6.1.1 Motivation 6.1.2 Related Work 6.2 The Design of VLCcube 6.2.1 Feasibility of Introducing VLC Links into DCNs 6.2.2 The Interference Among Transceivers 6.2.3 Topology Design of VLCcube 6.3 Routing and Congestion Aware Flow Scheduling in VLCcube 6.3.1 Hybrid Routing Scheme in VLCcube 6.3.2 Problem Formulation of Flow Scheduling 6.3.3 Scheduling the Batched Flows 6.3.4 Online Scheduling of Flows 6.4 Performance Evaluation 6.4.1 Settings of Evaluations 6.4.2 Topological Properties of VLCcube 6.4.3 Network Performance Under Trace Flows 6.4.4 Network Performance Under Stride-2k Flows 6.4.5 Network Performance Under Random Flows 6.4.6 Impact of Congestion Aware Flow Scheduling 6.5 Discussion References Part III Traffic Cooperation Management in Data Centers 7 Collaborative Management of Correlated Incast Transfer 7.1 Introduction 7.2 In-Network Aggregation of an Incast Transfer 7.2.1 Feasibility of the Inter-flow Data Aggregation 7.2.2 Minimal Incast Tree 7.2.3 Inter-flow Data Aggregation on an Incast Tree 7.3 Efficient Building Method of an Incast Tree 7.3.1 Online Construction of an Incast Tree 7.3.2 Construction of the Minimal Incast Tree 7.3.3 Dynamical Behaviors of Senders 7.3.4 Dynamical Behaviors of the Receiver 7.4 Discussion 7.4.1 Extension to General Incast Transfers 7.4.2 Extension to Other Network Topologies 7.4.3 Impact on the Job Execution Time 7.5 Performance Evaluation 7.5.1 The Prototype Implementation 7.5.2 Impact of the Data Center Size 7.5.3 Impact of the Incast Transfer Size 7.5.4 Impact of the Aggregation Ratio 7.5.5 Impact of the Distribution of Incast Members References 8 Collaborative Management of Correlated Shuffle Transfer 8.1 Introduction 8.2 In-Network Aggregation of Shuffle Transfers 8.2.1 Problem Statement 8.2.2 Construction of an Incast Aggregation Tree 8.2.3 Construction of a Shuffle Aggregation Subgraph 8.2.4 The Fault-Tolerance of Shuffle Aggregation Subgraph 8.3 Scalable Forwarding Schemes for Performing In-Network Aggregation 8.3.1 General Forwarding Scheme 8.3.2 In-Switch Bloom Filter Based Forwarding Scheme 8.3.3 In-Packet Bloom Filter Based Forwarding Scheme 8.4 Performance Evaluation 8.4.1 The Prototype 8.4.2 Impact of the Data Center Size 8.4.3 Impact of the Shuffle Transfer Size 8.4.4 Impact of the Aggregation Ratio 8.4.5 The Size of Bloom Filter in Each Packet References 9 Collaborative Management of Uncertain Incast Transfer 9.1 Introduction 9.2 Overview of Aggregating Uncertain Incast Transfer 9.2.1 Problem Statement of Uncertain Incast Transfer 9.2.2 Aggregating a Deterministic Incast Transfer 9.2.3 Aggregating an Uncertain Incast Transfer 9.3 Aggregation Tree Building Method for Uncertain Incast Transfer 9.3.1 The Diversity of In-Network Aggregation Gain 9.3.2 Initializing Senders for an Uncertain Incast Transfer 9.3.3 Aggregation Tree Building Method for Uncertain Incast 9.4 Performance Evaluation 9.4.1 Evaluation Methodology and Scenarios 9.4.2 Impact of the Incast Transfer Size 9.4.3 Impact of the Data Center Size 9.4.4 Impact of the Aggregation Ratio 9.4.5 Impact of Distributions of Incast Members 9.4.6 Impact of the Receiver Diversity References 10 Collaborative Management of Correlated Multicast Transfer 10.1 Introduction 10.2 Related Work 10.3 Problem Statement of Uncertain Multicast 10.3.1 Observations 10.3.2 Problem Statement 10.3.3 Mixed Integer Linear Programming 10.4 Efficient Building Methods of MCF 10.4.1 Primary Approximation Method 10.4.2 Enhanced Approximation Method 10.5 Performance Evaluation 10.5.1 Implementation of the Uncertain Multicast in SDN Testbed 10.5.2 Evaluation Based on Small-Scale Experiments 10.5.3 Evaluation Based on Large-Scale Simulations References