Data Mesh: Delivering Data-Driven Value at Scale

Description

Many enterprises are investing in a next-generation data lake, hoping to democratize data at scale to provide business insights and ultimately make automated intelligent decisions. In this practical book, author Zhamak Dehghani reveals that, despite the time, money, and effort poured into them, data warehouses and data lakes fail when applied at the scale and speed of today's organizations. A distributed data mesh is a better choice. Dehghani guides architects, technical leaders, and decision makers on their journey from monolithic big data architecture to a sociotechnical paradigm that draws from modern distributed architecture. A data mesh considers domains as a first-class concern, applies platform thinking to create self-serve data infrastructure, treats data as a product, and introduces a federated and computational model of data governance. This book shows you why and how. Examine the current data landscape from the perspective of business and organizational needs, environmental challenges, and existing architectures Analyze the landscape's underlying characteristics and failure modes Get a complete introduction to data mesh principles and its constituents Learn how to design a data mesh architecture Move beyond a monolithic data lake to a distributed data mesh Cover Starburst Data Copyright Table of Contents Part I. Why Data Mesh? Chapter 1. The Inflection Point Great Expectations of Data The Great Divide of Data Operational Data Analytical Data Analytical and Operational Data Misintegration Scale, Encounter of a New Kind Beyond Order Approaching the Plateau of Return Recap Chapter 2. After The Inflection Point Embrace Change in a Complex, Volatile and Uncertain Business Environment Align Business, Tech and Now Analytical Data Close The Gap Between Analytical and Operational Data Localize Data Change to Business Domains Reduce Accidental Complexity of Pipelines and Copying Data Sustain Agility in the Face of Growth Remove Centralized and Monolithic Bottlenecks of the Lake or the Warehouse Reduce Coordination of Data Pipelines Reduce Coordination of Data Governance Enable Autonomy Increase the Ratio of Value from Data to Investment Abstract Technical Complexity with a Data Platform Embed Product Thinking Everywhere Go Beyond The Boundaries Recap Chapter 3. Before The Inflection Point Evolution of Analytical Data Architectures First Generation: Data Warehouse Architecture Second Generation: Data Lake Architecture Third Generation: Multimodal Cloud Architecture Characteristics of Analytical Data Architecture Monolithic Monolithic Architecture Monolithic Technology Monolithic Organization The complicated monolith Technically-Partitioned Architecture Activity-oriented Team Decomposition Recap Part II. What is Data Mesh Chapter 4. Principle of Domain ownership Apply DDD’s Strategic Design to Data Domain Data Archetypes Source-aligned Domain Data Aggregate Domain Data Consumer-aligned Domain Data Transition to Domain Ownership Push Data Ownership Upstream Define Multiple Connected Models Embrace the Most Relevant Domain, and Don’t Expect the Single Source of Truth Hide the Data Pipelines as Domains’ Internal Implementation Recap Chapter 5. Principle of Data as a Product Apply Product Thinking to Data Baseline usability characteristics of a data product Transition to Data as a Product Include Data Product Ownership in Domains Recap Prospective Table of Contents (Subject to Change) Part I : Why Data Mesh? Chapter 1: The Inflection Point Chapter 2: After the Inflection Point Chapter 3: Before The Inflection Point Part II: What Is Data Mesh? Chapter 4: Principle of Domain Ownership Chapter 5: Principle of Data as a Product Chapter 6: Principle of Self-Serve Data Platform Chapter 7: Principle of Federated Computational Governance Part III: How to Design Data Mesh Architecture? Chapter 8: The Logical Architecture Chapter 9: Data Product Quantum Blueprint Chapter 10: The Multi-Plane Data Platform Part IV: How to Get Started With Data Mesh Chapter 11: Execution Model Chapter 12: Organization Design Chapter 13: What Comes Next