Simplifying Data Engineering and Analytics with Delta: Create analytics-ready data that fuels artificial intelligence and business intelligence

Description

Explore how Delta brings reliability, performance, and governance to your data lake and all the AI and BI use cases built on top of it Key FeaturesLearn Delta's core concepts and features as well as what makes it a perfect match for data engineering and analysisSolve business challenges of different industry verticals using a scenario-based approachMake optimal choices by understanding the various tradeoffs provided by DeltaBook Description Delta helps you generate reliable insights at scale and simplifies architecture around data pipelines, allowing you to focus primarily on refining the use cases being worked on. This is especially important when you consider that existing architecture is frequently reused for new use cases. In this book, you'll learn about the principles of distributed computing, data modeling techniques, and big data design patterns and templates that help solve end-to-end data flow problems for common scenarios and are reusable across use cases and industry verticals. You'll also learn how to recover from errors and the best practices around handling structured, semi-structured, and unstructured data using Delta. After that, you'll get to grips with features such as ACID transactions on big data, disciplined schema evolution, time travel to help rewind a dataset to a different time or version, and unified batch and streaming capabilities that will help you build agile and robust data products. By the end of this Delta book, you'll be able to use Delta as the foundational block for creating analytics-ready data that fuels all AI/BI use cases. What you will learnExplore the key challenges of traditional data lakesAppreciate the unique features of Delta that come out of the boxAddress reliability, performance, and governance concerns using DeltaAnalyze the open data format for an extensible and pluggable architectureHandle multiple use cases to support BI, AI, streaming, and data discoveryDiscover how common data and machine learning design patterns are executed on DeltaBuild and deploy data and machine learning pipelines at scale using DeltaWho this book is for Data engineers, data scientists, ML practitioners, BI analysts, or anyone in the data domain working with big data will be able to put their knowledge to work with this practical guide to executing pipelines and supporting diverse use cases using the Delta protocol. Basic knowledge of SQL, Python programming, and Spark is required to get the most out of this book. Table of ContentsAn Introduction to Data EngineeringData Modeling and ETLDelta – The Foundation Block for Big DataUnifying Batch and Streaming with DeltaData Consolidation in Delta LakeSolving Common Data Pattern Scenarios with DeltaDelta for Data Warehouse Use CasesHandling Atypical Data Scenarios with DeltaDelta for Reproducible Machine Learning PipelinesDelta for Data Products and ServicesOperationalizing Data and ML PipelinesOptimizing Cost and Performance with DeltaManaging Your Data Journey