Limitless Analytics with Azure Synapse: An end-to-end analytics service for data processing, management, and ingestion for BI and ML requirements

Description

Leverage the Azure analytics platform's key analytics services to deliver unmatched intelligence for your data Key FeaturesLearn to ingest, prepare, manage, and serve data for immediate business requirementsBring enterprise data warehousing and big data analytics together to gain insights from your dataDevelop end-to-end analytics solutions using Azure SynapseBook Description Azure Synapse Analytics, which Microsoft describes as the next evolution of Azure SQL Data Warehouse, is a limitless analytics service that brings enterprise data warehousing and big data analytics together. With this book, you'll learn how to discover insights from your data effectively using this platform. The book starts with an overview of Azure Synapse Analytics, its architecture, and how it can be used to improve business intelligence and machine learning capabilities. Next, you'll go on to choose and set up the correct environment for your business problem. You'll also learn a variety of ways to ingest data from various sources and orchestrate the data using transformation techniques offered by Azure Synapse. Later, you'll explore how to handle both relational and non-relational data using the SQL language. As you progress, you'll perform real-time streaming and execute data analysis operations on your data using various languages, before going on to apply ML techniques to derive accurate and granular insights from data. Finally, you'll discover how to protect sensitive data in real time by using security and privacy features. By the end of this Azure book, you'll be able to build end-to-end analytics solutions while focusing on data prep, data management, data warehousing, and AI tasks. What you will learnExplore the necessary considerations for data ingestion and orchestration while building analytical pipelinesUnderstand pipelines and activities in Synapse pipelines and use them to construct end-to-end data-driven workflowsQuery data using various coding languages on Azure SynapseFocus on Synapse SQL and Synapse SparkManage and monitor resource utilization and query activity in Azure SynapseConnect Power BI workspaces with Azure Synapse and create or modify reports directly from Synapse StudioCreate and manage IP firewall rules in Azure SynapseWho this book is for This book is for data architects, data scientists, data engineers, and business analysts who are looking to get up and running with the Azure Synapse Analytics platform. Basic knowledge of data warehousing will be beneficial to help you understand the concepts covered in this book more effectively. Table of ContentsIntroduction to Azure SynapseConsiderations for Your Compute EnvironmentBringing Your Data to Azure SynapseUsing Synapse Pipelines to Orchestrate Your DataUsing Synapse Link with Azure Cosmos DBWorking with T-SQL in Azure SynapseWorking with R, Python, Scala, .NET, and Spark SQL in Azure SynapseIntegrating a Power BI Workspace with Azure SynapsePerform Real-Time Analytics on Streaming DataGenerate Powerful Insights on Azure Synapse Using Azure Machine LearningPerforming Backup and Restore in Azure Synapse AnalyticsSecuring Data on Azure SynapseManaging and Monitoring Synapse WorkloadsCoding Best Practices Cover Copyright Contributors Table of Contents Preface Section 1: The Basics and Key Concepts Chapter 1: Introduction to Azure Synapse Technical requirements Introducing the components of Azure Synapse Creating a Synapse workspace Understanding Azure Data Lake Exploring Synapse Studio Summary Chapter 2: Considerations for Your Compute Environment Technical requirements Introducing SQL Pool Creating a SQL pool Understanding Synapse SQL Pool architecture and components Examining DWUs Understanding distributions in Synapse SQL Pool Understanding partitions in Synapse SQL Pool Using temporary tables in Synapse SQL Pool Discovering the benefits of Synapse SQL Pool Understanding Synapse SQL on-demand SQL on-demand architecture and components Learning about the benefits of Synapse SQL on-demand Understanding Spark pool Spark pool architecture and components Creating a Synapse Spark pool Learning about the benefits of a Synapse Spark pool Summary Section 2: Data Ingestion and Orchestration Chapter 3: Bringing Your Data to Azure Synapse Technical requirements Using Synapse pipelines to import data Bringing data to your Synapse SQL pool using Copy Data tool Using Azure Data Factory to import data Using SQL Server Integration Services to import data Using a COPY statement to import data Loading data from a public storage account Loading data from a private storage account using an SAS token Using authentication mechanisms Summary Chapter 4: Using Synapse Pipelines to Orchestrate Your Data Technical requirements Introducing Synapse pipelines Integration runtime Activities Pipelines Triggers Creating linked services Defining source and target datasets Using various activities in Synapse pipelines Scheduling Synapse pipelines Creating pipelines using samples Summary Chapter 5: Using Synapse Link with Azure Cosmos DB Technical requirements Enabling the analytical store in Cosmos DB Data storage Transactional store Analytical store Querying the Cosmos DB analytical store Querying with Azure Synapse Spark Querying with Azure Synapse SQL Serverless Summary Section 3: Azure Synapse for Data Scientists and Business Analysts Chapter 6: Working with T-SQL in Azure Synapse Technical requirements Supporting T-SQL language elements in a Synapse SQL pool CTEs SELECT – OVER clause Using dynamic SQL in Synapse SQL Learning GROUP BY options in Synapse SQL Using T-SQL loops in Synapse SQL Creating stored procedures and views in Synapse SQL Stored procedures Views Optimizing transactions in Synapse SQL Supporting system views in a Synapse SQL pool Using T-SQL queries on semi-structured and unstructured data Reading Parquet files Reading JSON documents External tables Summary Chapter 7: Working with R, Python, Scala, .NET, and Spark SQL in Azure Synapse Technical requirements Using Azure Open Datasets Using sample scripts PySpark (Python) Spark (Scala) .NET Spark (C#) Spark SQL Summary Chapter 8: Integrating a Power BI Workspace with Azure Synapse Technical requirements Connecting to a Power BI workspace Creating your own dashboard on Azure Synapse Creating new Power BI datasets Creating Power BI reports Connecting Azure Synapse data to Power BI Desktop Connecting to a Synapse-dedicated SQL pool Connecting to a Synapse serverless SQL pool Summary Chapter 9: Perform Real-Time Analytics on Streaming Data Technical requirements Understanding various architecture and components Bringing data to Azure Synapse Using Azure Stream Analytics Using Azure Databricks Implementation of real-time analytics on streaming data Ingesting data to Cosmos DB Accessing data from the Azure Cosmos DB analytical store in Azure Synapse Loading data to a Spark DataFrame Creating visualizations Summary Chapter 10: Generate Powerful Insights on Azure Synapse Using Azure ML Technical requirements Preparing the environment Creating a Text Analytics resource in the Azure portal Creating an Anomaly Detector resource in the Azure portal Creating an Azure key vault Creating an Azure ML linked service in Azure Synapse Machine learning capabilities in Azure Synapse Data ingestion and orchestration Data preparation and exploration Training machine learning models Use cases with Cognitive Services Sentiment analysis Anomaly detection Summary Section 4: Best Practices Chapter 11: Performing Backup and Restore in Azure Synapse Analytics Technical requirements Creating restore points Automatic restore points User-defined restore points Geo-backups and disaster recovery Geo-redundant restore through the Azure portal Geo-redundant restore through PowerShell Cross-subscription restore Summary Chapter 12: Securing Data on Azure Synapse Implementing network security Managed workspace virtual network Private endpoint for SQL on-demand IP firewall rules SQL authorization Azure Active Directory authorization Implementing RBAC in a Synapse SQL pool Enabling threat protection Azure SQL auditing Azure Defender for SQL Understanding information protection Summary Chapter 13: Managing and Monitoring Synapse Workloads Technical requirements Managing Synapse resources Analytics pools External connections Integration Security Source control Monitoring Synapse workloads Integration Activities Analytics pools Managing maintenance schedules Creating alerts for Azure Synapse Analytics Summary Chapter 14: Coding Best Practices Technical requirements Implementing best practices for a Synapse dedicated SQL pool Maintaining statistics Using correct distribution for your tables Using partitioning Using an adequate column size Advantages of using a minimum transaction size Using PolyBase to load data Reorganizing and rebuilding indexes Materialized views Using an appropriate resource class Implementing best practices for a Synapse serverless SQL pool Selecting the region to create a serverless SQL pool Files for querying Using CETAS to enhance query performance Implementing best practices for a Synapse Spark pool Configuring the Auto-pause setting Enhancing Apache Spark performance Summary Other Books You May Enjoy Index