Cloud-Native Microservices with Apache Pulsar: Build Distributed Messaging Microservices

Description

Apply different enterprise integration and processing strategies available with Pulsar, Apache's multi-tenant, high-performance, cloud-native messaging and streaming platform. This book is a comprehensive guide that examines using Pulsar Java libraries to build distributed applications with message-driven architecture. You'll begin with an introduction to Apache Pulsar architecture. The first few chapters build a foundation of message-driven architecture. Next, you'll perform a setup of all the required Pulsar components. The book also covers work with Apache Pulsar client library to build producers and consumers for the discussed patterns.  You'll then explore the transformation, filter, resiliency, and tracing capabilities available with Pulsar. Moving forward, the book will discuss best practices when building message schemas and demonstrate integration patterns using microservices. Security is an important aspect of any application; the book will cover authentication and authorization in Apache Pulsar such as Transport Layer Security (TLS), OAuth 2.0, and JSON Web Token (JWT). The final chapters will cover Apache Pulsar deployment in Kubernetes. You'll build microservices and serverless components such as AWS Lambda integrated with Apache Pulsar on Kubernetes.  After completing the book, you'll be able to comfortably work with the large set of out-of-the-box integration options offered by Apache Pulsar. What You'll LearnExamine the important Apache Pulsar components Build applications using Apache Pulsar client librariesUse Apache Pulsar effectively with microservicesDeploy Apache Pulsar to the cloud Who This Book Is For Cloud architects and software developers who build systems in the cloud-native technologies. Table of Contents About the Authors About the Technical Reviewer Acknowledgments Introduction Chapter 1: Introduction to Apache Pulsar Distributed Challenges Availability Scalability Data Consistency Diverse Integration Event-Driven Architecture Innovation Resilience Scalability Extensible Apache Pulsar Unified Messaging Model Efficient Data Exchange Geo-Replication Multi-Tenancy Schema Registry Infinite Storage Cloud Native Pulsar Architecture Brokers Bookies Pulsar Components ZooKeeper Service Discovery Pulsar Proxy Pulsar Setup Pulsar Command Line Single-Node Cluster Docker Multi-Node Cluster ZooKeeper Bookies Brokers Verify Cluster Pulsar Admin API Summary Chapter 2: Working with Messages Types of Messages Message Channel Breaking up of Messages Messaging Patterns Point-2-Point Pub/Sub Message Filtering Durable Subscriptions Non-Durable Subscriptions Pulsar Client Libraries Client Setup Phase Creating a Producer Creating Consumers Publish Messages Messaging in Pulsar Message Topic Format Subscriptions Shared Key_Shared Exclusive Failover Receiver Queues Sequencing Using Acknowledgments Message Retention Time-to-Live (TTL) Retention Policy Backlog Quota Batching and Chunking Batching Batch Index Acknowledgment Batch Receiving in Consumers Chunking Handle Chunked Messages with One Producer and One Ordered Consumer Handle Chunked Messages with Multiple Producers and One Ordered Consumer Summary Chapter 3: Working with Pulsar Functions Functional Design Pulsar Functions Developing Function Type Support Custom Serialization Pulsar Functions SDK External Configuration Logs State Storage Testing Functions Deploying Functions LocalRunner Programmatic API Cluster Mode Message Delivery Modes At-Most-Once At-Least-Once Effectively-Once Summary Chapter 4: Schema Registry Schema-on-Write vs. Schema-on-Read Schema Registry and Pulsar SchemaInfo Primitive Data Types Complex Data Types Key/Value Key/Value Test Cases Struct Auto Schema Schema Version Schema Evolution and Compatibility Schema Compatibility Check Strategy Managing Schemas Auto Update of Schema: Producer Schema AutoUpdate: Consumer Adjust AutoUpdate Strategy Schema Validation Manual Schema Management Getting a Schema Deleting a Schema Custom Schema Storage Deploying the Classes Created Summary Chapter 5: Building Microservices Using Pulsar Scenario: Portfolio Tracker Enterprise Integration Fan-out Fan-in Router Interservice Communication Resiliency Dead-Letter Queues Retries Throttling Publisher Throttling Subscriber Throttling Dispatch Throttling Backpressure Backlog Quotas Time-to-Live (TTL) Topic Compaction Circuit Breaker Summary Chapter 6: Pulsar Connectors Working with Connectors Processing Guarantees Using Connectors Running Sources Running Sinks Monitoring Connectors Sample Source Connectors File Source Connector MySQL Source Connector Sink Connectors Sample Sink Connector Developing IO Connectors Developing a Source IO Connector Developing a Sink IO Connector Summary Chapter 7: Pulsar Security Quick Overview of TLS TLS Termination Prerequisites Configuring the Broker Configuring the Client Mutual TLS Configuring Mutual TLS in the Client Configuring Mutual TLS in the Broker JSON Web Token Creating a Token Configuring JWT in the Client Configuring JWT in the Broker Authorization Role Grants Configuring Broker for Authorization Configuring the Client for Authorization Summary Chapter 8: Deploying Pulsar on Kubernetes Setup Prerequisites Configure Helm Exploring Pulsar Helm Chart Installation Pulsar Components Pulsar Toolset Pulsar Proxy Pulsar Manager Deployment Pulsar Storage Tiered Storage Configure Offloading Pulsar Geo-replication Pulsar Monitoring pulsar_function_process_latency_ms pulsar_storage_write_latency_le bookie_journal_JOURNAL_SYNC pulsar_source_written_total / pulsar_sink_written_total pulsar_ml_NumberOfMessagesInBacklog pulsar_storage_size pulsar_replication_backlog Monitoring Dashboard Summary Index