Windows Internals, Part 2

Description

The definitive guide to modern Windows internals: new coverage of virtualization, file systems, boot, security, and more.   For advanced computing professionals, this is the definitive up-to-date guide to how Windows core components behave “under the hood.” Using it, experienced developers can build more powerful and scalable software, administrators can debug complex system and performance  problems, and security researchers can harden their systems. This Seventh Edition is fully updated through the May 2021 (21H1/2104) updates to Windows 10 and Windows Server (2022, 2019, and 2016). It adds extensive content on Hyper-V, plus fully rewritten chapters on the boot process, new storage technologies, and Windows system and management mechanisms. As always, it delivers unparalleled insight based on insider access to Microsoft source code, with hands-on experiments using the latest debugging tools to show you Windows’ internal behaviors firsthand. With Windows 11 introducing new user interface design elements that build upon the same core technologies as Windows 10, readers will be well-prepared for this new chapter of computing.   Leading Windows insiders help you: • Discover system mechanisms for serving device drivers and applications, including ALPC, Object Manager, synchronization, WNF, WoW64, and the processor execution model • Explore underlying hardware architecture, including trap processing, segmentation, and side channel vulnerabilities • Understand Windows virtualization, and how virtualization-based security (VBS) protects against OS vulnerabilities • Delve into key management and configuration mechanisms, including the Registry, Windows services, WMI, and Task Scheduling • Explore diagnostic services such as Event Tracing for Windows (ETW) and DTrace • Learn how the cache manager and file system drivers interact to provide reliable support for files, directories, and disks, including on Persistent Memory (NVDIMM) DAX devices. • Understand NTFS, ReFS, and other Windows file systems • Review Windows startup/shutdown operations, and OS components involved in boot flow • Analyze UEFI-based Secure Boot, Measured Boot, and Secure Launch   About This Book • For experienced programmers, architects, software quality and performance specialists, administrators, security practitioners, and support professionals • Assumes you are a Windows power user Cover Title Page Copyright Page Contents at a Glance Contents About the Authors Foreword Introduction Chapter 8 System mechanisms Processor execution model Segmentation Task state segments Hardware side-channel vulnerabilities Out-of-order execution The CPU branch predictor The CPU cache(s) Side-channel attacks Side-channel mitigations in Windows KVA Shadow Hardware indirect branch controls (IBRS, IBPB, STIBP, SSBD) Retpoline and import optimization STIBP pairing Trap dispatching Interrupt dispatching Line-based versus message signaled–based interrupts Timer processing System worker threads Exception dispatching System service handling WoW64 (Windows-on-Windows) The WoW64 core File system redirection Registry redirection X86 simulation on AMD64 platforms ARM Memory models ARM32 simulation on ARM64 platforms X86 simulation on ARM64 platforms Object Manager Executive objects Object structure Synchronization High-IRQL synchronization Low-IRQL synchronization Advanced local procedure call Connection model Message model Asynchronous operation Views, regions, and sections Attributes Blobs, handles, and resources Handle passing Security Performance Power management ALPC direct event attribute Debugging and tracing Windows Notification Facility WNF features WNF users WNF state names and storage WNF event aggregation User-mode debugging Kernel support Native support Windows subsystem support Packaged applications UWP applications Centennial applications The Host Activity Manager The State Repository The Dependency Mini Repository Background tasks and the Broker Infrastructure Packaged applications setup and startup Package activation Package registration Conclusion Chapter 9 Virtualization technologies The Windows hypervisor Partitions, processes, and threads The hypervisor startup The hypervisor memory manager Hyper-V schedulers Hypercalls and the hypervisor TLFS Intercepts The synthetic interrupt controller (SynIC) The Windows hypervisor platform API and EXO partitions Nested virtualization The Windows hypervisor on ARM64 The virtualization stack Virtual machine manager service and worker processes The VID driver and the virtualization stack memory manager The birth of a Virtual Machine (VM) VMBus Virtual hardware support VA-backed virtual machines Virtualization-based security (VBS) Virtual trust levels (VTLs) and Virtual Secure Mode (VSM) Services provided by the VSM and requirements The Secure Kernel Virtual interrupts Secure intercepts VSM system calls Secure threads and scheduling The Hypervisor Enforced Code Integrity UEFI runtime virtualization VSM startup The Secure Kernel memory manager Hot patching Isolated User Mode Trustlets creation Secure devices VBS-based enclaves System Guard runtime attestation Conclusion Chapter 10 Management, diagnostics, and tracing The registry Viewing and changing the registry Registry usage Registry data types Registry logical structure Application hives Transactional Registry (TxR) Monitoring registry activity Process Monitor internals Registry internals Hive reorganization The registry namespace and operation Stable storage Registry filtering Registry virtualization Registry optimizations Windows services Service applications Service accounts The Service Control Manager (SCM) Service control programs Autostart services startup Delayed autostart services Triggered-start services Startup errors Accepting the boot and last known good Service failures Service shutdown Shared service processes Service tags User services Packaged services Protected services Task scheduling and UBPM The Task Scheduler Unified Background Process Manager (UBPM) Task Scheduler COM interfaces Windows Management Instrumentation WMI architecture WMI providers The Common Information Model and the Managed Object Format Language Class association WMI implementation WMI security Event Tracing for Windows (ETW) ETW initialization ETW sessions ETW providers Providing events ETW Logger thread Consuming events System loggers ETW security Dynamic tracing (DTrace) Internal architecture DTrace type library Windows Error Reporting (WER) User applications crashes Kernel-mode (system) crashes Process hang detection Global flags Kernel shims Shim engine initialization The shim database Driver shims Device shims Conclusion Chapter 11 Caching and file systems Terminology Key features of the cache manager Single, centralized system cache The memory manager Cache coherency Virtual block caching Stream-based caching Recoverable file system support NTFS MFT working set enhancements Memory partitions support Cache virtual memory management Cache size Cache virtual size Cache working set size Cache physical size Cache data structures Systemwide cache data structures Per-file cache data structures File system interfaces Copying to and from the cache Caching with the mapping and pinning interfaces Caching with the direct memory access interfaces Fast I/O Read-ahead and write-behind Intelligent read-ahead Read-ahead enhancements Write-back caching and lazy writing Disabling lazy writing for a file Forcing the cache to write through to disk Flushing mapped files Write throttling System threads Aggressive write behind and low-priority lazy writes Dynamic memory Cache manager disk I/O accounting File systems Windows file system formats CDFS UDF FAT12, FAT16, and FAT32 exFAT NTFS ReFS File system driver architecture Local FSDs Remote FSDs File system operations Explicit file I/O Memory manager’s modified and mapped page writer Cache manager’s lazy writer Cache manager’s read-ahead thread Memory manager’s page fault handler File system filter drivers and minifilters Filtering named pipes and mailslots Controlling reparse point behavior Process Monitor The NT File System (NTFS) High-end file system requirements Recoverability Security Data redundancy and fault tolerance Advanced features of NTFS Multiple data streams Unicode-based names General indexing facility Dynamic bad-cluster remapping Hard links Symbolic (soft) links and junctions Compression and sparse files Change logging Per-user volume quotas Link tracking Encryption POSIX-style delete semantics Defragmentation Dynamic partitioning NTFS support for tiered volumes NTFS file system driver NTFS on-disk structure Volumes Clusters Master file table File record numbers File records File names Tunneling Resident and nonresident attributes Data compression and sparse files Compressing sparse data Compressing nonsparse data Sparse files The change journal file Indexing Object IDs Quota tracking Consolidated security Reparse points Storage reserves and NTFS reservations Transaction support Isolation Transactional APIs On-disk implementation Logging implementation NTFS recovery support Design Metadata logging Log file service Log record types Recovery Analysis pass Redo pass Undo pass NTFS bad-cluster recovery Self-healing Online check-disk and fast repair Encrypted file system Encrypting a file for the first time The decryption process Backing up encrypted files Copying encrypted files BitLocker encryption offload Online encryption support Direct Access (DAX) disks DAX driver model DAX volumes Cached and noncached I/O in DAX volumes Mapping of executable images Block volumes File system filter drivers and DAX Flushing DAX mode I/Os Large and huge pages support Virtual PM disks and storages spaces support Resilient File System (ReFS) Minstore architecture B+ tree physical layout Allocators Page table Minstore I/O ReFS architecture ReFS on-disk structure Object IDs Security and change journal ReFS advanced features File’s block cloning (snapshot support) and sparse VDL ReFS write-through ReFS recovery support Leak detection Shingled magnetic recording (SMR) volumes ReFS support for tiered volumes and SMR Container compaction Compression and ghosting Storage Spaces Spaces internal architecture Services provided by Spaces Conclusion Chapter 12 Startup and shutdown Boot process The UEFI boot The BIOS boot process Secure Boot The Windows Boot Manager The Boot menu Launching a boot application Measured Boot Trusted execution The Windows OS Loader Booting from iSCSI The hypervisor loader VSM startup policy The Secure Launch Initializing the kernel and executive subsystems Kernel initialization phase 1 Smss, Csrss, and Wininit ReadyBoot Images that start automatically Shutdown Hibernation and Fast Startup Windows Recovery Environment (WinRE) Safe mode Driver loading in safe mode Safe-mode-aware user programs Boot status file Conclusion Contents of Windows Internals, Seventh Edition, Part 1 I