Linux System Programming: Talking Directly to the Kernel and C Library

Description

Write software that draws directly on services offered by the Linux kernel and core system libraries. With this comprehensive book, Linux kernel contributor Robert Love provides you with a tutorial on Linux system programming, a reference manual on Linux system calls, and an insider’s guide to writing smarter, faster code. Love clearly distinguishes between POSIX standard functions and special services offered only by Linux. With a new chapter on multithreading, this updated and expanded edition provides an in-depth look at Linux from both a theoretical and applied perspective over a wide range of programming topics, including: A Linux kernel, C library, and C compiler overview Basic I/O operations, such as reading from and writing to files Advanced I/O interfaces, memory mappings, and optimization techniques The family of system calls for basic process management Advanced process management, including real-time processes Thread concepts, multithreaded programming, and Pthreads File and directory management Interfaces for allocating memory and optimizing memory access Basic and advanced signal interfaces, and their role on the system Clock management, including POSIX clocks and high-resolution timers Copyright Table of Contents Foreword Preface Audience and Assumptions Contents of This Book Versions Covered in This Book Conventions Used in This Book Using Code Examples Safari® Books Online How to Contact Us Acknowledgments Chapter 1. Introduction and Essential Concepts System Programming Why Learn System Programming Cornerstones of System Programming System Calls The C Library The C Compiler APIs and ABIs APIs ABIs Standards POSIX and SUS History C Language Standards Linux and the Standards This Book and the Standards Concepts of Linux Programming Files and the Filesystem Processes Users and Groups Permissions Signals Interprocess Communication Headers Error Handling Getting Started with System Programming Chapter 2. File I/O Opening Files The open() System Call Owners of New Files Permissions of New Files The creat() Function Return Values and Error Codes Reading via read() Return Values Reading All the Bytes Nonblocking Reads Other Error Values Size Limits on read() Writing with write() Partial Writes Append Mode Nonblocking Writes Other Error Codes Size Limits on write() Behavior of write() Synchronized I/O fsync() and fdatasync() sync() The O_SYNC Flag O_DSYNC and O_RSYNC Direct I/O Closing Files Error Values Seeking with lseek() Seeking Past the End of a File Error Values Limitations Positional Reads and Writes Error Values Truncating Files Multiplexed I/O select() poll() poll() Versus select() Kernel Internals The Virtual Filesystem The Page Cache Page Writeback Conclusion Chapter 3. Buffered I/O User-Buffered I/O Block Size Standard I/O File Pointers Opening Files Modes Opening a Stream via File Descriptor Closing Streams Closing All Streams Reading from a Stream Reading a Character at a Time Reading an Entire Line Reading Binary Data Writing to a Stream Writing a Single Character Writing a String of Characters Writing Binary Data Sample Program Using Buffered I/O Seeking a Stream Obtaining the Current Stream Position Flushing a Stream Errors and End-of-File Obtaining the Associated File Descriptor Controlling the Buffering Thread Safety Manual File Locking Unlocked Stream Operations Critiques of Standard I/O Conclusion Chapter 4. Advanced File I/O Scatter/Gather I/O readv() and writev() Event Poll Creating a New Epoll Instance Controlling Epoll Waiting for Events with Epoll Edge- Versus Level-Triggered Events Mapping Files into Memory mmap() munmap() Mapping Example Advantages of mmap() Disadvantages of mmap() Resizing a Mapping Changing the Protection of a Mapping Synchronizing a File with a Mapping Giving Advice on a Mapping Advice for Normal File I/O The posix_fadvise() System Call The readahead() System Call Advice Is Cheap Synchronized, Synchronous, and Asynchronous Operations Asynchronous I/O I/O Schedulers and I/O Performance Disk Addressing The Life of an I/O Scheduler Helping Out Reads Selecting and Configuring Your I/O Scheduler Optimzing I/O Performance Conclusion Chapter 5. Process Management Programs, Processes, and Threads The Process ID Process ID Allocation The Process Hierarchy pid_t Obtaining the Process ID and Parent Process ID Running a New Process The Exec Family of Calls The fork() System Call Terminating a Process Other Ways to Terminate atexit() on_exit() SIGCHLD Waiting for Terminated Child Processes Waiting for a Specific Process Even More Waiting Versatility BSD Wants to Play: wait3() and wait4() Launching and Waiting for a New Process Zombies Users and Groups Real, Effective, and Saved User and Group IDs Changing the Real or Saved User or Group ID Changing the Effective User or Group ID Changing the User and Group IDs, BSD Style Changing the User and Group IDs, HP-UX Style Preferred User/Group ID Manipulations Support for Saved User IDs Obtaining the User and Group IDs Sessions and Process Groups Session System Calls Process Group System Calls Obsolete Process Group Functions Daemons Conclusion Chapter 6. Advanced Process Management Process Scheduling Timeslices I/O- Versus Processor-Bound Processes Preemptive Scheduling The Completely Fair Scheduler Yielding the Processor Legitimate Uses Process Priorities nice() getpriority() and setpriority() I/O Priorities Processor Affinity sched_getaffinity() and sched_setaffinity() Real-Time Systems Hard Versus Soft Real-Time Systems Latency, Jitter, and Deadlines Linux’s Real-Time Support Linux Scheduling Policies and Priorities Setting Scheduling Parameters sched_rr_get_interval() Precautions with Real-Time Processes Determinism Resource Limits The Limits Setting and Retrieving Limits Chapter 7. Threading Binaries, Processes, and Threads Multithreading Costs of Multithreading Alternatives to Multithreading Threading Models User-Level Threading Hybrid Threading Coroutines and Fibers Threading Patterns Thread-per-Connection Event-Driven Threading Concurrency, Parallelism, and Races Race Conditions Synchronization Mutexes Deadlocks Pthreads Linux Threading Implementations The Pthread API Linking Pthreads Creating Threads Thread IDs Terminating Threads Joining and Detaching Threads A Threading Example Pthread Mutexes Further Study Chapter 8. File and Directory Management Files and Their Metadata The Stat Family Permissions Ownership Extended Attributes Extended Attribute Operations Directories The Current Working Directory Creating Directories Removing Directories Reading a Directory’s Contents Links Hard Links Symbolic Links Unlinking Copying and Moving Files Copying Moving Device Nodes Special Device Nodes The Random Number Generator Out-of-Band Communication Monitoring File Events Initializing inotify Watches inotify Events Advanced Watch Options Removing an inotify Watch Obtaining the Size of the Event Queue Destroying an inotify Instance Chapter 9. Memory Management The Process Address Space Pages and Paging Memory Regions Allocating Dynamic Memory Allocating Arrays Resizing Allocations Freeing Dynamic Memory Alignment Managing the Data Segment Anonymous Memory Mappings Creating Anonymous Memory Mappings Mapping /dev/zero Advanced Memory Allocation Fine-Tuning with malloc_usable_size() and malloc_trim() Debugging Memory Allocations Obtaining Statistics Stack-Based Allocations Duplicating Strings on the Stack Variable-Length Arrays Choosing a Memory Allocation Mechanism Manipulating Memory Setting Bytes Comparing Bytes Moving Bytes Searching Bytes Frobnicating Bytes Locking Memory Locking Part of an Address Space Locking All of an Address Space Unlocking Memory Locking Limits Is a Page in Physical Memory? Opportunistic Allocation Overcommitting and OOM Chapter 10. Signals Signal Concepts Signal Identifiers Signals Supported by Linux Basic Signal Management Waiting for a Signal, Any Signal Examples Execution and Inheritance Mapping Signal Numbers to Strings Sending a Signal Permissions Examples Sending a Signal to Yourself Sending a Signal to an Entire Process Group Reentrancy Guaranteed-Reentrant Functions Signal Sets More Signal Set Functions Blocking Signals Retrieving Pending Signals Waiting for a Set of Signals Advanced Signal Management The siginfo_t Structure The Wonderful World of si_code Sending a Signal with a Payload Signal Payload Example A Flaw in Unix? Chapter 11. Time Time’s Data Structures The Original Representation And Now, Microsecond Precision Even Better: Nanosecond Precision Breaking Down Time A Type for Process Time POSIX Clocks Time Source Resolution Getting the Current Time of Day A Better Interface An Advanced Interface Getting the Process Time Setting the Current Time of Day Setting Time with Precision An Advanced Interface for Setting the Time Playing with Time Tuning the System Clock Sleeping and Waiting Sleeping with Microsecond Precision Sleeping with Nanosecond Resolution An Advanced Approach to Sleep A Portable Way to Sleep Overruns Alternatives to Sleeping Timers Simple Alarms Interval Timers Advanced Timers Appendix A. GCC Extensions to the C Language GNU C Inline Functions Suppressing Inlining Pure Functions Constant Functions Functions That Do Not Return Functions That Allocate Memory Forcing Callers to Check the Return Value Marking Functions as Deprecated Marking Functions as Used Marking Functions or Parameters as Unused Packing a Structure Increasing the Alignment of a Variable Placing Global Variables in a Register Branch Annotation Getting the Type of an Expression Getting the Alignment of a Type The Offset of a Member Within a Structure Obtaining the Return Address of a Function Case Ranges Void and Function Pointer Arithmetic More Portable and More Beautiful in One Fell Swoop Appendix B. Bibliography Books on the C Programming Language Books on Linux Programming Books on the Linux Kernel Books on Operating System Design Index About the Author