An Introduction to Kolmogorov Complexity and Its Applications

Description

This must-read textbook presents an essential introduction to Kolmogorov complexity (KC), a central theory and powerful tool in information science that deals with the quantity of information in individual objects. The text covers both the fundamental concepts and the most important practical applications, supported by a wealth of didactic features. This thoroughly revised and enhanced fourth edition includes new and updated material on, amongst other topics, the Miller-Yu theorem, the Gács-Kucera theorem, the Day-Gács theorem, increasing randomness, short lists computable from an input string containing the incomputable Kolmogorov complexity of the input, the Lovász local lemma, sorting, the algorithmic full Slepian-Wolf theorem for individual strings, multiset normalized information distance and normalized web distance, and conditional universal distribution. Topics and features: describes the mathematical theory of KC, including the theories of algorithmic complexity and algorithmic probability; presents a general theory of inductive reasoning and its applications, and reviews the utility of the incompressibility method; covers the practical application of KC in great detail, including the normalized information distance (the similarity metric) and information diameter of multisets in phylogeny, language trees, music, heterogeneous files, and clustering; discusses the many applications of resource-bounded KC, and examines different physical theories from a KC point of view; includes numerous examples that elaborate the theory, and a range of exercises of varying difficulty (with solutions); offers explanatory asides on technical issues, and extensive historical sections; suggests structures for several one-semester courses in the preface. As the definitive textbook on Kolmogorov complexity, this comprehensive and self-contained work is an invaluable resource for advanced undergraduate students, graduate students, and researchers in all fields of science. Preface to the First Edition Preface to the Second Edition Preface to the Third Edition Preface to the Fourth Edition How to Use This Book Outlines of One-Semester Courses Contents List of Figures List of Tables 1 Preliminaries 1.1 A Brief Introduction 1.2 Prerequisites and Notation 1.3 Numbers and Combinatorics 1.4 Binary Strings 1.5 Asymptotic Notation 1.6 Basics of Probability Theory 1.7 Basics of Computability Theory 1.8 The Roots of Kolmogorov Complexity 1.9 Randomness 1.10 Prediction and Probability 1.11 Information Theory and Coding 1.12 State x Symbol Complexity 1.13 History and References 2 Algorithmic Complexity 2.1 The Invariance Theorem 2.2 Incompressibility 2.3 C as an Integer Function 2.4 Random Strings 2.5 *Random Sequences 2.6 Statistical Properties of Strings 2.7 Algorithmic Properties of C 2.8 Algorithmic Information Theory 2.9 History and References 3 Algorithmic Prefix Complexity 3.1 The Invariance Theorem 3.2 Incompressibility 3.3 K as an Integer Function 3.4 Random Strings 3.5 *Random Sequences 3.6 Algorithmic Properties of K 3.7 *Complexity of Complexity 3.8 *Symmetry of Algorithmic Information 3.9 *Sizes of the Constants 3.10 History and References 4 Algorithmic Probability 4.1 Semicomputable Functions 4.2 Measure Theory 4.3 Discrete Sample Space 4.4 Universal Average-Case Complexity 4.5 Continuous Sample Space 4.6 Universal Average-Case Complexity, Continued 4.7 History and References 5 Inductive Reasoning 5.1 Introduction 5.2 Solomonoff’s Theory of Prediction 5.3 Simple Pac-Learning 5.4 Hypothesis Identification by MDL 5.5 Nonprobabilistic Statistics 5.6 History and References 6 The Incompressibility Method 6.1 Three Examples 6.2 High-Probability Properties 6.3 Combinatorics 6.4 Kolmogorov Random Graphs 6.5 Compact Routing 6.6 Average-Case Analysis of Sorting 6.7 Longest Common Subsequence 6.8 Formal Language Theory 6.9 Online CFL Recognition 6.10 Turing Machine Time Complexity 6.11 Communication Complexity 6.12 Circuit Complexity 6.13 Lov´asz Local Lemma 6.14 History and References 7 Resource-Bounded Complexity 7.1 Mathematical Theory 7.2 Language Compression 7.3 Computational Complexity 7.4 Instance Complexity 7.5 Kt and Universal Search 7.6 Time-Limited Universal Distributions 7.7 Logical Depth 7.8 History and References 8 Physics, Information, and Computation 8.1 Information Theory 8.2 Reversible Computation 8.3 Information Distance 8.4 Normalization 8.5 Information Diameter 8.6 Thermodynamics 8.7 Entropy Revisited 8.8 Quantum Kolmogorov Complexity 8.9 Compression in Nature 8.10 History and References References Index