The Equation of Knowledge: From Bayes' Rule to a Unified Philosophy of Science

Description

The Equation of Knowledge: From Bayes' Rule to a Unified Philosophy of Science introduces readers to the Bayesian approach to science: teasing out the link between probability and knowledge. The author strives to make this book accessible to a very broad audience, suitable for professionals, students, and academics, as well as the enthusiastic amateur scientist/mathematician. This book also shows how Bayesianism sheds new light on nearly all areas of knowledge, from philosophy to mathematics, science and engineering, but also law, politics and everyday decision-making. Bayesian thinking is an important topic for research, which has seen dramatic progress in the recent years, and has a significant role to play in the understanding and development of AI and Machine Learning, among many other things. This book seeks to act as a tool for proselytising the benefits and limits of Bayesianism to a wider public. Features Presents the Bayesian approach as a unifying scientific method for a wide range of topics Suitable for a broad audience, including professionals, students, and academics Provides a more accessible, philosophical introduction to the subject that is offered elsewhere Cover Half Title Title Page Copyright Page Contents Foreword Acknowledgment Preface Section I: Pure Bayesianism Chapter 1: On A Transformative Journey 1.1 STUMPED BY A STUDENT 1.2 MY PATH TOWARDS BAYESIANISM 1.3 A UNIFIED PHILOSOPHY OF KNOWLEDGE 1.4 AN ALTERNATIVE TO THE SCIENTIFIC METHOD 1.5 THE OBJECTIVITY MYTH 1.6 THE GOALS OF THE BOOK Chapter 2: Bayes’ Theorem 2.1 THE TROLL STUDENT PUZZLE 2.2 THE MONTY HALL PROBLEM 2.3 THE TRIAL OF SALLY CLARK 2.4 THE LEGAL CONVICTION OF BAYESIANISM 2.5 BAYES’ THEOREM 2.6 THE COMPONENTS OF BAYES’ RULE 2.7 BAYES TO THE RESCUE OF DIAGNOSIS 2.8 BAYES TO THE RESCUE OF SALLY CLARK 2.9 BAYES TO THE RESCUE OF THE TROLL STUDENT PROBLEM 2.10 A FEW WORDS OF ENCOURAGEMENT Chapter 3: Logically Speaking... 3.1 TWO THINKING PROCESSES 3.2 THE RULES OF LOGIC 3.3 ARE ALL QUEENS BLUE? 3.4 QUANTIFIERS AND PREDICATES 3.5 ARISTOTLE’S SYLLOGISM REINTERPRETED 3.6 AXIOMATIZATION 3.7 PLATONISTS VERSUS INTUITIONISTS 3.8 BAYESIAN LOGIC* 3.9 BEYOND TRUE OR FALSE 3.10 THE COHABITATION OF INCOMPATIBLE THEORIES Chapter 4: Let’s Generalize! 4.1 THE SCOTTISH BLACK SHEEP 4.2 A BRIEF HISTORY OF EPISTEMOLOGY 4.3 A BRIEF HISTORY OF PLANETOLOGY 4.4 SCIENCE AGAINST POPPER? 4.5 FREQUENTISM* 4.6 STATISTICIANS AGAINST THE p-VALUE 4.7 p-HACKING 4.8 WHAT A STATISTICS TEXTBOOK SAYS 4.9 THE EQUATION OF KNOWLEDGE 4.10 CUMULATIVE LEARNING 4.11 BACK TO EINSTEIN Chapter 5: All Hail Prejudices 5.1 THE LINDA PROBLEM 5.2 PREJUDICES TO THE RESCUE OF LINDA* 5.3 LONG LIVE PREJUDICES 5.4 xkcd’s SUN 5.5 PREJUDICES TO THE RESCUE OF xkcd 5.6 PREJUDICES TO THE RESCUE OF SALLY CLARK 5.7 PREJUDICES AGAINST PSEUDO-SCIENCES 5.8 PREJUDICES TO THE RESCUE OF SCIENCE 5.9 THE BAYESIAN HAS AN OPINION ON EVERYTHING 5.10 ERRONEOUS PREJUDICES 5.11 PREJUDICES AND MORAL QUESTIONS Chapter 6: The Bayesian Prophets 6.1 A THRILLING HISTORY 6.2 THE ORIGINS OF PROBABILITY 6.3 THE MYSTERIOUS THOMAS BAYES 6.4 LAPLACE, THE FATHER OF BAYESIANISM 6.5 LAPLACE’S SUCCESSION RULE 6.6 THE GREAT BAYESIAN WINTER 6.7 BAYES TO THE RESCUE OF ALLIES 6.8 BAYESIAN ISLANDS IN A FREQUENTIST OCEAN 6.9 BAYES TO THE RESCUE OF PRACTITIONERS 6.10 BAYES’ TRIUMPH, AT LAST! 6.11 BAYES IS UBIQUITOUS Chapter 7: Solomonoff’s Demon 7.1 NEITHER HUMAN NOR MACHINE 7.2 THE THEORY OF COMPUTATION 7.3 WHAT’S A PATTERN? 7.4 THE SOLOMONOFF COMPLEXITY* 7.5 THE MARRIAGE OF ALGORITHMICS AND PROBABILITIES 7.6 THE SOLOMONOFF PRIOR* 7.7 BAYES TO THE RESCUE OF SOLOMONOFF’S DEMON* 7.8 SOLOMONOFF’S COMPLETENESS 7.9 SOLOMONOFF’S INCOMPUTABILITY 7.10 SOLOMONOFF’S INCOMPLETENESS 7.11 LET’S BE PRAGMATIC Section II: Applied Bayesianism Chapter 8: Can You Keep A Secret? 8.1 CLASSIFIED 8.2 TODAY’S CRYPTOGRAPHY 8.3 BAYES BREAKS CODES 8.4 RANDOMIZED SURVEY 8.5 THE PRIVACY OF THE RANDOMIZED SURVEY 8.6 THE DEFINITION OF DIFFERENTIAL PRIVACY* 8.7 THE LAPLACIAN MECHANISM 8.8 ROBUSTNESS TO COMPOSITION 8.9 THE ADDITION OF PRIVACY LOSSES 8.10 IN PRACTICE, IT’S NOT GOING WELL! 8.11 HOMOMORPHIC ENCRYPTION Chapter 9: Game, Set and Math 9.1 THE MAGOUILLEUSE 9.2 SPLIT OR STEAL? 9.3 BAYESIAN PERSUASION 9.4 SCHELLING’S POINTS 9.5 MIXED EQUILIBRIUM 9.6 BAYESIAN GAMES 9.7 BAYESIAN MECHANISM DESIGN* 9.8 MYERSON’S AUCTION 9.9 THE SOCIAL CONSEQUENCES OF BAYESIANISM Chapter 10: Will Darwin Select Bayes? 10.1 THE SURVIVOR BIAS 10.2 CALIFORNIA’S COLORED LIZARDS 10.3 THE LOTKA-VOLTERRA DYNAMIC* 10.4 GENETIC ALGORITHMS 10.5 MAKE UP YOUR OWN MIND? 10.6 AARONSON’S BAYESIAN DEBATING 10.7 SHOULD YOU TRUST A SCIENTIST? 10.8 THE ARGUMENT OF AUTHORITY 10.9 THE SCIENTIFIC CONSENSUS 10.10 CLICKBAIT 10.11 THE PREDICTIVE POWER OF MARKETS 10.12 FINANCIAL BUBBLES Chapter 11: Exponentially Counterintuitive 11.1 SUPER LARGE NUMBERS 11.2 THE GLASS CEILING OF COMPUTATION 11.3 EXPONENTIAL EXPLOSION 11.4 THE MAGIC OF ARABIC NUMERALS 11.5 BENFORD’S LAW 11.6 LOGARITHMIC SCALES 11.7 LOGARITHMS 11.8 BAYES WINS A GO¨ DEL PRIZE 11.9 BAYES ON HOLIDAY 11.10 THE SINGULARITY Chapter 12: Ockham Cuts to the Chase 12.1 LAST THURSDAY 12.2 IN FOOTBALL, YOU NEVER KNOW 12.3 THE CURSE OF OVERFITTING 12.4 THE COMPLEX QUEST OF SIMPLICITY 12.5 NOT ALL IS SIMPLE 12.6 CROSS VALIDATION 12.7 TIBSCHIRANI’S REGULARIZATION 12.8 ROBUST OPTIMIZATION 12.9 BAYES TO THE RESCUE OF OVERFITTING* 12.10 ONLY BAYESIAN INFERENCES ARE ADMISSIBLE* 12.11 OCKHAM’S RAZOR AS A BAYESIAN THEOREM! Chapter 13: Facts Are Misleading 13.1 HOSPITAL OR CLINIC 13.2 CORRELATION IS NOT CAUSALITY 13.3 LET’S SEARCH FOR CONFOUNDING VARIABLES! 13.4 REGRESSION TO THE MEAN 13.5 STEIN’S PARADOX 13.6 THE FAILURE OF ENDOGENOUS STRATIFICATION 13.7 RANDOMIZE! 13.8 CAVEATS ABOUT RANDOMIZED CONTROLLED TRIALS 13.9 THE RETURN OF THE SCOTTISH BLACK SHEEP 13.10 WHAT’S A CAT? 13.11 POETIC NATURALISM Section III: Pragmatic Bayesianism Chapter 14: Quick And Not Too Dirty 14.1 THE MYSTERY OF PRIMES 14.2 THE PRIME NUMBER THEOREM 14.3 APPROXIMATING T 14.4 LINEARIZATION 14.5 THE CONSTRAINTS OF PRAGMATISM 14.6 TURING’S LEARNING MACHINES 14.7 PRAGMATIC BAYESIANISM 14.8 SUBLINEAR ALGORITHMS 14.9 DIFFERENT THINKING MODES 14.10 BECOME POST-RIGOROUS! 14.11 BAYESIAN APPROXIMATIONS Chapter 15: Wish Me Luck 15.1 FIVETHIRTYEIGHT AND THE 2016 US ELECTION 15.2 IS QUANTUM MECHANICS PROBABILISTIC? 15.3 CHAOS THEORY 15.4 UNPREDICTABLE DETERMINISTIC AUTOMATA 15.5 THERMODYNAMICS 15.6 SHANNON’S ENTROPY 15.7 SHANNON’S OPTIMAL COMPRESSION 15.8 SHANNON’S REDUNDANCY 15.9 THE KULLBACK-LEIBLER DIVERGENCE 15.10 PROPER SCORING RULES 15.11 WASSERSTEIN’S METRIC 15.12 GENERATIVE ADVERSARIAL NETWORKS (GANS) Chapter 16: Down Memory Lane 16.1 THE VALUE OF DATA 16.2 THE DELUGE OF DATA 16.3 THE TOILET PROBLEM 16.4 EFFICIENT BIG DATA PROCESSING 16.5 THE KALMAN FILTER 16.6 OUR BRAINS FACED WITH BIG DATA 16.7 REMOVING TRAUMATIC SOUVENIRS 16.8 FALSE MEMORY 16.9 BAYES TO THE RESCUE OF MEMORY 16.10 SHORTER AND LONGER-TERM MEMORIES 16.11 RECURRENT NEURAL NETWORKS 16.12 ATTENTION MECHANISMS 16.13 WHAT SHOULD BE TAUGHT AND LEARNED? Chapter 17: Let’s Sleep on It 17.1 WHERE DO IDEAS COME FROM? 17.2 CREATIVE ART BY ARTIFICIAL INTELLIGENCES 17.3 LATENT DIRICHLET ALLOCATION (LDA) 17.4 THE CHINESE RESTAURANT 17.5 MONTE CARLO SIMULATIONS 17.6 STOCHASTIC GRADIENT DESCENT (SGD) 17.7 PSEUDO-RANDOM NUMBERS 17.8 IMPORTANCE SAMPLING 17.9 IMPORTANCE SAMPLING FOR LDA 17.10 THE ISING MODEL* 17.11 THE BOLTZMANN MACHINE 17.12 MCMC AND GOOGLE PAGERANK 17.13 METROPOLIS-HASTING SAMPLING 17.14 GIBBS SAMPLING 17.15 MCMC AND COGNITIVE BIASES 17.16 CONSTRASTIVE DIVERGENCE Chapter 18: The Unreasonable Effectiveness of Abstraction 18.1 DEEP LEARNING WORKS! 18.2 FEATURE LEARNING 18.3 WORD VECTOR REPRESENTATION 18.4 EXPONENTIAL EXPRESSIVITY* 18.5 THE EMERGENCE OF COMPLEXITY 18.6 THE KOLMOGOROV SOPHISTICATION* 18.7 SOPHISTICATION IS A SOLOMONOFF MAP!* 18.8 THE BENNETT LOGICAL DEPTH 18.9 THE DEPTH OF MATHEMATICS 18.10 THE CONCISION OF MATHEMATICS 18.11 THE MODULARITY OF MATHEMATICS Chapter 19: The Bayesian Brain 19.1 THE BRAIN IS FORMIDABLE 19.2 MOUNTAIN OR VALLEY? 19.3 OPTICAL ILLUSIONS 19.4 THE PERCEPTION OF MOTION 19.5 BAYESIAN SAMPLING 19.6 THE SCANDAL OF INDUCTION 19.7 LEARNING TO LEARN 19.8 THE BLESSING OF ABSTRACTION 19.9 THE BABY IS A GENIUS 19.10 LEARNING TO TALK 19.11 LEARNING TO COUNT 19.12 THE THEORY OF MIND 19.13 NATURE VERSUS NURTURE Section IV: Beyond Bayesianism Chapter 20: It’s All Fiction 20.1 PLATO’S CAVE 20.2 ANTIREALISM 20.3 DOES LIFE EXIST? 20.4 DOES MONEY EXIST? 20.5 IS TELEOLOGY A SCIENTIFIC DEAD END? 20.6 THE CHURCH-TURING THESIS VERSUS REALITY 20.7 IS (INSTRUMENTAL) ANTIREALISM USEFUL? 20.8 IS THERE A WORLD OUTSIDE OUR BRAIN? 20.9 A CAT IN A BINARY CODE? 20.10 SOLOMONOFF DEMON’S ANTIREALISM Chapter 21: Exploring The Origins of Beliefs 21.1 THE SCANDAL OF DIVERGENT SERIES 21.2 BUT THIS IS FALSE, RIGHT? 21.3 CADET OFFICER 21.4 MY ASIAN JOURNEY 21.5 ARE WE ALL POTENTIAL MONSTERS? 21.6 STORIES OVER STATISTICS 21.7 SUPERSTITIONS 21.8 THE DARWINIAN EVOLUTION OF IDEOLOGIES 21.9 BELIEVING SUPERSTITIONS CAN BE USEFUL 21.10 THE MAGIC OF YOUTUBE 21.11 THE JOURNEY GOES ON Chapter 22: Beyond Bayesianism 22.1 THE BAYESIAN HAS NO MORAL 22.2 THE NATURAL MORAL 22.3 UNAWARE OF OUR MORALS 22.4 CARROT AND STICK 22.5 THE MORAL OF THE MAJORITY 22.6 DEONTOLOGICAL MORAL 22.7 SHOULD KNOWLEDGE BE A GOAL? 22.8 UTILITARIANISM 22.9 BAYESIAN CONSEQUENTIALISM 22.10 LAST WORDS Index