Probability Theory: The Logic of Science

Description

[Briggs](https://isidore.co/calibre/#panel=book_list&search=authors:%22%3DBriggs%2C%20William%20M.%22) cited this in [his 7 October 2022 lecture "Uncertainty" @16:03](https://youtu.be/-t5lHXAvuLQ?t=963). It's posthumously published; a small group of physicists followed Jaynes after his death.* * *The standard rules of probability can be interpreted as uniquely valid principles in logic. In this book, E. T. Jaynes dispels the imaginary distinction between'probability theory'and'statistical inference', leaving a logical unity and simplicity, which provides greater technical power and flexibility in applications. This book goes beyond the conventional mathematics of probability theory, viewing the subject in a wider context. New results are discussed, along with applications of probability theory to a wide variety of problems in physics, mathematics, economics, chemistry and biology. It contains many exercises and problems, and is suitable for use as a textbook on graduate level courses involving data analysis. The material is aimed at readers who are already familiar with applied mathematics at an advanced undergraduate level or higher. The book will be of interest to scientists working in any area where inference from incomplete information is necessary.* * *The theme of this book is summarized in a footnote on p. xix: "our topic is the optimal processing of incomplete information''. And again, "Our theme is simply: probability theory as extended logic'' [p. xxii]. Roughly, and perhaps slightly inaccurately put, this work is a development of probability and statistics from this point of view. While frequentists might easily dismiss this as another Bayesian text, statisticians of the Bayesian persuasion would have greater difficulty in seeing it as supporting more classical (Fisher, Neyman-Pearson) methods. Jaynes himself, however, has no hesitation, claiming that his method, based on the principle of maximum entropy, encompasses not only Bayesian and frequency calculations but also calculations in neither group. The book is in two parts: I. Principles and elementary applications, and II. Advanced applications. There are a number of challenging exercises, two indexes, a list of references and a bibliography. In Chapter 1, "Plausible reasoning'', Jaynes introduces the keystone of his theory: a robot that will "carry out useful plausible reasoning, following clearly defined principles expressing an idealized common sense'' [p. 8]. Three basic desiderata are given, which show not only the inner workings of the robot's brain, but also how the robot relates (or, more accurately, should relate) to the external world. In Chapter 2 the usual fundamental rules for probabilities are derived from the aforementioned desiderata. The method follows that of R. T. Cox [The algebra of probable inference, The Johns Hopkins Press, Baltimore, Md, 1961; [MR0130703](https://mathscinet.ams.org/mathscinet/search/publdoc.html?r=1&pg1=MR&s1=130703&loc=fromrevtext)]. "Elementary sampling theory'' is the subject of Chapter 3, while Chapter 4 is concerned with elementary hypothesis testing. The reader used to a more conventional approach may be surprised to find at the beginning of this chapter a discussion of likelihoods, prior and posterior probabilities. In Chapter 5, "Queer uses for probability theory'', matters as diverse as extra-sensory perception, the discovery of Neptune, horse racing and weather forecasting are considered. Chapter 6 is concerned with elementary parameter estimation. In $\S6.20$ Jaynes discusses the "taxicab problem'' and mentions his inability to find it in the orthodox literature. It is in fact discussed in [H. Jeffreys, Theory of probability, Third edition, Clarendon Press, Oxford, 1961; [MR0187257](https://mathscinet.ams.org/mathscinet/search/publdoc.html?r=1&pg1=MR&s1=187257&loc=fromrevtext)] (where it is concerned with tramcars) and in [C. P. Robert, The Bayesian choice, Translated and revised from the French original by the author, Springer, New York, 1994; [MR1313727](https://mathscinet.ams.org/mathscinet/search/publdoc.html?r=1&pg1=MR&s1=1313727&loc=fromrevtext)]. Notice, by the way, that while Jeffreys attributes the problem to Maxwell Herman Alexander Newman, a gremlin—dare we say Maxwell's demon?—causes Robert to attribute it to [Jerzy?] Neyman! Chapter 7 is concerned with the normal distribution, and Chapter 8 deals with sufficiency, ancillarity "and all that''. "Repetitive experiments: probability and frequency'' is the title of the ninth chapter, one in which Jaynes examines what predictions his robot can make with very limited information. In Chapter 10 Jaynes considers the fundamental difficulty associated with the notion of "random'' experiments, particularly in relation to physics. The entropy principle is introduced in Chapter 11, the first in the advanced applications section. Chapter 12 is concerned with ignorance priors and transformation groups. Bertrand's paradox is discussed here, the recognition of an element of rotational symmetry enabling Jaynes to arrive at a unique answer. Jaynes had considered this paradox in 1973 [Found. Phys. 3 (1973), 477–492; [MR0426227](https://mathscinet.ams.org/mathscinet/search/publdoc.html?r=1&pg1=MR&s1=426227&loc=fromrevtext)], and it is perhaps a pity that he does not respond here to later relevant work by others—e.g. K. S. Friedman [Found. Phys. 5 (1975), no. 1, 89–91] and A. Nathan [Philos. Sci. 51 (1984), no. 4, 677–684 [MR0781314](https://mathscinet.ams.org/mathscinet/search/publdoc.html?pg1=MR&s1=0781314&loc=fromrevtext)]. Jaynes devotes his thirteenth chapter, "Decision theory, historical background'', to showing that the procedures he has so far introduced are optimal in terms of some clearly defined criterion. In Chapter 14 the decision theory of the previous chapter is applied to the detection of signals in noise and the detection of systematic drift in machine characteristics. The word "paradox'' is ambiguous, and Jaynes advisedly gives a precise definition, perhaps different from that in common acceptance, of his usage of the term at the start of his fifteenth chapter, one devoted to some probabilistic paradoxes. The next chapter is devoted to the historical background of orthodox statistics. Jaynes being a physicist, we can perhaps see his view on scientific inference, similar to that of Daniel Bernoulli, Laplace and other mathematical physicists, as tainted, or at least tinctured, by professional interest. Jaynes briefly discusses the methods introduced by the Fisher and Neyman-Pearson schools, and mentions the attacks and defences of men like M. G. Kendall, Cramér, Feller, von Mises, "and even the putative Bayesian L. J. Savage'' [p. 493], etc.—the certainty expressed by these savants in their discussions Jaynes finds almost inexplicable "since they were all quite competent mathematically'' [p. 493]. O mysterious condescension! In Chapter 17, the "Principles and pathology of orthodox statistics'', Jaynes investigates the consequences of not using Bayesian methods in some very simple problems. Cox's theorems, as developed by Jaynes, facilitate the examination of how, and under what circumstances, orthodox and Bayesian results differ. In Chapter 18, "The $A_p$ distribution and rule of succession'', Jaynes investigates whether some machinery can be built into the robot that will enable it to store general conclusions. This is effected by the introduction of a new proposition $A_p$, interpreted as "regardless of anything else you may have been told, the probability of $A$ is $p$''. Jaynes returns to the matter of physical measurement in Chapter 19, paying especial attention to the problem of the great inequality of Jupiter and Saturn. Chapter 20 is concerned with model comparison, and "Outliers and robustness'' is the subject of Chapter 21. The final chapter presents an introduction to communication theory. That both the references and bibliography are annotated is most useful, Jaynes showing in some of his remarks that he is indeed fortiter in both re and modo. The anecdote about "Given $2+2=5$: prove that I am the Pope'' attributed in Appendix B to G. H. Hardy and J. E. McTaggart is ascribed in [G. Birkhoff, Lattice theory, Corrected reprint of the 1967 third edition, Amer. Math. Soc., Providence, R.I., 1979; [MR0598630](https://mathscinet.ams.org/mathscinet/search/publdoc.html?r=1&pg1=MR&s1=598630&loc=fromrevtext)] to Bertrand Russell. Further, note that Borel (1924) is in fact translated in the first edition (1964) of Kyburg & Smokler, and is not in the second. Jaynes notes that part of the problem between Jeffreys (a geophysicist) and Fisher (a geneticist) rose from their consideration of very different problems. One may well ask oneself whether there is one probabilistic/statistical methodology that is appropriate for all scientific inference, as this book in fact suggests. This is a work written by a scientist for scientists. As such it is to be welcomed. The reader will certainly find things with which he disagrees: but he will also find much that will cause him to think deeply not only on his usual practice but also on statistics and probability in general. Probability theory: the logic of science is, for both statisticians and scientists, more than just "recommended reading'': it should be prescribed.[Reviewed by A. I. Dale](https://mathscinet.ams.org/mathscinet-getitem?mr=1992316)