Information Theory: A Tutorial Introduction

Description

Originally developed by Claude Shannon in the 1940s, information theory laid the foundations for the digital revolution, and is now an essential tool in telecommunications, genetics, linguistics, brain sciences, and deep space communication. In this richly illustrated book, accessible examples are used to introduce information theory in terms of everyday games like ‘20 questions’ before more advanced topics are explored. Online MatLab and Python computer programs provide hands-on experience of information theory in action, and PowerPoint slides give support for teaching. Written in an informal style, with a comprehensive glossary and tutorial appendices, this text is an ideal primer for novices who wish to learn the essential principles and applications of information theory Content: What is information? -- Entropy of discrete variables -- Source coding theorem -- Noisy channel coding theorem -- Entropy of continuous variables -- Mutual information: continuous -- Channel capacity: continuous -- Thermodynamic entropy and information -- Information as Nature's currency -- Further reading -- A. Glossary -- B. Mathematical symbols -- C. Logarithms -- D. Probability density functions -- E. Averages from distributions -- F. The rules of probability -- G. The Gaussian distribution -- H. Key equations.