Lecture 2: Entropy and Data Compression (I): Introduction to Compression, Inf.Theory and Entropy

Lecture 2 of the Course on Information Theory, Pattern Recognition, and Neural Networks. Produced by: David MacKay (University of Cambridge) Author: David MacKay, University of Cambridge A series of sixteen lectures covering the core of the book "Information Theory, Inference, and Learning Algorithms" (Cambridge University Press, 2003, http://www.inference.eng.cam.ac.uk/mackay/itila/) which can be bought at Amazon (http://www.amazon.co.uk/exec/obidos/ASIN/0521642981/davidmackay0f-21), and is available free online (http://www.inference.eng.cam.ac.uk/mackay/itila/). A subset of these lectures used to constitute a Part III Physics course at the University of Cambridge. The high-resolution videos and all other course material can be downloaded from the Cambridge course website (http://www.inference.eng.cam.ac.uk/mackay/itprnn/). Snapshots of the lecture can be found here: http://www.inference.eng.cam.ac.uk/itprnn_lectures/ These lectures are also available at http://videolectures.net/course_information_theory_pattern_recognition/ (synchronized with snapshots and slides)