信息论基础

章节摘录

　　Philosophy of Science (Occam's Razor). William of Occam said "Causes shall not be multiplied beyond necessity," or to paraphrase it, "The simplest explanation is best". Solomonoff, and later Chaitin, argue persuasively that one gets a universally good prediction procedure if one takes a weighted combination of all programs that explain the data and observes what they print next. Moreover, this inference will work in many problems not handled by statistics. For example, this procedure will eventually predict the subsequent digits of π. When this procedure is applied to coin flips that come up heads with probability 0.7, this too will be inferred. When applied to the stock market, the procedure should essentially find all the "laws" of the stock market and extrapolate them optimally. In principle, such a procedure would have found Newton's laws of physics. Of course, such inference is highly impractical, because weeding out all computer programs that fail to generate existing data will take impossibly long. We would predict what happens tomorrow a hundred years from now.　　……

前言

　　信息理论是当代国内外大学电子工程系、计算机系和统计系等为研究生和高年级本科生开设的一门基础专业课程。自20世纪80年代以来，在国内外广为流传的教材有30多种。Thomas M.Cover-所著的“Elements of Information Theory”一书在时间上虽不是最新的，但在美国是获得最广泛应用的一本教科书，如麻省理工学院（MIT）、斯坦福大学（Stanford University）、加州大学伯克利分校（University of California at Berkeley）等美国一流大学均采用本书作为该课程的教材或主要参考书。因此本书很值得向国内推荐。　　本书的主要优点是：　　（1）概念清晰。信息论涉及很多数学问题，其概念很多淹没在数学推导中，而本书将讲清概念放在第一位，且能深入浅出，使读者很快得其要领。　　（2）数学工具的使用和数学推导过程的介绍恰到好处，既没有过于简化又没有拘泥于数学细节。　　（3）理论与应用并重，既保证理论的完整性和系统性，又突出理论研究面向应用的性质，使读者能带着问题学，具有启发性。　　（4）虽然此书出版较早（此后在美国又有三本教材问世），但从内容的覆盖面来讲此书仍有优势，且迄今仍有一定的先进性。　　（5）与美国的其他教材类似，本书也不可避免地带有作者爱好的印记，书中的某些内容是作者偏爱而放入的，在一般的信息论课程中大都不介绍这些内容。作为参考书，扩大学生视野，这也是很有益的。选用本书的教员可以选择其中的若干章讲授。

内容概要

Thomas M．Cover斯坦福大学电气工程系、统计系教授。曾任IEEE信息论学会主席，现任数理统计研究所研究员、IEEE高级会员。1 972年以论文“Broadcast Channels”荣获信息论优秀论文奖，1990年被选为“Shannon Lecturer”，这是信息论领域的最高荣誉。最近20年，他致力于研究信息论和统计学之间的关系。

书籍目录

list of figures 1 introduction and preview   1.1 preview of the book 2 entropy, relative entropy and mutual information   2.1 entropy   2.2 joint entropy and conditional entropy   2.3 relative entropy and mutual information   2.4 relationship between entropy and mutual information   2.5 chain rules for entropy, relative entropy and mutual information   2.6 jensen's inequality and its consequences   2.7 the log sum inequality and its applications   2.8 data processing inequality   2.9 the second law of thermodynamics   2.10 sufficient statistics   2.11 fano's inequality   summary of chapter 2   problems for chapter 2   historical notes 3 the asymptotic equipartition property   3.1 the aep   3.2 consequences of the aep: data compression   3.3 high probability sets and the typical set   summary of chapter 3  problems for chapter 3   historical notes 4 entropy rates of a stochastic process  4.1 markov chains   4.2 entropy rate   4.3 example: entropy rate of a random walk on a weighted graph   4.4 hidden markov models  summary of chapter 4   problems for chapter 4   historical notes 5 data compression6 gambling and data compression 7 kolmogorov complexity 8 channel capacity 9 differential entropy 10 the gaussian channel 11 maximum entropy and spectral estimation 12 information theory and statistics 13 rate distortion theory 14 network information theory 15 information theory and the stock market 16 inequalities in information theory index

作者简介

《国际知名大学原版教材•信息论基础》系统介绍了信息论基本原理及其在通信理论、统计学、计算机科学、概率论以及投资理论等领域的应用。作者以循序渐进的方式，介绍了信息量的基本定义、相对熵、互信息以及他们如何自然地用来解决数据压缩、信道容量、信息率失真、统计假设、网络信息流等问题。

图书封面

---

 信息论基础下载

---