| The Science of Information | p. 1 |
| Information Measures | p. 5 |
| Independence and Markov Chains | p. 5 |
| Shannon's Information Measures | p. 10 |
| Continuity of Shannon's Information Measures | p. 16 |
| Chain Rules | p. 17 |
| Informational Divergence | p. 19 |
| The Basic Inequalities | p. 23 |
| Some Useful Information Inequalities | p. 25 |
| Fano's Inequality | p. 28 |
| Entropy Rate of Stationary Source | p. 32 |
| Problems | p. 36 |
| Historical Notes | p. 39 |
| Zero-Error Data Compression | p. 41 |
| The Entropy Bound | p. 42 |
| Prefix Codes | p. 45 |
| Definition and Existence | p. 45 |
| Huffman Codes | p. 48 |
| Redundancy of Prefix Codes | p. 54 |
| Problems | p. 58 |
| Historical Notes | p. 59 |
| Weak Typicality | p. 61 |
| The Weak AEP | p. 61 |
| The Source Coding Theorem | p. 64 |
| Efficient Source Coding | p. 66 |
| The Shannon-McMillan-Breiman Theorem | p. 68 |
| Problems | p. 70 |
| Historical Notes | p. 71 |
| Strong Typicality | p. 73 |
| Strong AEP | p. 73 |
| Strong Typicality Versus Weak Typicality | p. 81 |
| Joint Typicality | p. 82 |
| An Interpretation of the Basic Inequalities | p. 92 |
| Problems | p. 93 |
| Historical Notes | p. 94 |
| The I-Measure | p. 95 |
| Preliminaries | p. 96 |
| The I-Measure for Two Random Variables | p. 97 |
| Construction of the I-Measure [mu]* | p. 100 |
| [mu]* Can be Negative | p. 103 |
| Information Diagrams | p. 105 |
| Examples of Applications | p. 112 |
| A Variation of the Inclusion-Exclusion Formula | p. 119 |
| Problems | p. 121 |
| Historical Notes | p. 124 |
| Markov Structures | p. 125 |
| Conditional Mutual Independence | p. 126 |
| Full Conditional Mutual Independence | p. 135 |
| Markov Random Field | p. 140 |
| Markov Chain | p. 143 |
| Problems | p. 146 |
| Historical Notes | p. 147 |
| Channel Capacity | p. 149 |
| Discrete Memoryless Channels | p. 153 |
| The Channel Coding Theorem | p. 158 |
| The Converse | p. 160 |
| Achievability of the Channel Capacity | p. 166 |
| A Discussion | p. 171 |
| Feedback Capacity | p. 174 |
| Separation of Source and Channel Coding | p. 180 |
| Problems | p. 183 |
| Historical Notes | p. 186 |
| Rate Distortion Theory | p. 187 |
| Single-Letter Distortion Measures | p. 188 |
| The Rate-Distortion Function R(D) | p. 191 |
| The Rate-Distortion Theorem | p. 196 |
| The Converse | p. 204 |
| Achievability of R[subscript I](D) | p. 206 |
| Problems | p. 212 |
| Historical Notes | p. 214 |
| The Blahut-Arimoto Algorithms | p. 215 |
| Alternating Optimization | p. 216 |
| The Algorithms | p. 218 |
| Channel Capacity | p. 218 |
| The Rate-Distortion Function | p. 223 |
| Convergence | p. 226 |
| A Sufficient Condition | p. 227 |
| Convergence to the Channel Capacity | p. 230 |
| Problems | p. 231 |
| Historical Notes | p. 231 |
| Single-Source Network Coding | p. 233 |
| A Point-to-Point Network | p. 234 |
| What is Network Coding? | p. 236 |
| A Network Code | p. 240 |
| The Max-Flow Bound | p. 242 |
| Achievability of the Max-Flow Bound | p. 245 |
| Acyclic Networks | p. 246 |
| Cyclic Networks | p. 251 |
| Problems | p. 259 |
| Historical Notes | p. 262 |
| Information Inequalities | p. 263 |
| The Region [Gamma]*[subscript n] | p. 265 |
| Information Expressions in Canonical Form | p. 267 |
| A Geometrical Framework | p. 269 |
| Unconstrained Inequalities | p. 269 |
| Constrained Inequalities | p. 270 |
| Constrained Identities | p. 272 |
| Equivalence of Constrained Inequalities | p. 273 |
| The Implication Problem of Conditional Independence | p. 276 |
| Problems | p. 277 |
| Historical Notes | p. 278 |
| Shannon-Type Inequalities | p. 279 |
| The Elemental Inequalities | p. 279 |
| A Linear Programming Approach | p. 281 |
| Unconstrained Inequalities | p. 283 |
| Constrained Inequalities and Identities | p. 284 |
| A Duality | p. 285 |
| Machine Proving--ITIP | p. 287 |
| Tackling the Implication Problem | p. 291 |
| Minimality of the Elemental Inequalities | p. 293 |
| The Basic Inequalities and the Polymatroidal Axioms | p. 297 |
| Problems | p. 298 |
| Historical Notes | p. 300 |
| Beyond Shannon-Type Inequalities | p. 301 |
| Characterizations of [Gamma]*[subscript 2], [Gamma]*[subscript 3], and [Gamma]*[subscript n] | p. 302 |
| A Non-Shannon-Type Unconstrained Inequality | p. 310 |
| A Non-Shannon-Type Constrained Inequality | p. 315 |
| Applications | p. 321 |
| Problems | p. 324 |
| Historical Notes | p. 325 |
| Multi-Source Network Coding | p. 327 |
| Two Characteristics | p. 328 |
| The Max-Flow Bounds | p. 328 |
| Superposition Coding | p. 330 |
| Examples of Application | p. 335 |
| Multilevel Diversity Coding | p. 335 |
| Satellite Communication Network | p. 336 |
| A Network Code for Acyclic Networks | p. 337 |
| An Inner Bound | p. 340 |
| An Outer Bound | p. 342 |
| The LP Bound and Its Tightness | p. 346 |
| Achievability of R[subscript in] | p. 350 |
| Approximation of Random Variables with Infinite Alphabets | p. 360 |
| Problems | p. 361 |
| Historical Notes | p. 364 |
| Entropy and Groups | p. 365 |
| Group Preliminaries | p. 366 |
| Group-Characterizable Entropy Functions | p. 372 |
| A Group Characterization of [Gamma]*[subscript n] | p. 377 |
| Information Inequalities and Group Inequalities | p. 380 |
| Problems | p. 384 |
| Historical Notes | p. 387 |
| Bibliography | p. 389 |
| Index | p. 403 |
| Table of Contents provided by Syndetics. All Rights Reserved. |