Coding Theorems of Information Theory

1. Heuristic Introduction to the Discrete Memoryless Channel.- 2. Combinatorial Preliminaries.- 2.1. Generated sequences.- 2.2. Properties of the entropy function.- Remarks.- 3. The Discrete Memoryless Channel.- 3.1. Description of the channel.- 3.2. A coding theorem.- 3.3. The strong converse.- 3.4. Strong converse for the binary symmetric channel.- 3.5. The finite-state channel with state calculable by both sender and receiver.- 3.6. The finite-state channel with state calculable only by the sender.- Remarks.- 4. Compound Channels.- 4.1. Introduction.- 4.2. The canonical channel.- 4.3. A coding theorem.- 4.4. Strong converse.- 4.5. Compound d.m.c. with c.p.f. known only to the receiver or only to the sender.- 4.6. Channels where the c.p.f. for each letter is stochastically determined.- 4.7. Proof of Theorem 4.6.4.- 4.8. The d.m.c. with feedback.- Remarks.- 5. The Discrete Finite-Memory Channel.- 5.1. The discrete channel.- 5.2. The discrete finite-memory channel.- 5.3. The coding theorem for the d.f.m.c..- 5.4. Strong converse of the coding theorem for the d.f.m.c.- 5.5. Rapidity of approach to C in the d.f.m.c.- 5.6. Discussion of the d.f.m.c.- Remarks.- 6. Channels with Arbitrarily Varying Channel Probability Functions.- 6.1. Introduction.- 6.2. Necessary and sufficient conditions for a positive rate of transmission.- 6.3. Remarks on the capacity of an arbitrarily varying channel.- 6.4. The capacity C of an arbitrarily varying channel when b = 2.- 6.5. Certain results for the general arbitrarily varying channel.- Remarks.- 7. General Discrete Channels.- 7.1. Alternative description of the general discrete channel.- 7.2. The method of maximal codes.- 7.3. The method of random codes.- 7.4. Weak converses.- 7.5. Digression on the d.m.c.- 7.6. Discussion of the foregoing.- 7.7. Channels without a capacity.- 7.8. Strong converses.- 7.9. The strong converse for the d.m.c. revisited.- Remarks.- 8. The Semi-Continuous Memoryless Channel.- 8.1. Introduction.- 8.2. A coding theorem and its strong converse.- 9. Continuous Channels with Additive Gaussian Noise.- 9.1. A continuous memoryless channel with additive Gaussian noise.- 9.2. Message sequences within a suitable sphere.- 9.3. Message sequences on the periphery of the sphere or within a shell adjacent to the boundary.- 9.4. Another proof of Theorems 9.2.1 and 9.2.2.- Remarks.- 10. Mathematical Miscellanea.- 10.1. Introduction.- 10.2. The asymptotic equipartition property.- 10.3. Admissibility of an ergodic input for a discrete finite-memory channel.- 11. Fundamentals of Rate Distortion Theory.- 11.1. Introduction.- 11.2. The approximation theorem.- 11.3. Converse of the approximation theorem.- 11.4. Summary of the previous results.- 11.5. The rate distortion function when side information is available.- Remarks.- 12. Source Coding.- 12.1. Separate coding to span the product of two spaces.- 12.2. Source coding with side information at the decoder.- 12.3. Encoding assisted by a common channel.- Remarks.- 13. Source Coding and Rate Distortion.- 13.1. The problem of Section 12.3 for rate distortion.- 13.2. The rate distortion function for source coding with side information at the decoder.- 14. Multiple Access Channels.- 14.1. Description of the problem.- 14.2. A coding theorem.- 14.3. Converse of the coding theorem.- 14.4. Miscellaneous remarks.- 15. Degraded Broadcast Channels.- 15.1. Formulation of the problem.- 15.2. A coding theorem.- 15.3. Beginning of the proof of the strong converse.- 15.4. Proof of the weak converse.- 15.5. Miscellaneous remarks.- References.