General Survey of the Mathematical Model for a Communication Channel
Information Theory is concerned with the generation, transmission and processing of certain quantities which are assumed to represent in some way an “amount of information”. Of course a definition of “information” in general terms is by no means obvious,
- PDF / 6,293,205 Bytes
- 110 Pages / 481.92 x 691.68 pts Page_size
- 30 Downloads / 332 Views
A ND
L E C T U R E S - No.
18
GIUSEPPE LONGO UNIVERSITY OF TRIESTE
SELECTED TOPICS IN INFORMATION THEORY
.LECTURES HELD AT THE DEPARTMENT OF AUTOMATION AND INFORMIA.TION SEPTEMBER - OCTOBER 1969
UDINE 1973
SPRINGER-VERLAG WIEN GMBH
This woIk is subject to copyright AII rights are reserved, whether the whole or part of the material ia concemed specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machine or similar means, and storage in data banks. © 1972 Springer-Verlag Wien Originally published by Springer-Verlag Wien-New York in 1972
ISBN 978-3-211-81166-5 ISBN 978-3-7091-2850-3 (eBook) DOI 10.1007/978-3-7091-2850-3
foreword. What I had in mind in preparing this course on " Selected Topics in Information Theory " was first of all to give a general idea of what Information Theory is to people who are not acquainted with it, and secondly to introduce the audience to a very limited number of particular problems which could be of interest also for those who are already a little more familiar with the theory. Although these problems are very few, nevertheless they have not been treated as deeply as they deserve, because time was limited. In fact I have preferred to sacrifice somewhat the completeness than to treat just one problern very deeply. I hope that in so doing I have made the listeners appreciate ideas and concepts of Information Theory more extensively. I am very indebted to CISM and particularly to the Secretary General, prof. Luigi Sobrero, for giving me the opportunity of delivering this course. I dedicate the course to prof. 0. Onicescu, who introduced me into the field of Information Theory.
Udine, 1969
G. Longo
Contents.
Pages
Foreward
3
Introduction
6
Part One - General Survey of the Mathematical model for a Communication Channel 1.1. Introduction
9 9
1.2. The Model for the Communication Link
10
1.3. Accuracy Requirements
12
1.4. Encoding for Minimum Average Length Source Rate
14
1.5. The Noisy Channel. Reduction of Transmission Errors
15
1.6. Entropy and Information. The Finite Case
20
1.7. Channel Equivocation and Channel Capacity
31
Part Two - Calculation of Channel Capacity
37
2.1. Case of Symmetrie Channel
37
2.2. Some Properties of I(x;y)
40
2.3. Capacity of a Discrete Memoryless Channel with Square Nonsingular Channel Matrix 2.4. A General Theorem Part Three - Measures for the Amount of Information 3.1. Introduction
45 50 55 55
3.2. Incomplete Schemes and Generalized Probability Distributions
57
Contents
5
3.3. Characterization of Shannon's Entropy among the Entropies of Positive Order
62
3.4. Pragmatic Approach to Uncertainty Measures
64
3.5. Informational Entropy and Informational Energy 3.6. Uncertainty Functions for Countahle Schemes
66 68
3.7. The Continuous Case. The Intensity of the Informational Energy 3.8. The Continuous Case. Entropy
70 78
3.9. An Interpretation for the Intensity of the Informational Energy
79
3.10. Dimension of a Probability Distribution and d-dimens
Data Loading...
