Fundamentals of Information Theory
This chapter introduces some elementary concepts regarding information theory. First, we present entropy and other measures of information. Then, we discuss a very important quantity in classical information theory, the capacity of a discrete noisy channe
- PDF / 713,878 Bytes
- 37 Pages / 439.36 x 666.15 pts Page_size
- 40 Downloads / 276 Views
Fundamentals of Information Theory
The statistical theory of communication introduced by Claude Shannon answers fundamental questions about a system consisting of agents exchanging classical information through a classical communication channel [20]. The proposition of information theory was made in the electrical engineering domain due to the problems with telegraphs, radios, and telephones that the engineers faced in order to attend a growing demand of customers. Such practical challenges motivated the search for answers to fundamental questions such as how to define and quantify information, bounds to information compression and also strategies to exchange information securely [9]. Nowadays, information theory lies at the heart of modern technology, underpinning all communications, networking, and data storage systems. The history of classical information theory began with Claude Shannon [28]. In this paper, he coined the essential terminology, and he stated and justified the main mathematical definitions, as well as the two fundamental theorems of information theory. The noiseless coding theorem quantifies the physical resources required to store the output of an information source; the second, called channel coding theorem, quantifies how much information it is possible to reliably transmit through a noisy communication channel [24]. We can say that information theory has two primary goals. The first is the development of fundamental theoretical limits on the achievable performance when communicating a given information source over a communication channel and using a specified coding scheme. The second goal is the development of coding schemes that provide reasonably good performance in comparison with the optimal theoretical limits established by the theory. [14]. While classical information theory refers to a mathematical framework for modeling the manipulation and transmission of classical information, quantum information theory studies fundamental problems related to the transmission of classical and quantum information over quantum communication channels. Quantum information theory promises to lead to a deeper understanding of fundamental © Springer International Publishing Switzerland 2016 E.B. Guedes et al., Quantum Zero-Error Information Theory, DOI 10.1007/978-3-319-42794-2_3
27
28
3 Fundamentals of Information Theory
properties of nature and, at the same time, support new and exciting applications [20]. Quantum information is a fascinating topic precisely because it shows that the laws of information processing are actually dependent on the laws of physics [36]. This chapter introduces some elementary concepts regarding information theory. We start with classical information theory, aiming at introducing the two important theorems of Claude Shannon. First, we present entropy and other measures of information in Sect. 3.1.1. Then, we discuss in Sect. 3.1.2 a very important quantity in classical information theory, the capacity of a noisy classical channel. In the second part of the chapter, we i
Data Loading...
