The Law of Large Numbers
We have mentioned (more than once) that the basis for probabilistic modeling is the stabilization of the relative frequencies.
- PDF / 462,210 Bytes
- 64 Pages / 439.37 x 666.142 pts Page_size
- 50 Downloads / 213 Views
The Law of Large Numbers
We have mentioned (more than once) that the basis for probabilistic modeling is the stabilization of the relative frequencies. Mathematically this phenomenon can be formulated as follows: Suppose that we perform independent repetitions of an experiment, and let X k = 1 if round k is successful and 0 otherwise, k ≥ 1. The relative frequency of successes is described by the arithmetic mean, n1 nk=1 X k , and the stabilization of the relative frequencies corresponds to n 1 Xk → p n
as n → ∞,
k=1
where p = P(X 1 = 1) is the success probability. Note that the convergence arrow is a plain arrow! The reason for this is that the first thing to wonder about is: Convergence in what sense? The basis for the probability model was the observation that whenever such a random experiment is performed the relative frequencies stabilize. The word “whenever” indicates that the interpretation must be “almost sure convergence”. The stabilization thus is translated into n 1 a.s. X k → p as n → ∞. n k=1
This, in turn, means that the validation of the Ansatz is that a theorem to that effect must be contained in our theory. The strong law of large numbers, which is due to Kolmogorov, is a more general statement to the effect that if X 1 , X 2 , . . . are arbitrary independent, identically distributed random variables with finite mean, μ, then the arithmetic mean converges almost surely to μ. Moreover, finite mean is necessary for the conclusion to hold. If, in particular, the summands are indicators, the result reduces to the almost sure formulation of the stabilization of the relative frequencies. There also exist weak laws, which means convergence in probability.
A. Gut, Probability: A Graduate Course, Springer Texts in Statistics, DOI: 10.1007/978-1-4614-4708-5_6, © Springer Science+Business Media New York 2013
265
266
6 The Law of Large Numbers
Other questions concern (i) moment convergence, that is, questions related to uniform integrability (recall Sect. 5.4), (ii) whether other limit theorems can be obtained by other normalizations under suitable conditions, and (iii) laws of large numbers for randomly indexed sequences. We shall also prove a fact that has been announced before, namely that complete convergence requires more than almost sure convergence. Applications to normal numbers, the Glivenko–Cantelli theorem, renewal theory, and records will be given, which for the latter two means a continuation of earlier visits to these topics. The final section, entitled “Some Additional Results and Remarks”, preceding the problem section, contains different aspects of convergence rates. This section may be considered as less “hot” (for the non-specialist) and can therefore be skipped, or skimmed through, at a first reading.
1 Preliminaries A common technique is to use what is called truncation, which means that one creates a new sequence of random variables which is asymptotically equivalent to the sequence of interest, and easier to deal with than the original one.
1.1 Convergence Equivalence The
Data Loading...
