The Central Limit Theorem
The Central Limit Theorem is one of the most impressive achievements of probability theory. From a simple description requiring minimal hypotheses, we are able to deduce precise results. The Central Limit Theorem thus serves as the basis for much of Stati
- PDF / 265,937 Bytes
- 8 Pages / 439 x 666 pts Page_size
- 76 Downloads / 200 Views
The Central Limit Theorem is one of the most impressive achievements of probability theory. From a simple description requiring minimal hypotheses, we are able to deduce precise results. The Central Limit Theorem thus serves as the basis for much of Statistical Theory. The idea is simple: let X1 , . . . , X j , . . . be a sequence of i.i.d. random variables with finite variance. n Let Sn = j=1 Xj . Then for n large, L(Sn ) ≈ N (nμ, nσ 2 ), where E{Xj } = μ 2 and σ = Var(Xj ) (all j). The key observation is that absolutely nothing (except a finite variance) is assumed about the distribution of the random variables (Xj )j≥1 . Therefore, if one can assume that a random variable in question is the sum of many i.i.d. random variables with finite variances, that one can infer that the random variable’s distribution is approximately Gaussian. Next one can use data and do Statistical Tests to estimate μ and σ 2 , and then one knows essentially everything! Theorem 21.1 (Central Limit Theorem). Let (Xj )j≥1 be i.i.d. n with E{Xj } = μ and Var(Xj ) = σ 2 (all j) with 0 < σ 2 < ∞. Let Sn = j=1 Xj . −nμ . Then Yn converges in distribution to Y , where L(Y ) = Let Yn = Sσn √ n N (0, 1). Observe that if σ 2 = 0 above, then Xj = μ a.s. for all j, hence
Sn n
=μ
a.s. Proof. Let ϕj be the characteristic function of Xj − μ. Since the (Xj )j≥1 are −nμ . Since the i.i.d., ϕj does not depend on j and we write ϕ. Let Yn = Sσn √ n Xj are independent, by Theorem 15.2 ϕYn (u) = ϕ
σ
1 √
n n
= ϕn
j =1
(Xj −μ) (u)
(Xj −μ)
u √
(21.1)
σ n u √ = ϕ(Xj −μ) σ n j=1
n u √ . = ϕ σ n j =1
n
J. Jacod et al., Probability Essentials © Springer-Verlag Berlin Heidelberg 2004
182
21 The Central Limit Theorem
Next note that E{Xj −μ} = 0 and E{(Xj −μ)2 } = σ 2 , hence by Theorem 13.2 we know that ϕ has two continuous derivatives and moreover ϕ (u) = iE (Xj − μ)eiu(Xj −μ) , ϕ (u) = −E (Xj − μ)2 eiu(Xj −μ) . Therefore ϕ (0) = 0 and ϕ (0) = −σ 2 . If we expand ϕ in a Taylor expansion about u = 0, we get (see Exercise 14.4) ϕ(u) = 1 + 0 −
σ 2 u2 + u2 h(u) 2
(21.2)
where h(u) → 0 as u → 0 (because ϕ is continuous). Recall from (21.1):
n u √ ϕYn (u) = ϕ σ n u n log ϕ( σ √ ) n
=e
2
n log(1− u2n +
=e
u2 nσ2
u h( σ √ )) n
,
where here “log” denotes the principal value of the complex valued logarithm. Taking limits as n tends to ∞ and using (for example) L’Hˆ opital’s rule gives that 2 lim ϕYn (u) = e−u /2 ; n→∞
L´evy’s Continuity Theorem (Theorem 19.1) then implies that Yn converges 2 in law to Z, where ϕZ (u) = e−u /2 ; but then we know that L(Z) = N (0, 1), using Example 13.5 and the fact that characteristic functions characterize distributions (Theorem 14.1). Let us now discuss the relationship between laws of large numbers and the central limit theorem. Let (Xj )j≥1 be i.i.d. with finite variances, and let μ = E{X1 }. Then by the Strong Law of Large Numbers, lim
n→∞
Sn =μ n
a.s. and in L2 ,
(21.3)
n where Sn = j=1 Xj . Thus we know the limit is μ, but a natural question is: How large must
Data Loading...
