Notes for Central Limit Theorem and Confidence Interval

These are some notes for reviewing the statistics knowledge while I was studying the lesson 1 of Udacity A/B testing. Specifically, it is for binomial distribution converging to normal distribution when n is large. Here is a more basic note for understanding the intuition of CLT and confidence interval I wrote previously, mostly assuming a normal distribution.

Central Limit Theorem

In Udacity A/B testing session 1, the instructors reviewed how to compute confidence interval of the estimated probability p of binomial distribution. When n is very large, binomial distribution tends to converge to normal distribution. Thus, the same formula to estimate the mean of normal distribution can be used to compute the margin of error, m = z(1-alpha/2 )* SE, where SE is computed from the variance of binomial distribution n*p*(1-p), and z is computed from a normal distribution given a confidence level alpha.

But why does binomial distribution converges to normal distribution when n is large? Because Central Limit Theorem (CLT)says so. CLT studies under what condition, the limit distribution of the sum of random variables is normal distribution. The first version of CLT is de Moivre–Laplace theorem. For example,

Xi ~ b(1, p)
Yn = sum(Xi) from 1 to n, n Bernoulli distributions
Thus, Yn ~ b(n, p)
When n -> infinity, Y ~ b(n, p) ~ N(np, np(1-p))
Y* = (Y - np) / sqrt(np(1-p)) ~ N(0, 1)
We can compute P(Yn > k)

Then, the most classical version of CLT is Lindeberg–Lévy CLT. If a sequence of random variables are identically and independently distributed, the sum of them converges to normal distribution when n is large.

Finally, even though the requirement of independent random variables can be easily satisfied, identically distributed random variables are not very common in practice. The more general version of CLT is Lyapunove CLT.

Why is it interesting to study CLT, that is, under what conditions, the limit distribution of the sum of random variables converges to normal distribution? Because there are many such cases in practice.

• In many applications, the error is considered following normal distribution, because the error is the sum of a large amount (n is large) of independent random effects (i.i.d random variables), such as wind, temperature, humidity, human difference, etc.
• Given a uniform distribution generator, generate a list of numbers from a normal distribution.
• Error analysis of numeric computation, for example, error from rounding individual numbers. Assume the error from each rounding follows uniform distribution.

Normal Distribution vs. Poisson Distribution

For a binomial distribution b(n, p), as n gets larger:

• when p is small, b(n, p) ~ P(np), Poisson distribution;
• when np > 5 and n(1-p) > 5, b(n, p) ~ N(np, np(1-p)).

Confidence interval for binomial distribution when n is large

As n gets larger, k ~ b(n, p) converges to N(np, np(1-p)), where k is the number of successes in n runs. If k is the fraction instead of the absolute number, k ~ b(n, p) converges to N(p, sqrt(p(1–p)/n)).

P(|(x - p) / sqrt(p(1-p)/n)| < u) = confidence level 
Margin of error = u * sqrt(x * (1 - x) / n)
Thus,the confidence interval is [x - u*sqrt(x(1 - x)/n), x + u*sqrt(x(1 - x)/n)]

The formula of the margin of error can also be used to estimate n (the sample size) given a interval range.