R Stats: Central Limit Theorem

Averages and Proportions

• Random variable times a constant
• • The expected value of a random variable multiplied by a constant is that constant times its original expected value:
    E[aX]=aμ
• • The standard error of a random variable multiplied by a constant is that constant times its original standard error:
    SE[aX]=aσ
• Average of multiple draws of a random variable
• • The expected value of the average of multiple draws from an urn is the expected value of the urn (μ).
• • The standard deviation of the average of multiple draws from an urn is the standard deviation of the urn divided by the square root of the number of draws (σ/sqrt(n)).
• The sum of multiple draws of a random variable
• • The expected value of the sum of n draws of a random variable is n times its original expected value:
    E[nX]=nμ
• • The standard error of the sum of n draws of random variable is sqrt(n) times its original standard error:
    SE[nX]= sqrt(n)σ
• The sum of multiple different random variables
• • The expected value of the sum of different random variables is the sum of the individual expected values for each random variable:
    E[X1+X2+⋯+Xn]=μ1+μ2+⋯+μn
• • The standard error of the sum of different random variables is the square root of the sum of squares of the individual standard errors:
    SE[X1+X2+⋯+Xn]=sqrt(σ1^2+σ2^2+⋯+σn^2)

Transformation of random variables If X is a normally distributed random variable and a and b are non-random constants, then aX+b is also a normally distributed random variable.

Law of Large Numbers

• The law of large numbers states that as n increases, the standard error of the average of a random variable decreases. In other words, when n is large, the average of the draws converges to the average of the urn.
• The law of large numbers is also known as the law of averages.
• The law of averages only applies when n is very large and events are independent. It is often misused to make predictions about an event being "due" because it has happened less frequently than expected in a small sample size.

Advanced: How Large is Large Enough for Central Limit Theorem

• The sample size required for the Central Limit Theorem and Law of Large Numbers to apply differs based on the probability of success.
• • If the probability of success is high, then relatively few observations are needed.
• • As the probability of success decreases, more observations are needed.
• If the probability of success is extremely low, such as winning a lottery, then the Central Limit Theorem may not apply even with extremely large sample sizes. The normal distribution is not a good approximation in these cases, and other distributions such as the Poisson distribution (not discussed in these courses) may be more appropriate.