FS1 Mean and variance of a normal distribution

Further statistics 1

Table of contents

1. Probability density functions
2. Moment generating functions
3. Moments
4. Edexcel FS1 Ch Exercise

1. Probability density functions

The probability dentity function for a normal distribution, $X \sim \textrm N \left( \mu, \sigma^2 \right)$, is $$\begin{align} \textrm f(x) = \frac{1}{ \sqrt{2\pi} \sigma} \textrm e^{ - \frac12 \left( \frac{ x - \mu }{ \sigma } \right)^2 } \end{align}$$ The probability dentity function for the standard normal distribution, $Z = \frac{ X - \mu }{ \sigma } \sim \textrm N \left( 0, 1^2 \right)$, is $$\begin{align} \textrm f(z) = \frac{1}{ \sqrt{2\pi} } \textrm e^{ - \frac12 z^2 } \end{align}$$

2. Moment generating functions

2.1 A normal distribution

For a normal distribution, $X \sim \textrm N \left( \mu, \sigma^2 \right)$, $$\begin{align} M_X(t) &= 1 + \langle X \rangle t + \frac{ 1 }{ 2! } \left\langle X^2 \right\rangle t^2 + \frac{ 1 }{ 3! } \left\langle X^3 \right\rangle t^3 + \cdots \\ &= \left\langle 1 + tX + \frac{ 1 }{ 2! } (tX)^2 + \frac{ 1 }{ 3! } (tX)^3 + \cdots \right\rangle \\ &= \left\langle \textrm{e}^{tX} \right\rangle \\ &= \frac{1}{ \sqrt{2\pi} \sigma} \int_{-\infty}^{\infty} \textrm{e}^{tx} \textrm e^{ - \frac12 \left( \frac{ x - \mu }{ \sigma } \right)^2 } \, \textrm dx \\ &= \frac{1}{ \sqrt{2\pi} \sigma} \int_{-\infty}^{\infty} \textrm e^{ - \frac1{ 2 \sigma^2 } \left[ ( x^2 - 2 ( \mu + \sigma^2 t ) x + \mu^2 \right] } \, \textrm dx \\ &= \frac{1}{ \sqrt{2\pi} \sigma} \int_{-\infty}^{\infty} \textrm e^{ - \frac1{ 2 \sigma^2 } \left[ [ x - ( \mu + \sigma^2 t ) ]^2 - ( \mu + \sigma^2 t )^2 + \mu^2 \right] } \, \textrm dx \\ &= \frac{1}{ \sqrt{2\pi} \sigma} \int_{-\infty}^{\infty} \textrm e^{ - \frac12 \left( \frac{ x - \mu - \sigma^2 t }{ \sigma } \right)^2 } e^{ - \frac1{ 2 \sigma^2 } \left[ - \left( \mu^2 + 2 \mu \sigma^2 t + \sigma^4 t^2 \right) + \mu^2 \right] } \, \textrm dx \\ &= e^{ \frac12 \sigma^2 t^2 + \mu t } \frac{1}{ \sqrt{2\pi} \sigma} \int_{-\infty}^{\infty} \textrm e^{ - \frac12 \left( \frac{ x - \mu - \sigma^2 t }{ \sigma } \right)^2 } \, \textrm dx \\ &= e^{ \mu t + \frac12 \sigma^2 t^2 } \end{align}$$

2.2 The standard normal distribution

For the standard normal distribution, $Z = \frac{ X - \mu }{ \sigma } \sim \textrm N \left( 0, 1^2 \right)$, $$\begin{align} M_Z(t) &= 1 + \langle Z \rangle t + \frac{ 1 }{ 2! } \left\langle Z^2 \right\rangle t^2 + \frac{ 1 }{ 3! } \left\langle Z^3 \right\rangle t^3 + \cdots \\ &= \left\langle 1 + tZ + \frac{ 1 }{ 2! } (tZ)^2 + \frac{ 1 }{ 3! } (tZ)^3 + \cdots \right\rangle \\ &= \left\langle \textrm{e}^{tZ} \right\rangle \\ &= \frac{1}{ \sqrt{2\pi} } \int_{-\infty}^{\infty} \textrm{e}^{tz} \textrm e^{ - \frac12 z^2 } \, \textrm dz \\ &= \frac{1}{ \sqrt{2\pi} } \int_{-\infty}^{\infty} \textrm e^{ - \frac12 \left( z^2 - 2zt \right) } \, \textrm dz \\ &= \frac{1}{ \sqrt{2\pi} } \int_{-\infty}^{\infty} \textrm e^{ - \frac12 \left[ ( z - t )^2 - t^2 \right] } \, \textrm dz \\ &= \textrm e^{ \frac12 t^2 } \frac{1}{ \sqrt{2\pi} } \int_{-\infty}^{\infty} \textrm e^{ - \frac12 ( z - t )^2 } \, \textrm dz \\ &= \textrm e^{ \frac12 t^2 } \end{align}$$ which agrees with $M_X(t)$ when $\mu = 0$ and $\sigma = 1$.

3. Moments

3.1 The standard normal distribution

The Maclaurin/Taylor expansion of the moment generating function: $$\begin{align} M_Z(t) &= \textrm e^{ \frac12 t^2 } \\ &= \sum_{n=0}^\infty \frac1{n!} \left( \frac{t^2}{2} \right)^n \\ &= \sum_{n=0}^\infty \frac{(2n)!}{2^n n!} \frac{t^{2n}}{(2n)!} \end{align}$$ which gives $$\begin{align} \Rightarrow \qquad \mathbb E \left( Z^{ \textrm{odd} } \right) &= 0 &&& \mathbb E \left( Z^{2n} \right) &= \frac{(2n)!}{2^n n!} \\ \\ \mathbb E \left( Z \right) &= 0 &&& \mathbb E \left( Z^2 \right) &= \frac{2!}{2^1 1!} = 1 \\ \mathbb E \left( Z^3 \right) &= 0 &&& \mathbb E \left( Z^4 \right) &= \frac{4!}{2^2 2!} = 3 \\ \mathbb E \left( Z^5 \right) &= 0 &&& \mathbb E \left( Z^6 \right) &= \frac{6!}{2^3 3!} = 15 \\ \mathbb E \left( Z^7 \right) &= 0 &&& \mathbb E \left( Z^8 \right) &= \frac{8!}{2^4 4!} = 105 \\ \mathbb E \left( Z^9 \right) &= 0 &&& \mathbb E \left( Z^{10} \right) &= \frac{10!}{2^5 5!} = 945 \end{align}$$

 

Aside. Alternatively, we can evaluate: $$\begin{align} \mathbb E \left( Z^{m} \right) &= \frac{1}{ \sqrt{2\pi} } \underbrace{ \int_{-\infty}^{\infty} }_{\textrm{even}} z^{m} \underbrace{ \textrm e^{ - \frac12 z^2 } }_{\textrm{even}} \, \textrm dz \\ \end{align}$$ and the integral vanishes when $m$ is odd. Thus, we may restrict to even $m$, i.e. $m=2n$. $$\begin{align} \mathbb E \left( Z^{2n} \right) &= \frac{1}{ \sqrt{2\pi} } \int_{-\infty}^{\infty} z^{2n} \textrm e^{ - \frac12 z^2 } \, \textrm dz \\ &= \frac{2}{ \sqrt{2\pi} } \int_{0}^{\infty} z^{2n} \textrm e^{ - \frac12 z^2 } \, \textrm dz \\ \end{align}$$ By the substitution $u = \frac12z^2$, we find $z^2 = 2u$ and $\textrm du = z \textrm dz$. This gives $$\begin{align} \Rightarrow \qquad \mathbb E \left( Z^{2n} \right) &= \frac{2}{ \sqrt{2\pi} } \int_{0}^{\infty} (2u)^n \textrm e^{ - u } \, \frac{ \textrm du }{ \sqrt{2u} } \\ &= \frac{ 2^{n+1} }{ 2 \sqrt{\pi} } \int_{0}^{\infty} u^{n-\frac12} \textrm e^{ - u } \, \textrm du \\ &= \frac{ 2^{n} }{ \sqrt{\pi} } \Gamma \left( n + \frac12 \right) \\ &= \frac{ 2^{n} }{ \sqrt{\pi} } \cdot \frac{ (2n)! }{ 2^{2n} n! } \sqrt{ \pi } \\ &= \frac{ (2n)! }{ 2^{n} n! } \qquad \checkmark \end{align}$$ since $$\begin{align} \Gamma \left( n + \frac12 \right) &= \left( n - \frac12 \right) \left( n - \frac32 \right) \left( n - \frac52 \right) \cdots \frac12 \underbrace{ \Gamma \left( \frac12 \right) }_{ \sqrt{ \pi } } \\ &= \frac{ (2n-1) (2n-3) (2n-5) \cdots 1 }{ 2^n } \sqrt{ \pi } \\ &= \frac{ (2n)! }{ 2^{2n} n! } \sqrt{ \pi } \end{align}$$ In summary, the standardised moments for a normal distribution, $X \sim \textrm N( \mu , \sigma^2 )$: $$\begin{align} \mathbb E \left( Z^n \right) = \mathbb E \left[ \left( \frac{ X - \mu }{ \sigma } \right)^n \right] &= \frac{1}{ \sqrt{2\pi} \sigma} \int_{-\infty}^{\infty} \left( \frac{ x - \mu }{ \sigma } \right)^n \textrm e^{ - \frac12 \left( \frac{ x - \mu }{ \sigma } \right)^2 } \, \textrm dx \\ \\ \mathbb E \left( Z^{ \textrm{odd} } \right) &= 0 \\ \mathbb E \left( Z^{ \textrm{2n} } \right) &= \frac{ (2n)! }{ 2^n n! } \\ \\ \mathbb E \left( Z^2 \right) &= 1 \\ \mathbb E \left( Z^4 \right) &= 3 \\ \mathbb E \left( Z^6 \right) &= 15 \\ \mathbb E \left( Z^8 \right) &= 105 \\ \mathbb E \left( Z^{10} \right) &= 945 \end{align}$$

3.2 A normal distribution

We can Maclaurin/Taylor expand $M_X(t)$: $$\begin{align} M_X(t) &= \textrm e^{ \mu t + \frac12 \sigma^2 t^2 } \\ &= \textrm e^{ \mu t } \textrm e^{ \frac12 \sigma^2 t^2 } \\ &= \left( 1 + \mu t + \frac{ (\mu t)^2 }{2!} + \frac{ (\mu t)^3 }{3!} + \frac{ (\mu t)^4 }{4!} + \cdots \right) \left[ 1 + \left( \frac12 \sigma^2 t^2 \right) + \frac1{2!} \left( \frac12 \sigma^2 t^2 \right)^2 + \cdots \right] \\ &= \left( 1 + \mu t + \mu^2 \frac{t^2}{2!} + \mu^3 \frac{t^3}{3!} + \mu^4 \frac{t^4}{4!} + \cdots \right) \left[ 1 + \sigma^2 \frac{t^2}{2!} + \frac{4!}{2!} \left( \frac{\sigma^2}2 \right)^2 \frac{t^4}{4!} + \cdots \right] \\ &= 1 + \mu t + \left( \mu^2 + \sigma^2 \right) + \left( \mu^3 + 3 \mu \sigma^2 \right) \frac{t^3}{3!} + \left[ \mu^4 + \frac{4!}{2!2!} \mu^2 \sigma^2 + \frac{4!}{2!} \left( \frac{\sigma^2}2 \right)^2 \right] \frac{t^4}{4!} + \cdots \\ &= 1 + \mu t + \left( \mu^2 + \sigma^2 \right) + \left( \mu^3 + 3 \mu \sigma^2 \right) \frac{t^3}{3!} + \left( \mu^4 + 6 \mu^2 \sigma^2 + 3 \sigma^4 \right) \frac{t^4}{4!} + \cdots \end{align}$$ Aside. Alternatively, we can evaluate: $$\begin{align} \mathbb E \left( X^n \right) &= \frac{1}{ \sqrt{2\pi} \sigma} \int_{-\infty}^{\infty} x^n \textrm e^{ - \frac12 \left( \frac{ x - \mu }{ \sigma } \right)^2 } \, \textrm dx \\ &= \frac{1}{ \sqrt{2\pi}} \int_{-\infty}^{\infty} ( \mu + \sigma z )^n \textrm e^{ - \frac12 z^2 } \, \textrm dz \\ &= \sum_{k=0}^n \binom{n}{k} \mu^{n-k} \sigma^{k} \left\langle Z^k \right\rangle \end{align}$$ By the binomial expansion and using $\left\langle Z^{ \textrm{odd} } \right\rangle = 0$ and $\left\langle Z^{2n} \right\rangle = \frac{(2n)!}{2^n n!}$, we find: $$\begin{align} \mathbb E \left( X \right) &= \frac{1}{ \sqrt{2\pi}} \int_{-\infty}^{\infty} ( \mu + \sigma z ) \textrm e^{ - \frac12 z^2 } \, \textrm dz \\ &= \mu \\ \\ \mathbb E \left( X^2 \right) &= \frac{1}{ \sqrt{2\pi}} \int_{-\infty}^{\infty} ( \mu + \sigma z )^2 \textrm e^{ - \frac12 z^2 } \, \textrm dz \\ &= \left\langle ( \mu + \sigma Z )^2 \right\rangle \\ &= \left\langle \mu^2 + 2 \mu \sigma Z + \sigma^2 Z^2 \right\rangle \\ &= \mu^2 + \sigma^2 \\ \\ \mathbb E \left( X^3 \right) &= \frac{1}{ \sqrt{2\pi}} \int_{-\infty}^{\infty} ( \mu + \sigma z )^3 \textrm e^{ - \frac12 z^2 } \, \textrm dz \\ &= \left\langle ( \mu + \sigma Z )^3 \right\rangle \\ &= \left\langle \mu^3 + 3 \mu^2 \sigma Z + 3 \mu \sigma^2 Z^2 + \sigma^3 Z^3 \right\rangle \\ &= \mu^3 + 3 \mu \sigma^2 \\ \\ \mathbb E \left( X^4 \right) &= \frac{1}{ \sqrt{2\pi}} \int_{-\infty}^{\infty} ( \mu + \sigma z )^4 \textrm e^{ - \frac12 z^2 } \, \textrm dz \\ &= \left\langle ( \mu + \sigma Z )^4 \right\rangle \\ &= \left\langle \mu^4 + 4 \mu^3 \sigma Z + 6 \mu^2 \sigma^2 Z^2 + 4 \mu \sigma^3 Z^3 + \sigma^4 Z^4 \right\rangle \\ &= \mu^4 + 6 \mu^2 \sigma^2 + 3 \sigma^4 \\ \\ \mathbb E \left( X^5 \right) &= \frac{1}{ \sqrt{2\pi}} \int_{-\infty}^{\infty} ( \mu + \sigma z )^5 \textrm e^{ - \frac12 z^2 } \, \textrm dz \\ &= \left\langle ( \mu + \sigma Z )^5 \right\rangle \\ &= \left\langle \mu^5 + 5 \mu^4 \sigma Z + 10 \mu^3 \sigma^2 Z^2 + 10 \mu^2 \sigma^3 Z^3 + 5 \mu \sigma^4 Z^4 + \sigma^5 Z^5 \right\rangle \\ &= \mu^5 + 10 \mu^3 \sigma^2 + 15 \mu \sigma^4 \end{align}$$ In summary, the moments for a normal distribution, $X \sim \textrm N( \mu , \sigma^2 )$: $$\begin{align} \mathbb E \left( X^n \right) &= \frac{1}{ \sqrt{2\pi} \sigma} \int_{-\infty}^{\infty} x^n \textrm e^{ - \frac12 \left( \frac{ x - \mu }{ \sigma } \right)^2 } \, \textrm dx \\ \\ \mathbb E \left( X \right) &= \mu \\ \mathbb E \left( X^2 \right) &= \mu^2 + \sigma^2 \\ \mathbb E \left( X^3 \right) &= \mu^3 + 3 \mu \sigma^2 \\ \mathbb E \left( X^4 \right) &= \mu^4 + 6 \mu^2 \sigma^2 + 3 \sigma^4 \\ \mathbb E \left( X^5 \right) &= \mu^5 + 10 \mu^3 \sigma^2 + 15 \mu \sigma^4 \\ \mathbb E \left( X^6 \right) &= \mu^6 + 15 \mu^4 \sigma^2 + 45 \mu^2 \sigma^4 + 15 \sigma^6 \\ \mathbb E \left( X^7 \right) &= \mu^7 + 21 \mu^5 \sigma^2 + 105 \mu^3 \sigma^4 + 105 \mu \sigma^6 \\ \mathbb E \left( X^8 \right) &= \mu^8 + 28 \mu^6 \sigma^2 + 210 \mu^4 \sigma^4 + 420 \mu^2 \sigma^6 + 105 \sigma^8 \\ \mathbb E \left( X^9 \right) &= \mu^9 + 36 \mu^7 \sigma^2 + 378 \mu^5 \sigma^4 + 1260 \mu^3 \sigma^6 + 945 \mu \sigma^8 \\ \end{align}$$