NOTE: Only some basic distributions and properties are mentioned. The Wikipedia pages on these distributions are a great resource.


Gaussian Distribution

Definition

Let \(X\) be a Gaussian random variable with expectation \(\mu\) and variance \(\sigma^2\): \[ X \sim \mathcal{N}(\mu, \sigma^2). \]

It admits a probability density function (pdf): \[ f(x) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x - \mu)^2}{2\sigma^2}\right) \quad \forall x \in \mathbb{R}, \]

And a characteristic function: \[ \phi_{X}(t) = \mathbb{E}[e^{itX}] = e^{i\mu t - \frac{\sigma^2 t^2}{2}} \quad \forall t \in \mathbb{R}. \]

Density plots

Moments

\[ \mathbb{E}[X] = \mu \quad \mathbb{Var}[X] = \sigma^2. \]

Some Properties

  • The Gaussian distribution with mean \(0\) and variance \(1\) is called the standard normal distribution.

  • If \(X \sim \mathcal{N}(\mu, \sigma^2)\), then for any \((a, b) \in \mathbb{R}^2\), then \(Y = aX + b \sim \mathcal{N}(a\mu + b, a^2\sigma^2).\)

  • The mode, median and mean of the distribution are all equal to \(\mu\).

  • If \(X_1 \sim \mathcal{N}(\mu_1, \sigma^2_1)\) and \(X_2 \sim \mathcal{N}(\mu_2, \sigma^2_2)\) are independent Gaussian random variables, then their sum \(Y = X_1 + X_2\) is also Gaussian, with mean \(\mu_1 + \mu_2\) and variance \(\sigma^2_1 + \sigma^2_2\): \(Y \sim \mathcal{N}(\mu_1 + \mu_2, \sigma^2_1 + \sigma^2_2)\)

  • … many more (see e.g. the wikipedia page).

R functions

The density function of the Gaussian distribution in R is dnorm.

For information on the distribution function (pnorm), quantile function (qnorm) and random generation (rnorm), see:

help(dnorm)

Chi Squared Distribution

Definition

Let \(X_1, \dotsc, X_p\) be \(p\) standard normal independent identically distributed (iid) random variables: \(X_i \sim \mathcal{N}(0, 1)\). Then \[ X = \sum_{i = 1}^p X_i^2 \sim \chi^2_p \]

is a Chi squared random variable with \(p\) degrees of freedom.

Density Plots

Moments

\[ \mathbb{E}[X] = p \quad \mathbb{Var}[X] = 2p. \]

Some Properties

  • The chi squared distribution converges towards the Gaussian distribution with matching moments:
    if \(X \sim \chi^2_{p}\), then the distribution of \(\frac{X - p}{\sqrt{2p}}\) converges toward a standard normal distribution when \(p\) goes to infinity.

  • If \(X_1 \sim \chi^2_{p_1}\) and \(X_2 \sim \chi^2_{p_2}\) are independent chi squared random variables with respective degrees of freedom \(p_1\) and \(p_2\), then their sum \(Y = X_1 + X_2\) is also a chi squared random variable, with \(p_1 + p_2\) degrees of freedom: \(Y \sim \chi^2_{p_1+p_2}\)

  • … (see e.g. the wikipedia page).

R functions

The density function of the chi squared distribution in R is dchisq.

For information on the distribution function (pchisq), quantile function (qchisq) and random generation (rchisq), see:

help(dchisq)

Student Distribution

Definition

Let \(Z\) be a standard normal random variable : \(Z \sim \mathcal{N}(0, 1)\), and \(X\) a chi squared random variable with \(p\) degrees of freedom: \(Z \sim \chi^2_p\), \(Z\) and \(X\) independent. Then \[ T = \frac{Z}{\sqrt{X/p}} \sim \mathcal{T}_p \]

is a Student (t) random variable with \(p\) degrees of freedom.

Density Plots

Moments

\[ \mathbb{E}[T] = 0 ~ (\text{for } p > 1) \quad \mathbb{Var}[T] = \frac{p}{p-2} ~ (\text{for } p > 2). \]

Some Properties

  • The student distribution converges towards the standard normal distribution:
    if \(T \sim \mathcal{T}_{p}\), then the distribution of \(T\) converges towards a standard normal distribution when \(p\) goes to infinity.

  • … (see e.g. the wikipedia page).

R functions

The density function of the Student distribution in R is dt.

For information on the distribution function (pt), quantile function (qt) and random generation (rt), see:

help(dt)

Fisher Distribution

Definition

Let \(U_1 \sim \chi^2_{p_1}\) and \(U_2 \sim \chi^2_{p_2}\), be two chi squared random variables, \(U_1\) and \(U_2\) independent. Then \[ F = \frac{U_1/p_1}{U_2/p_2} \sim \mathcal{F}^{p_1}_{p_2} \]

is a Fisher (F) random variable with \((p_1, p_2)\) degrees of freedom.

Density Plots

Moments

\[ \mathbb{E}[F] = \frac{p_2}{p_2 - 2} ~ (\text{for } p_2 > 2) \quad \mathbb{Var}[F] = \frac{2p_2^2(p_1 + p_2 - 2)}{p_1(p_2-2)^2(p_2-4)} ~ (\text{for } p_2 > 4) \]

Some Properties

  • If \(T \sim \mathcal{T}_{p}\), then \(T^2 \sim \mathcal{F}^{1}_{p}\).

  • If \(F \sim \mathcal{F}^{p_1}_{p_2}\), then \(F^{-1} \sim \mathcal{F}^{p_2}_{p_1}\).

  • If \(F \sim \mathcal{F}^{p_1}_{p_2}\), then the distribution of \(p_1 F\) converges to the chi squared distribution \(\chi^2_{p_1}\) when \(p_2\) goes to infinity.

  • … (see e.g. the wikipedia page).

R functions

The density function of the Student distribution in R is df.

For information on the distribution function (pf), quantile function (qf) and random generation (rf), see:

help(df)