Some Continuous Distributions-I (P-8)

We are going to discuss the continuous distributions in this blog. I will provide you some of their basic properties with pdfs of each without going into a lot of details. The topics of discussion are as follows:

Normal Distribution
Lognormal Distribution
Cauchy Distribution
Gamma Distribution
Beta Distribution

Normal Distribution

The normal distribution is the most widely used distribution in the statistics and many other applications from communication to finance, from sociology to all forms of science. It starts from the work of Abraham de Moivre who studied it as an approximation tool for binomial distribution, later which became Laplace-de Moivre theorem, an initial version of the central limit theorem. Also the great Gauss studied it as he was studying the movements of the planets and he figured out that the errors in measurement follow the normal curve or distribution, from then onwards many natural phenomenon were shown to follow normal distribution approximately. And in finance and many other fields modifications (read also as transformations) of normal distributions are generally good enough for the modeling purposes. It only requires two parameters to be defined, mean and variance or standard deviation. The pdf of normal distribution is given by, $f(x;\mu,\sigma)=\frac{1}{\sqrt{2\pi}\sigma}\exp^{\frac{-(x-\mu)^2}{2\sigma^2}}$ for $-\infty <x<\infty, -\infty<\mu<\infty, \sigma >0$ . It is a good exercise in integration to verify that this forms a probability density function (pdf in short),i.e.,

$\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi}\sigma}\exp^{\frac{-(x-\mu)^2}{2\sigma^2}} dx=1$

When $X$ follows a normal distribution, this is generally denoted by $X ~n(\mu,\sigma^2)$ . When $\mu$ and $\sigma$ are $0$ and $1$ respectively, it is called as standard normal. As you must have guessed that $E[X]=\mu$ and $Var(X)=\sigma^2$ . This pdf has iconic bell shape symmetric about the mean (which is same as the mode and median). As it is a symmetric distribution the skewness for it is zero and kurtosis value is $3$ (excess kurtosis = kurtosis – 3= 0, which is generally what you get in stat programs, so check carefully what is meant by kurtosis in your application). I wanted to write a blog on these but there is this beautiful blog explaining these two concepts in detail, I couldn’t have done better job than that.

Lognormal Distribution

The lognormal distribution basically means a distribution whose log is normal, i.e. we say a random variable $X$ follows a lognormal distribution if $Y=log X$ follows a normal distribution. The pdf for this is given by :

$f(x;\mu, \sigma)= \frac{1}{\sqrt{2\pi}\sigma x}e^{\frac{-(\ln{x}-\mu)^2}{2\sigma^2}}$ , where $0<x<\infty, -\infty<\mu<\infty, \sigma >0$ .

It is also defined using the mean and variance of the underlying normal distribution. It is an important distribution in finance because it is generally assumed that $log$ of stock price returns follows normal distribution, log(income_returns) follows normal distribution. So it is a good distribution when we need to model right-skewed observations. What is the expected value, $E[X]$ for this distribution? One might guess that it should be $e^{\mu}$ but actually that is not the case, it is $e^{\mu + \frac{\sigma^2}{2}}$ . In any case $e^{\mu}$ is important because it is the median for this distribution. The variance is given by $Var(X)= e^{2(\mu+\sigma^2)}-e^{2\mu+\sigma^2}$ . Suppose the stock price at time zero is given by $S_0$ and time $T$ by $S_T$ and let $r$ denote the continuously compounded growth rate of the stock between time zero and $T$ . Then we have $S_T=S_0 e^{rT}$ , further $\frac{S_T}{S_0}=e^{rT}$ . Taking natural log ln on both sides, $\ln{\frac{S_T}{S_0}}=rT$ . If we assume that the rate of growth $r$ is normally distributed then the stock return between time zero and $T$ is lognormally distributed.

Blue vertical line represent expectation of the blue curve while black vertical line is the median of all

Cauchy distribution

It is one of the interesting distribution in the sense that the mean (in fact, no moments exist) does not exist. It has bell-shaped symmetric curve centered at the median which in the following diagram is at the zero. It is similar to normal but it has thick tails. The general form of the pdf is given by the two parameters, $\mu$ and $\sigma$ where $\mu$ is the median of the distribution while $\sigma$ is called the scale, note that it is not variance. The pdf is as follows:

$f(x;\mu,\sigma)=\frac{1}{\sigma \pi(1+(\frac{x-\mu}{\sigma})^2)}$ , $-\infty<x<\infty, -\infty<\mu<\infty, \sigma>0$ .

When $\mu=0$ and $\sigma=1$ , we get standard Cauchy distribution. Let’s show that for standard Cauchy distribution mean does not exist. If mean exists then it could be either finite or infinite.

$E[X]=\frac{1}{\pi}\int_{-\infty}^{\infty}\frac{x}{1+x^2} dx=\frac{1}{\pi}\int_{0}^{\infty}\frac{x}{1+x^2} dx+\frac{1}{\pi}\int_{-\infty}^{0}\frac{x}{1+x^2} dx$ ,

here the first part of the integral is infinity and also the second part of the integral is negative infinity, so that the mean is neither finite nor it is infinite, hence it is undefined. On such kind of distributions, one can not apply Central limit theorem.

Actually there is interesting fact that this distribution was first published by the mathematician Poisson but Cauchy another mathematician got involved in some controversy and got the distribution named after him. This is an instance of Stigler’s law of Eponymy which basically says that no discovery is named after its actual discoverer, for examples you can see this list.

Gamma Distribution

The general form of the gamma distribution is given by the two parameters, $\alpha$ and $\beta$ often called as the shape and rate parameters respectively. The pdf is,

$f(x;\alpha,\beta)=\frac{1}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x}$ , where $0<x<\infty$ , $\alpha>0$ , $\beta >0$ and also $\Gamma(\alpha)$ is our familiar gamma function given by $\Gamma(\alpha)=\int_0^{\infty} t^{\alpha-1}e^{-t} dt$ .

It is one of the useful family of distributions, many distributions are formed using this distributions which would be the topic our next blog on continuous distributions, for example, for $\alpha=1$ we get exponential distributions for distinct values of $\beta$ . One can easily calculate the expectation and variance for this by solving the integrations. These are given as follows:

$E[X]=\alpha \beta$ and $Var(X)=\alpha \beta^2$ .

Beta Distribution

The beta distribution is a continuous distribution on $(0,1)$ given by two parameters $\alpha$ and $\beta$ . This distribution is denoted by $beta(\alpha, \beta)$ and it is given by:

$f(x;\alpha,\beta)=\frac{1}{B(\alpha,\beta)}x^{\alpha-1}(1-x)^{\beta-1}$ , $0<x<1,$ $\alpha >0$ , $\beta >0$ , where $B(\alpha,\beta)$ denotes the beta function,

$B(\alpha,\beta)=\int_0^1 x^{\alpha-1}(1-x)^{\beta-1} dx$ .

The expectation and variance of this is given by the following:

$E[X]=\frac{\alpha}{\alpha+\beta}$ and $Var(X)=\frac{\alpha\beta}{(\alpha+\beta)^2(\alpha_\beta+1)}$ .

It is used as conjugate prior for binomial distribution, where conjugate priors are prior probabilities used in Bayesian inference, because the posterior probability is again has the form of the beta distribution.