Binomial distribution

The binomial distribution is a discrete probability distribution that models the number of successes in a fixed number of independent Bernoulli trials, each with the same probability of success. It is one of the most fundamental distributions in probability theory and statistics, serving as the basis for the popular binomial test of statistical significance and forming the core of various statistical inference methods.

Definition and Probability Mass Function

The binomial distribution is characterized by two parameters: $n$, the number of independent trials, and $p$, the probability of success in a single trial. A random variable $X$ that follows a binomial distribution is denoted as $X \sim B(n, p)$. The possible values of $X$ are the integers from $0$ to $n$.

The probability mass function (PMF) gives the probability of getting exactly $k$ successes in $n$ trials and is defined by the formula:

$$ P(X = k) = \binom{n}{k} p^k (1 - p)^{n - k} $$

where $\binom{n}{k}$ is the binomial coefficient, representing the number of ways to choose $k$ successes out of $n$ trials. The term $p^k$ represents the probability of $k$ successes, and $(1 - p)^{n - k}$ represents the probability of the remaining $n - k$ failures.

Properties

The binomial distribution possesses several well-defined mathematical properties that describe its central tendency, dispersion, and shape.

Mean and Variance

The expected value (mean) of a binomial random variable $X$ is given by:
$$ E[X] = np $$
This intuitively represents the average number of successes expected over $n$ trials. The variance, which measures the spread of the distribution, is:
$$ \text{Var}(X) = np(1 - p) $$
The standard deviation is simply the square root of the variance, $\sqrt{np(1 - p)}$.

Mode

The mode of a binomial distribution is the value of $k$ that maximizes the probability mass function. It is generally given by the floor of $(n + 1)p$. If $(n + 1)p$ is an integer, the distribution has two modes: $(n + 1)p$ and $(n + 1)p - 1$.

Skewness and Kurtosis

The skewness of the binomial distribution is $\frac{1 - 2p}{\sqrt{np(1 - p)}}$. The distribution is symmetric when $p = 0.5$, positively skewed when $p < 0.5$, and negatively skewed when $p > 0.5$. The excess kurtosis is $\frac{1 - 6p(1 - p)}{np(1 - p)}$, which approaches zero as $n$ becomes large, indicating that the tails of the distribution become similar to those of a normal distribution.

Cumulative Distribution Function

The cumulative distribution function (CDF) of the binomial distribution gives the probability that the random variable $X$ is less than or equal to a specific value $k$. It is expressed as the sum of the probabilities of all outcomes from $0$ to $k$:

$$ F(k; n, p) = \sum_{i=0}^{\lfloor k \rfloor} \binom{n}{i} p^i (1 - p)^{n - i} $$

Alternatively, the CDF can be expressed in terms of the regularized incomplete beta function, $I_x(a, b)$, which is computationally more efficient for large values of $n$:

$$ F(k; n, p) = I_{1-p}(n - k, k + 1) $$

Approximations and Limit Theorems

Under certain conditions, the binomial distribution can be approximated by other continuous or discrete distributions, which simplifies calculations significantly.

Normal Approximation

According to the De Moivre–Laplace theorem, as the number of trials $n$ becomes large, the binomial distribution approaches a normal distribution. A common rule of thumb is that the normal approximation is adequate if both $np \geq 5$ and $n(1 - p) \geq 5$. The approximating normal distribution has a mean of $np$ and a variance of $np(1 - p)$. A continuity correction is often applied to improve the accuracy of this approximation.

Poisson Approximation

When $n$ is large and $p$ is small, such that the product $\lambda = np$ remains moderate, the binomial distribution can be approximated by a Poisson distribution with parameter $\lambda$. This is often referred to as the law of rare events and is highly useful in modeling scenarios like the number of defects in a large batch of products or the arrival of rare events over a fixed interval.

Related Distributions

The binomial distribution is closely related to several other probability distributions:

• Bernoulli Distribution: The binomial distribution is a generalization of the Bernoulli distribution. A Bernoulli distribution is simply a binomial distribution with $n = 1$.
• Poisson Binomial Distribution: If the independent trials do not have the same probability of success, the sum of these trials follows a Poisson binomial distribution. The standard binomial distribution is a special case where all success probabilities are equal.
• Negative Binomial Distribution: While the binomial distribution models the number of successes in a fixed number of trials, the negative binomial distribution models the number of trials needed to achieve a fixed number of successes.
• Beta Distribution: In Bayesian statistics, the beta distribution serves as the conjugate prior for the binomial distribution. This means that if the prior distribution of the success probability $p$ is a beta distribution, the posterior distribution after observing binomial data will also be a beta distribution.

Statistical Inference

In practical applications, the parameter $p$ is often unknown and must be estimated from observed data.

Point Estimation

The most common estimator for $p$ is the sample proportion, $\hat{p} = \frac{x}{n}$, where $x$ is the observed number of successes. This estimator is unbiased and achieves the Cramér–Rao lower bound, making it the minimum variance unbiased estimator (MVUE).

Confidence Intervals

Constructing confidence intervals for the binomial proportion $p$ is a classic problem in statistics. The standard Wald confidence interval relies on the normal approximation but can perform poorly for small sample sizes or proportions near 0 or 1. Alternative methods, such as the Agresti–Coull interval, the Wilson score interval, and the exact Clopper–Pearson interval, are generally preferred as they provide better coverage probabilities.

Applications

The binomial distribution is widely applied across various scientific and industrial fields. In quality control, it is used to model the number of defective items in a sample drawn from a production batch. In medicine and clinical trials, it helps determine the probability of a specific number of patients responding to a treatment out of a total cohort. In ecology, it models the presence or absence of a species across a fixed number of surveyed quadrats. Furthermore, in finance and economics, binomial trees are utilized to model the price evolution of options and other derivative securities over discrete time steps.