Autocorrelation

Autocorrelation, also known as serial correlation, is the correlation of a signal with a delayed copy of itself as a function of delay, serving as a fundamental mathematical tool for identifying repeating patterns, periodicities, or hidden frequencies obscured by noise in time series analysis and signal processing.

Mathematical Definition

The exact definition of autocorrelation varies depending on the field of study and whether the signal is continuous or discrete, deterministic or stochastic. In statistics and time series analysis, the autocorrelation of a stochastic process is typically defined using the Pearson correlation coefficient.

For a wide-sense stationary (WSS) stochastic process $X(t)$ with mean $\mu$ and variance $\sigma^2$, the autocovariance function at lag $\tau$ is defined as:
$C(\tau) = \mathbb{E}[(X(t) - \mu)(X(t+\tau) - \mu)]$

The autocorrelation function (ACF), often denoted as $R(\tau)$ or $\rho(\tau)$, is the normalized autocovariance:
$\rho(\tau) = \frac{C(\tau)}{\sigma^2} = \frac{\mathbb{E}[(X(t) - \mu)(X(t+\tau) - \mu)]}{\sigma^2}$

In signal processing, for a deterministic continuous-time signal $x(t)$, the autocorrelation is often defined as the integral of the product of the signal and its time-shifted complex conjugate:
$R_{xx}(\tau) = \int_{-\infty}^{\infty} x(t) x^*(t-\tau) dt$

For discrete-time signals $x[n]$, the definition becomes a summation:
$R_{xx}[k] = \sum_{n=-\infty}^{\infty} x[n] x^*[n-k]$

Properties

The autocorrelation function possesses several fundamental mathematical properties that make it highly useful in analytical applications:

1. Symmetry: For real-valued signals, the autocorrelation function is an even function, meaning $R(\tau) = R(-\tau)$. For complex-valued signals, it exhibits Hermitian symmetry: $R(\tau) = R^*(-\tau)$.
2. Maximum at Zero Lag: The autocorrelation reaches its absolute maximum at zero lag ($\tau = 0$). This value corresponds to the total energy of a deterministic signal or the variance (plus mean squared) of a stochastic process. No other lag can produce a higher correlation value.
3. Wiener-Khinchin Theorem: For a WSS random process, the autocorrelation function and the power spectral density (PSD) form a Fourier transform pair. This means the PSD can be obtained by taking the Fourier transform of the autocorrelation function, and vice versa. This property is foundational in spectral analysis.
4. Linearity: The autocorrelation of the sum of uncorrelated signals is equal to the sum of their individual autocorrelations.

Estimation and Computation

In practical applications, the true autocorrelation function of a stochastic process is unknown and must be estimated from a finite sample of data. For a discrete time series of $N$ observations $x_1, x_2, \dots, x_N$, the sample autocorrelation at lag $k$ is commonly estimated as:

$\hat{\rho}(k) = \frac{\sum_{t=1}^{N-k} (x_t - \bar{x})(x_{t+k} - \bar{x})}{\sum_{t=1}^{N} (x_t - \bar{x})^2}$

where $\bar{x}$ is the sample mean. This estimator guarantees that the resulting autocorrelation sequence is positive semi-definite and that $\hat{\rho}(k) \le 1$. However, it is a biased estimator. An unbiased estimator divides the numerator by $N-k$ instead of $N$, but this can lead to a covariance matrix that is not positive semi-definite, which is undesirable in many modeling contexts.

For large datasets, direct computation of the sample autocorrelation requires $O(N^2)$ operations. To improve computational efficiency, the Wiener-Khinchin theorem is leveraged alongside the Fast Fourier Transform (FFT). By computing the FFT of the signal, multiplying it by its complex conjugate to obtain the power spectrum, and then applying the inverse FFT, the autocorrelation can be computed in $O(N \log N)$ time.

Applications

Autocorrelation is a versatile tool utilized across numerous scientific and engineering disciplines.

Signal Processing and Acoustics: It is used to determine the pitch of musical notes or human speech by identifying the fundamental frequency of a periodic waveform. It is also employed in radar and sonar systems to measure the distance and velocity of targets by analyzing the time delay of reflected signals.

Econometrics and Time Series Analysis: In financial and economic data, autocorrelation helps identify trends, seasonality, and cyclical behaviors. It is a core component in identifying the parameters of Autoregressive Integrated Moving Average (ARIMA) models, where the autocorrelation function (ACF) and partial autocorrelation function (PACF) are plotted as correlograms to determine model orders.

Physics and Optics: In dynamic light scattering (DLS) and photon correlation spectroscopy, the autocorrelation of scattered light intensity fluctuations is used to determine the size distribution of nanoparticles or macromolecules in a suspension. In ultrafast optics, intensity autocorrelation is the standard technique for measuring the duration of femtosecond laser pulses.

Astronomy: The spatial autocorrelation of galaxy distributions (the two-point correlation function) is used to study the large-scale structure of the universe and the underlying dark matter distribution.

Autocorrelation in Regression Analysis

In the context of linear regression, one of the fundamental assumptions of the Ordinary Least Squares (OLS) method is that the error terms are uncorrelated. When the residuals exhibit autocorrelation (serial correlation), this assumption is violated.

The presence of autocorrelated errors does not bias the OLS coefficient estimates, but it renders them inefficient, meaning they no longer have the minimum possible variance. More critically, the standard errors of the coefficients are typically underestimated, leading to inflated t-statistics and an increased risk of Type I errors (false positives).

To detect autocorrelation in regression residuals, statisticians use tests such as the Durbin-Watson test or the Breusch-Godfrey test. If autocorrelation is detected, remedies include using Generalized Least Squares (GLS), incorporating lagged dependent variables into the model, or applying heteroskedasticity and autocorrelation consistent (HAC) standard errors, such as the Newey-West estimator, to correct the inference without altering the coefficient estimates.

Related Concepts

Cross-correlation: While autocorrelation measures the similarity of a signal with itself at different lags, cross-correlation measures the similarity between two distinct signals as a function of the displacement of one relative to the other.

Partial Autocorrelation: The partial autocorrelation function (PACF) measures the correlation between a variable and its lagged self, controlling for the values of the variable at all shorter lags. It is particularly useful in identifying the order of an autoregressive (AR) model.

Autoregressive (AR) Models: These are time series models where the current value of a variable is expressed as a linear combination of its past values plus a stochastic error term. The theoretical autocorrelation structure of an AR process decays exponentially or oscillates, which distinguishes it from moving average (MA) processes.