Likelihood Ratio Test Statistic [Equation of the Week]
From Protassov et al. (2002, ApJ, 571, 545), here is a formal expression for the Likelihood Ratio Test Statistic,
TLRT = -2 ln R(D,Θ0,Θ)
R(D,Θ0,Θ) = [ supθεΘ0 p(D|Θ0) ] / [ supθεΘ p(D|Θ) ]
where D are an independent data sample, Θ are model parameters {θi, i=1,..M,M+1,..N}, and Θ0 form a subset of the model where θi = θi0, i=1..M are held fixed at their nominal values. That is, Θ represents the full model and Θ0 represents the simpler model, which is a subset of Θ. R(D,Θ0,Θ) is the ratio of the maximal (technically, supremal) likelihoods of the simpler model to that of the full model.
When standard regularity conditions hold — the likelihoods p(D|Θ) and p(D|Θ0) are thrice differentiable; Θ0 is wholly contained within Θ, i.e., the nominal values {θi0, i=1..M} are not at the boundary of the allowed values of {θi}; and the allowed range of D are not dependent on the specific values of {θi} — then the LRT statistic is distributed as a χ2-distribution with the same number of degrees of freedom as the difference in the number of free parameters between Θ and Θ0. These are important conditions, which are not met in some very common astrophysical problems (e.g, one cannot use it to test the significance of the existence of an emission line in a spectrum). In such cases, the distribution of TLRT must be calibrated via Monte Carlo simulations for that particular problem before using it as a test for the significance of the extra model parameters.
Of course, an LRT statistic is not obliged to have exactly this form. When it doesn’t, even if the regularity conditions hold, it will not be distributed as a χ2-distribution, and must be calibrated, either via simulations, or analytically if possible. One example of such a statistic is the F-test (popularized among astronomers by Bevington). The F-test uses the ratio of the difference in the best-fit χ2 to the reduced χ2 of the full model, F=Δχ2/χ2ν, as the statistic of choice. Note that the numerator by itself constitutes a valid LRT statistic for Gaussian data. This is distributed as the F-distribution, which results when a ratio is taken of two quantities each distributed as the χ2. Thus, all the usual regularity conditions must hold for it to be applicable, as well as that the data must be in the Gaussian regime.
Alex:
A couple of points. First, the LR test statistic is applicable for dependent samples (ie time series). In such cases, the likelihood is typically written as the product of densities conditional on the previous observations, ie:
p(D|theta) = p(y_t|theta,y_t-1,…,y_1) * p(y_t-1|theta,y_t-2,…,y_1)*…*p(y_1|theta)
Second, even given the regularity conditions, the LR statistic is not necessarily distributed as a chi-square random variable for finite samples. This distribution holds asymptotically given regularity conditions (as n -> infinity), but it need not hold in finite samples. Likelihood-ratio type tests are actually the basis of many common tests; for example, the one-sided Z test is equivalent to a likelihood ratio test. The use of likelihood ratios tests is largely based on an important result in frequentist testing theory known as the Neyman-Pearson lemma.
06-18-2008, 10:20 pmvlk:
Thanks for those useful clarifications, Alex!
06-18-2008, 10:40 pm