Comments on: What is so special about chi square in astronomy?

By: hlee

hlee — Mon, 16 Jul 2007 18:06:50 +0000

All doubts are originated from my interests in multi-modality of globular clusters. Astronomers used luminosity functions (LF), which is in general represented by a histogram because of binning, and they visually identified multi-modality to explain multiple epochs of star formation history of a galaxy. Then, later Ashman, Bird and Zepf (1994) introduced the likelihood approach to resolve/prove the bimodality problem statistically. However, as Protossov, et. al. (2002) pointed out, there are regularity conditions such as finite expectation and identifiability, and I've seen papers on globular clusters using likelihood ratio tests (LRT) for the hypothesis testing of say, 2 generations of clusters vs 3 generations of clusters with gaussian mixture models, where these regularity conditions are violated and LRT cannot be applied for such a hypothesis testing.

Not all statistics are robust until someone proved its robustness case by case. The part that annoys me about the chi-square is out of sudden, researchers said, "due to chi-square..." What if the conditions on those chi-square methods do not satisfy the nature of the data set as some astronomers used LRT where it cannot be applied for the hypothesis testing on their data sets but already applied to other data sets of the same object type satisfying the mathematical conditions? Because of its fame and convenience, for me at least (most?), people tend to use chi-square blindly.

By: hlee

hlee — Mon, 16 Jul 2007 15:50:22 +0000

I never implied chi-square is wrong. Among so many statistics, I wonder how chi-square “almost” only caught astronomers’ eyes. I wanted to get rid of my prejudice by knowing what makes chi-square special to astronomer.

By: vlk

vlk — Sat, 14 Jul 2007 13:49:47 +0000

In response, perhaps I should ask, what is wrong with good old chi-square? As you point out, it is just a statistic, and has a meaning that is easily comprehensible even to statistically illiterate astronomers (to wit, the fractional deviations of the model from the data, relative to the inherent error, square added) — a very nice quantity to carry around in and of itself, without reference to its standard form distribution. That the distribution is also valid in a majority of the cases is a plus. With the advent of high speed computers, even when the chi-square function is not distributed as a chi-square distribution, it is fairly easy to calibrate it on a case by case basis. The ease of comprehension, ease of computation, and the wide applicability are significant advantages and outweigh the few cases where it is inapplicable. Admittedly, with the advent of telescopes like Chandra, those “few cases” are increasing. I agree that astronomers should pay attention to why they are using this particular function, but I should also say that what they are doing is usually not such a bad thing.