[ArXiv] Data-Driven Goodness-of-Fit Tests, Aug. 1, 2007
From arxiv/math.st:0708.0169v1
Data-Driven Goodness-of-Fit Tests by L. Mikhail
Goodness-of-Fit tests have been essential in astronomy to validate the chosen physical model to observed data whereas the limits of these tests have not been taken into consideration carefully when observed data were put into the model for estimating the model parameters. Therefore, I thought this paper would be helpful to have a thought on the different point of views between the astronomers’ practice of goodness-of-fit tests and the statisticians’ constructing tests. (Warning: the paper is abstract and theoretical.)
This paper began with presenting two approaches to constructing test statistics: 1. some measure of distance between the theoretical and empirical distributions like the Cramer-von Mises and the Komogorov-Smirnov statistics and 2. score test statistics, constructed in a way that the tests is asymptotically normal. As the second approach is preferred, the author confined his study to generalize the theory of score tests. The notion of the Neyman type (NT) test was introduced with very minimal assumptions to shape the statistics.
The author discussed the statistical inverse problems or the deconvolution problems of physics, seismology, optics, and imaging where noisy signals and measurements occur. These inverse problems induce the Neyman’s type statistics under appropriate regularity assumptions.
Other type of NT tests in terms of score functions and their consistency was presented in an abstract fashion.
Mikhail Langovoy:
Thank you very much for reviewing my article!
I think that up to now I haven’t seen a completely unbiased overview of how statistical tests of different types should be applied to a data from astronomy. If I will see such a paper, I will post the link here.
In general, I have an impression that there are still some powerful tests that weren’t used in statistical problems of astronomy yet. Also, I think that there are still many statistical problems in astronomy such that the statisticians haven’t constructed optimal tests for these problems. In principle, it is the aim of the theory of statistical inverse problems to provide such testing procedures. But this theory is still under development. There remains a lot of work both for theoreticians and practitioners.
—[hlee response] Thank you for your comment and promise for sharing information. It would be very valuable to know theoretical perspective of applying goodness-of-fit tests incorporating information from data. Relying on my impression, there’s not much understanding of powerful tests or optimal tests in the astronomical community mainly due to the complexity of physical models and data process.
09-24-2007, 2:07 pm