[Quote] When all the models are wrong
From page 103 of Bayesian Model Selection and Model Averaging by L. Wasserman (2000) Journal of Mathematical Psychology, 44, pp.92-107
… So does it make sense to compare any finite list of models when we do not literally believe any of them? The answer is: sometimes.
First, we would hope that, while none of the models is exactly correct, at least one is approximately correct. It behooves the data analyst to do common sense exploratory analysis — checking residuals, for example — to make sure that not all of the models are heinously wrong. [---snip---]
Second, even when all models are wrong, it is useful to consider the relative merits of two models. Newtonian physics and general relativity are both wrong. Yet it makes sense to compare the relative evidence in favor of one or the other. …
Any discussion between Newtonian physics and general relativity had been a very grief subject for physicists and philosophers. For nowadays statisticians, it’s just another example.
TomLoredo:
“Newtonian physics and general relativity are both wrong.”
While I agree in large part with Larry’s sentiments, I think he chose a bad example here. GR is probably the only physical theory that is “not wrong,” in the sense that there isn’t a shred of observable evidence that contradicts it (as far as I know), despite lots of looking. The challenge is that it’s a theory for the weakest force we know of, so it’s hard to really probe its limits in laboratory (or even astrophysical) settings. I think nearly all physicists think it (and all theories) must be “wrong” in some regime, but it’s beginning to get embarassing (from a philosophical perspective) that we can’t show it. Put another way, we believe GR is wrong, but don’t yet know it’s wrong.
“Electromagnetism and QED are both wrong” might have been a better example, in the sense that we know of regimes where both theories fail to describe nature accurately.
Harold Jeffreys made a related observation in his book, Probability Theory, many years ago, pointing out that there was probably seldom (if ever) a time in history when a Newtonian model of planetary motion would be found to be acceptable by a significance test (either due to classical model limitations such as assuming sphericity or neglecting many-body effects, or more recently, due to GR corrections). But at no time were scientists prepared to abandon Newtonian theory (there being nothing to replace it until GR).
02-20-2008, 3:25 pm