The Role of Model Assumptions in Statistics
Christian Hennig (UCL)
Friday 12th October, 2018 15:00-16:00 Maths 311B
In Statistics we learn that statistical methodology relies on certain model assumptions, and that we better try to make sure that these are fulfilled if we want to apply the methodology.
A famous statement by George Box is that "all models are wrong but some are useful". If we believe this, it actually seems hopeless to "make sure that model assumptions are fulfilled". Some would think that model assumptions could at least be "approximately" fulfilled and that we could test this. Surprisingly, Box's statement rules this out as well, at least if we take "model M holds approximately" to mean that some other model N holds (which, according to Box, is wrong) and M is in some sense close to N.
I agree with Box and I think that we use models not because they would be "true" or even "approximately true", but rather because they provide us with potentially useful ways of thinking about phenomena that we perceive as "random".
But this implies that the act of "model checking" and "model testing" doesn't do what it claims to do. We know that the model is false before checking it, so does checking it still make sense? What we actually should check is whether the model can be useful for us, or to what extent the data indicate that the method that we'd like to apply may be led astray.
I will present some elements of a philosophy of statistics and data analysis that attempts to make the role of models and the meaning of "checking them" clearer.
I will touch on how mathematical models are connected to reality, on observability and identifiability, the "goodness of fit"-paradox (meaning that, under some conditions that are almost always fulfilled, the act of checking model assumptions violates them automatically), on the necessity of making subjective decisions (other than choosing priors), and on the old controversy between frequentists and (various schools of) Bayesians.
C. Hennig: Falsification of propensity models by statistical tests and the goodness-of-fit paradox. Philosophia Mathematica 15: 166-192, 2007.
C. Hennig: Mathematical models and reality - a constructivist perspective. Foundations of Science 15: 29-49, 2010.
A. Gelman and C. Hennig: Beyond subjective and objective in statistics. Journal of the Royal Statistical Society Series A 180: 967–1033 (with discussion), 2017.