When researchers focus on model design, they often worry whether the model is correct or not. I believe that we should accept the fact that all the models are wrong. The world is too complex to be captured by a single model: we are never able to acknowledge all the variables. Therefore, the answer to the question “Is the model correct?” is always “No”. It should not bother us: from the pragmatic perspective, it is irrelevant whether the model is correct or not. If we embrace the model misspecification, we can switch our attention to the question “What is the impact of deviations from the model on the decision-making?”
Recently, I was reading cerreia2020. I am still in the process of understanding the technical part, but I was charmed by the Introduction, so I want to share quotes I liked from this paper and referenced box1976 and chatfield1995.
The consequences of a decision may depend on exogenous contingencies and uncertain outcomes that are outside the control of a decision maker. This uncertainty takes on many forms. Economic applications typically feature risk, where the decision maker knows the correct probabilistic model governing the contingencies but not necessarily the decision outcomes. Yet, this is a demanding assumption. As a result, statisticians and econometricians have long wrestled with how to confront ambiguity over models or unknown parameters within a model. Each model is itself a simplification or an approximation designed to guide or enhance our understanding of some underlying phenomenon of interest. Thus, the model, by its very nature, is misspecified, but in typically uncertain ways. How should a decision maker acknowledge model misspecification in a way that guides the use of purposefully simplified models sensibly? This concern has certainly been on the radar screen of statisticians and control theorists, but it has been largely absent in formal approaches to decision theory.
— Simone Cerreia-Vioglio, Lars Peter Hansen, Fabio Maccheroni, Massimo Marinacci Making Decisions Under Model Misspecification (2020) / The Models Are Misspecified
The protagonist of our analysis is a decision maker who is able to formulate models – for instance a policy maker having to decide a climate policy based on existing alternative climate models – but is concerned about their misspecification and wants to use a decision criterion which accounts for that. Our axiomatic analysis, which has a normative nature, aims to derive a criterion of this kind to help the decision maker to cope with model misspecification in a principled way.
— Simone Cerreia-Vioglio, Lars Peter Hansen, Fabio Maccheroni, Massimo Marinacci Making Decisions Under Model Misspecification (2020) / Decision Maker Protagonist
Since all models are wrong the scientist must be alert to what is importantly wrong. It is inappropriate to be concerned about mice when there are tigers abroad.
— George E. P. Box Science and Statistics (1976) / What is Importantly Wrong
Finally it does not seem helpful just to say that all models are wrong. The very word model implies simplification and idealization. The idea that complex physical, biological or sociological systems can be exactly described by a few formulae is patently absurd. The construction of idealized representations that capture important stable aspects of such systems is, however, a vital part of general scientific analysis and statistical models, especially substantive ones (Cox, 1990), do not seem essentially different from other kinds of model.
(D. R. Cox, Comments)
— Chris Chatfield Model Uncertainty, Data Mining and Statistical Inference (1995) / It Does Not Seem Helpful Just to Say that All Models Are Wrong