How can we define "Useful", when it comes to our models of reality?

Wikimedia Commons: "N-Gauge Cassiopeia E26 & EF81 from Kato"

A comment to "Useful models, model checking, and external validation: a mini-discussion" by Andrew Gelman - Statistical Modeling, Causal Inference, and Social Science

Gelman wrote (with Cosma Shalizi) a very fine philosophical justification for real-world, effective Bayesian techniques, which differs greatly from the usual philosophy associated with Bayesians.

Gelman, Shalizi: Philosophy and the practice of Bayesian statistics in the social sciences

My own recurring criticism of Gelman is of using Bayesian/statistical models to the exclusion of others.  I am more comfortable with the idea of models of different type in competition.

http://manuelmoeg.blogspot.com/search/label/Model%20building

The multiple models you then have will now compete in different uses - based on predictive power, accuracy, ability to calculate meaningful error ranges, cost of collecting data, cost of computation, cost of comparison, ability to predict outcomes from interventions, cost of understanding, etc.

quoting Gelman:

"We always talk about a model being "useful" but the concept is hard to quantify."

My comment:
Simply build a model of costs and gains and methods of comparison between models!  If a model is good enough for your work, a model must be good enough as a working definition of "useful"!

Sometimes the best answer to "Why" is "Just because".  Sometimes the best mechanism for rating different models is another model.  The Skeptics will always howl, so you simply have to demonstrate that their own behavior is consistent with putting undue confidence in their own model, whether a conscious model or unconscious.  (And, it must simply terminate with a model, because of the limits of the tools available to the human brain.  Only a model, probably over-simplified, can be manipulated with the agility needed to predict future outcomes of the universe from actions considered now, in real-time.)

Just keep asking the Skeptic "Why" with regards to their own personal actions, and when they hit the "Just because" point, they probably have described a model of utility, assumed true without proof, as an answer to "Why" in the previous step.

If the Skeptic refuses ultimate responsibility over their personal actions, and tries to plead pure capriciousness or mystery, then their model is simply statistical, based on stimulus and internal states (like stress) that can be approximately discovered with objective external measures (like galvanic skin response).  Of course, it is easier to plead pure capriciousness or mystery than demonstrate it - if their behavior is well predicted by a deterministic model suggested by another, the Skeptic is shut up.  Most times the reason for behaviors is gross and banal, no matter how elevated the sophistry of the Skeptic.

Can it be Irrational to prize Rationality? What is Rationality?

http://en.wikipedia.org/wiki/Melancholia

I am a big fan of Scott Plous's The Psychology of Judgment and Decision Making, because not only does it call out cognitive failure modes, but it also suggests remedies.  The book is written in a non-technical style, so uses the conventional language of modes of thought being "Rational" or "Irrational", and "biases" leading to "Irrational" decisions.

I found a blog post "Is postdecisional dissonance functional?" that takes exception to calling "Post-decision dissonance" irrational (post-decisional dissonance is where the self-judged value of a chosen item increases, and the value of a declined item decreases, compared to the self-judged values before the choice is made: http://en.wikipedia.org/wiki/Cognitive_dissonance#Post-decision_dissonance ).

"Is postdecisional dissonance functional?" seems like a yes/no question, but the answer can change from situation to situation.  We can construct a situation where this bias is "Irrational/dysfunctional", or is "Rational/functional".

Example: If postdecisional dissonance is the way that one "stops" the decision process, instead of endlessly revisiting a decision and wasting time and energy, then postdecisional dissonance is functional (this is the point raised by Konrad Talmont-Kaminski).  If postdecisional dissonance keeps you from switching decisions when later you are offered the alternate choice along with a small but real payment, because you deny yourself the additional payment even though the options were judged to be identical in value, postdecisional dissonance is dysfunctional.

Which is the most likely scenario?  What is the cost of a more rigorous and rational analysis?  Different answers from subtle changes to these questions...

All of these biases, because they are manifest in humans today, cannot absolutely prevent reproductive success or success in cultural transmission of ideas, obviously.  So you are on very shaky ground calling these biases non-adaptive.  And if you cannot call them non-adaptive, what is the exact basis for calling them "Irrational/dysfunctional"?

Modeling, instead of using the language of Bias and Rationality and Functionality

George Mason University,
Dept of Statistics,
Gallery of Great Statisticians,
George E. P. Box
http://statistics.gmu.edu/pages/famous.html

That is why it is not always wise to use the culturally defined notion of rationality, or assume an implied sound situationally defined notion of rationality, and why *sometimes* there is benefit to specifically stating:

(1) the failure mode of decision that you are trying to avoid and

(2) how you are modeling the
(2A) cost of falling victim the failure mode and the
(2B) cost of remedy

(3) how you are modeling the likelihood of different scenarios taking place.

And different models will give different answers.  As George E. P. Box says "All models are false but some models are useful."

[Edit 7/29/2010]

Very helpful reply [ http://deisidaimon.wordpress.com/2010/07/25/is-postdecisional-dissonance-functional/#comment-1675] from academic Konrad Talmont-Kaminski, but my profound ignorance prevents me from getting much from it.  I am self-taught exclusively from an engineer's perspective of decision making from Decision Analysis texts [ term coined in 1964 by Ronald A. Howard ].

I fixed the post above, to add

1) specific examples as to how postdecisional dissonance can be functional or dysfunctional,

2) why one is unjustified to call manifest biases non-adaptive, and

3) the need to model the likelihood of different situations arising, or else the the analysis is nonsensical.

[Edit #2 7/29/2010]


Konrad Talmont-Kaminski recommends the writings of Herbert Simon, Nobel Prize winner in Economics 1978.

All models are wrong, but some are useful

Image by kevindooley via Flickr

"all models are wrong, but some are useful."

-- George E. P. Box

reddit.com/r/science/comments