Showing posts with label Scientific consensus. Show all posts
Showing posts with label Scientific consensus. Show all posts

Friday, February 12, 2010

Andrew Gelman and Climate Change Concern Trolling

New Troll and Old Troll (Front)Image by Dunechaser via Flickr
Andrew Gelman's blog [ http://www.stat.columbia.edu/~cook/movabletype/mlm/ ] has some of the best writing on climate change when Phillip Price submits a post [ see http://manuelmoeg.blogspot.com/2009/12/talking-about-climate-change-publishing.html ] and has some of the worst writing when Andrew Gelman himself posts.

Andrew Gelman seems to be sympathetic to conspiratorial thinking about the scientific culture around climate change.  My feeling is that in his own field of statistics, Bayesian techniques have been actively suppressed and misrepresented, so Gelman is open to the idea that investigators who don't see human global warming could be actively suppressed and misrepresented too.

Which is fine, if the arguments were not so lame.

http://www.stat.columbia.edu/~cook/movabletype/archives/2010/02/stabilizing_fee.html
(Anonymous concern troll says any paper contradicting Arrhenius' 1896 climate model is likely to be self suppressed. And if this is not the point of the anonymous question, what is?)
http://www.stat.columbia.edu/~cook/movabletype/archives/2009/12/how_do_i_form_m.html
(Before Andrew Gelman steps onto a subway train, he ponders that the civil engineers that designed the train might be laboring under thoughts so stupid that they can only come from some aspect of the civil engineering consensus in a particularly ugly undigested form. And he is paralyzed by fits of panic. And if this is not the point of Gelman's "Beyond my limited sphere of scientific compresension, I dunno", what is?)

It could be more convincingly argued that:

(1) human activity is making an ice age less likely

(2) global warming skepticism and preference for inaction plays a semi-rational role in the debate, not because of the poor quality of their arguments, but as a brake against premature solutions:

365 Days - Day 71 - Hippy Tree HuggersImage by Auntie P via Flickr
Al Gore holding up a mercury light bulb in "An Inconvenient Truth",
Ed Begley, Jr. installing semiconductor solar panels to all exposed surfaces of his home,
the US government giving loans to Tesla electric cars

... all of which are very likely contributing to burning *more* fossil fuels, not less, because the total costs over those product's entire lifetime (including manufacturing plants that themselves burn fossil fuels, including safe disposal of wastes) are unrepresented by the relatively small price charged to consumers.  Better to tax fossil fuels in rich countries and spend that money on research placed in the public domain so even the poorest countries can benefit.  At a gradual rate of increase, so as to have minimum harm to productivity.

But, instead, we get lame concern trolling about scientific conspiracies.  I get the tiresome feeling that the global warming skeptics need to be saved from themselves, to have the people convinced by the scientific consensus to make their best arguments for them, because they cannon help but simply echo the last thing they heard from someone with a direct financial link to oil companies.  Tiresome.

My reply to the anonymous concern troll:

> If the prediction of a climate model is very much outside the consensus predictions, it is not likely to be published.

More arguing that climate science is a nonesuch science.  Taken to the logical extreme, we can argue that Einstein's papers on Special and General Relativity are not likely to be published (and that is why we still use epicycles today).  Taken to the logical extreme, we can posit that Alex Rodriguez is not likely to swing for the fences.  The blockbuster behavior of the players in the 99.999% percentile is poorly predicted by the tentative behavior of the average player.

Secondly, science publishing is not the only market for climate modeling.  Commodities traders and the reinsurance market for hedging risk on multi-year massive construction projects have a need for accurate climate modeling, because on those time scales a long range weather report would be worthless.  Those players are willing to leave millions on the table just so their hired gun scientists can parrot safe results that are unlikely to rattle tea cups at the next faculty function?  Unlikely.

I beg your forgiveness for the following snarkiness.  Can your anonymous concern troll name a single branch of science that has remained on a strictly linear trajectory since 1896?  Besides phrenology.

[ Edit 2/15/10 ]

The goalposts have been moved in the comments is Andrew Gelman's post on February 12.  It moved to always having the models subjected to a growing set of data, never casting out past data (good, good).  It moved to meta-analysis of all available models, over time (good, good).  It moved to the variance of published models compared to the subjective guessed distributions of the individual practicing scientists (fine, fine).  But where happened to the original claim "If the prediction of a climate model is very much outside the consensus predictions, it is not likely to be published."?

My comment submitted:
Marc Levy

> They find that if you ask climate experts to characterize their subjective best guess as to the distribution of key climate change parameters, you observe far more variance ... than you observe when you look at the distribution of all the climate model outputs.


This is to be expected, because no one would represent *any* model as perfectly describing reality - if it was perfect, it would no longer be a model, anymore.  Only pure mathematics has the benefit of being able to switch the analysis to a proved isomorphism that is easier to compute.  Every model is an adequate simplification, over a domain, and it is hoped the failure modes are understood so the model is not misused.  But the option of "proving" the model a perfect representation of reality is not available.

Useful scientific models typically give sharp results - sharper results than field readings, even counting for input precision or rounding in iteration, etc.  The models are useful *because* they give sharp results - or else you would have the perverse consequence of improving the usefulness of a model by adding slop into it to increase the variance.  A bound on the error is useful to track, but no one would actually mix in slop in a model to force the variance wider, even if the model's variance doesn't match field readings.

Any expert would know very well all the possible failure modes and other limitations of a particular model, that so their subjective guessed distribution would have greater variance than the considered model because of that knowledge.  The scientist possess what humans value as knowledge, the documented model cannot (and so scientists cannot be replaced with the models of their creation).  Why else might the variance be greater?  -- perhaps the scientist is in possession of they consider to be a better model, not yet published.  Or perhaps, the scientist is simply aware of the possibility of a better model.

The relatively uncontroversial model of satellite orbits is informative.  They are tighter because they, of course, consider fewer particles than Mother Nature is able to consider.  So they can consider events in the future, because they run faster than reality, and so they can run on economically available hardware.  No one would consider their tighter variance than the variance of observatory readings to be surprising, much less consider it a failure of the model.  Only if there was misrepresentation of best knowledge of the model's error bound, or failure modes, or applicable domain, and then, it would not be a failure of the model, it would be a misapplication by a human agent.

Can I note that the goalposts have been moved?  The original issue was "stabilizing feedback" and the original question contained the assertion "If the prediction of a climate model is very much outside the consensus predictions, it is not likely to be published."  There are other interesting issues to consider, but only after the parties admit that the goalposts have been moved and the focus of the argument shifted.
Reblog this post [with Zemanta]

Monday, December 14, 2009

How do I form my attitudes about scientific questions? - Statistical Modeling, Causal Inference, and Social Science


Crank it up!!Image by De Shark via Flickr
Andrew Gelman at Statistical Modeling, Causal Inference, and Social Science.

How do I form my attitudes about scientific questions? - Statistical Modeling, Causal Inference, and Social Science: "
It's not that the scientific consensus is stupid, it's that some statements are so stupid that they only come because the speaker has processed some aspect of the consensus in a particularly ugly undigested form.
...
...To me, it's another case where the existence of the consensus has switched off people's brains.
"

This point is valid, and well put, but if you read the whole post, I think Andrew Gelman is being far too pessimistic.

> "What do I recommend you all do? On subjects where Phil and I are the experts, I suggest you listen to what we have to say. Beyond that, I dunno."

This is very pessimistic and skeptical of considered consensus, and contradicted by Andrew Gelman's daily life. Before I step into a subway train, I don't form opinions about the quality of considered consensus of civil engineers, and Mr. Gelman does not either.

Commenter "jonathan" makes the point:

> I think you've raised two separate issues. One is the process by which consensus builds, entrenches, shifts, etc. The other is how rational people make rational decisions about information.
> It's interesting to me how in a few notable areas the two are lumped together: the idea that biologists are maintaining some (evil) consensus in favor of evolution and that climate scientists, etc. are doing the same with regard to climate change.
> ...


A highly resolved Tree Of Life, based on compl...Image via Wikipedia
If you step back and compare "Skepticism of Human Activity Causing Global-Warming/Climate-Change" to established cases of motivated obscurantism, like denying evolution and natural selection, and tobacco carcinogenicity, and the Jewish Holocaust of WWII, and the efficacy of the polio vaccine, and perhaps less established cases of motivated obscurantism like controlled demolition taking down the Twin Towers and HIV/AIDS, you see familiar patterns and similar techniques and motivations both sinister and innocent-by-way-of-ignorance/gullibility. It will seem like bad form to the self-described "Skeptics", but they could bring doubters into their fold by work - the work of authoritatively publishing their opposing immutable thesis, and welcoming that to be subjected to the highest standard of scrutiny. And what are we to make of the "Skeptics" doing everything _except_ that work?

The considered consensus of the scientific experts, here, is slowly growing and publishing an opposing authoritative immutable thesis - far too slowly and too messily and with too much initial unwarranted speculation for an impatient world - but at least they are building something up for possible future champion to knock down. And if it resists being knocked down - we have a consensus where it would be "perverse to withhold provisional consent", using Sagan's phrase.


Astroturf GreenImage by sbisson via Flickr
As for motivation within this possible case of motivated obscurantism, how can I discount the astroturf and sympathetic goodwill David Koch has purchased and does purchase?

If you draw the boundary of consideration small enough "I dunno" seems like honest skepticism of considered consensus. But what is the compelling reason to draw the boundary of consideration so small as to ignore case for motivated obscurantism?
Reblog this post [with Zemanta]

Tuesday, November 10, 2009

Scientific software quality and Global Warming

Instrumental temperature record of the last 15...Image via Wikipedia

Very well written response by Michael Tobis to this question from Jon Pipitone "Do likely bugs in the software of climate models cast doubt on the scientific consensus on human factors in global warming?"
jon pipitone: Scientific software quality: what would it take to convince software engineers?: "
Michael Tobis said...

regebro asks what looks like a reasonable question, but it's based on a fundamental misunderstanding of a question that is roughly equivalent to what the difference between weather and climate is. We have very little skill predicting one year out, even assuming no volcanoes and such. Most of the predictability of the detailed state atmosphere vanishes in three weeks or so. But at a multidecadal time scale the problem changes character. Technically, we are no longer dealing with an initial value problem but with a boundary value problem, even though the underlying dynamics are the same. We are not in that case looking for details in any specific year, but for the statistics over an extended period. In a mathematical sense that is an 'easier' problem; it is more constrained by energy balances than by nonlinear fluid dynamics. The messy stuff basically averages out and the residual is what we try to predict. In fact, maybe I'll use that as a definition of climate. It's 'what you can say about the system after the messy unpredictable part gets averaged out'. That's the basis for climate change modeling.

Temperature predictions from some climate mode...Image via Wikipedia

As for Jon's question, the models are wretched pieces of engineering. No commercial shop would release anything nearly as balky, hard to deploy, or prone to failure. It makes open source look good. And that has almost no bearing on whether they are suitable for the purpose. In fact, sometimes models are used well and sometimes they are used badly. This in turn is a scientific, not a software question. All this said, I desperately wish the software were better, and I think we could address many more scientifically meaningful problems much more effectively if it were. Finally, if you think the question is 'global warming, yes or no' the large models in question are much less relevant than many people would have you believe. The answer to that question is yes, to the extent of about 3 degrees per CO2 doubling. The idea that such a conclusion comes from complex models is wrong.
"
I will forgive Mr. Tobis for the dig against open source software ;) I maintain the difference in quality between the very best and the very worst engineered open source software projects makes it very difficult to say anything sensible about the totality.
Also, metrics in software development cannot predict the quality of output of any particular group working on a particular problem - too many confounding issues. For example, the developer with the highest bug count tends to be the best developer on the team - nobody else is trusted to tackle the hardest coding issues.

The geographic distribution of surface warming...Image via Wikipedia

Another issue is the issue of making scientific computations reproducible. Even more basic than if a particular computation is correct is making sure that computation can be reproduced by another group.
Making scientific computations reproducible
Computing in Science and Engineering archive
Volume 2 , Issue 6 (November 2000)
Pages: 61 - 67
ISSN:1521-9615
Authors
Matthias Schwab
Martin Karrenbach
Jon Claerbout
Reblog this post [with Zemanta]