worked out system of decision about intervention/action

\\cserve3\e$\MMG_Docs\worked_out_system_of_decision_about_intervention00.txt

http://www.stat.columbia.edu/~cook/movabletype/archives/2011/05/improvement_of.html

Typing this out for my own benefit.  Probably to nobody else's.

You need to build 4 models.  [1] A model of the car manufacturers and car market.  [2] A model of your negotiation for purchasing a car, and the resources you can bring to bear to the transaction, and the economic constraints you are under for money, time, etc.  [3] A model of your hierarchy and compromises between your values and goals.  And [4] a model of your rational decision making processes.

Lets call these models [1] "Car-Make", [2] "Car-Buy", [3] "My-Values", [4] "My-Rational"

Per Pearl, since the models "Car-Buy" "My-Values" "My-Rational" need to allow intervention experiments, they have to be expressible as directed acyclic graphs (DAGs).  So "Car-Buy" "My-Values" "My-Rational" may be sub-optimal - they are best candidate (or candidates) out of all models that can be expressed as a DAG.

Literature exists to help with the construction of "Car-Make" "Car-Buy".  "Car-Make" will model different ways goals can be achieved: you can make a car safer with more weight, or you can make a car safer with superior engineering and judicious use of materials, etc.  "Car-Buy" could be informed by a few issues of Consumer Reports, etc.

"My-Values" and "My-Rational" will be constructed with a combination of introspection, objective evidence, tests, and quizzing others.  If "My-Values" and "My-Rational" sooth your ego, they will probably perform badly for the task.  You want to take every opportunity to have a model be truthful above being a mere advertisement of you being a nice and super nifty guy.

It would seem that "My-Rational" might involve an infinite regress - you have a model, a model of the process that you judge models, a model of the process that you judge models of models, etc.  But this flatters the human mind.  Per Herbert Simon, Gerd Gigerenzer, Peter M. Todd, Bounded rationality, Ecological Rationality, your really existing system of "My-Values" "My-Rational" is bristling with irresistible fast and frugal heuristics.  You can discover these from poor decisions you make time and time again - fast and frugal heuristics work very well in their preferred ecological setting, but are susceptible to failure modes in other settings because they are not ideally rational.  (Sure sign of an irresistible fast and frugal heuristic: these are betrayed by poor decisions made time and time again, that are not followed by a relentless implementation of disciplines and restraints to prevent those poor decisions from being made in the future.)

A fast and frugal heuristic expressed as a preference for blondes is not revealed by a string of successful relationships with blondes, because this might be due to good qualities innate in all blondes.  A fast and frugal heuristic expressed as a preference for blondes would be revealed by a string of difficult relationships with blondes, and forever indulging a habit of buying drinks for blondes at bars.

There is a tendency to be self-serving in introspection, so objective evidence and third-party observations and judgements are crucial.

Now you can simulate the outcome of a particular choice of automobile, by combining all 4 models into 1 large model and using intervention experiments on the model, suggested by the particular choice of automobile.

Not fully succumbing to the infinite regress, but incorporating a model of how you judge models, could be helpful (call it "Judge-Models").  That way, if multiple suitable models can be imagined and the top candidate does not immediately reveal itself, you have an analytic recourse to determine best fitness.  As above, your really existing "Judge-Models" is also bristling with irresistible fast and frugal heuristics, which you can discover... etc...

It would be silly to say a satisfactory decision cannot be made without this level of rigor.  But the rigorous fully generalized system can suggest adequate "quick and dirty" substitutes, surely.

I am a faithful reader of Gelman's blog, but I am constantly irritated by his willingness to fashion models of everything _except_ "My-Values" "My-Rational" "Judge-Models", which is the same as crossing the moat and killing the dragon and entering the castle, but refusing to climb up the stairs to the princess in the tower - just sitting there on the first step.

Without discussion of "My-Values" "My-Rational" "Judge-Models", you have done so much preparation for a decision about an intervention (calling inaction its own kind of intervention)... but then dropped the bride at the threshold.

Supplying "My-Values" "My-Rational" "Judge-Models" violates the stereotypical separation of work/concern/responsibility between the academic and the decision maker and the action taker, so the reluctance to discuss them is completely understandable, and my irritation is unreasonable, I know.

If You Want to Stop Procrastinating, Give Yourself a Break

Ten Psychology Studies from 2010 Worth Knowing About

Neuronarrative - David DiSalvo

Ten Psychology Studies from 2010 Worth Knowing About

If You Want to Stop Procrastinating, Give Yourself a Break

Most of us inveterate procrastinators are also world-class self punishers. You miss a deadline because you put something off for too long and your mind instantly turns into the Grand Inquisitor, complete with a studded whip to flog you into self-induced terror. But a study of the past year tells us that we've got this all wrong. If you want to get yourself out of the procrastination trap, stop beating yourself up and try a little self forgiveness instead. Researchers followed first year college students through their first and second midterm exams with an eye toward tracking the effects of procrastination and self forgiveness. They found that students who procrastinated before the first midterm were significantly less likely to do so before their second midterm if they gave themselves a break.

This runs counter to the conventional assumption that letting ourselves off easy will foster more procrastination, but the result actually makes a lot of sense for a very practical reason: self-forgiveness allows you to get past your mistake and concentrate energy on correcting your behavior. When you punish yourself, you're also draining energy, sapping focus and taking on too much mental baggage. Not to mention, you also make trying to do whatever you failed at the first time a horrible experience because of its association with self punishment. Instead, acknowledge your procrastiantion and its ill-effects, forgive yourself for screwing up, and get on with the tasks at hand.

Decision Science News - communicating risks

Some ideas on communicating risks to the general public

Decision Science News - communicating risks - 12/03/10

Representations to use less often

Representations to use more often

speak of natural frequencies, and the models they are based on

cute ass comment in style of denialists

cute ass comment in style of denialists

http://www.easterbrook.ca/steve/?p=1954&cpage=1#comment-4175

How dare you point to these extreme weather events as proof of global warming. How unscientific of you! Whoever taught you statistics should have their 2.49 children shot, on average.
If we consider all possible worlds, real and fictional, we should be not surprised by extremes. I notice you didn’t mention all the fictional Earths where the planet froze solid. How convenient of you!
I am obliged to end this note castigating you for writing about dead trees when you should have been playing up all the brave scientists working on future speculative Breakthroughs, that allow us to continue burning oil at growing rates, which is the only way we can sustain our current Utopia of Limitless Wealth. It is a strange kind of Utopia – the kind that can stay hidden from 1.7 billion people who live in absolute poverty (our Utopia is such a playful rascal) – but it is a Utopia nonetheless.

A Mashey Gem

http://initforthegold.blogspot.com/2010/10/mashey-gem.html

http://www.realclimate.org/index.php/archives/2009/02/on-replication/langswitch_lang/in/

People are making an error common to those comparing science to commercial software engineering.

Research: *insight* is the primary product.
Commercial software development: the *software* is the product.

Of course, sometimes a piece of research software becomes so useful that it gets turned into a commercial product, and then the rules change.

===
It is fairly likely that any “advanced version control system” people use has an early ancestor or at least inspiration in PWB/UNIX Source Code Control System (1974-), which was developed by Marc Rochkind (next office) and Alan Glasser (my office-mate) with a lot of kibitzing from me and a few others.

Likewise, much of modern software engineering’s practice of using high-level scripting languages for software process automation has a 1975 root in PWB/UNIX.

It was worth a lot of money in Bell labs to pay good computer scientists to build tools like this, because we had to:

- build mission-critical systems
- support multiple versions in the field at multiple sites
- regenerate specific configurations, sometimes with site-specific patches
- run huge sets of automated tests, often with elaborate test harnesses, database loads, etc.

This is more akin to doing missile-control or avionics software, although those are somewhat worse, given that “system crash” means “crash”. However, having the US telephone system “down”, in whole or in part, was not viewed with favor either.

We (in our case, a tools department of about 30 people within a software organization of about 1000) were supporting software product engineers, not researchers. The resulting *software* was the product, and errors could of course damage databases in ways that weren’t immediately obvious, but could cause $Ms worth of direct costs.

It is easier these days, because many useful tools are widely available, whereas we had to invent many of them as we went along.

By late 1970s, most Bell Labs software product developers used such tools.

But, Bell Labs researchers? Certainly no the physicists/ chemists, etc, an usually not computing research (home of Ritchie & Thompson). That’s because people knew the difference between R & D and had decent perspective on where money should be spent and where not.

The original UNIX research guys did a terrific job making their code available [but "use at your own risk"], but they’d never add the overhead of running a large software engineering development shop. If they got a bunch of extra budget, they would *not* have spent it on people to do a lot of configuration management, they would have hired a few more PhDs to do research, and they’d have been right.

The original UNIX guys had their own priorities, and would respond far less politely than Gavin does to outsiders crashing in telling them how to do things, and their track record was good enough to let them do that, just as GISS’s is. They did listen to moderate numbers of people who convinced them that we understood what they were doing, and could actually contribute to progress.

Had some Executive Director in another division proposed to them that he send a horde of new hires over to check through every line of code in UNIX and ask them questions … that ED would have faced some hard questions from the BTL President shortly thereafter for having lost his mind.

As I’ve said before, if people want GISS to do more, help get them more budget … but I suspect they’d make the same decisions our researchers did, and spend the money the same way, and they’d likely be right. Having rummaged a bit on GISS’s website, and looked at some code, I’d say they do pretty well for an R group.

Finally, for all of those who think random “auditing” is doing useful science, one really, really should read Chris Mooney’s “The Republican War on Science”, especially Chapter 8 ‘Wine, Jazz, and “Data Quality”‘, i.e., Jim Tozzi, the Data Quality Act, and “paralysis-by-analysis.”

When you don’t like what science says, this shows how you can slow scientists down by demanding utter perfection. Likewise, you *could* insist there never be another release of UNIX, Linux, MacOS, or Windows until *every* bug is fixed, and the code thoroughly reviewed by hordes of people with one programming course.

Note the distinction between normal scientific processes (with builtin skepticism), and the deliberate efforts to waste scientists’ time as much as possible if one fears the likely results. Cigarette companies were early leaders at this, but others learned to do it as well.

Breakthrough Narrative and Time Machines, and Carbon Eating Trees

http://thingsbreak.wordpress.com/2010/10/26/stop-the-presses-climate-journos-think-the-emissions-reduction-issue-looks-an-awful-lot-like-a-narrative-problem-no-word-yet-on-just-how-nail-shaped-people-wielding-hammers-see-it/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+ThingsBreak+(The+Way+Things+Break)

Wow. Not to put too fine a point on it: """
Did I mention that this New Narrative meme is being pushed by the same people who are arguing against any sort of meaningful emissions pricing? They wouldn’t have a vested interested in framing emissions legislation as a dead, would they?
...
It’s nice that you have a meme. It’s nice that some journalists bit. When you feel like getting around to actually hooking some grassroots support, give us a reason to support your Narrative besides an appeal to novelty.
"""

This Breakthrough meme is exactly like a call for more time-machine funding. The only reason they are pushing for stratospheric saline geysers and carbon-eating mega-trees is that they would seem more plausible to the uninformed than a time-machine. Their plausibility is the foremost attraction, their actual viability is a far distant concern -- and that is morally noxious.

Education: "Waiting for Superman", Union Busting, and Obscuring the role of Parental Responsiblity

http://www.democracynow.org/2010/10/1/waiting_for_superman_critics_say_much

I agree that the teachers' union get hit with cheap shots in the school debate. I agree that the teachers' union is full of professionals dedicated to providing excellent education in the "really existing" world, and the construction of alternative scenarios (maybe fanciful) for education that blink the teachers' union out of existence has less to do with delivering quality education in the real world and have more to do with crude union busting.

But it silly to pretend that there is no conflict whatsoever between the education needs of the students and the political convenience of the teachers' union. Those are two distinct entities, and they have different political needs. For example: the mechanism that prevents capricious termination of union employees is in conflict with the discretion a school principal would wish to assert to fire a lacking teacher to replace with a potentially better suited teacher. For example: the mechanism that prevents threat of pay reduction being used to punish a union employee is in conflict with the discretion a school principal would want be able to better compensate superior teachers working from a constrained budget for teacher compensation.

The key point as I see it: the *existence* of teaching as a professional discipline is obscuring the primary role of *the responsibility of parents* for America's education of children. The parents are all too happy to shed responsibility, and push the responsibility to teaching professionals. If parents appropriately shouldered the responsibility for the quality of education of children, the primary role of teaching professionals would diminish, plainly.

Taking responsibility does not necessarily mean home schooling. It DOES mean attending PTA meetings and supervising children's homework every school night and being aware of trends of grades and children's enthusiasm or frustration in coursework -- parents taking every and all opportunity to take a full role in their children's education, paid for in *time*. If that time is then not available for television and amusement or if that time is then not available for working to support a certain level of consumerism, so be it.

In America, teaching professionals take the primary role, for praise and for blame, fair and unfair, because of the sloughing off of responsibility by America's parents.

race on the brain

http://www.stat.columbia.edu/~cook/movabletype/archives/2010/10/racism.html

I would much rather deal with racists than people who have "race on the brain".
I am not interested in searching the world for people free of racism, because it is hard to imagine people who *really* don't allow race to inform *any* judgement whatsoever.  I have met some children and adults who I would guess come very close, but so very very few to make the effort not worthwhile.
So I would much rather deal with racists, because, honestly, I must judge myself a racist.
I *do* have a problem with people who have "race on the brain" -- when the topic of race comes up they are reduced to blithering idiocy and vile reactionary tribalism.  White males haven't cornered the market on this particular form of idiocy -- it is embarrassing when Latino candidates win office on nothing more than their publicized ethnicity (such as the insubstantial Los Angeles mayor Antonio Villaraigosa, aka Tony Villar).
Political correctness makes physical and verbal violence against traditionally disadvantaged groups less likely, and that is good, but it cannot do much to lesson "race on the brain" on both sides of the racial divide.  Only the self-imposed discipline of critical thinking can do that, and people get too much pleasure from their vile reactionary tribalism to self constrain their thought.

good summary gelman philosophy of Bayesian statistics

Bayesian statistical pragmatism

http://www.stat.columbia.edu/~cook/movabletype/archives/2011/04/bayesian_statis_1.html

Thank you for writing this.  This is the clearest and quite comprehensive (even though short and to the point) support for your philosophical views, and I am inclined to agree on all counts.

I read Gelman and Shalizi 2010, and enjoyed it a lot, what my novice brain could understand.  But the summary above hits and handles all the difficulties, and is easy to read.

I would recommend people read Gelman and Shalizi 2010 "Philosophy and the practice of Bayesian statistics" [ http://www.stat.columbia.edu/~gelman/research/unpublished/philosophy.pdf ] for the section on Mayo's "severe" testing of models, Section 4 "Model checking" - the only lack of the summary above that I can see.

size of genome

http://sandwalk.blogspot.com/2011/03/how-big-is-human-genome.html?showComment=1300992550197#c1663047036846499582

manuel "moe" g said...

[Part 1 of 2]

Forgive my ignorance, but I am trying to make sense of different descriptions of the human genome, and different descriptions of the information needed to fully specify a large mammal, like a man.

You talk about 3.5 Gb for the genome, and, giving Ray Kurzweil the benefit of the doubt, 50 million bytes after loss-less compression.

If someone made extravagant claims about a computer program that runs on some unknown hardware and unknown OS, I would be unamused if they handed me a thumb-drive containing the compressed binary executable, and nothing more. This single file would demonstrate nothing.

I would demand the original source code, the specification for the code (including the business decisions the code is meant to automate, at the very least), some documentation demonstrating that I can move back and forth between points in the specification and the source code lines encoding that part of the specification, and the code for the automated tests (so an automated test can demonstrate what changes to the code will still keep it within specification, at the very least).

And maybe the same for some of the libraries and hardware - maybe needing the full specification if the libraries, OS, and hardware if they all are very novel, quite unlike any I have worked with before.

So there would be a dramatic explosion of information needed, moving from the binary executable to a bare minimum specification of a computer program as defined above.

THURSDAY, MARCH 24, 2011 2:49:00 PM

manuel "moe" g said...

[Part 2 of 2]

In the debate between PZ and Kurzweil, PZ makes this point:

http://scienceblogs.com/pharyngula/2010/08/ray_kurzweil_does_not_understa.php

"""

Let me give you a few specific examples of just how wrong Kurzweil's calculations are. Here are a few proteins that I plucked at random from the NIH database; all play a role in the human brain.

First up is RHEB (Ras Homolog Enriched in Brain). It's a small protein, only 184 amino acids, which Kurzweil pretends can be reduced to about 12 bytes of code in his simulation. Here's the short description.

MTOR (FRAP1; 601231) integrates protein translation with cellular nutrient status and growth signals through its participation in 2 biochemically and functionally distinct protein complexes, MTORC1 and MTORC2. MTORC1 is sensitive to rapamycin and signals downstream to activate protein translation, whereas MTORC2 is resistant to rapamycin and signals upstream to activate AKT (see 164730). The GTPase RHEB is a proximal activator of MTORC1 and translation initiation. It has the opposite effect on MTORC2, producing inhibition of the upstream AKT pathway (Mavrakis et al., 2008).

Got that? You can't understand RHEB until you understand how it interacts with three other proteins, and how it fits into a complex regulatory pathway.

"""

I am inclined to grant PZ the point, and say his understanding of the immensity of the task outstrips Kurzweil's understanding.

Would the explosion of information needed to move from the complete genome to the complete specification of a large mammal be on the same order of the explosion of information needed to move from the binary executable to a bare minimum specification of a computer program as defined above? Did I capture the gist of it, or am I hopelessly mistaken?

THURSDAY, MARCH 24, 2011 2:49:00 PM

SOCIALISM AND TEMPORARILY EMBARRASSED MILLIONAIRES

"Socialism never took root in America because the poor see themselves not as an exploited proletariat but as temporarily embarrassed millionaires." John Steinbeck (self.politics)


http://www.reddit.com/r/politics/comments/g877k/socialism_never_took_root_in_america_because_the/#


http://www.reddit.com/r/politics/comments/g877k/socialism_never_took_root_in_america_because_the/c1lngjo

cephas_rock 106 points 5 hours ago[-]

"Don't forget that most men with nothing would rather protect the possibility of becoming rich than face the reality of being poor." - John Dickinson, "1776" (the musical)

Nice sarcastic invite for blog comments

Image via WikipediaWhile reading Reddit Politics, I spotted this comment pointing out the solicitation for comments on the site "The Big Picture":

http://www.reddit.com/r/politics/comments/ftpis/president_of_fox_news_will_be_indictedmaybe_even/c1ijv3p

The Big Picture - http://www.ritholtz.com/blog/

"Please use the comments to demonstrate your own ignorance, unfamiliarity with empirical data, ability to repeat discredited memes, and lack of respect for scientific knowledge. Also, be sure to create straw men and argue against things I have neither said nor even implied. Any irrelevancies you can mention will also be appreciated. Lastly, kindly forgo all civility in your discourse . . . you are, after all, anonymous."

The importance of stupidity in scientific research

Image via WikipediaFrom a comment to

Only In It For The Gold - Michael Tobis - "Bad Media Advice"

by Dan Olner March 4, 2011 1:11 AM:

Here's Feynman on the 'terrible uncomfortable feeling called confusion'. And here's a great little paper on 'the importance of stupidity in scientific research' - "actively seek out new opportunities to feel stupid."

The importance of stupidity in scientific research

Martin A. Schwartz - 9 April 2008 - jcs.biologists.org

Excerpt:

I recently saw an old friend for the first time in many years. We had been Ph.D. students at the same time, both studying science, although in different areas. She later dropped out of graduate school, went to Harvard Law School and is now a senior lawyer for a major environmental organization. At some point, the conversation turned to why she had left graduate school. To my utter astonishment, she said it was because it made her feel stupid. After a couple of years of feeling stupid every day, she was ready to do something else.

I had thought of her as one of the brightest people I knew and her subsequent career supports that view. What she said bothered me. I kept thinking about it; sometime the next day, it hit me. Science makes me feel stupid too. It's just that I've gotten used to it. So used to it, in fact, that I actively seek out new opportunities to feel stupid. I wouldn't know what to do without that feeling. I even think it's supposed to be this way. Let me explain.

For almost all of us, one of the reasons that we liked science in high school and college is that we were good at it. That can't be the only reason – fascination with understanding the physical world and an emotional need to discover new things has to enter into it too. But high-school and college science means taking courses, and doing well in courses means getting the right answers on tests. If you know those answers, you do well and get to feel smart.

A Ph.D., in which you have to do a research project, is a whole different thing. For me, it was a daunting task. How could I possibly frame the questions that would lead to significant discoveries; design and interpret an experiment so that the conclusions were absolutely convincing; foresee difficulties and see ways around them, or, failing that, solve them when they occurred? My Ph.D. project was somewhat interdisciplinary and, for a while, whenever I ran into a problem, I pestered the faculty in my department who were experts in the various disciplines that I needed. I remember the day when Henry Taube (who won the Nobel Prize two years later) told me he didn't know how to solve the problem I was having in his area. I was a third-year graduate student and I figured that Taube knew about 1000 times more than I did (conservative estimate). If he didn't have the answer, nobody did.

That's when it hit me: nobody did. That's why it was a research problem. And being my research problem, it was up to me to solve. Once I faced that fact, I solved the problem in a couple of days. (It wasn't really very hard; I just had to try a few things.) The crucial lesson was that the scope of things I didn't know wasn't merely vast; it was, for all practical purposes, infinite. That realization, instead of being discouraging, was liberating. If our ignorance is infinite, the only possible course of action is to muddle through as best we can.

Links on Rational Discussion

Image via Wikipedia

Currently, I see that Rationality is all about a very high standard for yourself and for your allies. Worrying about the standard of rationality of enemies and opponents should be a very small part. There is a finite amount of energy, and that energy is best used to keep the self from deluding the self with comfortable ideas.

It is interesting that nobody wants to be seen as irrational, but very few happy assume the burden of a very high standard for rationality for their own thoughts.

John Wilkins - Evolving Thoughts - A Code of Conduct for Effective Rational Discussion

1. The Fallibility Principle
2. The Truth-Seeking Principle
3. The Clarity Principle
4. The Burden of Proof Principle
5. The Principle of Charity
6. The Relevance Principle
7. The Acceptability Principle
8. The Sufficiency Principle
9. The Rebuttal Principle
10. The Resolution Principle
11. The Suspension of Judgement Principle
12. The Reconsideration Principle
13. Fleck’s Addendum

Some of my own posts on Rational Discussion and Rationality:

• The next tack in Climate Disruption denialism - Scientists must be Nihilists
• The difference between Skeptics and Deniers
• Shifting between the genial general and the hard-nosed specific as a form of the "Shell and Pea game" - A juicy example of The Art of Controversy
• Uncertainty is not an argument for the status quo

How trusted is the "Big Five" Personality Traits in mainstream psychology?

flickr | Graela "Bathroom Personality Assessment - Part 2"
http://www.flickr.com/photos/9441263@N04/3984047524/

From Andrew Gelman, Statistical Modeling, Causal Inference, and Social Science, December 29, 2010 9:10 AM

Comments to "Brain Structure and the Big Five"

Sanjay Srivastava | December 29, 2010 10:36 AM

Walter Mischel's famous critique predated the Big Five. His critique was of the concept of a personality trait more broadly. If you ask around at Mischel's home department they'll probably tell you that Mischel won the argument, but that's not the mainstream view among personality psychologists elsewhere. In fact, I don't think there's a single mainstream view on traits or the Big Five, but I'd guess that many personality psychologists would endorse, at a minimum, "useful enough until a better model comes along." Some might go a lot farther. (Some of my own views on the Big Five are in this paper, if you're curious.)

Russell Almond | December 29, 2010 11:15 AM

A quick comment about the Big-5. A couple of years ago, I did some consulting with a psychologist who was developing new personality measures. The standard practice for validating the new measure was to give it as part of a battery to a sample of the target population, along with the Big 5, and other known measure that were similar to the target. I got the impression that it wasn't that the Big 5 were thought of as the answer to everything, but that it was a starting point that most people in the field understood. The burden of proof was to show that your proposed measures was something other than a composite of the 5 factors in the Big 5.

My comments: trying to get a fix on the Big Five. George E. P. Box: "All models are wrong, some models are useful"

Fetishizing p-Values - The Cult of Statistical Significance

en.wikipedia.org William Sealy Gosset

Fetishizing p-Values; Tom Leinster - The n-Category Cafe

Recovering the insight of "Student" Gosset from the over-simplification of Ronald A. Fisher

http://golem.ph.utexas.edu/category/2010/09/fetishizing_pvalues.html

Leinster: Now there’s a whole book making the same point: The Cult of Statistical Significance, by two economists, Stephen T. Ziliak and Deirdre N. McCloskey. You can see their argument in this 15-page paper with the same title. Just because they’re economists doesn’t mean their prose is sober: according to one subheading, ‘Precision is Nice but Oomph is the Bomb’.

http://www.statlit.org/pdf/2009ZiliakMcCloskeyASA.pdf

Leinster: it is true that p-value does not measure the magnitude of the effect (but then, anyone who has taken at least one course in statistics should know that)

I think Jost, Ziliak and McCloskey would completely agree that anyone who has taken at least one course in statistics should know that. They’re pointing out, open-mouthed, that this incredibly basic mistake is being made on a massive scale, including by many people who should know much, much better. Bane used the term ‘collective self-deception’; one might go further and say ‘mass delusion’. It’s a situation where a fundamental mistake has become so ingrained in how science is done that it’s hard to get your paper accepted if you don’t perpetuate that mistake.

That last statement is probably putting it too strongly, but as I understand it, the point they’re making is along those lines.

From the 15-page paper "The Cult of Statistical Significance":

In 1937 Gosset, the inventor and original calculator of “Student’s” t-table told Egon, then editor of Biometrika, that a significant finding is by itself “nearly valueless”:

...obviously the important thing in such is to have a low real error, not to have a "significant" result at a particular station. The latter seems to me to be nearly valueless in itself. . . . Experiments at a single station [that is, tests of statistical significance on a single set of data] are almost valueless. . . . What you really want is a low real error. You want to be able to say not only "We have significant evidence that if farmers in general do this they will make money by it", but also "we have found it so in nineteen cases out of twenty and we are finding out why it doesn't work in the twentieth.” To do that you have to be as sure as possible which is the 20th—your real error must be small...

Gosset to E. S. Pearson 1937, in Pearson 1939, p. 244.

Gosset, we have noted, is unknown to most users of statistics, including economists. Yet he was proposing and using in his own work at Guinness a characteristically economic way of looking at the acquisition of knowledge and the meaning of “error.” The inventor of small sample econometrics focused on the opportunity cost of each observation; he tried to minimize random and non-random errors, real errors.

Edit 11/12/10

A very nice write-up here, along same lines: Significance Tests in Climate Science -- Maarten H. P. Ambaum -- http://www.met.reading.ac.uk/~sws97mha/Publications/jclim_ambaum_rev2.pdf

Consider a scientist who is interested in measuring some effect and who does an experiment in the lab. Now consider the following thought process that the scientist goes through:

1. My measurement stands out from the noise.
2. So my measurement is not likely to be caused by noise.
3. It is therefore unlikely that what I am seeing is noise.
4. The measurement is therefore positive evidence that there is really something happening.
5. This provides evidence for my theory.

This apparently innocuous train of thought contains a serious logical fallacy, and it appears at a spot where not many people notice it.

To the surprise of most, the logical fallacy occurs between step 2 and step 3. Step 2 says that there is a low probability of finding our specific measurement if our system would just produce noise. Step 3 says that there is a low probability that the system just produces noise. These sound the same but they are entirely different.

This can be compactly described using Bayesian statistics...

This comes from a summary of the paper: How significance tests are misused in climate science -- Guest post by Dr Maarten H. P. Ambaum from the Department of Meteorology, University of Reading, U.K. -- http://www.skepticalscience.com/news.php?n=456#

Edit 11/21/10

Significance Tests, frequentist vs. bayesian

Open Mind | tamino.wordpress.com | much-ado-about-something

Dikran Marsupial | November 14, 2010 at 10:49 am

When we perform a test of statistical significance test, what we
would really like to ask is “what is the probability that the
alternative hypothesis is true?”. A frequentist analysis
fundamentally cannot give a direct answer to that question, as
they cannot meaningfully talk of the probability of a hypothesis
being true – it is not a random variable, it is either true or
false and has no “long run frequency”. Instead, the frequentists
gives a rather indirect answer to the question by telling you the
likelihood of the observations assuming the null hypothesis is
true and leaving it up to you to decide what to conclude from
that. A Bayesian on the other hand can answer the question
directly as the Bayesian definition of probability is not based
on long run frequencies but on the state of knowledge of the
truth of a proposition. The problem with frequentist statistical
test is that there is a tendency to interpret the result as if it
were the result of a Bayesian test, which is natural as that is
the form of answer we generally want, but still wrong.

The frequentist approach avoids the “subjectivity” of the
Bayesian approach (although the extent of that “subjectivity” is
debatable), but this is only achieved at the expense of not
answering the question we would most like to ask. It could be
argued that the frequentist approach merely shifts the
subjectivity from the analysis to the interpretation (what should
we conclude based on our p-value). Which form of analysis you
should use depends on whether you find the “subjectivity” of the
Bayesian approach or the “indirectness” of the frequentist
approach most abhorrent! ;o)

At the end of the day, as long as the interpretation is
consistent with the formulation, there is no problem and both
forms of analysis are useful.

This was my favorite comment, the whole sub-thread underneath is interesting. The original Open Mind | tamino.wordpress.com article has good qualifications to Dr Maarten H. P. Ambaum's Skeptical Science post.

Valuing stewardship of the environment for future generations, or not

commons.wikimedia.org Sheep_eating_grass_edit02.jpg

Attempted to post comment to Offsetting Behaviour - Eric Crampton : Yer either fer us or agin us

[ I am not speaking to the game-theoretic analysis of New Zealand leaving Kyoto -- Bjorn's swaying this way or that notwithstanding, there is no rational reason for NZ to stay inside Kyoto, unless it was seen as the price for signaling environmental concern. ]

The same climate scientists that Lomborg disparaged for stating evidence of high sensitivities for carbon emissions are now the same climate scientists he will trust to run geo-engineering. This is the the most embarrassing contradiction of Lomborg's evolving stance.

The Copenhagen Consensus cost-benefit analysis put carbon taxes at the bottom by valuing stewardship of the environment for future generations at zero. The same way pre-school for my toddler would be at the bottom of a cost-benefit analysis of all uses of my money, if I valued his own future earnings and quality of life at zero.

If you are standing on the train tracks with a freight train coming in five minutes, you have the choice to leap off the tracks. A "compromise" position of shifting over a few inches will have no effect, no matter how much you value "moderation and reasonableness". If you limit your analysis to only the next step minutes and fifty-nine seconds, the energy expended in the leap is a waste.

I wish we had the choice to live in an "warmer average" world -- it would be nice. If you put two bullets in a six chambered gun to play Russian roulette, on "average", you are still alive but with a headache. But the "average" is an abstraction, and in reality you have to deal with the consequences of the spun barrel. The risk is not a warmer world -- the risk is an over-energetic world that no longer has the climate stability that allowed civilization and large-scale agriculture and inexpensive & quick transportation to be developed and maintained.

It is fine to consider all possible humanitarian uses of scarce capital. The weight that stewardship of the environment for future generations should not be infinite, lest you indulge in pointless profligacy towards but a single goal. But that does not imply that stewardship of the environment for future generations should be weighted at zero.

[ This implies value placed on trying to give future generations a "western/first world" standard of living much like what we currently enjoy. If we are satisfied with a few hundred thousand on each continent living under conditions like indigenous peoples, living along the new raised coastlines and grasslands freed from permafrost, with climate instability but the net warmth & wetness still giving the ability to feed from the meat of small grazing animals, the costs we would bare would be slight. ]

Edit 9/21/10: Reply via Google Buzz from Eric Crampton:

If investing in tech reduces more warming per dollar spent than do other things, what's the problem with redirecting spending towards tech?

Copenhagen valued future generations the same way that cost-benefit analysis typically values future generations: by applying a standard discount rate. That doesn't say that future people don't count; rather, it says that future people might prefer being given cash.

My reply:

"""That doesn't say that future people don't count; rather, it says that future people might prefer being given cash"""

If I am the victim of blunt trauma, I may not value a cash dispersal later over a medical intervention now. There is a rational case to be made that the two are hardly substitutes in some circumstances.

I agree that I should have been more careful and said "valuing stewardship of the environment for future generations, *particularly* in reducing the risk of the very worst outcomes". I will be more careful in future.

"""If investing in tech reduces more warming per dollar spent than do other things, what's the problem with redirecting spending towards tech?"""

No argument here. But the lack of breakthrough tech *now* implies non-zero carbon taxes *now* (and there is a moral argument for quite substantial taxes now). I am certain it will take a few decades of people seeing global military preparation for the worst possible outcomes of climate disruption before it is plain that environmental stewardship may be worth 5 or more points of global economic activity. It is not surprising that substantial carbon taxes have near zero political traction in the two largest economies, now.

Selling Fantasies: Breakthrough Institute

http://en.wikipedia.org/wiki/File:Emperor_Clothes_01.jpg

Comment on Michael Tobis post "The Breakthrough Idea"

Breakthrough Institute works like the Copy Protection technology wizards selling their tech to record companies. It cannot work, because the pirates will always find a workaround towards copy protection - you are merely punishing your customers and training them to be pirates when they try to use your product in convenient ways. The Copy Protection technology wizards are not selling a working solution - because a solution is impossible - they are selling a pleasant fantasy to the record companies in the few years their business model has left.

People do not confine themselves to buying working products. Sometimes they will purchase fantasies. Look at the exercise gizmos that people buy from TV.

The Breakthrough Institute doesn't have to provide solutions that work - it will provide fantasies that it can sell. So lets try and figure out who their customers are.

If you are in the top 0.5% of incomes, you are intelligent and you may be slightly distressed that your great grandchildren will be born into a boiling world (when you can be bothered to consider the issue). You have the ability to direct funding, and in these few years before the climate disruption really hits human agriculture and infrastructure, you are in the market for fantasies, sold to you by the semi-knowledgeable folk (who are probably sincere, because their confidence in their tech solutions surpasses their scientific capabilities). That is what people like the Breakthrough Institute are selling. For example, Warren Buffet doesn't consider himself a bad person, and he cares for his grandchildren. But he has also made a huge bet on coal transport infrastructure. He would love to support the Breakthrough Institute by some means, to reconcile his position on the responsibility of environmental stewardship for future generations.

"The Breakthrough Institute, a project of Rockefeller Philanthropy Advisors, Inc." lets you know about the customers they are after. How did good ol' John D. make is money?

Lets predict their structure. They will rarely speak in absolute moral terms - they will never flatly state that it is craven to leave future generations a boiling world just because a handful of generations could not bare to lower their standard of living. The absolute moral issues will always be left unspoken. Those that talk about the moral issues will be marginalized as "un-serious" or "alarmist".

They will strive to distance themselves from the worst of the denialists. Pielke Jr and Fuller practically fell over their own feet trying to run away from Virginia State Attorney General Cuccinelli. But they will take "warmist" commentators that have a record of limiting themselves to the published science, like Romm, and equate them with denialists that spout off bat-shit nonsense - even thought the implication of equivalence is ridiculous. But you will know them by their actions, because they will spend most of their energy arguing against those with the clearest grasp of the facts, and moral issues, and political challenges.

It is the foolish "moderate" position of shifting your stance a few inches when you are standing on tracks, freight train coming. The half measure doesn't leave you just half-dead.

All you can do is make the case to ethical decision makers that they are being sold a bill of goods, by comparing the statements and techniques and rhetorical stances of the Breakthrough Institute to bunglers that stood in the way of decisions of moral courage, and the weavers of the Emperors New Clothes. These are the "moderate" apologists for moral failures - like those who stood in the way of eradicating slavery, or were the audience for the Letter from a Birmingham Jail, or were willing to negotiate with Hitler, or were willing to overlook Stalin's crimes. In all these cases, you could find "moderates" that participated in moral failures, and argued for positions with shabby facts and shabby rhetorical devices.

Edit: 09/14/10

Reply by Steve Bloom:

Moe, Rockefeller is the BTI fiscal sponsor; the main funder throughout has been the Nathan Cummings Foundation. By itself the fiscal sponsorship doesn't mean much,although it may well in this instance.

How can we define "Useful", when it comes to our models of reality?

Wikimedia Commons: "N-Gauge Cassiopeia E26 & EF81 from Kato"

A comment to "Useful models, model checking, and external validation: a mini-discussion" by Andrew Gelman - Statistical Modeling, Causal Inference, and Social Science

Gelman wrote (with Cosma Shalizi) a very fine philosophical justification for real-world, effective Bayesian techniques, which differs greatly from the usual philosophy associated with Bayesians.

Gelman, Shalizi: Philosophy and the practice of Bayesian statistics in the social sciences

My own recurring criticism of Gelman is of using Bayesian/statistical models to the exclusion of others.  I am more comfortable with the idea of models of different type in competition.

http://manuelmoeg.blogspot.com/search/label/Model%20building

The multiple models you then have will now compete in different uses - based on predictive power, accuracy, ability to calculate meaningful error ranges, cost of collecting data, cost of computation, cost of comparison, ability to predict outcomes from interventions, cost of understanding, etc.

quoting Gelman:

"We always talk about a model being "useful" but the concept is hard to quantify."

My comment:
Simply build a model of costs and gains and methods of comparison between models!  If a model is good enough for your work, a model must be good enough as a working definition of "useful"!

Sometimes the best answer to "Why" is "Just because".  Sometimes the best mechanism for rating different models is another model.  The Skeptics will always howl, so you simply have to demonstrate that their own behavior is consistent with putting undue confidence in their own model, whether a conscious model or unconscious.  (And, it must simply terminate with a model, because of the limits of the tools available to the human brain.  Only a model, probably over-simplified, can be manipulated with the agility needed to predict future outcomes of the universe from actions considered now, in real-time.)

Just keep asking the Skeptic "Why" with regards to their own personal actions, and when they hit the "Just because" point, they probably have described a model of utility, assumed true without proof, as an answer to "Why" in the previous step.

If the Skeptic refuses ultimate responsibility over their personal actions, and tries to plead pure capriciousness or mystery, then their model is simply statistical, based on stimulus and internal states (like stress) that can be approximately discovered with objective external measures (like galvanic skin response).  Of course, it is easier to plead pure capriciousness or mystery than demonstrate it - if their behavior is well predicted by a deterministic model suggested by another, the Skeptic is shut up.  Most times the reason for behaviors is gross and banal, no matter how elevated the sophistry of the Skeptic.

Thoughts on Roger Pielke Jr. | Stand-Up Economist - Yoram Bauman

wikipedia.org - Childe's_Tomb

Thoughts on Roger Pielke Jr. | Stand-Up Economist - Yoram Bauman

As an economist, I found Roger’s lack of discussion of climate impacts to be extremely disturbing. If—totally hypothetically—the science said that hitting 450ppm would cause the planet to explode, I’m pretty sure Roger’s talk would have looked different. (At least I hope so!) The economic point here is that cost-benefit analysis has two halves—costs and benefits—and you can’t do it by just talking about one of the two halves. Why Roger failed to talk about both halves has me totally perplexed and leaves me questioning how much he actually knows about economics. (For the record, he’s not an economist, so I think this is a legitimate question, not an insulting one. He’s a political scientist, but his talk was not about the intersection of science and politics; his talk was fundamentally about economics.)

RPJr's rhetorical trick "there is a lot of misunderstanding and misrepresentation displayed in this post... Fortunately, my new book covers all of these points so that there should be no ambiguity in my views." is annoying.

As if a book is a tomb for the ideas of a public intellectual, and it makes them incapable of stating plainly their views in public forums.

Y. Bauman refuses to play ball:

"Okay, here are some questions: (1) What did you say about the tenets of climate science? (Then I’ll try to get a video of your talk and see if I owe you an apology.) (2) How would you quickly characterize the main points of your talk? (3) Since you note above that you “did not discuss costs or benefits”, I’m curious about why. Do you not think cost-benefit analysis is important? (4) How (if at all) would your talk have been different if the scientific consensus was that 450ppm would destroy the planet?"

Is RPJr so craven as to simply "hit-and-run" from this forum, now that the questions are specific?  Stay tuned!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Related posts:

• The Post-Email-Hacking phase of climate denialism
• Complicity and Sympathy for Comforting but False & Dangerous Ideas
• It is time for some humility - Oh Lordy...
• Is using the term "Denialist" unfair?

[Edit 7/31/10]

To my surprise, RPJr replied; his answers:

1. I used a “bathtub” model to describe the challenge of stabilization and I argued that everyone in the debate on all sides agree that CO2 has impacts. Where there are debates is when those impacts become dangerous (the height of the bathtub, e.g., 450 ppm) and the consequences of spilling over. Such debates are of course legitimate.

2. Three points: A. Targets and timetables for reducing emissions now being discussed or even enacted in law (e.g., in the UK) are not credible (I think I proved this), B. Stabilizing concentrations requires advances in technology deployment and innovation rather than GDP contraction (shown a bit, but largely asserted), C. Acccelerating decarbonization requires much greater public investments in technology (asserted not proven).

3. I’ve written a lot of CBA, and teach it as well. This talk was not about CBA, but policy evaluation. I am happy to discuss the topic.

4. I have no idea.

flickr.com/photos/psd/1806225034
"Moral Compass" by "psd"

My comments to (1): "Such debates are of course legitimate."  RPJr has a problem with the debate coming to provisional conclusion, on the side of the science and the moral question of future generations being left a livable world - a provisional conclusion where we begin work on drastically reducing carbon emissions and mitigate previous carbon emissions, where GDP takes a major haircut if need be.  RPJr's fretting and fussing is consistent with the moral question of future generations being left a livable world always taking a backseat to today's GDP/standard of living - but he doesn't have the guts to admit that, or he realizes that if his cravenness is so obvious, he gives up any chance of political effect.

My comments to (2): Actually, this is the first sensible thing RPJr has ever said, to my knowledge.  It is very true: we have exactly zero experience with asking citizens to voluntarily cut their standard of living for the moral outcome of  future generations being left a livable world.  "Warmists", it can be argued, don't have the guts to admit this, or they realize that if they are so truthful they give up any chance of political effect.

My comments to (3): Why talk to economists about policy evaluation without reference to cause and effect?  That was Y. Bauman's original puzzlement.

My comments to (4): Pathetic.  Again, RPJr cannot deal with science predicting catastrophe (catastrophe, because it is hard to imagine the moral monsters that could cheerfully leave future generations an unlivable world), because his goal is that he moral question of future generations being left a livable world always taking a backseat to today's GDP/standard of living.  That can be used to perfectly predict his reaction to anything.