Statistics versus Causality - A predictable impasse

Image by Cedric Favero via Flickr

My undignified reply to Andrew Gelmans's take on Causality and Statistical Learning

http://www.stat.columbia.edu/~cook/movabletype/archives/2010/03/causality_and_s.html
follow-up here:
http://www.stat.columbia.edu/~cook/movabletype/archives/2010/03/no_comment.html

The causality people and the statistics people are talking past each other, your [Andrew Gelman's] 12 page magnum opus included.

Point 0) Sense of responsibility → decision → commitment to action/inaction → action/inaction ⇒ implies you possess a general description of reality, unless you are limiting yourself to a very narrow sphere of responsibility.

Point 1) Statistics cannot be the basis for a general description of reality because of Simpson's Paradox.  When it arises, the paradox can only be eliminated by an appeal to plausible causality, directly or indirectly.  Also, no statistical test exist, for a static situation, to make a prediction of what relationships would prevail if conditions change -- again, only an appeal to causality can do such.  (See Judea Pearl's book Causality, chapter 6)

Point 2) Causality cannot be the basis for a general description of reality because reality violates the assertion of independent variables needed for effective causal analysis ("no true zeroes" as you put it).  Reality doesn't even adhere to the laws of conditional probability [ http://www.stat.columbia.edu/~cook/movabletype/archives/2009/09/the_laws_of_con.html ] much less the structure of independence needed for causal analysis.

Image via Wikipedia

Point 3) There are no other contenders for general descriptions of reality besides statistics or causality.

Conclusion) SOL

So people, under the burden of responsibility, must maintain several models of reality, over smaller and larger domains of applicability, some statistical, some causal, some based on symmetry & curve fitting, some based on the laws of probability, some based on scientific laws, some based on economic laws, some based on rules of thumb, some based on multiple simulation runs, some hybrids.  These models compete against each other, at the cost of maintenance, data collection, computation, and comparison, with the benefit of correct probabilistic predictions of consequences of action/inaction, or the benefit of demonstrations of broad range of uncertainty that swamps discernment of effects between decisions.

And the sense of responsibility is made of shifting sands, and human values and goals are not static.  So you could pay all the costs for a model, just to dispense with it.

Image via Wikipedia

But all this *still* can be done for individuals or small groups.  Once you get past 30 members, what is rewarded are techniques for rubber stamping decisions already taken by the politically powerful, under the name of "objective analysis" for political cover.

So "small" decisions can be made quite well, with effort.  And "large" decisions are made quite poorly, because evidence of a cold calculated analysis would be blood on the hands of the politically powerful (besides, the ability to perform such analysis is in opposition to dumb loyalty, which is the most prized character trait of the privileged in-group).  But these "large" lousy decisions possess notoriety, and thus human appeal.  So a thousand pages each over describing a thousand theories chase after a relative small number of very poor decision making processes.

The consequences of all this may dim my sparkling optimism, so I must leave that as an exercise for others.

Math - could slack off, and still get good grades

To be honest, what I liked best about math was that I could slack off and still be at or near the top of the class. The harder the math got, the better I did, relative to the rest of the class.

Image via Wikipedia

Ultimately, I got a pretty decent undergrad math education from UCLA, took some graduate level algebra as an undergrad. I never had to rise above a loping slacker's pace.

Lacked and would have appreciated: category theory (zip zero zilch taught), better differential equations (I got a decent grade with no need to understand the subject, which is lame), continuous distributions and how they relate to the Fourier transform (if this was taught to me, I don't remember it).

It is shocking how much I need to learn now was invented/developed after I graduated from college. Before 1993 - No distributed revision control, no distributed operating systems, no decent theory of just-in-time compiling, no decent published real-world examples of Bayesian probability, Judea Pearl's theory of causality wasn't invented yet, no decent cryptographic hashes, no Bloom filters, no decent introductory texts on decision analysis. My wife says that is why she doesn't want our daughter to go into computers - you have to keep learning and forget the crap that just doesn't matter anymore. Whatever. I think it is inescapable that you have to keep learning just to keep somebody from eating your lunch.

Image by hyperboreal via Flickr

Frankly, I am glad I didn't do well enough, overall, to attempt a post-graduate degree in math or computers, back in 1993. A lot of what passes in academia today is pretty weak sauce, in applied math and computer science. God bless the IntarWebs. I can just grab the info I need and go. One problem, the tempo has increased.

Everything about causal inference in 40 pages

I will go over this with a fine tooth comb. I am looking forward to Judea Pearl's 2nd editon of _Causality_. I will have to transfer a whole bunch of margin notes!

Image via Wikipedia

My take. Causality is a hack. The structure of the universe doesn't guarantee anything that would causality a fool-proof technique. But it is very effective, and it is a built-in hack in the brains of humans. Humans handle causal relationships quite naturally and usually correctly in simple (and not so simple) situations. But, very effective or not, it is still a hack.

Hack or not, you have to use causality to understand the world, and make decisions about the world, and make rational actions inside of the world.

Image via Wikipedia

I would make the cycle-free directed graph the _definition_ of causality. Then there is the "do" operator on the graph, where a node is constrained to a particular value, chosen by analysis or chosen by a randomizer (from a range suggested by analysis).

In this causal graph, you can have "nodes" that, inside, contain

** relations from statical properties of historical data

** relations from statical properties of simulations, however the simulation is constructed

** relations from scientific laws

** relations from non-scientific laws - like "rules of thumb" and other relations that no one would defend as being laws of nature

** relations suggested by symmetry

** relations suggested by "smoothness" - filling in gaps or smoothing errors with "simple" equations

Image by disownedlight via Flickr

** relations suggested by naive Bayesian - using the Bayesian equation to add missing information, possibly quite naively

The analysis begins with 3 or more of these causality graphs - each chosen from a different discipline of analysis, and each chosen as the "simplest" thing that could possibly work, to start.

The graphs suggest experiments to carry out, and the graphs can be scrutinized.

The graphs change, more are added, some are deleted, just keeping 3 or more, with examples from different effective domains of analysis (one graph may be suggested by the laws of physics, another by the laws of economics, another strictly statistical from historical data, another from political science storytelling, etc.)

Some quite optimistic (borrowing the analysis of the most active participants) and some quite pessimistic (even so pessimistic so that a majority of current activity would not be advised by the model).

Each is must simpler than reality, and, thus, suitable to run against many many different future eventualities. Keep them simple so they are nimble and responsive tools.

None would really be defended as the complete, because so simple. The cost of any complexity is considered to be great. Simple models, easily manipulated, quickly manipulated.

We would have a personal standard of rationality, and a lower standard for semi-rational actions in a group (economic/social groups are demonstratively loath to hold themselves to the highest disciplines of rationality). For the personal standard of rationality - think Socrates. For the lower standard of group semi-rationality - think Rush Limbaugh and Glenn Beck. Neither is better or worse than another, each is to be used in their own domain. The real-world economic/social group would immediately feel paralyzing anxiety when presented arguments from people who hold themselves to the highest disciplines of rationality - so it is irrational to present that to them.

Everything about causal inference in 40 pages: "
Judea Pearl describes his new article Causal inference in statistics: An Overview as 'a recent submission to Statistics Survey which condenses everything I know about causality in only 40 pages.' That seemed like a bold claim, but after reading it I'm sold. I don't come from Pearl's 'camp' per se, but I found this a really impressive overview of his approach to causation. His overtures to folks like me who use the potential outcomes framework were much appreciated, although it is clear throughout that there is still intense debate on some of the issues. The bottom line: if you've ever wondered what the structural equation modeling approach to causal inference is all about, this is your one-stop, must-read introduction (and an insightful, engaging, and thorough one at that).
"

[Public Service Message] Jury Nullification: Please inform yourself about this fundamental right

[Public Service Message] Jury Nullification: Please inform yourself about this fundamental right

First a public service message... Please inform yourself of the right of Jury Nullification. This is a fundamental right you posses as a juror, that doesn't get a lot said about it. If the law is unjust, you have the right to acquit - to say "Not Guilty" - even in the face of the defendant's violation of the letter of the law.

Image via Wikipedia

You can read more about Jury Nullification here: http://www.law.umkc.edu/faculty/projects/ftrials/zenger/nullification.htmlt http://en.wikipedia.org/wiki/Jury_nullification I will not lie - exercising your right to Jury Nullification takes guts. I have never yet practiced jury nullification. In my jury duty career I have only had the guts to admit to the prosecuting attorney during jury selection that I could not find any defendant guilty of a drug offense, even if the defendant was a drug dealer (which was the nature of this trial). I was excused, and all I did was make the jury selection for this particular drug trial slightly harder to complete. I felt a little conspicuous back in the juror duty waiting room, when I returned, because the majority of people in my area are very conservative, and I don't welcome people pointing at me behind my back. But in the end, I felt not too bad, because through the whole process I was simply being completely truthful. If I was a twelfth juror, I am not sure I would have the guts to be the lone holdout and have the whol

Image via Wikipedia

e weight of the trial proceedings on my shoulders. So, what I did was a compromise, by being truthful to the prosecuting attorney during jury selection. But no matter what your personal values, it is important to not throw away your right to Jury Nullification, without some consideration first. After consideration, you can proceed as you see fit. OK, done with the public service message...

The book I would take if I had jury duty today would be Judea Pearl's _Causality: Models, Reasoning, and Inference_, because I have to review the book again (and review the notes I wrote in the margins in my own copy). It is a very difficult book - I was only able to read about six pages a day. The book is about Causality -- effect following cause. We take the concept for granted, but it has been on shaky ground since David Hume in the 18th century. But now, because of researchers like Judea Pearl, it is on solid footing. Here is the 5 minute summary: Write down all the possible causes and effects on a piece of paper - we will call these "points". Draw arrows from things you think are direct causes to the things you think are direct effects. Now you will have a bunch of "points" and "arrows" between those points. Look for loops - search for any "loops" that can be made by tracing your finger from point to point, always tracing your finger from arrow tail to arrow head. Are there zero loops? If so - good! - you have a description of causality, and you can use this diagram to understand the casual effects and interactions. Now, how can you tell if your diagram is, in fact, representative of the real world? Well, on the points of the graph, there is a "DO operation" linked with a manipulation you can do in the real world. Manipulations can be like fixing different variables in a scientific experiment, or like making a careful experiment that isolates the object of interest from anything that might mess things up. Consider these points: (A) Lawn Sprinkler near sidewalk; (B) Rain cloud over sidewalk; (C) wetness of sidewalk; (D) slipperiness of sidewalk; (E) number of people falling on slippery sidewalk Consider these arrows: (A -> C); (B -> C); (C -> D -> E) Draw it, run finger over graph, cannot find any loops - good! - ready to begin. We will try the DO operation on (C). On the graph, I just set "wetness" to "very wet" or "very dry" or whatever, and make a prediction. In the real world, I can either water down the sidewalk with a garden hose, or shield the sidewalk from the sprinklers with a tarp and shield from rain with a canopy and dry the sidewalk with a towel then a hairdryer. If the predictions match between the graph and number of people falling on the slippery sidewalk, I feel good that I have a graph representative of the real world. (This is a dumb example, because humans can handle the casual analysis of this situation with no problem. It only gets interesting as the graph gets more complicated.) Check the book out of your nearest university library (there is a 2nd edition coming out soon, but I wouldn't worry about it - no major revisions). Read only the last chapter, which is the text and slides from an informal lecture Judea Pearl gave. It is enough. The rest of the book is very difficult, or at least it was for me. I will probably have to read it two or three more times, on top of already reading it once carefully, to fully understand everything developed. My copy is heavily marked up, and I think I made a lot of mistakes, because I needed to think through it all more. So you probably don't want to borrow my marked up copy (not that I would actually lend it to you - the guy who lends out my books is sick today :P )