This started as a posting about Statistical Causality...

Interesting blog posts back and forth between Don Rubin, Judea Pearl, and Andrew Gelman. I cannot understand the original point of contention between Don Rubin and Judea Pearl. I have no idea what "M-bias", "controlling for pre-treatment versus post-treatment variables" mean. http://www.stat.columbia.edu/~cook/movabletype/archives/2009/07/disputes_about.html

Image via Wikipedia

Happily, the fundamental point of disagreement seems to be getting clear with a comment by Philip Dawid (who offers a very nice on-line document "Principles of Statistical Causality" which covers a lot of concepts in only 94 pages). This is a very clear summary of what is meant by "causal inference":

Like Pearl, I like to think of "causal inference" as the task of inferring what would happen under a hypothetical intervention, say F_E = e, that sets the value of the exposure E at e, when the data available are collected, not under the target "interventional regime", but under some different "observational regime". We could code this regime as F_E = idle. ... It should be obvious that, even to begin to think about the task of using data collected under one regime to infer about the properties of another, we need to make (and should attempt to justify!) assumptions as to how the regimes are related.

(Emphasis on the turn of phase that caught Andrew Gelman's eye) What is the nature of the discipline of "statistical causal modeling"? Beats me. I have the idea of several DAGs (gaps filled in with plausible causal contributors), several multi-dimensional collected sample distributions (gaps filled in with smoothing or naive-Bayesian techniques), several simulations based on plausible natural or un-natural laws (simulations providing multi-dimensional distributions). Between creatures of all three types, we test them against historical data, future data, and intervention experiments. They fight against themselves, and only a few remain. I have the idea of defining a distribution as the opportunity to open a sample portal to an alternate universe to the same sample stream. You can open sample portals at a certain rate, so the total number of samples available to you has a rate that increases as the square of time. You desire the sample portals to all have the same correct driving distribution. Now, obviously, you cannot actually create sample portals to alternate universes. So you create more fake sample portals. Each has the possibility of one or more failure modes. Taken all together, these form a pessimistic analysis. Human action is inherently optimistic, so, take the opportunity to make explicit biases of personality and human real-world population studies of life outcomes consistent with living a happy and fulfilled life. The third leg of the stool is a model of human effectiveness:

• rationality,
• choice,
• free will,
• changes in capability,
• changes in habitual action,
• goal-directed action,
• effectiveness,
• consciousness,
• self-knowledge/introspection,
• knowledge of the world,
• consistency between behavior and professed mental abstractions of desired behavior,
• and morality -

leading to the outcome of a happy and fulfilled life. That is what I got right now. The way to study how "human action is inherently optimistic" is with happiness research (covered in Daniel Gilbert's Stumbling On Happiness, and the sections on happy outcomes in Dan Ariely's Predictably Irrational). The model of human effectiveness is based on there being only a few chances over time for exercising free-will. We greatly limit the role of free-will, because human behavior is better explained by:

• habitual actions,
• personality,
• IQ,
• daily exercised capability,
• daily exercised responses,
• environment,
• etc.

We expect to see meaningful change caused by free-will events over the course of 10 years - just because there are so few meaningful free-will events during those many years.

Image via Wikipedia

Like Julian Jaynes arguing in The Origin of Consciousness in the Breakdown of the Bicameral Mind that consciousness only came into existence 3000 years ago from stresses on growing populations searching and competing for sustaining resources, I don't think that free-will is a capability of all humans since Homo sapiens originated 200,000 years ago. I think meaningful free-will also is a recent phenomenon. And I don't think it is expressed in all people - it only gets exercised under a peculiar stress of personality and environment, when the capability exists. I reject the idea that free-will is simply random behavior, or the absence of a mechanism for the deterministic prediction of behavior. It cannot be meaningfully separated from these: rationality, goal-directed action, effectiveness, consciousness, self-knowledge/introspection, consistency between behavior and professed mental abstractions of desired behavior, morality, and other issues. The reason for the need to consider time periods on the scale of a decade is that free-will events are the residue of actions and behaviors that cannot be explained by simpler means. Because it is a residue, we are only interested if there is a over-riding consistency and progression - because we don't want to give the label of "free-will" to an odd-ball collection of junk. That's enough for now. Added: Andrew Gelman explores issues further: More on Pearl's and Rubin's frameworks for causal inference He describes the idea of using Minimal Rubin, Full Rubin, Minimal Pearl, Full Pearl as techniques for statistical causality. And this note on the benefit of Rubin's approach:

Be explicit about data collection. For example, if you're interested in the effect of inflation on unemployment, don't just talk about using inflation as a treatment; instead, specify specific treatments you might consider (adding these to the graphs, in keeping with Pearl's principles). This also goes for missing data. ...

I don't understand the section on "Controlling for intermediate outcomes". Pearl then contributes a long and difficult comment. Gelman and Pearl argue over this: "the correct thing to do is to ignore the subgroup identity" - I just don't understand this at all.

Poopland Duck

Drew this on the back page of the Logitech desktop speaker LS11 Quick-start guide. I like these speakers because they have a headphone jack and over-sized power/volume control knob. Unlike the image on the Logitech website, they do not hover in space. USD 19.99

Group Selection - Altruism born from warring, genocidal tribes

Article in The Economist June 6th 2009, p. 77. Taking about research of Samuel Bowles (in Science) "Did Warfare Among Ancestral Hunter-Gatherers Affect the Evolution of Human Social Behaviors?"

Science 5 June 2009: Vol. 324. no. 5932, pp. 1293 - 1298 DOI: 10.1126/science.1168112

(Also work by Mark Thomas "Late Pleistocene Demography and the Appearance of Modern Human Behavior", summarized by NPR "Larger Populations Triggered Stone Age Learning". About population density needed to support technologies, and how population fluctuations caused technologies to be lost, only to be independently discovered later, again and again.)

Image via Wikipedia

Group selection takes place by warring, genocidal tribes (a process described as "genetically terminal" for the losers). This group selection was found to be mathematically compatible with a hard "selfish gene" approach to natural selection. With this approach, there is selection for altruistic traits. Is this the basis for all actual group selection? The more I read, the more I see that a hard "selfish gene" is the correct way to approach natural selection. Is this mechanism of warring, genocidal tribes all we have to make actual group selection?

Searing Acids Of Pain

Original drawing by my daughter. The problems of many acids.

Pajama Railroads

This drawing is a collaboration between my daughter and myself.

Why Dynamic Languages? Why Strongly Typed Languages?

From Reddit commenter "munificent", the best analogy of why productive coders use Dynamic Programming Languages and Strongly Typed Programming Languages (best analogy I have ever seen...):

> > So, the problem is that software developers have poor foresight and a complete lack of self-awareness? No, the problem is that the success of most software projects can not be predicted.

Image via Wikipedia

When you go camping, you just pitch a tent. When you're moving, you build a house. With many software projects, it's impossible to tell beforehand if you're just going camping or will be taking permanent residence. It doesn't make sense to spend a month building a new house every time you go somewhere, if 90% of the time you end up leaving after a couple of days. What does make sense, and is becoming a common pattern is this: 1. Stake out a new territory and pitch a tent (v1 in a dynamic language) 2. If it turns out to be a hospitable place, start building a new house next door (migrate back-end services to more strongly-typed languages). 3. Once that's done, ditch the tent and move in (move the production system over to the new back-end). See: twitter (Ruby -> Scala), Facebook (PHP -> Erlang), etc.

Image by ThaRainbowRaider. via Flickr

Yes, a lot of academic computer science completely ignores the economic constraints of software engineering.

United States Republicanism in 2009

A good summary from Reddit commenter "jimmyvanl" (who deleted his account the day after he posted this - strange), about the "really-existing" structures and properties of United States Republicans and Democrats:

Image by Roberto Rizzato ►pix jockey◄ via Flickr

American political parties were historically more organized along power coalitions than ideology. Back before the 1960s, the Democratic Party was an uneasy coalition of Southern racists, who still hated the Republicans for Lincoln, and the Northern urban working class and intelligentsia, which supported Roosevelt's New Deal and hated the Republicans for their refusal to fund social spending. These two groups didn't have much in common, but they cooperated to win national elections. When Presidents Kennedy and Johnson supported Civil Rights, though, the Southern Democrats bolted the Party and became Republicans. As Southerners and fundamentalist Christians took over the GOP, Northern and Midwestern professionals, who were the core of the old Republican Party, began to join the Democrats. So today's Democratic Party is basically a coalition between racial minorities, trade unionists and educated professionals. The GOP, conversely, is a coalition between the ultra-rich who don't want to pay taxes, beneficiaries of the military-industrial complex who want America in more wars, and lower-class Southern whites who believe they're somehow victims of an "elitist" conspiracy because they didn't apply themselves in school. Most of today's racists are in that last category.

Creepy to have it broken down all in one place. Now, from an article from SomethingAwful accurately describing how Sarah Palin burst unto the United States political scene (hat-tip to Reddit commenter "cthulhulou"):

Image by Getty Images via Daylife

Brutalized by eight years of Bush prosperity, the American people experienced a fit of sanity and elected the center-right corporatist and his goofball sidekick instead of the warmongering economic neophyte and the winking, know-nothing, hockey mom hate shit he took on the face of America.

Politics is an infection of the soul, and devours the ability to exercise personal free will and rationality. I am trying to push my internal political beast into a tiny cage.

Free will - Search for it among failure

What does free will look like?

• Image by greekadman via Flickr

  Externally Perceived Failure (or at least lack of perceived success)
• Externally Perceived Failure
• Externally Perceived Failure
• Externally Perceived Failure
• Private Moral Success
• Externally Perceived Failure
• Externally Perceived Failure
• Externally Perceived Failure
• Externally Perceived Failure
• Higher, Private Moral Success
• Externally Perceived Failure... etc.
• Rinse and Repeat

Anything else is indistinguishable from simple diversity of habitual actions, or actions triggered by environment. This has to do with having the guts to risk failure, to rise to the challenge of moral goals.

Armin Ronacher knows the correct way to use Python's "super"... Do you?

Tweet from "mitsuhiko" (Armin Ronacher) on the mis-uses of Python's "super".

Image of Armin Ronacher

http://twitter.com/mitsuhiko/status/2438234176 "super" is the built-in function that traverses the Method Resolution Order of bases classes for delegating work inside of a class method. This could be non-trivial, because Python supports multiple inheritance. It is very nice to have a built-in function to do this, but this is the correct way to call "super":

class TypeOutTheClassNameHere(Class1, Class2):

    # methodname - method provided to support delegation

    def methodname(self, x, y, z):
    
        # If you wish for Side-Effects...
        heavy_lifting1(x, y, z)
        
        result = super(TypeOutTheClassNameHere, self).methodname(x, y, z)
        
        return heavy_lifting2(result)
        
# greetings to library users:
# TypeOutTheClassNameHere is ready to be sub-classed!

class OtherClass(TypeOutTheClassNameHere):

    def methodname(self, x, y, z):
    
        heavy_lifting3(x, y, z)
        
        result = super(OtherClass, self).methodname(x, y, z)
        
        return heavy_lifting4(result)

Really Important Point: without unit-testing of the implementation and the success of delegation, using "super" is pure vanity. "super", by itself, cannot magically make your code handle multiple inheritance correctly! Not only do you have to test your class, you have to test that future users of your code, inheriting from your class, will get correct behavior. Yup, your unit-test suite may include creating one-off classes for testing! Armin's tweet demonstrates the prevalence of incorrect use of "super". If you fail to "TypeOutTheClassNameHere", any sub-class to your class will break. http://www.google.com/codesearch?q="super(type(self)"&hl=en&btnG=Search+Code Michele Simionato has a great three-part write-up called "Things to Know About Python Super". ( Part 1, Part 2, Part 3 ) The complexity in any given case is not great, the complexity comes only from considering every single corner case. My view is that unit-testing is a must to make sure the corner cases you are interested in is correctly implemented. Really Important Point: If you have no interest in supporting multiple-inheritance, and you have no interest in supporting sub-classing, don't use "super". It will simply mislead users of your code as a library that you investing in engineering and testing.

Image via Wikipedia

"super" is an advertisement to the world that you invested in the engineering and testing to support multiple-inheritance and sub-classing! Don't use it if you don't mean it!

I like "super". I only user "super" when I am supporting multiple-inheritance and sub-classing. I write unit-tests whenever I write code with "super". Please take these issues into consideration, for your library code.

Using an inner function for breaking out of nesting

From Fuzzyman, a blog post on different ways of handling breaking out of nesting. Here is a snippet:

def find_match():
    for x in range(max_x):
        for y in range(max_y):
            if match(x, y):
                return x, y
result = find_match()
if result is None:
    # match not found
else:
    x, y = result

Yup, looks clean to me, and the named "result" and "inner_function" give the opportunity for self-documentation, with appropriate names instead of "result" and "inner_function".

Image by mikebaird via Flickr

For your esthetical enrichment, a picture of a nest!