Revenge of the Duct Tape Programmer - skipping unit tests to ship faster

Image via Wikipedia

Hey, something like proof that going without unit-tests will let you ship faster, but at the penalty of technical debt in the code base. (I realize that isn't _exactly_ what was demonstrated ;)

Exploding Software-Engineering Myths - Microsoft Research: "

“The nice thing about working at Microsoft,” Nagappan says, “is that the development organization is large enough that we could select teams that allowed for an apples-to-apples comparison. We picked three development projects under the same senior manager and looked at teams that used TDD and those that didn’t. We collected data from teams working on Visual Studio, Windows, and MSN and also got data from a team at IBM, since the project was a joint study.”

The study and its results were published in a paper entitled Realizing quality improvement through test driven development: results and experiences of four industrial teams, by Nagappan and research colleagues E. Michael Maximilien of the IBM Almaden Research Center; Thirumalesh Bhat, principal software-development lead at Microsoft; and Laurie Williams of North Carolina State University. What the research team found was that the TDD teams produced code that was 60 to 90 percent better in terms of defect density than non-TDD teams. They also discovered that TDD teams took longer to complete their projects—15 to 35 percent longer.

Image by Getty Images via Daylife

“Over a development cycle of 12 months, 35 percent is another four months, which is huge,” Nagappan says. “However, the tradeoff is that you reduce post-release maintenance costs significantly, since code quality is so much better. Again, these are decisions that managers have to make—where should they take the hit? But now, they actually have quantified data for making those decisions.”"

The Duct Tape Programmer vs. The Test Infected Programmer

Image via Wikipedia

Joel likes JWZ. They both don't care for unit tests. Joel points to JWZ as a guy who ships. You could argue that JWZ was a guy who shipped, and became a burn-out poster child. The Netscape stuff just wasn't good enough to withstand the Microsoft onslaught. If you take JWZ + (Some) Unit Test + (Some) Test Infected Attitude, you really got something. I am happy to compete against coders who religiously avoid writing unit tests, so I am glad there is considerable variation in opinion. So, keep it up, Joel! The Duct Tape Programmer:

"Zawinski didn’t do many unit tests. They “sound great in principle. Given a leisurely development pace, that’s certainly the way to go. But when you’re looking at, ‘We’ve got to go from zero to done in six weeks,’ well, I can’t do that unless I cut something out. And what I’m going to cut out is the stuff that’s not absolutely critical. And unit tests are not critical. If there’s no unit test the customer isn’t going to complain about that.”

Image via Wikipedia

Remember, before you freak out, that Zawinski was at Netscape when they were changing the world. They thought that they only had a few months before someone else came along and ate their lunch. A lot of important code is like that.

...

Duct tape programmers have to have a lot of talent to pull off this shtick. They have to be good enough programmers to ship code, and we’ll forgive them if they never write a unit test, or if they xor the “next” and “prev” pointers of their linked list into a single DWORD to save 32 bits, because they’re pretty enough, and smart enough, to pull it off."

32 bits saved on a linked list? I'm convinced! Duct tape ahoy! OK, what is the alternative?

• Look at the Software Carpentry website: http://software-carpentry.org/
• Look at the automated testing sections.
• Experience automated testing improving product.

• Image via Wikipedia

  Image via Wikipedia
• Become "test infected", and stay away from people who are religious "Test Driven Developing" time-wasters.

[Edited 10/23/09] Found a very cute comment thread on jwz's own website:

It seems to me more like you use foresight and pessimism to avoid getting into situations where you need to demonstrate exceptional programming ability. Absolutely no offense, even by faint praise, intended. ... "Use foresight and pessimism" This needs to become some sort of meme metric. ... The formulation owes something to a pilot's maxim: "a superior pilot uses his superior judgment to avoid having to exercise his superior skill."

This sums up Spolsky's mistake - confounding hard-won pessimism about fruity techniques with making a conscious decision to accrue short term technical debt. [exhaustive definition of technical debt here by Brad Appleton http://bradapp.blogspot.com/2009/06/technical-debt-definition-and-resources.html ] If practically 100% of the value of a piece of code comes from the speed of publishing, and you are skilled enough to take on some technical debt, confident you can pay it back later, by all means, go ahead. Publish the code, and forget the unit-tests and forget test-driven-development. But only if you earned that confidence. Here is an excellent diagram about who can and who cannot make this judgement, by Martin Fowler: http://martinfowler.com/bliki/TechnicalDebtQuadrant.html [Edited 10/26/09 ] I found this comment from Reddit by terror406: http://www.reddit.com/r/programming/comments/9nipa/joel_on_software_the_duct_tape_programmer/c0dj2dt

Architecture Astronauts and Duct Tape programmers are fictional characters. There are only competent and incompetent programmers. And competent programmers love and understand elegant architecture, but also know when to go into 'duct tape mode' on order to get the job done and ship the damn product. Edit: There is an easy way to tell the two apart. Ask them about their Technical Debt. A competent programmer will start talking about all the stuff that could have been done better, the incompetent programmer will just give you a blank stare.

Again, the issue is having the skill and judgment to take on short-term technical debt.

Armin Ronacher knows the correct way to use Python's "super"... Do you?

Tweet from "mitsuhiko" (Armin Ronacher) on the mis-uses of Python's "super".

Image of Armin Ronacher

http://twitter.com/mitsuhiko/status/2438234176 "super" is the built-in function that traverses the Method Resolution Order of bases classes for delegating work inside of a class method. This could be non-trivial, because Python supports multiple inheritance. It is very nice to have a built-in function to do this, but this is the correct way to call "super":

class TypeOutTheClassNameHere(Class1, Class2):

    # methodname - method provided to support delegation

    def methodname(self, x, y, z):
    
        # If you wish for Side-Effects...
        heavy_lifting1(x, y, z)
        
        result = super(TypeOutTheClassNameHere, self).methodname(x, y, z)
        
        return heavy_lifting2(result)
        
# greetings to library users:
# TypeOutTheClassNameHere is ready to be sub-classed!

class OtherClass(TypeOutTheClassNameHere):

    def methodname(self, x, y, z):
    
        heavy_lifting3(x, y, z)
        
        result = super(OtherClass, self).methodname(x, y, z)
        
        return heavy_lifting4(result)

Really Important Point: without unit-testing of the implementation and the success of delegation, using "super" is pure vanity. "super", by itself, cannot magically make your code handle multiple inheritance correctly! Not only do you have to test your class, you have to test that future users of your code, inheriting from your class, will get correct behavior. Yup, your unit-test suite may include creating one-off classes for testing! Armin's tweet demonstrates the prevalence of incorrect use of "super". If you fail to "TypeOutTheClassNameHere", any sub-class to your class will break. http://www.google.com/codesearch?q="super(type(self)"&hl=en&btnG=Search+Code Michele Simionato has a great three-part write-up called "Things to Know About Python Super". ( Part 1, Part 2, Part 3 ) The complexity in any given case is not great, the complexity comes only from considering every single corner case. My view is that unit-testing is a must to make sure the corner cases you are interested in is correctly implemented. Really Important Point: If you have no interest in supporting multiple-inheritance, and you have no interest in supporting sub-classing, don't use "super". It will simply mislead users of your code as a library that you investing in engineering and testing.

Image via Wikipedia

"super" is an advertisement to the world that you invested in the engineering and testing to support multiple-inheritance and sub-classing! Don't use it if you don't mean it!

I like "super". I only user "super" when I am supporting multiple-inheritance and sub-classing. I write unit-tests whenever I write code with "super". Please take these issues into consideration, for your library code.

Joel Spolsky sometimes sucks. But we already knew that...

Joel Spolsky says "100% unit test coverage" should not be a Joel Rule:

From Podcast 38: starting at [42:28]...

... "You should have a 13th thing on here: Unit Testing, 100% unit tests of all your code."

And that strikes me as being just a little bit too doctrinaire about something that you may not need.

Glyph (Mr. Twisted) nails it, about why Joel is wrong, wrong, wrong.

Deciphering Glyph: The Joel Un-test

He goes on and on about how the real measure of quality is whether your code is providing value to customers, and sure you can use unit tests if that's working for you, but hey, your code probably works anyway.

It's a pretty weaselly argument, and I think he knew it, because he kept saying how he was going to get flamed.  Well, Mr. Spolsky, here at least that prediction has come true ;-).

It's weaselly because any rule on the Joel Test could be subjected to this sort of false equivalence.

However, I think Glyph drives his argument over a cliff.  In comments, Glyph says some pretty indefensible things responding to the valid concerts of commenter Andrew Dalke:

(Note: I am butchering Glyph's comments, to make him appear ridiculous.  I would feel guilty, if it wasn't my absolute goal in life, to make everyone appear ridiculous.)

You might be right about this test being so hard to write. If so, MySQL and C++ are bad places to work, because testing there is unnecessarily difficult.

...

Should the developers stop working in C?

Yes.

Frankly, I think 100% unit test coverage can sometimes be completely at odds with delivering quality code to customers.  Completely at odds, because sometimes it takes considerable time to cover that last 1% of the code paths, and your customer are practically begging you to put that time on new functionality.

I would write the "über-Joel" rule as:

"100% unit test coverage, unless you are willing to advertise to your customers exactly which sections of code only have 99% test coverage of all the code paths"

If I was a customer, and the developer told me that the last 1% of comprehensive unit test coverage would take 20 hours of development away from new features, I would give him a pass.  If other customers are not as lenient, then the developer should make a choice between that particular customer's patronage and the effort to build a truly comprehensive test rig (in this case, 20 hours of developer time).