Often it is the means that justify the ends: Goals advance technique and technique survives even when goal structures crumble.

- Alan Perlis, Epigram 64, "Epigrams in Programming"

How does a computer programmer get better at computer programming?

In a very broad sense, there is some consensus on the answer to this question.  It's like the answer any other skill.  "Do a lot of it."  Much like other skills, though, while necessary, this condition is not sufficient.  If you want to get really good at punching, you can punch a brick with your thumb inside your first ten thousand times, and someone who has learned to do it correctly and had a fraction of the equivalent amount of practice will be a lot better at punching than you.

So, to expand on that consensus, to get better at programming, you have to learn to do it right, and then do a lot of it, hopefully with someone correcting your mistakes and helping you do it better each time.  Like any other kind of serious training.

This concept is pretty old.  Shortly after the dawn of time, the dragon god-emperor pioneered this concept when he invented a martial art that is now popularly known as "Kung Fu", or, literally, "achievement through great effort".  The term does not necessarily refer specifically to martial arts, and can (in both Chinese and English) refer simply to the fact that someone has achieved something exceptional through a lot of effort and practice.

Specious metaphors between computer programming and martial arts (and various¹ eastern philosophies) have existed for almost as long as computer programming itself.  As a cocky and callow youth, I occasionally had these metaphors generously applied to me, but they rang hollow even in my hubristic state of mind.  I knew that despite any talents I might have, at the very least, I hadn't spent a lifetime of effort achieving a near-supernatural mastery of computers.  I had just messed around with C++ and Java a little bit.  The appellation of "master" was inappropriate, but I didn't know how one really did go about getting to that point.  I had already met a few folks in the programming profession who were old, but certainly not yet wise, and I wanted to learn how not to end up like that.

Whenever two programmers meet to criticize their programs, both are silent.

- Alan Perlis, Epigram 101, "Epigrams in Programming"

I first noticed that the parallel between UQDS and these various arts and philosophies when a friend observing Divmod from afar mentioned that our review process was intimidating.  Intrigued, I conducted a straw poll among my programming friends who are not deeply immersed in the eternal struggle of the Twisted or Divmod codebases, and found that there was a fairly consistent impression: the exchange of comments between authors and reviewers was highly unusual, intimidating, and impressive.

It's unusual because most people cannot read code well enough to comment in as much depth.  Code review is becoming more popular, but it's still rare.  Even intense, newfangled "Agile" processes like Extreme Programming focus more on discussing code as it's being created than reading code which has already been written.

The exchange is intimidating because many of the comments are extremely fine-grained and not very forgiving.  It isn't uncommon to see comments about a tiny private method missing documentation, or test coverage doesn't hit a single error condition.  This causes the reader to muse on the ways in which their own code would fail if it were subjected to such intense scrutiny.

Finally, the exchange is impressive because the factors which make it intimidating make the programmers doing it seem extremely skilled.  If you're looking at work which is being subjected to criticism only on details which seem insignificant, it implies that it is in every other way correct.  To some extent this impression is illusory, because the reviewers are only human and may be missing more important things.  For something as complex and intricate as software, though, it's impressive assuming that the reviewers tried at all.

I can't pretend that UQDS is an entirely unique process in this regard.  The Python and Subversion projects practice regular code reviews in public, and those are just the ones I know off the top of my head.  This isn't a phenomenon unique to open source either - I know that both Canonical and Google do fairly comprehensive code review on their closed-source products.  I've heard milder expressions of awe - although perhaps they were only milder because they came from friends - about those projects and companies.

While it isn't unique, UQDS is particularly intense.  Some other processes try to be "reasonable" - which is, on the face of it, a reasonable thing to do.  Small, "obviously correct" fixes are sometimes allowed without tests.  "Security" fixes are sometimes allowed in for expediency because of their urgency.  Some committers are sometimes exempt.  Some reviews are done after the fact.  Documentation is often optional, since reviews are meant to enforce correctness, not every aspect of quality.  For better or worse, UQDS sets out a much stricter guidelines: everything must be tested.  Everything must be documented.  Nobody is allowed to bypass the process.

It isn't all roses.  Especially when a contributor needs to update an area of code not originally developed under this regime, it can be frustrating to need to document and test and correct code that isn't really related to their goal.  Sometimes the overabundance of the opportunity for review creates interminable discussions about a design, when there is really a need to just go with a solution that works for now.  There may be some room to adjust the constraints to place less of a burden on those who want to contribute small improvements, and that's difficult without compromising on overall quality.

Despite their drawbacks, these rules plus a public ticket tracker add up to something that the industry of martial arts has known about the power of for quite some time: the exhibition.  If you're going to run a martial arts studio you need to attract students, and to do that you need to "wow" people every so often.  Martial arts are about defense, mind-body integration, health and fitness, and self-discipline, but putting that on a poster isn't going to fill up the thursday evening class.  Sometimes you need to get your school together and show off just how bad you would be able to kick some ass with your skills, you know, just in case anybody was wondering.

I am quite proud of the fact that I've received numerous unsolicited comments about the quality of the code itself within certain parts of Divmod and Twisted.  Like any programmer I've received my share of complaints about my work, too (especially the stuff that precedes all this intense review), but as Steve Yegge famously observed, most programmers who are famous for their programs don't actually write programs; they write about programs, or they wrote some programs nobody has the code to.

I'm almost more pleased with the idea that I could help pave the way for programming to have some real rock-stars on the merit of their code than the idea of being a rock-star myself.

Dealing with failure is easy: Work hard to improve. Success is also easy to handle: You've solved the wrong problem. Work hard to improve.

- Alan Perlis, Epigram 101, "Epigrams in Programming"

Or, 72/72 to be precise.

Twisted is once again in danger of losing the engineship of your Internet!  If you haven't voted on this Jyte claim already, please do so now!

Unfortunately, if you have questions about how to write and run tests using Twisted's "trial" tool, this work in progress is still the best answer I can give you.  It's a good work in progress, but it's definitely not finished.

If you need some documentation, please feel free to provide feedback and patches at Twisted's ticket number 2443.

I am working on putting together some ramblings about test-driven development of my own, but at this rate it's going to be quite a while until I'm ready to post them here.

If you have some public SSH keys, you should really upload them to Launchpad.

This is one of the lesser-known but extremely useful features of Launchpad.  When I'm adding accounts for people to one of my many, various machines, I always have to figure out a secure channel to upload/download keys or deal with the temporary exposure of a plaintext password.  Since Launchpad takes great care to encrypt both upload and download of sensitive data (including keys) it makes an ideal place to exchange them.

So, if you think you are the sort of person who I might need to give an account to for some reason one day, save us both some time and go upload your keys right now.

One of the first inklings the world had of the upcoming release of the game "Halo" was a cryptic website showing various obscure quotations which appeared to be from some ancient religious text.  Pre-acquisition, I always loved their sense of style; I am a huge fan of the Marathon series.  My favorite quote from the collection, which stuck with me for years, was:

Our conviction is like an arrow already in flight.
Your life will only last until it reaches you.

Today, I recalled this quotation as I noted a few things going on in my daily trip around the tubes:

• Dell will be offering an option to preinstall Ubuntu on its PCs.
  

  "Linux-based PCs was an "overwhelming" request" ... "We heard loud and clear from customers that they wanted this," Cook said. And of those who wanted Linux, "80 percent came back and said Ubuntu," Cook said.

• While Microsoft is putting a substantial marketing push behind Silverlight, even mainstream press are asking: what about Linux?  Of course, developer after developer are immediately asking the same thing.
  

  "Linux lets us say to the customer, 'This thing is ubiquitous,'" ... "Otherwise, we have to build a version of the Web site for Linux, which is expensive."

• The Twisted website's traffic from Linux has broken 20%.  (non-Windows traffic, including MacOS, is at 36%). The interesting thing about this is not the number - I realize that we are on the cutting edge of technology and so on - but the trend.  People are migrating away from Windows.  Nobody is going back.
  

Of course, Paul Graham already called this, so if anyone feels like they should be keeping score, by all means give him the credit.  But I think that Microsoft's problems are both a lot worse than the malaise that Mr. Graham describes.  It's not a question of large, abstract shifts in the focus of new technology developers.  I wouldn't say that they're dead, but they're definitely dying.

Microsoft's real problem is the much simpler business problem that their flagship product is clearly inferior its lower cost alternatives.  It doesn't help them much that the "lower" cost is, in many cases, zero, but that's beside the point.  I have made, and will make again, arguments about freedom and scaling cost and a number of other social issues surrounding code in general and operating systems especially, but all that is going to end up being peripheral.  I'm pretty sure that I'd continue to use the Ubuntu-derived operating systems for my own computers even if they somehow cost more than the Microsoft equivalents.

Of course, we've all heard this kind of thing before from the archetypical linux snobs on slashdot and other forums, people who seem incapable of recognizing that any value can be created by people who disagree with them.  Personally, I have purchased a Windows license - not just a computer which happened to have it included - as recently as last month.  (Although I must admit that my sense of practical compromise goes only so far.)

I can certainly understand some skepticism that Microsoft will topple and Linux will be ubiquitous in the home and office any time soon.  Certainly that is not going to happen soon  (for sufficiently software-industry-ish definitions of "soon" — which means they could be out of business but for cash in the bank within a year).  With Windows marketshare hovering somewhere between 80 and 95 percent, depending on who you believe, Canonical has a long way to go to fix bug 1.

On the other hand, one could make the case that it's happened already.  In the office?  Dozens of large companies, Red Hat foremost among them, make a comfortable living on corporate Linux deployments.  Sure, in most cases it's not on anyone's desktop, but it is making inroads.

In the home?  There are already 1 million PS3s in the US, and the current generation console market is staying rather cool - if it heats up to the levels of the previous one, the numbers could be more like 50 million.  Even when it's running games, the PS3 is a linux box, and to some people, it is in a much more literal sense.

Mark Shuttleworth himself says that there are about 8 million Ubuntu users alone.  That's not just a measurement error.

As with the Twisted site, however, the interesting thing here is not the number, but the trend.  Of course pundits and the mainstream press are going to look at the numbers they can see and not even consider the possibility that Windows could be dislodged.  By the time the idea seems reasonable and possible, though, the adoption curves will be in the process of going exponential as the new network effects combine with the advantages of the platform that already, now, have people migrating away from the current network effects.

If you're watching the arc of an arrow in flight it's quite fast.  Hard to predict, and often effectively invisible thanks to its motion.  By the time it's stopped moving and is easy for everyone to see, it's already arrived.