Overtime makes programmers less productive.

You have to be in the right mood, called "flow" by many, to be a good programmer. Different hackers can get into and out of this state more or less easily, and it is a large part of what determines their quality. When this blog was just a simple paper journal and not a multi-billion dollar media empire *, I spilled huge bottles of ink on wondering what the magic formula for instantly dropping into flow was. I've talked about it a lot with my friends in the years since then.

As the Tao and most physics teaches us, though, any powerful force has an equally powerful opposite. So it is with Flow, and I have experienced it: the Anti-Flow.

There's a lot of hacker mythology about working late into the night, eyes glazed over, blissfully hacking away on some deep problem, unaware of the outside world. It is generally acknowledged that this is a state of Being At One With The Machine. I believe that this is a false enlightenment. After this last round of crunch time, I have realized its true nature. After a certain number of hours of working, you think you're in flow. It seems as though everything is going well. What's really happening is that your mind is playing tricks on you: you're tired, you're making mistakes, but you're just not seeing them.

I think this is why the first programming methodology to get really bothered about programmer-written tests was also the first one to start complaining loudly that we should stick to a forty hour week. When you're just writing and writing and writing code, it doesn't occur to you that you're producing garbage, or if not garbage, at least substantially lower-quality code than usual. You hit your testing phase later, and you fix your bugs, and you shrug and think "that's software for you". It seems like you're productive on both ends, since you're producing lots of code when you're tired and you're fixing lots of bugs when you're awake.

But when you're sitting there bleary-eyed and trying to get your tenth test to pass and it just won't pass no matter what you do, you know it's time to quit. You have a real-time feedback loop that's telling you "go home". After a while, you put two and two together and realize that you're not really understanding the machine as much as you thought you were.

Of course, if you're not as rigorous as you should be, you just stop writing tests.

This week I watched a programmer who is normally incredibly fastidious about testing give up on his tests, work tons of overtime, jang it, and endure the consequent failure. I have done this myself many times when working overtime. I am much less likely to test code when I'm burning myself out than when I'm relaxed. It seems like the tests are just so much less likely to pass anyway, so why bother?

The moral of the story is that crunch time really doesn't work in the small any better than it does in the large. If a deadline looms in the near future and it looks like we're going to need overtime to make it, I am going to have to admit to myself and to my team earlier on that the deadline has already been blown.

I wonder how many more times I'll need to learn this lesson before it sticks.

---

*:

International guests: a gratuity of 15-20% is appreciated if you enjoyed the service, and may be included with the check for parties larger than 10.

I worked quite a lot over the last few days. Normally I really don't like crazy overtime, having had a very bad experience with it at Origin. Although we haven't really committed to show anything (and I'm not presenting a paper), I do feel as though our demo should be strong for OSCon. I'm sorry I have been a bit hard to reach because when I'm not working, I'm sleeping. I'll still be working for some of this weekend, and in keeping with recently-established tradition, when I'm not working or sleeping I'll be driving.

As soon as this server upgrade is over, I'm going to have a nice, long contemplation (along with my coworkers) of how things went wrong, and hopefully this crunch will be an isolated occurrence.

I've lost some data on my LJ. The last entry, in particular, and two comments associated with it were particularly annoying to lose, since I'd deleted my offline backups recently, assuming that LJ was never going to lose anything.

My support request is here, and a lj_maintenance entry detailing the fix that didn't work for me is here.

Let this be a lesson to you all: backups are not a way to restore your data, they are a magic talisman against data loss.

I'll start posting these to the community blog in the future, but since more people read this one right now, I'll put it here. Since to some extent our process is public, and I get a lot of questions, I feel like I should mention what I'll be focusing on in the coming week. I hope that those of you looking into the fishbowl find this interesting ;-).

This week I am endeavoring to get out of the process business. Over the last month or two, I've been drawing up schedules for the team, trying to get us (and myself) organized. While I seem to be able to imagine the process quite well, I'm not organized enough to pull it off, and my coding has been slipping behind while I'm doing this. So, I'm turning as much of it as I can over to Mark, our new "marketing" guy. While this is slightly out-of-scope for someone doing general marketing, it should give him the necessary insight into how the product works to describe it clearly to our audience.

So, that's what I won't be working on. What I will be working on is our scalability solution, so that we can start adding hardware when we get more customers. While simple in principle, I've been banging my head against it, with Allen's help, for several weeks now. I think I will be able to finish it up this week, if everything goes well.

Whether everything usually goes well or not is left as an exercise for the reader.

Well, I went to the mall to price some noise-cancelling headphones. WOW. Those are some absurdly expensive little toys. I still think I want some, but it's going to require a little more careful consideration.

The ideal headset for me would be around-the-ear, collapsable, USB, linux- and mac-compatible (e.g. a standard USB audio device, not some crazy third-party thing that requires drivers), with active noise cancelling earpieces, a noise cancelling microphone, and of course standard features like in-line volume control and a microphone mute switch that doesn't produce a deafening bang when used. (I did have a headset which did that.) It should also cost one dollar and be bullet-proof.

Obviously I'm willing to make some compromises, since I can't even find the union of "noise cancelling earpieces" and "microphone". Any suggestions?