(Unfortunately Brief) Austin Retrospective

The week before last I took a trip to Austin, Texas. This was unfortunately in lieu of both PyCon 2006 and ETech 2006 since we couldn't get the whole Divmod team to either of those events. Sorry, disappointed conference-goers! We'll make it next year for sure.

It was a half-vacation half-bizdev trip. It was really good to get a break from the day-to-day minutia of everything that needs to be done on Divmod's infrastructure, and take a look at the view from a thousand feet up, and think more about how awesome our team and technology are. Luckily we are pretty awesome so it was a good boost to my energy level to focus on that for a while.

The ostensible purpose for this trip was to cement some kind of formal working relationship between Divmod and Roxor Games. On that front I think things went spectacularly. We had a series of meetings that lasted all week long, and were, I think, a credit to the very concept of a "meeting". I had hoped that this post would be longer, but looking back over the week, it seems that most of the details of those meetings involved deals that aren't final yet and are technically secret. I'm kind of out of practice at keeping secrets, having been on the open-source circuit for so long, but even I am pretty clear that putting things on a syndicated blog that gets tens of thousands of hits per post is not exactly in the spirit of an NDA.

If you know me, or you know J---- (the malign overlord of Roxor), you can probably guess at least part of what's going on. Don't spoil it for everyone, please! If you, for example, posted "Oh snap are you guys making a [REDACTED] for the [REDACTED] market???" in the comments here, I think I might be violating confidentiality agreements by proxy. If you're not one of the folks In The Know, I will give you one hint: Divmod's technology has certain applications outside of email.

To give you some of the flavor of the week, though, here is a cell-phone snapshot of the one formal presentation I gave while I was out there. For a "projector", I used a prototype of an racing arcade machine Roxor will be releasing soon.
Divmod Racing Machine


It was also surreal, not to mention a ton of fun, to get a whirlwind tour of the gaming industry rumor mill, not having been involved for a while myself. At various points throughout the week I found myself, a visitor from two thousand miles away, relaying messages between folks who are within a stone's throw of each other. Thank you to everyone who had a great story for me.

Most of all it was fantastic to get to visit a bunch of old friends. Shout outs to my homies: thanks for making time for me! Thank you Allen for driving me around town for a week. Thanks also to Jason, John, Al, Travis, Brian, Jason[2], Matt, Melanie, Seneca, Dave, Damion, and Sean. To those of you who I missed - especially Andy, Skiv, Adam, and Spencer, and of course anyone who I forgot to list - sorry about that. I'll be down again sometime soon, drop me a line at glyph@divmod.com and I'll make sure to meet up with you next time.

I know you're all expecting an update...

One is forthcoming, but this has been a crazy week for me. There was a lot to do upon my return from Austin, and I have been sick for the last 2 days.  I do have quite a bit of backlogged posting to do here; I should have at least one up over the weekend.

PyCon's Anti-Prom

I didn't get to go to PyCon this year. I'm just now back from Austin, where I was discussing secret plans. More on that soon.

However, before I forget, I'd just like to thank Ted Leung for specifically mentioning Divmod's absence in his PyCon wrap-up. It's good to be missed. We missed interacting with the PyCon crowd this year too - and of course, specifically the Chandler team ;-).

(PS, it's "Divmod", not "DivMod" - although Ted is hardly the only person I've seen make this mistake. I have a feeling it's fated to be a common one, like the way some people say "PC or MAC" as if "MAC" were an acronym for Macintosh Apple Computer or something.)

Push F9 To Continue

F9 is the default keybinding for "run the unit tests for the current file" when using the Twisted Emacs bindings.

As her Windows/Divmod tutorial indicates, Ying is getting started with coding using Twisted again. This is the third time that she's gotten started with it - tries #1 and #2 involved Woven and Nevow, respectively. I was cringing as she started to get her development environment set up, because those previous attempts were shockingly painful. Every step of the way was a challenge - version skew. Path problems. Obscure deprecation warnings. Un-debuggable template interactions. Undocumented assumptions. DOCTYPE declarations.

This time she's got a bit more of a domain-model problem to attack. It was staggering to me how different it was. While getting started with a web-based development project with Twisted was painful, and it wasn't necessarily clear how to proceed, getting started with a domain model was trivial. With a 4-line unit test template, she was productive within 5 minutes. Of course, having yours truly in the room (and emotionally dependent upon you) when you start to work on a Twisted program helps, but I have been largely uninvolved - as opposed to previous attempts where she would ask a question every 5 minutes and I would start the answer with, "You have to understand the history of the multiple projects involved here..." or "There are some unresolved issues...", now she asks a question once per day or so, and the answers are short, direct sentences like "use twisted.web.client.getPage" or "return a Deferred from your test method".

I have long despaired of Twisted development being easy to newcomers, and the community (both friendly and hostile to Twisted) reinforces this assumption. "Asynchronous programming is too hard", "learning Twisted is a serious investment of time", "there's so much you have to know to get started", etc. However, I've now seen that it can be easy. We just need more tutorials that blitz through the introductory steps on a particular platform without explaining anything, and get straight to coding something.

For example, if I had written the tutorial that Ying posted to her blog, I probably would have explained each step, so as to give users maximal flexibility in their setup. "Make a folder to hold your projects. We'll call this the 'combinator container' folder from now on. You can place this anywhere on your sys.path. Now, make a folder /path/to/combinator-container/Divmod ..." Ying chose a much more direct route. Nobody really cares what the folder is named, they just want things to work. "Make a folder C:\Projects. Now make a folder inside that called Divmod. Then run 'svn co ...'"

Once you have gotten to the point where you are hitting F9 every five minutes, watching your code run, and fixing problems, you don't really care that you don't know how to put C:\Projects at H:\Documents and Settings\%USER%\My Documents\Programming\MyNiftyProject\Infrastructure. You don't care that you had to run svn command lines rather than installers. You certainly don't care why version 0.9.8a of OpenSSL is required or where ZopeInterface was installed.

That last step - where you just push a button, rather than starting up a terminal and typing a command line - is an important one. It makes the experience feel complete, and it removes a point in the development process (that happens every 5 minutes or so) where a new Twisted user thinks, "this is a pain in the ass, these tools are terrible".

twisted-dev.el is the best-kept secret of Twisted developers. It needs to be more front-and-center. We should eschew the bits that never really worked, and are no longer maintained, i.e. the PB/Emacs integration, and include the core one-button-unit-test functionality with Twisted itself.

The Opposite Test

Whether or not I've used macs, I've always been a big Guy Kawasaki fanboy. Getting someone with such integrity and seemingly boundless enthusiasm to evangelize for them was one of the best things Apple ever did.

Since I am going to be "evangelizing" quite a bit while in Austin, I am thrilled to see that recently Mr. Kawasaki started blogging, sharing his wisdom about evangelizing, and he's saying things that I already agree with.

He's a big fan of top ten lists, and everything I've read so far I agree with. One thing stuck out for me in particular though, because I just said something similar about open source project descriptions:
Apply the opposite test. How many times have you read a product description like this? “Our software is scalable, secure, easy-to-use, and fast?” Companies use these adjectives as if no other company claims its product is scalable, secure, easy-to-use, and fast. See if your competition uses the antonyms of the adjectives that you use to describe your product. If it doesn't, your description is useless. For example, I've never seen a company say that its product was limited, full of leaks, hard-to-use, and slow.
I wouldn't mind so much if people wrote such descriptions and then moved to substantiate them. Sometimes it's really important to have software that is scalable, easy-to-use, and fast. Sometimes you really do want a fast, clean, dark theme.

Presumably, in such a situation, your users know how fast, or how clean, or how dark they want the theme to be - how many users it has to scale to or what their training costs are going to be. Talk about that. Measure it, and write your advertising like a thesis you are going to have to defend. If you're writing such literature, even if sales isn't your job, you are in the role of a salesman and your readers know it, even if you don't. That means they are going to assume that every single word you say is a lie. Provide examples, show screenshots, compare to other things that they might be familiar with. Try to avoid graphs without meaningful numbers and units - for example, don't do this. Apple's Intel Core Duo site includes an "application performance" graph that has bars that say "4.1x faster", but don't explain the benchmarking very well, and while there are 4 tests there is only one "baseline" bar. (Also, they do say that they used a beta of the Cinebench software, which means that the results aren't even going to be comparable to something that customers looking at this site will have access to to run themselves on their own hardware, even if they were available, which they aren't. But I digress.)

I'm picking on apple because I'm considering maybe buying a MacBook this year, but the open source world has even more to learn about this than corporate marketroids. Every open source database project claims to be efficient, but how efficient? At least Oracle provides benchmarks during sales pitches. Let's say that I am going to build a system where database efficiency is really important - what is the most efficient open source database? Even the Open Source Database Benchmark site doesn't list results - their Project Status page (which is admittedly ancient, but they are still the first google hit for "open source database benchmarks") is only detailed enough to show that certain databases work with the benchmark, if you want to run it yourself.

When you're describing your open source project, think about your users, not about you. I think that the temptation to say software is "efficient" or "scalable" comes from the fact that programmers have to spend time doing optimizations and thinking about scalability. Even if it takes the bulk of your time, that's a base-level requirement, not something that's going to make your project better than its competition. Sure, if you walk up to a database user and ask them, "What would influence your choice of a database on future projects?" they might say something about efficiency, but if you think about what a database user is going to do with it, how they are going to experience the utility of the database both during development and on a running service, they are not going to be benchmarking and tuning constantly. They are going to be debugging problems with the database, when they inevitably get something wrong. People don't like to think about themselves making mistakes, or the database failing, but I think you will find people responding more positively to a database that provides gobs of useful information about what's going on than a database that is 8% faster than its nearest competitor.

While it's not perfect, I am a big fan of SQLite, and I think that (in addition to having an excellent technology) the "marketing message" on the website is very good. It begins by describing the database as "small, zero-configuration, self-contained", which is more interesting than the performance characteristics - to most users. I happened to be concerned about both issues, and despite being out of date, they provide a long, detailed page on database performance which clearly indicates that it is not slow.

Since I have been thinking about these sorts of issues, I am starting to formulate a plan for Twisted's marketing and future directions, too, but that's enough blogging for one night. Watch this space...