The Olestra of the Web

Tuesday January 31, 2006
I suppose if you do not know what Python is or what the Web is, maybe you don't know this, but Guido has been asking about web frameworks.

I wish I had something more substantial to throw into the discussion here, but if you're interested in my work on the subject, you know where to find it.

Python's web situation is a mess, there's no denying it. Even if there were some totally awesome framework which everyone should be using that was head and shoulders above everything else, it would take a while to figure that out at this point, and to achieve some consensus, just by the sheer amount of work required to get everyone's attention, and once the right people had found it, to focus the larger community on it. Unfortunately I think it's clear that no such silver bullet exists, and that there is unlikely to be one any time soon. Even if there were, larger projects like Mantissa and Zope would still exist, because it's not likely that most people's criterea for "web framework silver bullet" includes "integrated digital telephony".

The general frustration with (and bewilderment at) this situation has been breeding a poisonous feeling among Python developers though, I fear. I'm calling it "disenframeworkentarianism", and it is best typified by this post (unrelated, but linked-to by Guido's discussion). There is a strong desire for all the benefits of frameworks without any of the overhead. "Frameworks are too fattening for my code!", we exclaim, "Let's come up with something that has all the delicious flavor but none of the calories." This is the same line of reasoning which lead to Olestra, and various other food-substitutes.

While superficially the idea of a food that tastes great and has no impact on your weight is a great idea, it turns out that there is a word for things that you ingest which don't do anything for your body: poison. For example, I discovered a wonderful article recently: Why No One Should Eat Products That Contain Olestra.

Frameworks with zero "fat", that is, overhead imposed on your code, are probably not actually doing anything for you. It looks like they're "lighter-weight" than other frameworks, because they don't provide any of the "fat", that is, the cognitive overhead required to learn the framework, or as Guido puts it, "lavish quantities of kool-aid". By not providing a consistent view of the world from the framework's perspective, you end up having to do more of that work yourself. You have saved time learning someone else's vision of the world, but instead of actually saving time overall, you've had to construct your own island of functionality, totally isolated from other developers and integration opportunities. It may well be that you needed to do that work, because the requirements of your project are sufficiently bizarre and integration-heavy that you are really going to have to tie things together yourself anyway - that doesn't mean it's not more work.

I'm not saying this as a defense of super-heavy-weight frameworks either. The opposite of Olestra is not a bucket of fried lard, it's a balanced diet, with plenty of low-calorie things like fresh leafy vegetables. J2EE is the Big Mac here; plenty of calories, but most of them empty. I think this metaphor could be extended well into absurdity, with the requirements of your project being its "metabolism", unit tests being "exercise"... but let me get back to the actual point here.

No project has come along yet that really tries as hard as it can to be completely modular; i.e. let you plug in your own solution for any layer of any stack with no overhead. There's good reason - whether it's a good idea or not, it's just too hard. When frameworks do try to let you choose, it turns out that somebody has to come along and pick up the pieces, and do the work of getting the disparate pieces that you want to work together, to actually work together. The trendy thing these days is to call this a "megaframework", but I think it's a symptom for this frameworks-proliferation disease, not the cure. It's largely a matter of perspective. In some cases aggregation works wondefully - Ubuntu is the mega-est megaframework of them all, and it's fantastic. At some level it just falls apart though - nobody tries to cobble together a word processor from "best of breed" word-wrap algorithms and icons; you use AbiWord or you use OpenOffice, and you don't complain that you can't use AbiWord's tables feature in OpenOffice documents - they're just different tools, and you have to decide on one or the other.

I think that the community would mostly be better served right now by some straightforward competition - try to create the best full-stack experience, and maybe eventually it will become clear that project X really does have the best ORM and project Y has the best templating system, and the user communities can exert some pressure on the development teams to merge, but until then we need good, full, complete experiences for developers trying to build sites, whether they're megaframeworks or kiloframeworks or gigaframeworks, not half-finished chunks of functionality you are expected to finish yourself. It's much easier to extract just the part that you need from a polished, working solution than it is to create a polished, working solution from a handful of parts that by themselves do nothing useful, even if they're well-built parts.

Since it probably needs to be said... While I'm a "framework" guy, and I obviously have my own axe to grind here, note that I'm not commenting on Guido's blog (or responding to it here) and saying "Use Nevow!!!!" (Although, it seems like there are enough people doing that for me ;-)) There are serious problems with all current web frameworks, Nevow included. I think Nevow could be really awesome, but there are parts of it which really need to be removed and streamlined, cleaned up, and hidden from view. Let's face it, Nevow guys - the sequence renderer is a totem of Nevow's failure, at least from the perspective of comprehensibility to new users. Also, as Tim Peters famously said, the "it's surely too much for a lesser being" argument is pretty weak - I trip over sequence so regularly that I've stopped bothering to use it.

So, I've got to say, if you're in the market for one more tool in your build-your-own-web-framework toolbox, Nevow is probably not for you. If you're looking at a larger system, with object publishing, multi-protocol support, sessions, authentication, modular AJAX, and a variety of other features, it is a good tool to use, though. Especially the AJAX thing - most of Nevow's unnecessary complexity is worth wading through just to get to Athena, the AJAX-plus-javascript-modules-plus-pluggable-page-fragments Web 2.0 bonanza feature. There aren't enough examples, but I really haven't seen anything like it before. You can extend that even further with Mantissa, to include an integrated database, sign-up, account maintenance, administrative tools, per-user plug-ins, navigation, and so on - but Mantissa and Nevow are separated for good reason; Mantissa is still much more heavily in development, and is a much larger and more ambitious project. I am really looking forward to it stabilizing so that it is a more robust example to show other framework developers some of my ideas.