Divmod won't be going to PyCon this year.
Unfortunately, even a small conference like PyCon can cost too much when you
are an un-funded startup. When we evaluated whether or not it was a
worthwhile idea to go, we just couldn't make it add up - by the time PyCon
rolls around, we are going to need to be doing a lot of deployment,
marketing, and sales work, and PyCon is more like a vacation than a sales
venue for us. In previous years we've been able to rationalize the cost
because we were mostly on the east coast, and all in the US, so it was
generally a cheap trip. This year we have an overseas developer and the
conference is on average farther away from all of us.
I also wanted to say something publicly because this isn't a comment
on the conference - I'm definitely going to miss going, and I am sure it
will be a lot of fun. I apologize to anyone who thought they were going to
see us there. We were planning to go before we re-evaluated our
priorities. I hope we will be there again next year.
Ironically I will be on a business trip to Texas anyway at around the
same time as PyCon - in Austin, which is to say, the part of Texas that
isn't totally awful. (Seriously, I know the conference had to move because
it was outgrowing the venue, but Dallas? I'm still confused about that.) I
hope to post something about the super secret projects I am going
there to discuss, but don't hold your breath - they are super secret after
all.
That is not dead, which can eternal lie
and in strange aeons, even death may die
- HPL
Apparently some guy is trying to resurrect Medusa. I wasn't going to say anything until he mentioned Twisted directly, calling it "unprofitable", and "a complex maze of orthogonal but impractical interfaces", but IT'S ON NOW.
Medusa was great for its time. I read the whole thing before I started work on Twisted. In fact, Medusa can be credited with opening my eyes to the problems with threads. Somewhere, locked in a lead drum, encased in concrete, the very first version of Twisted Reality for Python can be found, using threads for concurrency and pickle for serialization.
However, even when I was starting with Twisted - circa 1999 - Medusa was already showing its age.
The main problem with Medusa is that it does too little. For example, Windows sockets are different from UNIX sockets in a variety of ways you probably aren't going to anticipate if you are calling self.recv and such in an
asyncore.dispatcher. Those are just the differences in sockets
- Twisted handles threads, pipes, and processes too, which a cursory look at
the new "Allegra"
reveals only rudimentary support for.95% of the time, you won't notice these problems. As they're related to operating-system error reporting, they tend to show up only when your program is under load. In other words, you won't notice any issues with your program until a lot of people are using it and there is a huge amount of pressure on you to fix it immediately. Luckily Allegra has no automated tests, so you won't have to be troubled by finding out about bugs too early. Twisted has more than 2500 tests run on 4 different platforms with each commit.
Don't believe me that there will be issues with your super-simple cross-platform select loop? By way of an example, after years of maintaining their own networking core, BitTorrent has started using Twisted, with the unobtrusive changelog message of "TCP Stack flaking out bug fixed, using Twisted".
Even given the testing situation, Medusa (I'm sorry, "Allegra") is, indeed, quite a bit simpler than the full Twisted suite, and an adequate solution to 95% of the asynchronous-socket-programming problems out there. However, it's only adequate, whereas I think Twisted is excellent.
As per my last post, there is a lot of complexity in some projects surrounding Twisted, and some of it is unnecessary. That complexity isn't there if you are just trying to get basic asynchronous I/O going, though. If you are considering Medu^HAllegra for some reason, have a look here first: Writing Servers with Twisted. If you can't figure that out, I've just saved you some time: you should just stop trying to write networked programs. "a complex maze", "impractical"??? The simplest working server is just 4 lines of code: an import, a class statement, a method definition, and a method body. You can really be productive that fast. I don't think it's even possible to be simpler than that in a Python framework.
A similar document for Medusa, Asynchronous Socket Programming, has more than twice as many words, and the simplest example is more than 4x longer.
As to "unprofitable", well, let me just direct you to a page that describes people's profitable experiences with Twisted: including the quote "The quality of the Twisted networking core is unmatched in the open- or closed-source arenas".
I'm glad to see someone doing a project that aims to do something similar to Twisted, since that indicates it's a thing that people need doing. I'd love to push for some interoperability standards if a different framework were to emerge that had different implementation strengths than Twisted does (and goodness gracious, it could certainly be faster. Twisted can only handle a few tens of thousands of requests per second on high-end hardware, C++ servers are in the millions these days).
However, there are a few things such a framework would have to get right first:
- Write some automated tests. Seriously, it's 2006 already, unit tests are an accepted best practice pretty much anywhere that people care about writing programs that work even some of the time.
- Be clear about what you're trying to do differently, or don't say anything at all. Here's a hint: any phrase involving the word "lightweight" is not specific. Reading the descriptions of Python frameworks (although this mainly applies to tiny one-off web frameworks) I am reminded of the descriptions of WindowMaker themes at the beginning of the themes craze five years or so ago. "It's a lightweight, simple, fast, clean, dark theme, with a picture of Neo on the desktop" probably described 4 out of every 5 themes to hit the site. The first four adjectives regularly show up in descriptions of frameworks. Where were all the heavy, slow, dirty, frameworks that these are in response to? If these are valid design tradeoffs, why don't people ever advertise as the opposite? Those four adjectives are basically meaningless - when an author of some software says it's "fast" and "lightweight" they just mean "I know what it does, so it doesn't surprise me when it's slow, and I can get things done quickly with it, because I wrote it." If it were measurably fast or small they would be quoting statistics at you about how many requests per second it can perform or how small the image can be compressed.
- Have an application. It looks like Mr. Allegra is, in fact, driving the requirements for his new-old framework from an application, and that's good. If his approach doesn't work well, he'll find that out, rather than assuming that it is fine.
- Finally, specific to asynchronous I/O frameworks: separate your transport from your protocols. Don't force your users to fetch data themselves from the OS; there is just no reason for that. It is a real pain to implement a low-level enough API in Twisted for compatibility, if your users expect to diddle the sockets themselves.
I suppose if you do not know what Python is or what the Web is, maybe you
don't know this, but Guido
has been asking about web frameworks.
I wish I had something more substantial to throw into the discussion here, but if you're interested in my work on the subject, you know where to find it.
Python's web situation is a mess, there's no denying it. Even if there were some totally awesome framework which everyone should be using that was head and shoulders above everything else, it would take a while to figure that out at this point, and to achieve some consensus, just by the sheer amount of work required to get everyone's attention, and once the right people had found it, to focus the larger community on it. Unfortunately I think it's clear that no such silver bullet exists, and that there is unlikely to be one any time soon. Even if there were, larger projects like Mantissa and Zope would still exist, because it's not likely that most people's criterea for "web framework silver bullet" includes "integrated digital telephony".
The general frustration with (and bewilderment at) this situation has been breeding a poisonous feeling among Python developers though, I fear. I'm calling it "disenframeworkentarianism", and it is best typified by this post (unrelated, but linked-to by Guido's discussion). There is a strong desire for all the benefits of frameworks without any of the overhead. "Frameworks are too fattening for my code!", we exclaim, "Let's come up with something that has all the delicious flavor but none of the calories." This is the same line of reasoning which lead to Olestra, and various other food-substitutes.
While superficially the idea of a food that tastes great and has no impact on your weight is a great idea, it turns out that there is a word for things that you ingest which don't do anything for your body: poison. For example, I discovered a wonderful article recently: Why No One Should Eat Products That Contain Olestra.
Frameworks with zero "fat", that is, overhead imposed on your code, are probably not actually doing anything for you. It looks like they're "lighter-weight" than other frameworks, because they don't provide any of the "fat", that is, the cognitive overhead required to learn the framework, or as Guido puts it, "lavish quantities of kool-aid". By not providing a consistent view of the world from the framework's perspective, you end up having to do more of that work yourself. You have saved time learning someone else's vision of the world, but instead of actually saving time overall, you've had to construct your own island of functionality, totally isolated from other developers and integration opportunities. It may well be that you needed to do that work, because the requirements of your project are sufficiently bizarre and integration-heavy that you are really going to have to tie things together yourself anyway - that doesn't mean it's not more work.
I'm not saying this as a defense of super-heavy-weight frameworks either. The opposite of Olestra is not a bucket of fried lard, it's a balanced diet, with plenty of low-calorie things like fresh leafy vegetables. J2EE is the Big Mac here; plenty of calories, but most of them empty. I think this metaphor could be extended well into absurdity, with the requirements of your project being its "metabolism", unit tests being "exercise"... but let me get back to the actual point here.
No project has come along yet that really tries as hard as it can to be completely modular; i.e. let you plug in your own solution for any layer of any stack with no overhead. There's good reason - whether it's a good idea or not, it's just too hard. When frameworks do try to let you choose, it turns out that somebody has to come along and pick up the pieces, and do the work of getting the disparate pieces that you want to work together, to actually work together. The trendy thing these days is to call this a "megaframework", but I think it's a symptom for this frameworks-proliferation disease, not the cure. It's largely a matter of perspective. In some cases aggregation works wondefully - Ubuntu is the mega-est megaframework of them all, and it's fantastic. At some level it just falls apart though - nobody tries to cobble together a word processor from "best of breed" word-wrap algorithms and icons; you use AbiWord or you use OpenOffice, and you don't complain that you can't use AbiWord's tables feature in OpenOffice documents - they're just different tools, and you have to decide on one or the other.
I think that the community would mostly be better served right now by some straightforward competition - try to create the best full-stack experience, and maybe eventually it will become clear that project X really does have the best ORM and project Y has the best templating system, and the user communities can exert some pressure on the development teams to merge, but until then we need good, full, complete experiences for developers trying to build sites, whether they're megaframeworks or kiloframeworks or gigaframeworks, not half-finished chunks of functionality you are expected to finish yourself. It's much easier to extract just the part that you need from a polished, working solution than it is to create a polished, working solution from a handful of parts that by themselves do nothing useful, even if they're well-built parts.
Since it probably needs to be said... While I'm a "framework" guy, and I obviously have my own axe to grind here, note that I'm not commenting on Guido's blog (or responding to it here) and saying "Use Nevow!!!!" (Although, it seems like there are enough people doing that for me ;-)) There are serious problems with all current web frameworks, Nevow included. I think Nevow could be really awesome, but there are parts of it which really need to be removed and streamlined, cleaned up, and hidden from view. Let's face it, Nevow guys - the
So, I've got to say, if you're in the market for one more tool in your build-your-own-web-framework toolbox, Nevow is probably not for you. If you're looking at a larger system, with object publishing, multi-protocol support, sessions, authentication, modular AJAX, and a variety of other features, it is a good tool to use, though. Especially the AJAX thing - most of Nevow's unnecessary complexity is worth wading through just to get to Athena, the AJAX-plus-javascript-modules-plus-pluggable-page-fragments Web 2.0 bonanza feature. There aren't enough examples, but I really haven't seen anything like it before. You can extend that even further with Mantissa, to include an integrated database, sign-up, account maintenance, administrative tools, per-user plug-ins, navigation, and so on - but Mantissa and Nevow are separated for good reason; Mantissa is still much more heavily in development, and is a much larger and more ambitious project. I am really looking forward to it stabilizing so that it is a more robust example to show other framework developers some of my ideas.
I wish I had something more substantial to throw into the discussion here, but if you're interested in my work on the subject, you know where to find it.
Python's web situation is a mess, there's no denying it. Even if there were some totally awesome framework which everyone should be using that was head and shoulders above everything else, it would take a while to figure that out at this point, and to achieve some consensus, just by the sheer amount of work required to get everyone's attention, and once the right people had found it, to focus the larger community on it. Unfortunately I think it's clear that no such silver bullet exists, and that there is unlikely to be one any time soon. Even if there were, larger projects like Mantissa and Zope would still exist, because it's not likely that most people's criterea for "web framework silver bullet" includes "integrated digital telephony".
The general frustration with (and bewilderment at) this situation has been breeding a poisonous feeling among Python developers though, I fear. I'm calling it "disenframeworkentarianism", and it is best typified by this post (unrelated, but linked-to by Guido's discussion). There is a strong desire for all the benefits of frameworks without any of the overhead. "Frameworks are too fattening for my code!", we exclaim, "Let's come up with something that has all the delicious flavor but none of the calories." This is the same line of reasoning which lead to Olestra, and various other food-substitutes.
While superficially the idea of a food that tastes great and has no impact on your weight is a great idea, it turns out that there is a word for things that you ingest which don't do anything for your body: poison. For example, I discovered a wonderful article recently: Why No One Should Eat Products That Contain Olestra.
Frameworks with zero "fat", that is, overhead imposed on your code, are probably not actually doing anything for you. It looks like they're "lighter-weight" than other frameworks, because they don't provide any of the "fat", that is, the cognitive overhead required to learn the framework, or as Guido puts it, "lavish quantities of kool-aid". By not providing a consistent view of the world from the framework's perspective, you end up having to do more of that work yourself. You have saved time learning someone else's vision of the world, but instead of actually saving time overall, you've had to construct your own island of functionality, totally isolated from other developers and integration opportunities. It may well be that you needed to do that work, because the requirements of your project are sufficiently bizarre and integration-heavy that you are really going to have to tie things together yourself anyway - that doesn't mean it's not more work.
I'm not saying this as a defense of super-heavy-weight frameworks either. The opposite of Olestra is not a bucket of fried lard, it's a balanced diet, with plenty of low-calorie things like fresh leafy vegetables. J2EE is the Big Mac here; plenty of calories, but most of them empty. I think this metaphor could be extended well into absurdity, with the requirements of your project being its "metabolism", unit tests being "exercise"... but let me get back to the actual point here.
No project has come along yet that really tries as hard as it can to be completely modular; i.e. let you plug in your own solution for any layer of any stack with no overhead. There's good reason - whether it's a good idea or not, it's just too hard. When frameworks do try to let you choose, it turns out that somebody has to come along and pick up the pieces, and do the work of getting the disparate pieces that you want to work together, to actually work together. The trendy thing these days is to call this a "megaframework", but I think it's a symptom for this frameworks-proliferation disease, not the cure. It's largely a matter of perspective. In some cases aggregation works wondefully - Ubuntu is the mega-est megaframework of them all, and it's fantastic. At some level it just falls apart though - nobody tries to cobble together a word processor from "best of breed" word-wrap algorithms and icons; you use AbiWord or you use OpenOffice, and you don't complain that you can't use AbiWord's tables feature in OpenOffice documents - they're just different tools, and you have to decide on one or the other.
I think that the community would mostly be better served right now by some straightforward competition - try to create the best full-stack experience, and maybe eventually it will become clear that project X really does have the best ORM and project Y has the best templating system, and the user communities can exert some pressure on the development teams to merge, but until then we need good, full, complete experiences for developers trying to build sites, whether they're megaframeworks or kiloframeworks or gigaframeworks, not half-finished chunks of functionality you are expected to finish yourself. It's much easier to extract just the part that you need from a polished, working solution than it is to create a polished, working solution from a handful of parts that by themselves do nothing useful, even if they're well-built parts.
Since it probably needs to be said... While I'm a "framework" guy, and I obviously have my own axe to grind here, note that I'm not commenting on Guido's blog (or responding to it here) and saying "Use Nevow!!!!" (Although, it seems like there are enough people doing that for me ;-)) There are serious problems with all current web frameworks, Nevow included. I think Nevow could be really awesome, but there are parts of it which really need to be removed and streamlined, cleaned up, and hidden from view. Let's face it, Nevow guys - the
sequence renderer is a totem of Nevow's failure, at least
from the perspective of comprehensibility to new users. Also, as Tim Peters famously
said, the "it's surely too much for a lesser being" argument is pretty
weak - I trip over sequence so regularly that I've
stopped bothering to use it.So, I've got to say, if you're in the market for one more tool in your build-your-own-web-framework toolbox, Nevow is probably not for you. If you're looking at a larger system, with object publishing, multi-protocol support, sessions, authentication, modular AJAX, and a variety of other features, it is a good tool to use, though. Especially the AJAX thing - most of Nevow's unnecessary complexity is worth wading through just to get to Athena, the AJAX-plus-javascript-modules-plus-pluggable-page-fragments Web 2.0 bonanza feature. There aren't enough examples, but I really haven't seen anything like it before. You can extend that even further with Mantissa, to include an integrated database, sign-up, account maintenance, administrative tools, per-user plug-ins, navigation, and so on - but Mantissa and Nevow are separated for good reason; Mantissa is still much more heavily in development, and is a much larger and more ambitious project. I am really looking forward to it stabilizing so that it is a more robust example to show other framework developers some of my ideas.
According to the sitemeter OS stats for twistedmatrix.com, 43% of Twisted users are using some form of Microsoft Windows.
I don't have any such graphs for Twisted developers, but off the top of my head I would guess that 2 out of 40 developers actually use Windows for anything except games and grudgingly booting into it to fix a couple of Twisted issues per year. I am not some kind of math genius, but even I can tell that is not 43%.
So - where are you Windows/Twisted people? If you're really out there, we could use a few more dedicated maintianers that don't totally hate the platform.
Mary has an
interesting post about IRC.
I've had the same problem, although IRC is so integral to my work and my life that it's not feasible for me to eliminate it. Most of the time it's infeasible for me to even log out for longer than a few hours during a normal day. I don't really have a reasoned essay about this, but a few thoughts are floating around.
I've had the same problem, although IRC is so integral to my work and my life that it's not feasible for me to eliminate it. Most of the time it's infeasible for me to even log out for longer than a few hours during a normal day. I don't really have a reasoned essay about this, but a few thoughts are floating around.
- IRC, as both software and a communication standard, is utter garbage.
I could write a post twenty pages long just cataloguing its obvious
flaws. That makes it very difficult to implement any of these suggestions.
It might be reasonable to do this with Jabber, but somebody would have to
write a reasonable Jabber server in Twisted first.
- There should be a way to broadcast the temperature of an IRC
discussion. Twisted is largely just random noise - I want to be able to
minimize my IRC client without fear that I'm missing anything, until
someone raises the "temperature" to alert everyone "an interesting
discussion is happening, tune in now".
- In that vein, more discussions should be scheduled, rather than
happening spontaneously. An automatic moderator bot could probably help
with that.
- It should be a lot easier to fork off a political conversation into a
new room. I can identify with the rage reaction to various IRC
conversations.
- There should be some sort of "comforting social gesture" built in to
the protocol so that you can greet people, thank them, and generally make
polite noises without having to dress up as a cat, weaponize trout,
chew off someone's arm, or wrestle them to the ground and lick their face.
Those are all "normal" greetings I've seen otherwise normal and
professional IRC users use to greet each other. In my experience, women
who engage in this sort of thing tend to be inappropriately affectionate,
but men tend to concoct extremely violent, bizarre and intricate
rituals, sometimes with a defined back-and-forth that might be five or six
steps.
World of Warcraft's emotes system achieves this by being really limiting, and thus far, from what I can see, that's a great thing. WoW players don't spend hours developing byzantine in-character rituals to greet each other - you have your option of /wave, /clap, or /bow.