Asynchronous - better than synchronous? Who would have thought?
Asynchronous - better than synchronous? Who would have thought?
http://www.ubuntulinux.org/logo.jpg
Best. Distro. Ever.
I am not exaggerating.
I ran out of disk space over the weekend, and after buying a new hard drive, I figured I'd better give Ubuntu a try. I installed the latest snapshot of Hoary Hedgehog and I was up and running in under an hour.
There are still some familiar Linux-y problems, for example, configuring X to support my 12-button mouse is still an unmitigated disaster, and some people just cannot write drivers to save their lives.
Before you Mac naysayers in the crowd start to gloat, at least I can burn a god damned CD without using a command line, okay?
Still, the drivers work. Installing Hoary somehow increased the performance of all my disk drives - I don't know how that happened, but it was automatic, and I can now use my USB drive without looking at my watch. Ying's tablet, after not working with 5 different distros over 4 years, magically started to function properly. My USB headset was detected without complaint. My laptop can suspend and resume (though there are still a few kernel problems there).
I know that many of the tools that they use were already available, but Ubuntu is the first Linux distribution that I have found which actually unleashes the full power of the free software community in a coherent way.
By the way, did I mention that it includes Twisted as part of the default install? ;-)
Best. Distro. Ever.
I am not exaggerating.
I ran out of disk space over the weekend, and after buying a new hard drive, I figured I'd better give Ubuntu a try. I installed the latest snapshot of Hoary Hedgehog and I was up and running in under an hour.
There are still some familiar Linux-y problems, for example, configuring X to support my 12-button mouse is still an unmitigated disaster, and some people just cannot write drivers to save their lives.
Before you Mac naysayers in the crowd start to gloat, at least I can burn a god damned CD without using a command line, okay?
Still, the drivers work. Installing Hoary somehow increased the performance of all my disk drives - I don't know how that happened, but it was automatic, and I can now use my USB drive without looking at my watch. Ying's tablet, after not working with 5 different distros over 4 years, magically started to function properly. My USB headset was detected without complaint. My laptop can suspend and resume (though there are still a few kernel problems there).
I know that many of the tools that they use were already available, but Ubuntu is the first Linux distribution that I have found which actually unleashes the full power of the free software community in a coherent way.
By the way, did I mention that it includes Twisted as part of the default install? ;-)
Today Exarkun and I added some features to Divmod that I am style="font-style: italic;">really looking forward to actually
using. They haven't made it to production yet, but I don't write
about our product near often enough, so I think a bit of an
introduction is in order.
The first feature we added was the bookmark manager:
![]()
src="http://www.divmod.org/users/glyph/RSSShots/Screenshot-Bookmarklet%20Manager%20-%20Mozilla%20Firefox.png">
By itself this wouldn't be a particularly interesting screenshot, but I
wanted to stress how it kind of looks like the "bookmark" window in
Mozilla. After adding the "bookmark to Divmod" button to your
browser, you can bookmark stuff similarly to del.icio.us.
Similarly, your list of bookmarks becomes a feed. style="font-style: italic;">Unlike del.icio.us, this feed is
private: the "live bookmark" only works when you're logged in, or if
you have explicitly given a client program your password as part of the
URL. One of the next things I'll be working on is finishing up
our sharing code and integrating it with this, so that you can share
certain portions of your feed with different groups of people.
![]()
src="http://www.divmod.org/users/glyph/RSSShots/Screenshot-Divmod.Org%20%3A%3A%20Home%20-%20Mozilla%20Firefox.png">
We hit a website in the browser, click on "bookmark to Divmod", and
voila: it has been bookmarked (note that it's been bookmarked style="font-style: italic;">securely, even...)
![]()
src="http://www.divmod.org/users/glyph/RSSShots/Screenshot-Divmod.Org%20%3A%3A%20Home%20-%20Mozilla%20Firefox-1.png">
The bookmark completes,
We update our RSS feed,
and the new bookmark has appeared.
I like that feature since it means that I can bookmark whatever I want
and store it centrally in the same place I keep my email, but an even
better feature is that Divmod can now turn style="font-style: italic;">any Pool (equivalent of a "folder",
or "label" depending on your point of view) into a private
feed. Here we can see the test inbox with the "live
bookmark" icon and all the same subjects showing up over RSS and
HTTP.
My favorite part of this was that it only took about 4 hours to do, and
although it's for a demo I feel like the code is pretty
production-worthy: it's a very simple re-application of our existing
back-end.
using. They haven't made it to production yet, but I don't write
about our product near often enough, so I think a bit of an
introduction is in order.
The first feature we added was the bookmark manager:
By itself this wouldn't be a particularly interesting screenshot, but I
wanted to stress how it kind of looks like the "bookmark" window in
Mozilla. After adding the "bookmark to Divmod" button to your
browser, you can bookmark stuff similarly to del.icio.us.
Similarly, your list of bookmarks becomes a feed. style="font-style: italic;">Unlike del.icio.us, this feed is
private: the "live bookmark" only works when you're logged in, or if
you have explicitly given a client program your password as part of the
URL. One of the next things I'll be working on is finishing up
our sharing code and integrating it with this, so that you can share
certain portions of your feed with different groups of people.
We hit a website in the browser, click on "bookmark to Divmod", and
voila: it has been bookmarked (note that it's been bookmarked style="font-style: italic;">securely, even...)
The bookmark completes,
We update our RSS feed,
and the new bookmark has appeared.
I like that feature since it means that I can bookmark whatever I want
and store it centrally in the same place I keep my email, but an even
better feature is that Divmod can now turn style="font-style: italic;">any Pool (equivalent of a "folder",
or "label" depending on your point of view) into a private
feed. Here we can see the test inbox with the "live
bookmark" icon and all the same subjects showing up over RSS and
HTTP.
My favorite part of this was that it only took about 4 hours to do, and
although it's for a demo I feel like the code is pretty
production-worthy: it's a very simple re-application of our existing
back-end.
... but I
know somebody who does.
A question worth asking: why does Python have an "integer" type, but no "character" type? Especially when the serious users of numbers in pythonenhanced it, they did everything in terms of arrays.
A question worth asking: why does Python have an "integer" type, but no "character" type? Especially when the serious users of numbers in pythonenhanced it, they did everything in terms of arrays.
So, I set out to provide a slightly more cogent debunking, since various
members of the audience were unsatisfied with my previous, dismissive
response. I haven't gotten terribly far, since I'm pretty busy, but I did
cook up some rudimentary results for you.
My goal was to show that threads don't scale terribly well in terms of processing concurrent I/O, and in particular, Twisted scales better, even in its slowest, most naive mode than a simple threaded implementation of a similar concept.
Unfortunately I was unable to do this easily, because Debian GNU/Linux (in its default configuration, using a 2.6.7 kernel) is not able to even spawn enough threads to generate a fair test.
My test program was to write a Twisted protocol handler that would take some simple input, perform a trivial computation, generate some output, and close the connection. I then wrote a Twisted client for it, and write a simple threaded reactor which would handle each connection in a separate thread, to compare the amount of time each multiplexor took to finish connecting vs. the amount of time it took to complete the entire test. At around 256 threads, though, I could no longer spawn more threads to handle simultaneous connections, a problem which others have experienced.
This indicates in part the cultural issues surrounding the performance of threads, to wit, nobody really cares enough about good threading performance to even bother making it possible to spawn a lot of them.
This test was carried out on an otherwise quiescent Linux machine, a 2000XP+ Athlon with a gig of RAM, using python 2.3. Without further ado:
I don't expect these graphs to be taken as serious performance measurements - after all, I'm not even providing source code yet. Still, the fact that the threading connection-start performance is so abysmally bad (even compared to
It has brought to light the fact that although Twisted would be an excellent test platform for different multiplexing mechanisms, nobody has bothered to put together a survey of those yet. I will do a more rigorous benchmark of threads vs. Twisted later this week, release the code, and hopefully some useful work will come of this if it can be used as a simple reactor benchmark.
My goal was to show that threads don't scale terribly well in terms of processing concurrent I/O, and in particular, Twisted scales better, even in its slowest, most naive mode than a simple threaded implementation of a similar concept.
Unfortunately I was unable to do this easily, because Debian GNU/Linux (in its default configuration, using a 2.6.7 kernel) is not able to even spawn enough threads to generate a fair test.
My test program was to write a Twisted protocol handler that would take some simple input, perform a trivial computation, generate some output, and close the connection. I then wrote a Twisted client for it, and write a simple threaded reactor which would handle each connection in a separate thread, to compare the amount of time each multiplexor took to finish connecting vs. the amount of time it took to complete the entire test. At around 256 threads, though, I could no longer spawn more threads to handle simultaneous connections, a problem which others have experienced.
This indicates in part the cultural issues surrounding the performance of threads, to wit, nobody really cares enough about good threading performance to even bother making it possible to spawn a lot of them.
This test was carried out on an otherwise quiescent Linux machine, a 2000XP+ Athlon with a gig of RAM, using python 2.3. Without further ado:
I don't expect these graphs to be taken as serious performance measurements - after all, I'm not even providing source code yet. Still, the fact that the threading connection-start performance is so abysmally bad (even compared to
select() - remember, this is using Twisted's
default configuration, with no tuning or optimization) and the fact
that with an otherwise excellent development machine I don't even have the
software to test larger numbers of concurrent connections might give you an
idea of why I find it so hard to take this kind of comparison seriously.It has brought to light the fact that although Twisted would be an excellent test platform for different multiplexing mechanisms, nobody has bothered to put together a survey of those yet. I will do a more rigorous benchmark of threads vs. Twisted later this week, release the code, and hopefully some useful work will come of this if it can be used as a simple reactor benchmark.