This is posted mainly for my own benefit, so that I won't have to re-remember the command-line options to make this work for the nth time in a row, but some of you may enjoy it:

    easy_install --prefix ~/.local/ --site-dirs ~/.local/lib/python2.5/site-packages your_package_here

Now that I've gone through and understood why that's necessary, I think I might be able to fix it in Combinator one day...

I have recently been working on some applications that make use of the features provided by notification-daemon.  I installed Jaunty to check out how the much-vaunted new notifications framework works and how it would affect these applications.  I was aware that it sacrificed some features, but I wasn't worried.  I am generally a fan of the GNOME philosophy of dropping functionality and configurability when it doesn't really serve the user.  I also appreciated the new, more minimal and sleek-looking graphic design of the notifications.

I feel like I need to preface my reactions with an apology.  I really like Ubuntu.  In many ways Jaunty looks really slick, and I'm generally enthusiastic to upgrade.

But this notifications stuff?  Wow.  What a disaster.

First, a little background.  I recently started working on two different applications which made heavy use of the notification API. While I originally thought I'd need to implement some of the notification features that I wanted on my own, I was pleasantly surprised to discover that notification-daemon provided almost all of them.

I want to present the user with a time-critical notification, one which didn't grab the focus and interrupt their work.  I want to show the user how much time they had to respond, and provide a few different options for responding to certain notifications.  In one application, I'm implementing a sort of dead-man's switch, where failing to respond within the time limit also counts as a negative response.  I also want to emphasize certain notifications, and provide hyperlinks to their web-based origin so the user can jump straight into the application at the appropriate point if a notification is interesting.

Some of the notifications I want to generate are notifications of events, some are indications of a change in status of my application; so sometimes I want to point at a particular status icon and sometimes I wanted to just drop the notification into a queue - hopefully a global queue which would intermix with other notifications.

I was assuming that I'd need to implement some of these features myself, but to my pleasant surprise notification-daemon handled every single use-case, more or less exactly how I envisioned it working.  I was thrilled.  As part of searching around for "notification" stuff, I learned that Jaunty will have some newer, even cooler notifications stuff, and I was really excited.  Unfortunately, Notify-OSD, the new Ubuntu-specific notification daemon, drops nearly all of these features.  So, I'm back to square one.

Here are some of the specific things which bothered me about it.

1. Applications which emit a notification that prompts for an action ­— something which they only would have done if they explicitly wanted to avoid grabbing focus, since popping up a dialog box is easy enough — will have a modal dialog box pop up and grab the user's focus while they're working.
   
2. Timeouts are no longer honored.  It so happens that in my application I have an operation which takes 2 minutes to time out; I would really like my notification to stay on-screen for this entire time.  I can still do this with notify-osd, but in order to do so I have to watch for the "closed" event and constantly create new notifications.  A smoothly animating timer was a much nicer interface than a sequence of bubbles saying "In 30 seconds I will time out.  In 25 seconds I will time out.  In 20 seconds I will time out."  Yes, I realize that the Ubuntu desktop team will say that I should do something different in this situation, but they're not designing my application and I don't like the options they've suggested.  This would be much less of a problem if the provided notification timeout weren't so distressingly fast.  As a user, by the time I've realized that a notification has popped up and taken the time to focus my eyes on it, it disappears halfway through my reading its message.  I am a very fast reader, and I'm pretty sure that there are a lot of people who are never really going to notice any notify-osd bubbles; they're so fleeting, they're just visual noise.
   
3. Markup is now silently ignored.  I can't emphasize portions of a notification with a larger font, or provide a hyperlink to the origin of the notification.  Similarly, since actions have been broken, I can't provide an action to jump to what caused the notification.  Notifications have thus become disconnected UI element which tell user something while providing them absolutely no tools to deal with it or respond to it.  For example, when Pidgin tells me that a user has signed on, I can no longer interact with the notification to say "yes, I would like to talk to that person".  I have to switch windows to the buddy list, locate the person who just signed on, and click on their name, rather than just clicking on the notification itself.  Or, I have to click on the tiny notification-daemon status icon that's a hard target to hit with my mouse, rather than a big friendly button.
   
4. Notifications are now displayed in the upper-right-hand corner of the screen (where important chrome, like close boxes, search boxes, toolbars, and menus frequently reside) rather than in the lower-right, where less important application features are traditionally located.  Granted, this is a damned-if-you-do-damned-if-you-don't situation, because some applications put important action buttons in the lower right, but I have been learning where to position my windows to avoid that area of the screen for years, and now I have to learn new habits.
   
5. Notifications can no longer be positioned on screen, relative to either widgets in windows or status icons.  This removes the ability for an application to use notifications to draw attention to a particular area of the screen.  Instead, users must make some connection between the notification bubble and the status icon themselves, perhaps by identifying some common graphical element.  If you're already using the "icon" area of the notification to display a picture (such as a person's portrait) it's a bit cramped to also show a copy of the status icon, especially given that the icon will now be squished for you so it's hard to get a pixel-accurate rendering of the status icon anyway.
6. Some of these problems are nominally addressed by the new "indicator applet" facility in Jaunty, but...
   
   1. The indicator applet and libindicate library appear to be almost completely undocumented; the "reference manual" looks like it was an auto-generated stub.  The automatically generated API documentation isn't hosted online anywhere.
   2. The python bindings for libindicate are similarly undocumented, and they aren't packaged anywhere, not even a PPA.  They also use autoconf, rather than distutils, for installation, so their build process doesn't produce a usable extension module and thus they resist installation anywhere but in /usr.
      
   3. I tried to read what passes for documentation — the patch to Pidgin's libnotify plugin that switches it to use libindicate for some things.  I tried it out because I wanted to see if maybe the indicator applet could address some of my concerns, but my misuse of the API caused the indicator applet to instantly segfault.
   4. From what I can tell, you can't just provide an icon and some text, you need to actually create a .desktop file, which means that packaging applications which want to use the indicator applet automatically gets two additional layers of complexity: first, you need to create a .desktop entry, and second, you need to figure out a way to have your application include it during installation.
      

It's interesting to read the version history for Growl, the OS X notification tool which so clearly provided the visual inspiration for Notify-OSD, and notice that many of the features now being removed (application level positioning, close buttons on notifications) are features which were added to later versions of Growl.

The "notification design guidelines" provide some very vague suggestions for ad-hoc mechanisms to work around these regressions.  These suggestions are unhelpful.  I want an API that I can call, not a picture of a window I need to re-create myself.  If the desktop team wants to change the look of my application in the future, I don't want them to submit a giant pile of patches against it.

Not only are the suggestions not a library, they don't include sample code, either.  How do I actually create an alert box which doesn't take focus, doesn't include any window manager controls, but stays above other windows?  Describing it this way is an invitation for applications to be inconsistent.  My interpretation of this specification will inevitably be different from other application authors.  For example, the specification doesn't say anything about compositing, but the window in the screenshot clearly appears to be alpha blended with the background.  It also doesn't say anything about WM controls, but some pictures have minimize/close buttons and some don't.  Some applications will have alpha blending for their notifications, some won't.  And since there's no library here, there's no reasonable way to consistently control the behavior of many applications.

With these features in libnotify, we have a single uniform queue for users' attention.  A single point of control which might be adjusted, bugfixed, and tweaked across the desktop as a whole, without writing tons of patches for individual applications.  Notify-OSD even takes advantage of that point of control.  But the suggestions for many of these features is to take control out of a single easily-managed client/server protocol and push it into a bunch of ad-hoc application-specific widgetry.

To add insult to injury, the one place that I do get hard examples of how to do things, on the notification development guidelines page, the Python code samples have yet to be written.  This is a minor nit, as I can definitely figure out what's going on from the C# code, but it's endemic of the same systemic problem with the Linux desktop ecosystem that brought us this half-baked replacement for notification-daemon.  Jamie Zawinski identified this problem as the "Cascade of Attention-Deficit Teenagers", but Canonical demonstrates that you can create this same problem with a medium-size company that employs highly competent, adult engineers.

Notification-daemon isn't perfect.  It could clearly stand to be improved, especially in the face of notification spam.  So please, improve it!  Or, at worst, if upstream is not cooperative, fork it.  I'm pretty sure that the solution to the rate limiting problem is not "then... what?".  Notify-OSD cuts the gordian knot of notify-spam by only letting you see one thing at a time, but that has its own problems.

Now, to be fair, all these regressions haven't really cost me any work.  I want my applications to be cross-platform, so I was going to have to implement most of this functionality for Windows anyway, using animating borderless windows.  Now I'm just going to be using my own notification widgetry on Ubuntu as well, rather than elegantly integrating with the platform and providing all of my notification interaction through a familiar UI.  But I'm sad that the superior notification infrastructure on Linux in general and Ubuntu specifically is no longer something that makes my application easier to write first.

So, beyond this one little screed, I'm really not going to complain too much.  I'll implement some of my own ideas for notification, try to come up with some way to be friendly to Notify-OSD in the meanwhile, and I'll still eventually upgrade all of my computers to jaunty and enjoy the other eye-candy and performance improvements.  This is not too terrible of a price to pay, and I do keep it in perspective.  I also understand that I'm not the guy who has to make the hard decisions for what goes into Ubuntu or GNOME or whatever.  I understand that sometimes, in order to make an omelette, you have to kill a few people.

But, if a decision maker for Ubuntu were to care about my entirely irrelevant opinion, as both an application developer and heavy user of notification systems, I would say this to them:

You guys have done some great work on Notify-OSD.  It's a worthy prototype.  In many ways it is better than notification-daemon: it looks nicer, it makes notifications between applications more consistent.  I can appreciate the uncompromising vision you have for cleaning up the sometimes confusing pile of notifications that users see.

You should package Notify-OSD in Jaunty, so that people start using it.  Start updating applications to honor the capabilities that it provides.  But please, don't make it the default in the first release where it's included, and yes, include a preference for the period of transition.  Write some libraries to support the other use-cases which Notify-OSD right now ignores.  Document and stabilize the indicator applet.  Package the Python bindings, please.  Make it not crash when applications abuse it.

Regardless of this new notification system's unpolished state, I'm sure many users will update and start experimenting with Notify-OSD, much as many started with Compiz for years before it became the default window manager.  Most users can keep using the regular notification bubbles until Karmic, though.  When Karmic comes along, you'll have had the time you need to finish the documentation and provide application developers better alternatives to a good notification API before yanking the carpet out from under them.

If this advice is ignored, as I'm almost sure it will be, it won't bother me - notification is hardly the most important API that an OS provides.  The thing that I really hope someone will take away from this is the general theme that platforms should evolve experimental features slowly, and you should always have a well-documented, better alternative ready before you remove something.  Notify-OSD removes a half-dozen features and informally, halfheartedly gestures at some ways you can make your window pop up to address what some of those features used to do.  That's fine, as a scrappy new competitor to notification-daemon, but not as a core part of a major platform.

My real hope is not specifically that Notify-OSD will actually be pulled.  Of course I'd be happy if it were, but Jaunty going to be released in just a few days.  Again, I feel like I need to qualify these statements: if I'd really wanted to impact decisions like this I should really be regularly using beta releases.  Not to mention the fact that if I'd actually finished these hypothetical applications I'm thinking about, I'm sure my voice would carry more weight.

My real hope is that you, gentle reader, will take this message, and the next time you are contemplating boiling your favorite ocean, you'll stop and reflect.  Break down the changes you are planning on into individual, incremental improvements, rather than sweeping, break-everything lateral movement.  Make radical improvements, but make them behind the stable facade of a system which is only lifted when the radical improvement is clearly both radical and an improvement.

I tried to write a conference wrap-up, but there's just too much.  Besides, Ted Leung already wrote a better one than I could.  (I'm pleased to note that the first talk he mentions is the only Twisted talk at the conference: "Twisted AMQP and Thrift".  Go Esteve!)

I am once again impressed by the conference organizers.  Every year it seems like the conference gets better.  This year, it seemed like nothing went wrong - which wouldn't be remarkable if I didn't know just how difficult it is to create that impression.  I registered for the conference online, reserved my hotel room through the housing committee, and when I showed up everything was ready.  The food was good, all of the staff were helpful and efficient.  There was plenty of room for everything, even though the Open Space board was completely packed.

The video was particularly awesome.  I didn't get to see many talks, as I was moving from one conversation to another in the hallways, but I was impressed to discover that some of the talks I was hearing about during the conference were already online so I could catch up on the buzz.  The camera work and editing are really good.  I particularly liked that they often used picture-in-picture to show both the presenter and their slide.  But, you don't have to take my word for it; check them out now on pycon.blip.tv.

My personal experience of the conference is of course defined by hallway chats, open spaces and sprints.  This year we had a great Twisted open space, followed by a great Twisted Web open space the next day.  Thank you to everyone who came.

These sessions helped to reinforce for me the need to repeat this frequently for our users: if Twisted is doing something which confuses you but seems wrong, please go ahead and file a bug on twistedmatrix.com.  If you're wrong, and Twisted is working properly, there's still a bug, it's just a bug in the documentation.  If the documentation were perfect presumably you would understand why it's doing what it's doing.  Even if we ultimately decide that the documentation is sufficient, by adding a bug in a tracker, you've told Google that people with your question should find the document we refer to in our comments, so there's still value.

The sprints also went really smoothly this year.  Power and wifi were in abundance, so we could all just get to work.  I'll tell you a secret, though: I never plan to get much work done at the PyCon sprint, especially with monthly Twisted sprints here in Boston.  The value in this nation-wide gathering is in helping new users get up to speed with hacking on Twisted, and in running around talking to people sprinting on other projects, getting them to integrate with Twisted.  This year I had a pretty focused message that I wanted to get out: Twisted is a WSGI container, and it's one you can invoke from the command line with "twistd web --wsgi=your.application.here".  In other words, even if you prefer to write synchronous, blocking code, you can still use Twisted to run your web application, and you don't need to write any additional code to do it.  As a result of this communication, David Reid fixed a few minor bugs in our WSGI container and Twisted trunk now runs Django as well as Pinax.  I hope that this will drive even more adoption of Twisted in the Python world.

Also, it seems like for the last few years I've gotten started thinking about PyCon talks too late, and by the time the deadline rolls around I've got nothing.  In fact it seems like this has happened to most Twisted devs over the last few years.  This year I plan to get started right now.  I hope you'll join me, so that we can have a "Twisted comeback tour" at PyCon 2010.

One last thing: at least three people approached me at various points during the conference to ask me about using Mantissa and Axiom, and I didn't have a chance to catch up with any of them.  I feel bad about this.  If you tried to talk to me about using some Divmod technology at pycon, and still want to talk about it, please feel free to send me a personal email; we can set up a Skype session or something.

A friend of a friend has had some trouble with the thugs who run the recording industry.

Whatever you feel about file-sharing, I hope we can all agree that it's wrong to legally attack, and bankrupt, a poor college student on a full scholarship, for copying four songs.

While I would, of course, love you all to donate to Twisted, if you only have fifty dollars to donate this year, give it to Fabiola.  We'll muddle through.

As bloggers are often wont to observe, the best thing about PyCon is the "hallway track".  This is especially true for old hands like me who are already familiar with most of the projects and techniques being introduced and described in the scheduled talks.

Usually I just wander around in the hallways and let random encounters occur as they will.  That's worked out reasonably well in the past, but I think I can get more out of the conference with a little preparation.  In particular, I'd like to use the open-space rooms to do some of this discussion so that we can really come to some conclusions rather than standing awkwardly in hallways, seeming as if we're about to leave.  So, with your help, I'd like to plan things out a little more.  Here are the things that I'd really like to do.

I am going to dedicate the sprint to thinking and talking about development on Twisted itself.  Of course I'm up for having a few meals during the conference with the Twisted developers who I don't get to see in person too often, but I talk to you guys all the time — I'd rather spend the conference talking to people I wouldn't otherwise see.

During the conference, I'd like to try to talk to people who are working on other projects.  Of particular interest to me are PyGame and Django.  There are currently no good, well-documented answers to "how do you use Twisted to make a networked game with PyGame" or "how do you use Twisted to network non-HTTP protocols with an existing Django app".  Both of those seem to be increasingly common questions.  If you are a leader of either of those projects, or can help me schmooze with one of them, I'd appreciate it.  I am, of course, just as interested in hearing from other projects that I don't even know about their twisted-integration stories.

In general I'd like to encourage other projects unrelated to Twisted to do this kind of cross-project jam session if you can.  There's a great opportunity for cross-pollenation at PyCon.

I'd also like to hear from users of Twisted, Nevow, Mantissa, et cetera, especially those who have not been particularly vocal on the mailing lists or IRC.  The best kind of user is a prospective volunteer, of course (stay for the sprints!), but both praise and constructive criticism are also welcome.

Don't underestimate the value of cheerleading!  Open source projects thrive on donated labor, and the fuel for donated labor is enthusiasm.  I can say for myself that a few kind comments at prior PyCons have motivated lots of hacking.  I don't just mean for me, though: take the opportunity to thank your favorite open source Python hacker, whoever that may be.

I'll be raising funds for the Twisted project.  If you're considering sponsorship, I would like to talk to you about how it will make you and everyone in your organization healthier, smarter, and better-looking.  (Mind-control powers and teleportation are reserved for sponsors at the gold level and higher, and only while supplies last.)

Finally, I'd like to talk about some of Divmod's less well-known projects, like Imaginary and Vertex, especially with people who are interested in helping to work on such things.  I am thinking about maybe doing a lightning talk or open space session, if it seems like there is enough interest to justify the preparation effort.

If any of these things are interesting to you, please leave a comment or drop me a note.  I doubt that I'm popular enough, or organized enough, to really do this properly, but I'd like to have at least a vague calendar of pre-planned meetings and open space sessions on my phone so I can avoid wasting any time once I'm actually there.