What the hell has gotten into these GNOME people's minds

As the "Mood;" header implies, I'm annoyed. I just installed the GNOME 2.6 release onto 2 computers.



The most visible feature of this release is that it completely broke my keyboard layout. Nicholas Petrely and others have already commented harshly upon this release. While I like the "spatial nautilus" feature that Petreley and others have complained loudly about, and I agree that he should have done his homework just a little better, maybe by clicking on the "Applications" menu, I do think that there is an upsetting trend in GNOME development; the GNOME developers see themselves as the final arbiters of the user's desktop experience, and are trying to subsume all parts of X as part of their configuration. The switch from sawfish to metacity was the first step down this path, and others have been fast following.

Today's adventure was rooted in the new "Keyboard" control panel. Visually an appealing feature, there is now a "Layout Options" tab, which lets you fix a few of my favorite keyboard layout problems (control key in an inconvenient position, multi / compose key not configured on many keyboards, etc). However, there are options that are left out.

For example, I use a Mac keyboard, because it is the only decent scissor-action USB keyboard I can find. Macs have the odd habit of placing the 'alt' key in an unfamiliar location and the 'command' key (interpreted as the 'windows' key by linux) in the normal meta-key position. So I want to swap those because this is a very unpleasant finger-memory habit to be constantly correcting. Then, I would like to assign Hyper to the meta keys.

I did this with an ~/.Xmodmap file. Upon logging in to Gnome 2.6 for the first time, however, I was confronted with a pleasant dialog that alerted me to the fact that this was no longer supported, and I'd have to use the new Keyboard control panel. "Surely," I thought, "the ever-wise GNOME developers will have considered this case and provided an option for me in a handy new GUI control panel."

They had not.

In fact, not only had they broken a standard X behavior, they seem to have gone to great lengths to make it impossible to correct. As I am "Advanced", I thought that I would use the "sessions" control panel to run a command upon login, at the right point (after the session manager proxy but before my window manager). No dice! This hangs the login process with no debug information, as it appears to be infinitely attempting to restart the xmodmap program and waiting for it to map a window.

Also, I can't run ~/.xmodmap in my ~/.xsession, because gnome-smproxy resets both any system-wide or any local xkb configuration changes. I think. I also couldn't find any documentation of this anywhere on short notice.

I eventually fixed the problem by inserting a shell script into my $PATH that looks like this:


glyph@kazekage:~% cat UNIX/bin/sawfish
#!/bin/bash

xmodmap ~/.Xmodmap
exec /usr/bin/sawfish $*


There is a lesson here, for GNOME as well as other developers.

DO NOT INTENTIONALLY DISABLE FUNCTIONALITY IN AN "UPGRADE" UNTIL THERE IS A REPLACEMENT AVAILABLE. In software development we call this a "regression" and it is why we have "regression tests", so that things like this do not happen.

I understand the new GNOME philosophy that this is a bug, and the correct solution is to fix the bug (by providing a more robust GUI keyboard-layout editor) and not to provide some kind of backward-compatibility hack that allows me to turn on ~/.Xmodmap again. However, the reality of integrated software is an ugly place. These kinds of hacks exist because it's really hard to re-implement functionality that's been around for ten or twenty years and do it correctly so you won't inadvertently break half of your users. And yes, in the UNIX world, you are much more likely to encounter heinous edge cases, weird custom hardware, and abrasive, know-it-all users than elsewhere. It's a niche where these sorts of things accrete.

I don't intend to call the GNOME developers names or insult their intelligence and motives as seems to be common when critiquing these decisions. Still, if they are interested in me (and the users for which I am sure I am a proxy here) using their software, they have to have a less cavalier attitude towards edge-cases and outside-of-GNOME configuration options.

These configuration options have real uses. Some examples: I use sawfish (and have sawfish depend on the hyper key) in part because I use the alt-tab key combination in emacs, which binds it to lisp-complete-symbol by default. I also have several keybindings useful for managing large quantities of windows, which Metacity is woefully inadequate for. What if you open every image on a website in GIMP and you want to iconify the resulting window group? Not only will this take you several minutes of real time thanks to the mandatory animation, it's not possible to do it automatically. You also can't tile or cascade just the group, shade them and stack them, move them all to the lower-left hand corner of the screen so that the close-boxes line up, or do any kind of batch window manager operation. It's the same bone-grindingly slow window management routine I've had to get used to on Windows - worse even than the Mac, where at least I have application groups provided by the OS.

All in all, I think that GNOME 2.6 is an improvement over previous versions. There was a lot of great work that went into it both from the GNOME developers and the Debian packagers who made my upgrade otherwise seamless. Still, I have yet to upgrade GNOME and waste less than 2 hours getting my configuration working again, though, and at some point the time consumed by constant maintenance of my setup and creation of gross hacks and workarounds is going to lead me to just run sawfish and fspanel or something.

Unfortunately, thanks to extremely tight coupling somewhere in the configuration framework, running with that light of an environment also seems to break all my GTK themes, so I have to make the choice between a working keyboard and a nice-looking desktop. Maybe I'll spend the two hours it takes to get a gtk configuration without gnome-session working at some point.

Update: there is a Debian bug and a GNOME bug already filed about this. Apparently it's equally destructive if you just happen to run one gnome application. To be fair, this is almost as much the fault of the XKB system as it is of GNOME. XKB seems to provide ten times the complexity of xmodmap, with none of the functionality. A quote from the documentation:
You can think of a group as of a vector of columns per each keycode (naturally the dimension of this vector may differ for different keycodes).

Naturally.

Triumphant Return

At least for the next few weeks, Ying is here again!

Modulo a trip to China in 3 weeks, she is back for good.

I have to say it, it is the law

I'm sorry. I know you've heard it already. But, this is a blog. I am practically required to comment upon the MovableType fracas.



I suppose I should start with that quintessentially bloggish idiom of a clever phrase where every word is a hyperlink detailing the community disaster they have on their hands. I might mention that every link there was taken from the trackback of the corporate weblog where they announced this "improvement"; the top title right now is "Looks Like I'll be Dumping MovableType Soon", and they get worse from there.

I feel bad for the Six Apart team. It must be very painful to be dragged over the coals like this for a change that they perceive as an improvement, and one which they are already rushing to fix. But there is an important point here, not to be lost among the recriminations and the loud declarations of switching.

In Mark Pilgrim's dramatically titled Freedom Zero posting, he explains what's going wrong. Essentially, "free enough" is not free enough. When MovableType was initially released, the licensing structure was not generally perceived as a good thing. It was a shameful and annoying legal detail that MT fans tried hard to ignore because the functionality was tempting.

What's happening here is that Six Apart has taken this small hypothetical problem and made it large, real and relevant for all their users. Although users may complain about the details of the licensing strategy, the real issue is that if it's changing now, it will change again. The message getting sent is: if Six Apart feels that you can afford more money for their tool, then they will cut you off from future upgrades if you don't agree.

This is a particularly bad idea in a market where free alternatives are both plentiful and arguably superior. Blogging tools are largely a matter of taste, since there are dozens of them with comparable feature sets.

I hope that this is a precedent, though, for more and more users to start being vocal about the fact that source secrecy, license key managers, and the like are not features, but barely-tolerated impositions. It may be that some categories of software will always be closed-source, but maybe we can work towards an understanding that that is a tradeoff.

Say Hello To Our New Robotic Masters

http://asimo.honda.com/

I'm starting my "collapse of civilization" timer now. Also, I really want one of these!

Update: was the one who showed this to me, and she wants one too.

man what the heck

I should have different blogs for different things. Today, I'm going to be talking about technology. Specifically, the implications of time travel on the quality of life in post-submergence California.

...

Okay, our sponsors don't like that one, and would like me to inform you that I've never been anywhere in the future, let alone to any periods after the 2295 collapse of the Pax Hegemonia world government. Apparently hillbillies have all the fun. Just because I don't have ten billion dollars and a hobo-hunting range they think they can tell me what to do! It is a good thing my crafty markup confuses them!

Instead, I have a few features that I've been thinking about for Twisted that have implications for my favorite application.



Subsystems



Some kinds of processing are better done in a blocking, synchronous, program-at-a-time kind of way. It's always a challenge when you're in a non-blocking environment and you want to do something that just involves a whole crapload of math or data copying that you can't split up easily.

In these cases it would generally better if you had a subprocess which could do all the nasty blocking work and then return a small result, copying the large chunks of data out of band (say, to files, or to a database).

The particular use case I'm thinking of is using Lupy to index and search for messages in Quotient. In this case, there are 3 operations: flush your cache, index this thing into that index, and query that index for some things based on this text. Additionally, as the first operation implies, indexes have quite a bit of stuff that can be cached, so it's good to keep the subprocess alive for long periods of time and not attempt to shut it down too often.

I am probably going to do the inter-process communication with pickle, because I don't need safety between processes in this case, but I imagine that will be pluggable. The general interface I want is to have a global subsystem manager that I can ask for a particular service (by Interface) and then request that a method be called on it and given certain arguments. This will be done in a subprocess, blocking, and the result from the subsystem manager in the parent process will be a Deferred that will fire when the operation is done.

The main thing that I want this to do is to manage the IPC and to start and stop processes as necessary. The real trick, I think, will be spawning a process that has a reactor of a variety similar to that of the superprocess; it should run twistd, with an appropriate reactor argument, and then load a subsystem module into it, but I don't know what that looks like.

Faceted



I decided I'd have a name for the new kind of Componentized. I think that "Faceted" is nice because rather than having an Adapter class which is intended for both a generally useful superclass of all adapters AND the magical getComponent-makes-a-U-turn behavior, you can have a "Facet" which does magical stuff and an "Adapter" elsewhere that is simply utility.

from twisted.python import reflect

class Facet(object):
def __init__(self, original):
self.original = original

def getComponent(self, interface, registry=None, default=None):
return self.original.getComponent(self, interface, registry, default)

class Faceted(dict):
__slots__ = ()
def getComponent(self, interface, registry=None, default=None):
return self.get(interface, default)


This is actually a version that I believe is compatible with the existing (crummy) component system behavior. It's just a prototype; it doesn't do adapter lookup, but most of the time you want to be explicit about putting adapters on something stateful like this anyway. I believe we'll want adapter registry lookup, but this example illustrates how simple it can be.

System Plug Ins



In Atop, I'm unhappy with the way Powerups turned out. They are way too stateful, because they have to procedurally add and remove themselves from sources of events; they can't just declare their areas of interest and let the framework manage the state.

There's a general need, I think, for the powerup interface to fill two roles: one is to provide the functionality it already does, which is access mechanisms from various cred-enabled protocols via credup, and the other is responses to events in the system. This is also just a sketch, but I think it might make a good central event broadcasting mechanism for something like that:

from twisted.python import log

class Broadcaster:
def __init__(self):
self.listeners = {}

def addListener(self, interface, listener):
l = self.listeners
if not l.has_key(interface):
l[interface] = []
l[interface].append(listener)

def removeListener(self, interface, listener):
l = self.listeners
if not l.has_key(interface):
return
l[interface].remove(listener)

def getListeners(self, imeth):
return self.listeners[imeth.im_class]

def callMethod(self, listener, imeth, *a, **kw):
return getattr(listener, imeth.im_func.__name__)(*a,**kw)

def broadcast(self, interfaceMethod, *a, **kw):
m = self.getListeners(interfaceMethod)
for i in m:
self.callMethod(i, interfaceMethod, *a, **kw)

def safeBroadcast(self, interfaceMethod, *a, **kw):
m = self.getListeners(interfaceMethod)
for i in m:
try:
self.callMethod(i, interfaceMethod, *a, **kw)
except:
log.err()

def collect(self, interfaceMethod, *a, **kw):
m = self.getListeners(interfaceMethod)
for i in m:
yield self.callMethod(i, interfaceMethod, *a, **kw)

def safeCollect(self, interfaceMethod, *a, **kw):
m = self.getListeners(interfaceMethod)
for i in m:
try:
yield self.callMethod(i, interfaceMethod, *a, **kw)
except:
log.err()


This uses interfaces - I might have the getListeners method also adapt to the interface for extra flexibility, I'm not sure - but this, plus some kind of hierarchical channel-managing API, would allow the powerup to add itself to the appropriate broadcaster.

This would also be a nice way to encapsulate pool events through a general mechanism rather than having pools do everything themselves.

Execution Context



Every major system in Quotient requires its own execution context:

  • Atop relies on a transaction being in twisted.python.context.

  • Twisted relies on the reactor being instantiated

  • Nevow relies on an explicitly-passed 'context' object to every method it interacts with, which is generated by the page render. This generally also requires that some data, and an IRequest implementor, be in scope at render time.


These three systems are all currently ad-hoc. They should be unified, and the context should use the same method of identifying fragments of context for all three: interface names. For example,

# in all new style examples, something like
from twisted.python.excctx import context

# Twisted
from twisted.internet import reactor
# would become
from twisted.internet.interfaces import IReactor
reactor = IReactor(context)

# Nevow
from iwhatever import IA, IB, IC
from nevow.inevow import IRequest
def doit(context, data):
context.remember(Foo(), IA, IB, IC)
request = context.locate(IRequest)
# would become
def doit(data):
context[IA, IB, IC] = Foo()
request = IRequest(context)

# Atop
c = context.get("CursorFactory").cursor()
# would become
c = ICursorFactory(context).cursor()


I think this would let us sort out a lot of thorny problems, for example, by putting the currently running services into context when they are being invoked; by saving context across Deferreds, one can do things like automatically put both parts of a Nevow page rendering with a Deferred in the middle into two different (potential) transactions. Capturing context would also be useful for a protocol to initialize itself without storage, set its storage in the login process, and then automatically have transactions from then forward with any protocol notification switching into the protocol's saved context.

For the mandatory evil associated with a rambling post like this, I was thinking about how to implement a convenient context.clone() which would push another level onto the context stack without requiring a context.call - so that you could use the new context variables in the rest of your function. This is evil enough for par, I think, and it should be obvious how one would implement such a thing from here:

import gc
import inspect
import weakref

_wrd = {}
_wrc = 0

def whencallerexits(x, stacklevel=2):
global _wrd, _wrc
up = inspect.currentframe()
for ig in range(stacklevel):
up = up.f_back
n = _nothing()
up.f_locals[' nothing '] = n
cap = _wrc
def curry(w):
del _wrd[cap]
x()
_wrd[cap] = weakref.ref(n, curry)
_wrc += 1

def printit():
print 'aaa'

def somestuff():
print 'doing some stuff'
whencallerexits(printit)
print 'done doing some stuff'

def horrible():
print 'get ready for some horribleness'
somestuff()
print 'here we go...'

horrible()
print "wasn't that horrible?"


I think there may still be some remaining issues with managing which bits of context want to be remembered when captured and which don't -- for example, you still want to be able to switch the logging backend for the entire application, not just the protocols which haven't remembered their own logging frosting features -- but I believe it's possible to provide answers to a lot of questions regarding system-wide configuration by providing this context as pseudo-global state which can be addressed as an object rather than as actual global variables. This way multiple applications could still run entirely separated in the same process, just by referring to different context roots; in fact, properly done, this would enable multiple reactors to run in the same process WITHOUT any crazy bootstrapping, by using multi-threading and specifying the context root with a different reactor from the .callInThread from the initial reactor.

I hope that some of that made sense. Have a happy weekend.