Simple Made Variadic

Thursday July 12, 2012
Last night I made a snarky tweet about how Clojure is doomed.  Out of context, it didn't really make a lot of sense, and Ivan Krstić replied asking what the heck I was talking about.  I tried to fit the following into a tweet but it kinda broke the tweeterizer I was using right in half, and so I had to put it here.

I love a good snark as much as the next person - some might say more - but it really bothers me when people make snide comments denigrating others' free work without at least offering a cogent criticism to go with it, and I don't want to be that guy.  So, hopefully before the whole Clojure community finds said tweet and writes me off as an arrogant Python bigot, I would like to explain what I meant in a bit more detail.

Right off the bat I should say that this was a bit tongue-in-cheek.  I actually rather like Clojure and I think Rich Hickey has some very compelling ideas about programming in general.  I watch his talk "Simple Made Easy" once every month or two, contemplating its deeper meaning, and I still usually come away with an insight or two.

I should also make it clear that I was linking to the recur special form specifically, and not just the special forms documentation in general.  Obviously having reference docs isn't a bad thing.

Ivan, (or should I say, "@radian"?) you may be right; that documentation you linked to may indeed one day spell Python's doom.  If Python does eventually start to suck, it will be because it collapsed under the weight of its own weird edge cases like the slightly-special behavior of operator dispatch as compared to method dispatch, all the funkiness of descriptors, context managers, decorators, metaclasses, et cetera.

A portion of my point that was serious, though.  The documentation for recur does highlight some problems with Clojure that the Python docs can play an interesting counterpoint to.

The presence of the recur form at all is an indication of the unhealthy level of obsession that all LISPs have with recursion.  We get it: functions are cool. You can call them. They can call themselves. Every other genre of language manages to use this to a reasonable degree of moderation without adding extra syntactic features to their core just so you can recurse forever without worrying about stack resources. Reading this particular snipped of documentation, I can almost hear Rich Hickey cackling as he wrote it, having just crowned himself God-Emperor of the smug LISP weenies, as he gleefully points out that Scheme has it wrong and the CL specifications had it wrong with respect to tail call elimination, and that it should be supported by the language but also be explicit and compiler-verified.

The sad thing is, my hypothetical caricature of Clojure's inventor is actually right! This is a uniquely clever solution to a particularly thorny conceptual problem with the recursive expression of algorithms. The Scheme folks and the Common Lisp folks did both kinda get it wrong.  But the fact that this has to be so front-and-center in the language is a problem. Most algorithms shouldn't be expressed recursively; it is actually quite tricky to communicate about recursive code, and anyway most systems that really benefit from it have to be mutually, dynamically reentrant anyway and won't be helped by tail call elimination.  (My favorite example of this is still the unholy shenanigans that Deferreds have to get up to to make sure you don't have to care about callback chain length or Deferred return nesting depth.)

Also, if you want to be all automatically parallelizable and web scale and "cloud"-y, recursion and iteration are both the wrong way to do it; they're both just ways of tediously making your way down a list of things one element at a time.  What you want to do is to declaratively apply a computation to your data in such a way as to avoid saying anything about the order things have to happen in.  To put it more LISPily, (map) is a better conceptual foundation for the future than (loop) or (apply).  Of course you can do the naive implementation of (map) with (recur), but smarter implementations need application code to be written some other way.

The language style choices of the manual in this case is also telling. The Python docs that Ivan linked to go into excruciating detail, rephrasing and explaining the same concept in a few different ways, linking to other required concepts in depth so the reader can easily familiarize themselves with any prerequisites, while still essentially explaining a nerdy part of the language that you can ignore while still using it productively. Every Python programmer ignores descriptors while they're learning to write classes and methods, despite that they're using them all the time; this ability to be understood at different levels of complexity is a strength of every good language, and python does particularly well in that regard. Of course one could also make the case that this is just because Python has so many dusty corners hidden behind double-underscores, and a better language would just have less obscure junk in it, not make understanding the obscure junk optional, but I digress.

The description of recur, by contrast, is deeply flawed.  It is terse, to a fault. It introduces the concept of "recursion points" without linking to any kind of general reference.  It uses abbreviations all over the place ("exprs", "params", "args", "seq") without even using typesetting to illuminate whether they are using a general term or a specific variable name.

But, by far, the worst sin of this document is the use of the words "variadic" and "arity". There is really no excuse for the use of these words, ever. Take it from me, I am exactly the kind of pedantic jerk who will drop "arity" into a conversation about, for example, music theory, just to demonstrate that I can, and as that kind of jerk I can tell you with certainty: I have no excuse.

It should say: "a function that takes a variable number of arguments". Or possibly: "it must take exactly the same number of arguments as the recursion point".

This was particularly disappointing example to me because Clojure strikes me as a particularly... for lack of a better word, "optimistic" lisp, one that looks forward rather than back, one that is more interested in helping people find comprehensible and composeable ways to express programs than in revisiting obscure debates over reader macros or lisp-1/lisp-2 holy wars.  But the tone of the documentation for recur aims it straight at the smug lisp weenie crowd.

As I hope is obvious, if not initially, then at least by now, I don't think that Clojure will fail (or succeed) on the merits of one crummy piece of documentation. It's a much younger language than Python, so it may have a ways to go in its documentation practices. It also comes from an illustrious heritage that I can't expect to see none of in the way that it talks about itself, no matter how unfortunate certain details of that heritage are.  Heck, at the parallel point in Python's lifetime, it didn't even have descriptors yet, let alone the documentation for them!

Still, I don't think that this issue is entirely trivial, and I hope that the maintainers for the documentation for Clojure, at least the documentation for the parts of the language you have to see every day, take care to improve its accessibility to the less arcane among us.