For various reasons, I find myself spending more and more time in win32 these days.  Blech.

Luckily lots of useful programs have been ported to Windows lately so I am not entirely without tools.  However, there is one glaring problem which I can't seem to work around.

Rumour has it that the command-line interface to Windows isn't a program at all, but some kind of awful kernel service.  cmd.exe is also some kind of horrid abomination.  Here are some things that are wrong with it:

• Command Window
• • It doesn't draw the same style of window borders as every other window.
  • You can't resize it horizontally, interactively.
  • • You can't maximize it horizontally
    • On a system with multiple desktops, you can only maximize it on one desktop
  • There is no keyboard shortcut for pasting text.
  • "QuickEdit mode" isn't the default, so the window is unresponsive to the mouse
  • • and even when it is turned on, it behaves extremely bizarrely, unlike xterm or any other Windows program
  • The scrollback is unbelievably pathetic - only 999 lines?
  • You can't select monospaced fonts such as ProFont, only Lucida Console and the default VGA font
  • Regular window keyboard shortcuts don't work: you can't close it with Alt-F4.
  • It ... conflicts ... with some video card drivers.  I had to update my nvidia drivers just so I could drag command windows around!! How is that even possible!??
• CMD.EXE
• • no tab-completion of commands
  • no tab-completion of commands
  • you can't tab-complete commands
  • history behaves weirdly: you seem to maintain a global position in it, except... you don't
  • going back to your home directory is harder than starting a new terminal, seriously:
    X:\>%HOMEDRIVE%
    X:\>cd %HOMEPATH%
  • there is no shortcut like ~ to refer to your home path in other commands, either
  • there is no shell-startup file
  • .bat language is almost perversely crippled
  • the shell doesn't do expansion, so tricks like 'echo *' don't work.
  • the whole idea of %PATHEXT% is weird; how do I make this compatible with scripts for any other platform?
  • did I mention that tab-completion is broken?

Now, I realize that you can use cygwin's bash.exe to correct some of the deficiencies of the shell, but it has its own issues.  Is there anything that can replace the command window, so that I can use cywin's bash, Python, and other command-line tools without cringing constantly?  In particular the horizontal resize and maximize issues are the worst.  I have already tried xterm under cygwin as well as local SSH with PuTTY - both of these don't work very well with actual Windows command-line programs (such as Python) so I'd like something designed specifically for Windows.

Cyan Worlds has laid off all but two employees. Wow.

This company was a huge inspiration to me when I was growing up. The first time I ever thought I'd get involved with a software company, rather than making games as a hobby on my own, was watching the "Making of Myst" video that came as a companion disc with the first Myst CD. Apparently I'm not the only person who feels strongly about this company.

I've been really busy for the last few years in general, and haven't had time to play a Myst game since Myst III (which I haven't finished) - so this feels a little like finding out that a friend that I've been "too busy" to call has gone and died. Of course the murmurs of how this was caused by the inhuman churning of the mechanism that drives the game industry seem to echo the recently-published angst of other prominent game designers, and that doesn't make it any easier to hear.

I hope that this giant's demise at least warrants a comic strip.

Today I realized what Q2Q is. It is a (I swear, this just came to me, I was not even trying to make it sound like anything) Self-Certifying Remote Endpoint Authentication Mechanism, or "SCREAM".

A SCREAM in this sense is a mechanism whereby connections are authenticated by cryptographic means; where the handshake includes information identifying the connector to an arbitrary level of precision (in Q2Q's case, via an SSL certificate, that the connection is authenticated with)

It is self-certifying because the connection itself identifies itself, via both an in-band nonce and by TLS. All security is transport security.

It refers to a remote endpoint which is the other end of a networked communication. It identifies not only the user, but their agent, and optionally the capabilities and permissions of their agent.

It is an authentication mechanism because you use it to prove that your connection is authentic.

Also, Vertex will blow a hole in your NAT device the size of a watermelon: no kidding. Vertex is the Divmod implementation of Q2Q. We really want Q2Q to become a standard so we are making a big deal out of the separation between product and protocol.

(I really feel like there are some uses for this thing that I've missed. I really hope I have enough time to work on it in the next 6 months to see something through to fruition: other, less focused, worse P2P and identity solutions are starting to get some traction, and it bothers me.)

Alan Cox on Twisted:

"6Mbytes of unauditable weirdness"

"First they laugh at you", etc. :)

There's no such thing as magic. So when someone tells you that you can magically transform blocking code into Deferreds, as in this Python Cookbook posting, From blocking functions to Deferred functions, you should be suspicious.

As Itamar suggested, this particular goal can be accomplished with Twisted's standard twisted.internet.threads.deferToThread, which lacks the horrible, possibly crashing bugs present in the recipe presented above.

But, I'm not really here to talk about the recipe, or to impugn its author, Michele Simionato, who has written several other excellent recipes on ASPN; I have even personally used the DOT-grapher for inheritance hierarchies. I doubt Michele spent much time on this quick hack, or considered it a statement in the holy war I'm about to bring up, so please don't interpret what follows as a personal attack.

What concerns me is that there is a persistent meme around the periphery of the Twisted community that asynchronous programming is too hard, and that things would be easier if it looked like it were multi-threaded. This recently came up in a mailing list post I wrote as well.

My personal opinion on this, and I believe this is a matter of public record, is as follows: CONCURRENCY IS HARD. If you are going to write concurrent programs you need to think about it all the time; you need to plan for race conditions and draw your state-transition diagrams and have big explicit comments in any section of the code that has critical-section requirements even if you don't have to "lock" it as with an event driven system. No inventions have really significantly eased the cognitive difficulty of writing scalable concurrent applications and it is unlikely that any will in the near term. Systems like Twisted and Erlang have both provided powerful tools, but only if you are smart and willing to invest energy in learning to use them properly; they don't make the basic problems any easier. Most of all, threads do not help, in fact, they make the problem worse in many cases. To plagiarize a famous Lisp fellow, if you have a concurrency problem, and you decide to use threads, now you have two problems.

Let's put that aside for the moment, though.

Whether you agree with me about threads or not, though, Twisted was written by, and is maintained by a large group of people who feel basically the same way about this. We have some subtle differences about it, the consensus is the same. Threads are bad. Only use them when you have to, and understand clearly what that means, don't loudly provide "conveniences" for threads or use those "conveniences" for code which could otherwise be written as non-blocking.

Please, Twisted users, please stop trying to turn Twisted into something it isn't. If you want to use threads, write a multi-threaded program and please stop trying to write infrastructure for Twisted to turn it into a big multi-threaded application platform. WSGI and Zope efforts are excluded from this comment, by the way: that's not trying to help people to write threaded Twisted code, that's about trying to help Twisted be the container for code written using a totally different paradigm, on a different framework, and not written to directly use the Twisted libraries.

Programs written with these kinds of thread-happy conveniences are generally the ones which end up the buggiest, the hardest to test, and most likely the least efficient as well. Worst of all, when you do run into those problems, if you ask the Twisted dev team, you are likely to get a lot of smug "I told you so", and very little actual help, since we have seen the problem before and we keep trying to tell folks not to get started down this path. Personally, It's frustrating to have that advice disregarded again and again and still to get help requests from people who ignore it.

Imagine a man walks into a doctor's office, and says, "Doctor doctor, it hurts when I do this OW", promptly shooting himself in the hand with a nailgun. If this is the third time this week the doctor has removed such a nail, do you think the doctor is going to show this patient much patience? Now, imagine the doctor isn't getting paid for his services. The fellow would be lucky to walk out without a second nail...

You may have some awesome ideas about how multi-threaded programs should work. Good for you. I love reading the work of people who think differently than I do and succeed. It's a good way to learn. However, if you ask me, or if you use Twisted, you are going to run into a lot of advice to discard those ideas, a lot of roadblocks related to pervasive multi-threading, and general "impedance mismatch" problems with the differences between the way you think and the way Twisted works. It would probably be less work for you to start from scratch, or to use a system that has threads as a fundamental part of its programming model.

So, please, I'm not offended if you don't like Twisted, but if you like it, appreciate it for what it is, and if you don't, don't bother with it at all. Trying to use it while sweeping the most central parts of it under the rug isn't going to help anyone, least of all you.