A Rich Source of Gremlinium

Wednesday June 21, 2006
Today, Tycho makes an interesting point in Penny Arcade, which is to say, he agrees with me.

A few months ago, when I was first pimping Divmod's Vertex generic peer-to-peer system, the first application I proposed was a BitTorrent-like protocol, but with explicit support for giving users credit. There is a proof-of-concept implementation of this called "sigma" in vertex (but don't try to use it yet, it's full of horrible known bugs - it's just there if you want to help implement it).

When I first mentioned this to a few potentially interested parties, they assumed that the advantage would be increased bandwidth savings on the part of the "content provider". It doesn't provide that, and I'm not sure that one can improve on BT's savings; as Steve Holden said, the only platform Avalanche runs on is PowerPoint. When I revealed what I thought were substantial advantages, this prospective audience sighed and said that BitTorrent was great and that nobody cared about any of my features.

The advantages of a Vertex-implemented BitTorrent would be as follows:
  • Vertex provides extremely aggressive NAT-traversal code. If it can get through your firewall, it will. This is important because huge numbers of people using BitTorrent don't care that they're not transmitting data; the seeds are "fast enough" and they don't have the technical knowledge required to configure their firewalls. For some BitTorrent-based applications (such as the Blizzard downloader) such configuration is intentionally impossible; the downloader uses a fixed port number and would therefore require smarts on your gateway to forward properly. I cannot fathom how they could make such a stupid decision - that's certainly not a flaw in BitTorrent itself but it seems to be a common implementation mistake.
  • Vertex totally separates the application (file downloading) from the transport mechanism. This means a vastly reduced implementation complexity for implementors (if, like Blizzard, they think they need a totally customized "user experience", they can still use common vertex libraries, or at least implement the protocol separately).
  • Most relevant to Tycho's post, Vertex provides strong identity verification for peers. This means that you can provide non-bandwidth-related incentives for people to upload. Imagine that your character got one gold piece for every megabyte of the patch that you uploaded!

This last point I have had an especially difficult time communicating, because I thought that Tycho's point was obvious and I didn't bother to explain it.

So here's the thing: if you're a "content provider" and you use BitTorrent, it's great that it decreases your bandwidth costs, but the message you're sending to your users is, "I don't care about the experience you get using my site. I care about saving money on bandwidth." Especially given that the percentage of full-duplex capable seeders is so poor (see my point about NAT traversal) it is often significantly slower to download a file over BitTorrent than over a straight-up high-upstream link, and of course it is vastly better to simply offer it via a service like Akamai's media delivery.

If you want your users to shoulder part of the burden for you, that's great, and it can improve the experience for them provided you're using the money you've saved for something worthwhile, but at least say thank you for using their upstream. To do that, you have to know who is actually helping you more or less. Since BitTorrent has started using bits of Twisted now, hopefully by the time this project gets off the ground, it won't be some custom file-download protocol that offers this feature, it will simply be the next version of BT itself.

Thankfully Divmod has a client right now who is motivating a bit more maintenance on Vertex. I hope I can present it at the next CodeCon.