I realized during the huge thread this week on TPML that one of the major problems with Twisted's documentation is the lack of any roadmap.
I am prone to starting overly ambitious projects, so of course my first idea was to start on a new "roadmap website" where I would document the 5-year-plan.
Through some force of will I am managing not to do this and I will try to break up a few ideas that I have and post them here.
Twisted as a library
This closely matches how people are using Twisted right now.
There should be a simple template from which all 'twistified' programs are taken. It should provide support to your command for a useful subset of the same command-line options that twistd supports, and should provide a nice easy way to identify which reactor needs to be installed in order to use it.
It should be easy to identify a 'twistified' program in the same way that it is easy to identify a GNU program. There should be a large set of consistent options and a consistent configuration file format (probably some subset of Python).
Twisted as a framework
There should be very good plugin support, and good plugin detection support for every protocol. There should be a plugin management interface (echoes of COIL anyone?) available through the web UI.
There should be a program included with the Twisted sumo distribution, that uses the startup library described above, which starts up and binds a port for every protocol Twisted supports. At the least, it should do IRC, OSCAR, HTTP, FTP, SSH, LDAP*, IMAP, POP, and SMTP.
Each of these servers should have some level of application running on it - a thin enough layer that implements the semantics of some protocol that it's possible to find a interface point for plugins. Installed on each should be (for properly authenticated users) a plugin which executes Python commands. For example, manhole/conch under SSH, an AJAX-y web form that serves as an interactive prompt, a chatterbot over OSCAR and IRC that does the same.
Cred should also be extended to handle basic user creation and there should be a provided web control panel UI for manipulating a user-database of your choice.
I feel that this latter objective is neglected because the design is difficult, but it is still very important. Without some "official" way of making protocols interoperate, Twisted will be an underachiever, since it will only provide a simpler way to write applications that you could have built from scratch. When people can write components that hook together in novel and unexpected ways though - that is when an entirely new class of application can be enabled.
That doesn't mean that people should stop writing applications, either - simply that applications which embed the Twisted webserver have the option (default to on, but of course can be turned off) of allowing Twisted web plugins to be loaded into that server. Twisted chat server applications can similarly embed in-server bots.
Eventually, if the APIs for these respective application domains are really good, applications would be small wrappers that launched the plugin daemon with a pre-set configuration. The optimal situation would be if I could unconditionally declare that using plugins is a better idea than writing your own Twisted mainpoint. In order for that to be the case though, there will have to be a plugin system which loads UI modules in a variety of different toolkits and that is an even bigger project I don't want to talk about today.
*: Yes, I realize this is not a currently supported protocol. We'd have to write something.