If I am creating a new feature in library code, I have two choices with the implementation details: I can make them public - that is, exposed to application code - or I can make them private - that is, for use only within the library.

If I make them public, then the structure of my library is very clear to its clients. Testing is easy, because the public structures may be manipulated and replaced as the tests dictate. Inspection is easy, because all the data is exposed for clients to manipulate. Developers are happy when they can manipulate and test things easily. If I select "public" as the general rule, then developers using my library will be happy, because they'll be able to inspect and test everything quite easily whether I specifically designed in support for that or not.

However, now that they're public, I have to support them in their current form into the forseeable future. Since I tend to maintain the libraries I work on, and maintenance necessarily means change, a large public API surface means a lot of ongoing changes to exposed functionality, which means a constant stream of deprecation warnings and deprecated feature removals. Without private implementation details, there's no axis on which I can change my software without deprecating older versions. Developers hate keeping up with deprecation warnings or having their applications break when a new version of a library comes out, so if I adopt "public" as the general rule, developers will be unhappy.

If I make them private, then the structure of my library is a lot easier to understand by developers, because the API surface is much smaller, and exposes only the minimum necessary to accomplish their tasks. Because the implementation details are private, when I maintain the library, I can add functionality "for free" and make internal changes without requiring any additional work from developers. Developers like it when you don't waste their time with trivia and make it easier to get to what they want right away, and they love getting stuff "for free", so if I adopt "private" as the general rule, developers will be happy.

However, now that they're private, there's potentially no way to access that functionality for unforseen use-cases, and testing and inspection may be difficult unless the functionality in question was designed with an above-average level of care. Since most functionality is, by definition, designed with an average level of care, that means that there will inevitably be gaps in these meta-level tools until the functionality has already been in use for a while, which means that developers will need to report bugs and wait for new releases. Developers don't like waiting for the next release cycle to get access to functionality that they need to get work done right now, so if I adopt "private" as the general rule, developers will be unhappy.

Hmm.

I (along with about 6 million other people, according to the little statistics widget alongside it) just saw this rather heartbreaking post from Lennart Poettering.

I have not had much occasion to interact with Lennart personally, and (like many people) I have experienced bugs in the software he has written. I have been frustrated by those bugs. I may not have always been charitable in my descriptions of his software. I have, however, always assumed good faith on his part and been happy that he continues making valuable contributions to the free software ecosystem. I haven’t felt the need to make that clear in the past because I thought it was understood.

Apparently, not only is it not understood, there is active hostility directed against his participation. There is constant, aggressive, bad-faith attempts to get him to stop working on the important problems he is working on.

So, Lennart,

Thank you for your work on GNOME, for working on the problem of getting free software into the hands of normal people.

Thank you for furthering the cause of free software by creating PulseAudio, so that we can at least attempt to allow users to play sound on their Linux computers from multiple applications simultaneously without writing tedious configuration files.

Thank you for your work on SystemD, attempting to bring modern system-startup and service-management practices to the widest possible free software audience.

Thank you, Lennart, for putting up with all these vile personal attacks while you have done all of these things. I know you could have walked away; I’m sure that at times, you wanted to. Thank you for staying anyway and continuing to do the good work that you’ve done.

While the abuse is what prompted me to write this, I should emphasize that my appreciation is real. As a long-time user of Linux both on the desktop and in the cloud, I know that my life has been made materially better by Lennart’s work.

This shouldn’t be read as an endorsement of any specific technical position that Mr. Poettering holds. The point is that it doesn’t have to be: this isn’t about whether he’s right or not, it’s about whether we can have the discussion about whether he’s right in a calm, civil, technical manner. In fact I don’t agree with all of his technical choices, but I’m not going to opine about that here, because he’s putting in the work and I’m not, and he’s fighting many battles for software freedom (most of them against our so-called “allies”) that I haven’t been involved in.

The fact that he felt the need to write an article on the hideous state of the free software community is as sad as it is predictable. As a guest on a podcast recently, I praised the Linux community’s technical achievements while critiquing its poisonous culture. Now I wonder if “critiquing” is strong enough; I wonder if I should have given any praise at all. We should all condemn this kind of bilious ad-hominem persecution.

Today I am saying “thank you” to Lennart because the toxicity in our communities is not a vague, impersonal force that we can discuss academically. It is directed at specific individuals, in an attempt to curtail their participation. We need to show those targetted people, regardless of how high-profile they are, or whether they’re paid for their work or not, that they are not alone, that they have our gratitude. It is bullying, pure and simple, and we should not allow it to stand.

Software is made out of feelings. If we intend to have any more free software, we need to respect and honor those feelings, and, frankly speaking, stop trying to make the people who give us that software feel like shit.

Update 2021: While I still stand by many of the ideas expressed in this essay — particularly “software is made out of feelings” — my views have been significantly changed by two follow-ups. If you're interested in this topic, you should read them both; they’ll teach you more than this will.

The first, “Reverse Ungineering”, by LVH, was a direct response to my post, based on personal experience being trained as a civil engineer and working as a software engineer. Reverse Ungineering changed my opinion almost immediately, so I actually held the view expressed in the summary for a very short period of time after publishing.

The second, “Are We Really Engineers?”, by Hillel Wayne, is a small but comprehensive ethnographic study of people who have done both jobs. It’s extremely eye-opening, and made me realize just how much of my idea of “engineering” was derived from a mixture of fiction and popular culture, and not at all on any reality.

Both of these posts bring to bear informative facts based on direct personal experience, as opposed to my unsubstantiated hypothesizing. While I often still call myself a software “developer” or “author”, and I think that comparisons to fields like writing and research can also be illuminating, I do now call myself an engineer as well. The experience of writing this post and reading its rebuttals taught me an important lesson about not drawing conclusions from an imagined experience that some unfamiliar category of person — in this case, civil engineers — might have.

I am not an engineer.

I am a computer programmer. I am a software developer. I am a software author. I am a coder.

I program computers. I develop software. I write software. I code.

I’d prefer that you not refer to me as an engineer, but this is not an essay about how I’m going to heap scorn upon you if you do so. Sometimes, I myself slip and use the word “engineering” to refer to this activity that I perform. Sometimes I use the word “engineer” to refer to myself or my peers. It is, sadly, fairly conventional to refer to us as “engineers”, and avoiding this term in a context where it’s what everyone else uses is a constant challenge.

Nevertheless, I do not “engineer” software. Neither do you, because nobody has ever known enough about the process of creating software to “engineer” it.

According to dictionary.com, “engineering” is:

“the art or science of making practical application of the knowledge of pure sciences, as physics or chemistry, as in the construction of engines, bridges, buildings, mines, ships, and chemical plants.”

When writing software, we typically do not apply “knowledge of pure sciences”. Very little science is germane to the practical creation of software, and the places where it is relevant (firmware for hard disks, for example, or analytics for physical sensors) are highly rarified. The one thing that we might sometimes use called “science”, i.e. computer science, is a subdiscipline of mathematics, and not a science at all. Even computer science, though, is hardly ever brought to bear - if you’re a working programmer, what was the last project where you had to submit formal algorithmic analysis for any component of your system?

Wikipedia has a heaping helping of criticism of the terminology behind software engineering, but rather than focusing on that, let's see where Wikipedia tells us software engineering comes from in the first place:

The discipline of software engineering was created to address poor quality of software, get projects exceeding time and budget under control, and ensure that software is built systematically, rigorously, measurably, on time, on budget, and within specification. Engineering already addresses all these issues, hence the same principles used in engineering can be applied to software.

Most software projects fail; as of 2009, 44% are late, over budget, or out of specification, and an additional 24% are cancelled entirely. Only a third of projects succeed according to those criteria of being under budget, within specification, and complete.

What would that look like if another engineering discipline had that sort of hit rate? Consider civil engineering. Would you want to live in a city where almost a quarter of all the buildings were simply abandoned half-constructed, or fell down during construction? Where almost half of the buildings were missing floors, had rents in the millions of dollars, or both?

My point is not that the software industry is awful. It certainly can be, at times, but it’s not nearly as grim as the metaphor of civil engineering might suggest. Consider this: despite the statistics above, is using a computer today really like wandering through a crumbling city where a collapsing building might kill you at any moment? No! The social and economic costs of these “failures” is far lower than most process consultants would have you believe. In fact, the cause of many such “failures” is a clumsy, ham-fisted attempt to apply engineering-style budgetary and schedule constraints to a process that looks nothing whatsoever like engineering. I have to use scare quotes around “failure” because many of these projects classified as failed have actually delivered significant value. For example, if the initial specification for a project is overambitious due to lack of information about the difficulty of the tasks involved, for example – an extremely common problem at the beginning of a software project – that would still be a failure according to the metric of “within specification”, but it’s a problem with the specification and not the software.

Certain missteps notwithstanding, most of the progress in software development process improvement in the last couple of decades has been in acknowledging that it can’t really be planned very far in advance. Software vendors now have to constantly present works in progress to their customers, because the longer they go without doing that there is an increasing risk that the software will not meet the somewhat arbitrary goals for being “finished”, and may never be presented to customers at all.

The idea that we should not call ourselves “engineers” is not a new one. It is a minority view, but I’m in good company in that minority. Edsger W. Dijkstra points out that software presents what he calls “radical novelty” - it is too different from all the other types of things that have come before to try to construct it by analogy to those things.

One of the ways in which writing software is different from engineering is the matter of raw materials. Skyscrapers and bridges are made of steel and concrete, but software is made out of feelings. Physical construction projects can be made predictable because the part where creative people are creating the designs - the part of that process most analagous to software - is a small fraction of the time required to create the artifact itself.

Therefore, in order to create software you have to have an “engineering” process that puts its focus primarily upon the psychological issue of making your raw materials - the brains inside the human beings you have acquired for the purpose of software manufacturing - happy, so that they may be efficiently utilized. This is not a common feature of other engineering disciplines.

The process of managing the author’s feelings is a lot more like what an editor does when “constructing” a novel than what a foreperson does when constructing a bridge. In my mind, that is what we should be studying, and modeling, when trying to construct large and complex software systems.

Consequently, not only am I not an engineer, I do not aspire to be an engineer, either. I do not think that it is worthwhile to aspire to the standards of another entirely disparate profession.

This doesn’t mean we shouldn’t measure things, or have quality standards, or try to agree on best practices. We should, by all means, have these things, but we authors of software should construct them in ways that make sense for the specific details of the software development process.

While we are on the subject of things that we are not, I’m also not a maker. I don’t make things. We don’t talk about “building” novels, or “constructing” music, nor should we talk about “building” and “assembling” software. I like software specifically because of all the ways in which it is not like “making” stuff. Making stuff is messy, and hard, and involves making lots of mistakes.

I love how software is ethereal, and mistakes are cheap and reversible, and I don’t have any desire to make it more physical and permanent. When I hear other developers use this language to talk about software, it makes me think that they envy something about physical stuff, and wish that they were doing some kind of construction or factory-design project instead of making an application.

The way we use language affects the way we think. When we use terms like “engineer” and “builder” to describe ourselves as creators, developers, maintainers, and writers of software, we are defining our role by analogy and in reference to other, dissimilar fields.

Right now, I think I prefer the term “developer”, since the verb develop captures both the incremental creation and ongoing maintenance of software, which is so much a part of any long-term work in the field. The only disadvantage of this term seems to be that people occasionally think I do something with apartment buildings, so I am careful to always put the word “software” first.

If you work on software, whichever particular phrasing you prefer, pick one that really calls to mind what software means to you, and don’t get stuck in a tedious metaphor about building bridges or cars or factories or whatever.

To paraphrase a wise man:

I am developer, and so can you.

Sometimes, sea sickness is caused by a sort of confusion. Your inner ear can feel the motion of the world around it, but because it can’t see the outside world, it can’t reconcile its visual input with its proprioceptive input, and the result is nausea.

This is why it helps to see the horizon. If you can see the horizon, your eyes will talk to your inner ears by way of your brain, they will realize that everything is where it should be, and that everything will be OK.

As a result, you’ll stop feeling sick.

I have a sort of motion sickness too, but it’s not seasickness. Luckily, I do not experience nausea due to moving through space. Instead, I have a sort of temporal motion sickness. I feel ill, and I can’t get anything done, when I can’t see the time horizon.

I think I’m going to need to explain that a bit, since I don’t mean the end of the current fiscal quarter. I realize this doesn’t make sense yet. I hope it will, after a bit of an explanation. Please bear with me.

Time management gurus often advise that it is “good” to break up large, daunting tasks into small, achievable chunks. Similarly, one has to schedule one’s day into dedicated portions of time where one can concentrate on specific tasks. This appears to be common sense. The most common enemy of productivity is procrastination. Procrastination happens when you are working on the wrong thing instead of the right thing. If you consciously and explicitly allocate time to the right thing, then chances are you will work on the right thing. Problem solved, right?

Except, that’s not quite how work, especially creative work, works.

I try to be “good”. I try to classify all of my tasks. I put time on my calendar to get them done. I live inside little boxes of time that tell me what I need to do next. Sometimes, it even works. But more often than not, the little box on my calendar feels like a little cage. I am inexplicably, involuntarily, haunted by disruptive visions of the future that happen when that box ends.

Let me give you an example.

Let’s say it’s 9AM Monday morning and I have just arrived at work. I can see that at 2:30PM, I have a brief tax-related appointment. The hypothetical person doing my hypothetical taxes has an office that is a 25 minute hypothetical walk away, so I will need to leave work at 2PM sharp in order to get there on time. The appointment will last only 15 minutes, since I just need to sign some papers, and then I will return to work. With a 25 minute return trip, I should be back in the office well before 4, leaving me plenty of time to deal with any peripheral tasks before I need to leave at 5:30. Aside from an hour break at noon for lunch, I anticipate no other distractions during the day, so I have a solid 3 hour chunk to focus on my current project in the morning, an hour from 1 to 2, and an hour and a half from 4 to 5:30. Not an ideal day, certainly, but I have plenty of time to get work done.

The problem is, as I sit down in front of my nice, clean, empty text editor to sketch out my excellent programming ideas with that 3-hour chunk of time, I will immediately start thinking about how annoyed I am that I’m going to get interrupted in 5 and a half hours. It consumes my thoughts. It annoys me. I unconsciously attempt to soothe myself by checking email and getting to a nice, refreshing inbox zero. Now it’s 9:45. Well, at least my email is done. Time to really get to work. But now I only have 2 hours and 15 minutes, which is not as nice of an uninterrupted chunk of time for a deep coding task. Now I’m even more annoyed. I glare at the empty window on my screen. It glares back. I spend 20 useless minutes doing this, then take a 10-minute coffee break to try to re-set and focus on the problem, and not this silly tax meeting. Why couldn’t they just mail me the documents? Now it’s 10:15, and I still haven’t gotten anything done.

By 10:45, I manage to crank out a couple of lines of code, but the fact that I’m going to be wasting a whole hour with all that walking there and walking back just gnaws at me, and I’m slogging through individual test-cases, mechanically filling docstrings for the new API and for the tests, and not really able to synthesize a coherent, holistic solution to the overall problem I’m working on. Oh well. It feels like progress, albeit slow, and some days you just have to play through the pain. I struggle until 11:30 at which point I notice that since I haven’t been able to really think about the big picture, most of the test cases I’ve written are going to be invalidated by an API change I need to make, so almost all of the morning’s work is useless. Damn it, it’s 2014, I should be able to just fill out the forms online or something, having to physically carry an envelope with paper in it ten blocks is just ridiculous. Maybe I could get my falcon to deliver it for me.

It’s 11:45 now, so I’m not going to get anything useful done before lunch. I listlessly click on stuff on my screen and try to relax by thinking about my totally rad falcon until it’s time to go. As I get up, I glance at my phone and see the reminder for the tax appointment.

Wait a second.

The appointment has today’s date, but the subject says “2013”. This was just some mistaken data-entry in my calendar from last year! I don’t have an appointment today! I have nowhere to be all afternoon.

For pointless anxiety over this fake chore which never even actually happened, a real morning was ruined. Well, a hypothetical real morning; I have never actually needed to interrupt a work day to walk anywhere to sign tax paperwork. But you get the idea.

To a lesser extent, upcoming events later in the week, month, or even year bother me. But the worst is when I know that I have only 45 minutes to get into a task, and I have another task booked up right against it. All this trying to get organized, all this carving out uninterrupted time on my calendar, all of this trying to manage all of my creative energies and marshal them at specific times for specific tasks, annihilates itself when I start thinking about how I am eventually going to have to stop working on the seemingly endless, sprawling problem set before me.

The horizon I need to see is the infinite time available before me to do all the thinking I need to do to solve whatever problem has been set before me. If I want to write a paragraph of an essay, I need to see enough time to write the whole thing.

Sometimes - maybe even, if I’m lucky, increasingly frequently - I manage to fool myself. I hide my calendar, close my eyes, and imagine an undisturbed millennium in front of my text editor ... during which I may address some nonsense problem with malformed utf-7 in mime headers.

... during which I can complete a long and detailed email about process enhancements in open source.

... during which I can write a lengthy blog post about my productivity-related neuroses.

I imagine that I can see all the way to the distant horizon at the end of time, and that there is nothing between me and it except dedicated, meditative concentration.

That is on a good day. On a bad day, trying to hide from this anxiety manifests itself in peculiar and not particularly healthy ways. For one thing, I avoid sleep. One way I can always extend the current block of time allocated to my current activity is by just staying awake a little longer. I know this is basically the wrong way to go about it. I know that it’s bad for me, and that it is almost immediately counterproductive. I know that ... but it’s almost 1AM and I’m still typing. If I weren’t still typing right now, instead of sleeping, this post would never get finished, because I’ve spent far too many evenings looking at the unfinished, incoherent draft of it and saying to myself, "Sure, I’d love to work on it, but I have a dentist’s appointment in six months and that is going to be super distracting; I might as well not get started".

Much has been written about the deleterious effects of interrupting creative thinking. But what interrupts me isn’t an interruption; what distracts me isn’t even a distraction. The idea of a distraction is distracting; the specter of a future interruption interrupts me.

This is the part of the article where I wow you with my life hack, right? Where I reveal the one weird trick that will make productivity gurus hate you? Where you won’t believe what happens next?

Believe it: the surprise here is that this is not a set-up for some choice productivity wisdom or a sales set-up for my new book. I have no idea how to solve this problem. The best I can do is that thing I said above about closing my eyes, pretending, and concentrating. Honestly, I have no idea even if anyone else suffers from this, or if it’s a unique neurosis. If a reader would be interested in letting me know about their own experiences, I might update this article to share some ideas, but for now it is mostly about sharing my own vulnerability and not about any particular solution.

I can share one lesson, though. The one thing that this peculiar anxiety has taught me is that productivity “rules” are not revealed divine truth. They are ideas, and those ideas have to be evaluated exclusively on the basis of their efficacy, which is to say, on the basis of how much stuff that you want to get done that they help you get done.

For now, what I’m trying to do is to un-clench the fearful, spasmodic fist in my mind that gripping the need to schedule everything into these small boxes and allocate only the time that I “need” to get something specific done.

Maybe the only way I am going to achieve anything of significance is with opaque, 8-hour slabs of time with no more specific goal than “write some words, maybe a blog post, maybe some fiction, who knows” and “do something work-related”. As someone constantly struggling to force my own fickle mind to accomplish any small part of my ridiculously ambitions creative agenda, it’s really hard to relax and let go of anything which might help, which might get me a little further a little faster.

Maybe I should be trying to schedule my time into tiny little blocks. Maybe I’m just doing it wrong somehow and I just need to be harder on myself, madder at myself, and I really can get the blood out of this particular stone.

Maybe it doesn’t matter all that much how I schedule my own time because there’s always some upcoming distraction that I can’t control, and I just need to get better at meditating and somehow putting them out of my mind without really changing what goes on my calendar.

Maybe this is just as productive as I get, and I’ll still be fighting this particular fight with myself when I’m 80.

Regardless, I think it’s time to let go of that fear and try something a little different.