The uncanny practicality of impracticality

The uncanny practicality of impracticality

I think one of the big cultural disconnects between me & current mainstream dev is the idea that it’s a burden to know something.

It’s not totally alien: it’s a burden to need to know something, and it’s a burden to need to know it on a deadline. So, I understand wanting a minimum understanding when it comes to doing professional work. But, I have never considered professional work to be the most meaningful part of software development.

I got into this field because I consider it a joy to learn about the ugly twisty obscure corners of the domain. So, the “real work” is the stuff I do in my free time out of interest, while the stuff I do on a deadline is just leveraging my existing skills to make a couple bucks. I’m up for learning anything, as long as it happens off company time and happens because I feel like learning it.

Knowledge’s association with effectiveness is complicated. I suspect that you literally cannot benefit fully from knowledge-gathering if your goal is to benefit from knowledge-gathering.

The most valuable bits of knowledge, in terms of productive work, are the ones that nobody could have reasonably expected to apply, because anything that can be expected to apply will be learned by everybody and become common sense. We have an intellectual monoculture in tech driven by capitalism.

If you learn whatever seems marginally interesting, each individual thing is unlikely to be relevant but the number of things means that the likelihood that one of them will be a real breakthrough is pretty high.

This is sort of a tangential response to the recent Signal v. Noise post called “Conceptual compression means developers no longer need to know SQL”.

My take is basically: it’s great to not need to know SQL, pointers, garbage collection, and whatever else, but if someone doesn’t want to know those things in the abstract, it’s a problem. The benefits of knowing those things goes far beyond the circumstances wherein knowing them is strictly necessary.

To say “C means developers no longer need to learn assembly” rather than “C means developers no longer need to write assembly” is upsetting — it considers knowledge by itself burdensome, rather than the work it takes to apply that knowledge and the awkwardness involved in that application. From my perspective, learning is only something to minimize when your time is someone else’s, and even then, only when they don’t value learning appropriately.

This is the main reason I don’t trust developers who don’t write code in their free time — it’s a good proxy for determining whether or not somebody is curious enough to have learned anything beyond what was required when they first started, because it’s very easy in this industry to stop learning long enough to become completely useless without HR noticing. However, it applies to any kind of knowledge work: someone who isn’t intellectually curious can easily stagnate, as whatever knowledge they do have becomes devalued either through sheer redundancy or through a sudden and complete loss of relevance. If you stand upon blocks of knowledge then it’s easy for your tower to fall as small bits of it lose relevance, but if you float along a stipple of knowledge cross the whole conceptual landscape, any piece losing relevance is pretty likely to be accompanied with another piece gaining relevance.

This is not strictly a defense of intrinsic motivation (or a defense strictly of intrinsic motivation, or a defense of strictly intrinsic motivation). Feeling effective or accomplished is another form of intrinsic motivation. I think the key is curiosity and generalism.

I also think that an important component of generalism is not having a strong emotional connection to your sense of identity. It makes us dismiss interesting things on the (often absolutely accurate) grounds that they’re terrible — which is dumb, because terrible things are great. Terrible things teach us lessons in entertaining ways, and when they do something right, it’s surprising enough to be memorable. In a mostly-open possibility space, knowing one thing you shouldn’t do is more valuable than knowing one thing it’s desirable to do — we have made many novel varieties of turing tarpits but we haven’t even gotten close to figuring out the ideal programming language, for instance, so knowing not to make PHP is valuable.

This is also not a defense of poor documentation strategies or purity tests. Making things easier to learn helps everybody. Specifically, optimizing documentation for usage makes it easier for people to learn by doing, whether or not they want to continue learning beyond that material. A lot of projects (think java-style object orientation), systems (think Mac OS), and documentation sets (think the ElasticSearch documentation or any other form of purely-tutorial-oriented documentation) are structured so that certain ideas are easy to learn and everything else is hidden, because the distinction between “doesn’t need to” and “can’t” is lost — so we end up with systems where use cases not considered by the original authors are on the spectrum from hacky to literally impossible, because a particular attitude has been imposed upon all users.

I would also be wary of the idea that incidental complexity is inherently burdensome while only essential complexity is interesting. Knowing about incidental complexity makes things that are made difficult by incidental complexity easy, and makes things that are made very difficult by incidental complexity possible.

We live in a world where the apparent complexity of almost all tasks is absolutely dominated by incidental complexity, and removing that incidental complexity requires understanding it.

I find many forms of incidental complexity fascinating, and understanding it is just as useful.

There’s also the problem that what constitutes essential vs incidental complexity is a matter of the problem statement & domain.

If I’m told to do something in three hours, then the essential and incidental elements are inverted from what they would be if I was given three months for the problem, because the time constraint means that a naive solution based on a shallow understanding is the only possibility.

If I’m told to do it with a particular tool, I can’t ignore the tool’s sharp edges just because another tool could make the problem easy to solve. What is easy in TCL is difficult in Prolog and vice versa, and these constraints are not platonically incidental or essential — all that matters is whether or not the choice to do something with the appropriate tools is open. Today, in software development, tool choice is rarely fully open and the selection of acceptable tooling rarely includes anything appropriate for solving unusual problems. This largely comes down to the expectation that all tools should already be familiar to all developers — in other words, being able to treat employees as disposable is more valuable than having efficient solutions and giving employees a little time to understand them. So, understanding incidental complexity is extremely important, and the more of it you learn, the easier a time you’ll have.

Every piece of essential complexity is surrounded by a thick shell of incidental complexity. That shell is nigh impenetrable without a grasp of whatever tools. However, as a generalist, you will probably find that the shell is hollow: essential complexity comes in a handful of forms and while the names change the same basic structures play different roles across different domains. Things that seem distinct only do so because of their context — nobody had noticed that two fundamental ideas are really exactly the same thing because nobody has gotten to the same level of understanding in those two specific fields. At this point, the generalist looks like a genius — but of course, all that has happened is that he has saved time and effort by being sufficiently eclectic. His aimless wandering has prepared him to encounter familiar ideas in alien costumes, and his impressive ability to identify new ways of manipulating some piece of essential complexity is really just rudimentary memory of how the same structure was manipulated in a seemingly unrelated field.

The value of this kind of cross-disciplinary connection is in its unpredictability, and so the only way to guarantee it is to avoid having exactly the same set of interests as anybody else. The easiest way to go about doing that is to pursue whatever sticks out to you about whatever domain you find yourself in, without concern for relevance, appropriateness, or order.