Different Kind of Engineering

Have you ever heard the term “architecture” applied to the realm of software? Pretty sure you have. Our discipline, at least on the surface level, seems to borrow a lot of nomenclature from fields related to the construction of buildings. We talk about designs, frameworks, foundations, sometimes even scaffolds or facades.

And, of course, there is this whole convention of referring to programmers as “engineers”. Guess it simply sounds too good as a job title to let it go!

But frivolous as it may be, there is something deeply wrong about drawing analogies between erecting real structures and hacking virtual ones. It’s not even about attributing undue significance to the “mere” activity of sitting behind a screen and hammering at the keyboard; at this point, software development is probably at least as important as civil engineering – perhaps even more so.

The problem is that thinking of developing software as analogous to constructing buildings promotes flawed, unsustainable approaches. What is even more insidious is that they appear perfectly reasonable at first, mature even. It seems intuitively right, after all, to think of software project as something akin to construction project. In both cases you need to satisfy certain requirements, accommodate for interactions with external environment, and fit everything within time and/or budget constrains.

If it seems so right, then why it is so wrong? Well, because the similarity is only apparent, and thus largely superficial.

Design can only be arrived at by building

How common it is to begin construction of a building before its blueprint is finalized? Although few shovels might have tilled the ground here and there, construction engineers generally do not begin laying foundations until the structure’s design is ready. Before even the first bricks form a nascent wall, you can have pretty good idea of how the final thing will look like.

Can you say the same about software project? Certain methodologies (*cough* waterfall) claim to achieve almost the same degree of robustness and predictability, but most of us know how patently ridiculous they are. Not only is the final shape of system unknown when you start writing it, but it is practically unknowable. The design only reveals itself while you are already building, and this is something I’m sure would make the blood of traditional engineers run cold.

Laws of physics are of no concern

There are reasons for those unknowns, however. While real world engineering deals with real world limitations, bits and pixels are often completely unconstrained by those. More than by climate conditions, size of electromagnetic spectrum or the speed of light, software systems are limited by other, existing systems and processes: not necessarily software ones, but often unstable and changing.
As a result, dealing with them can get arbitrarily complex, not to mention that it transitively affects future systems interacting with us.

In comparison, the ground you build upon is quite literally rock solid.

Code is not concrete

Thankfully, there is one aspect of programming that makes all those obstacles somewhat surmountable. Unlike skeletons of skyscrapers, made of reinforced concrete, code is always modifiable. Regardless of how far you’ve progressed with current implementation, you can change and redesign it. Sure, there is typically a requirement to support outside clients through existing API but rarely, if ever, you are limited in how you’re gonna support them.

Frankly, this is something that I often see people struggling to come to terms with. Maybe because of those construction analogies I’ve mentioned, the structure of a program is often perceived as rigid and almost constant.
You are not allowed to tamper with it. You can only extend it in certain, predefined ways. And you better get it right the first time, as there is no going back. It’s almost as if the code was literally set in stone.

This, however, is a massive misconception. Code is meant to be poked and prodded; extended and removed; modified and reverted; and most importantly, hacked on and refactored. The notion of “technical debt” is a beautiful concept that captures all those mechanics. You don’t want to be in debt which is too large, because the interest rates (maintenance costs) will hinder you in the long run. But you also don’t want to have too small credit, or you won’t be able to make new investments (features) and expand your “business”.

Everybody Should Learn to Code!… Wait, What?

You may have seen this video, starring Mark Zuckerberg, Drew Houston (from Dropbox) and a couple of other tech stars. They promote the Code.org initiative, advocating that every school kid in the U.S. should have programming classes as part of their educational offer.

It’s a recent development, but the whole idea of promoting computer programming among broad audience is in the spotlight for quite some time now. Not only in the U.S., mind you. In Estonia, for example, children are to be offered coding classes starting from early elementary school.

The message seems to be pretty clear: everyone should learn to code, the earlier the better. In the future, it will be an obvious part of basic literacy, like reading and writing natural language. Everyone will get to do some really cool and useful things with their computers.

Sounds great, doesn’t it?…

So why I can’t help but to roll my eyes when I hear about this? Am I just an obnoxious, elitist jerk who doesn’t want others to experience the feeling of empowerment and satisfaction that creating your own computer programs can evoke every so often?… D’oh, I really hope not.

So maybe there are some actual problems with this idea. I would hesitate to say that it just doesn’t, ahem, compute, but I can certainly see a few issues, misconceptions and dubious assumptions.

Ego-Driven Technologies

Last week – while still on the other side of the pond – I attended a meet-up organized by the local Google Developers Group. The meeting included a presentation about Go, aimed mostly at newcomers, which covered the language from the ground up but at very fast pace. This spurred a lot of survey questions, as people evidently wanted to assess the language’s viability in general and fitness for particular domain of applications.
One of them was about web frameworks that are available to use in Go. Answer mentioned few simple, existing ones, but also how people coming from other languages are working to (re)build their favorite ones in Go. The point was, of course, that even though the language does not have its own Django or Rails just yet, it’s bound to happen quite soon.

And that’s when it dawned on me.

See, I wondered for a while now why people are eager to subject themselves to a huge productivity drop (among other hardships) when they switch from one technology, that they are proficient in, to a different but curiously similar one.
Mind you, I’m not talking about exploratory ventures intended to evaluate language X or framework Y by doing one or two non-trivial projects; heck, I do it very often (and you should too). No, I’m talking about all-out switching to a new shiny toy, especially when decided consciously and not through a gradual slanting, in a kind of “best tool for the job” fashion.

Whenever I looked for justification, usually I’d just find a straightforward litany of perks and benefits of $targetTechnology, often having a not insignificant intersection with analogous list for the old one. Add the other, necessary part of risk-benefit calculation – drawbacks – and it just doesn’t balance out. Not by a long shot.

So, I notice that I am confused. There must some be other factor in play, but I couldn’t come up with any candidates – until that day.
As I speculate now, there is actually a big incentive to jump ship whenever a new one appear. And it seems to be one of dirty secrets of the hacker community, because it directly questions the esteemed notion of meritocracy that we are so eager to flaunt.

California State of Mind

I can think of multiple ways to spend half a day – twelve hours – enjoyably and/or productively. Sitting in a slightly uncomfortable position inside a scarcely lit tin can soaring a few miles above the ground is unlikely to make it into top 10. But if that’s what is needed to get to the other side of the planet, so be it. Slightly torpid back muscles are hardly a set back, after all :)

That other side is San Francisco Bay Area, of course, where I’ve went to visit the Google mothership. Finding yourself so far away from home is uniquely surreal experience, I must say, but this place makes it a whole lot more unusual.

For a European like me, it’s nothing short of bizarre.

Imagine a fairly typical, middle-class suburban area, where buildings reach at most three or four floors. Some apartment complexes, detached houses, shops, restaurants, cinemas, maybe a mall and a park… Now imagine this goes on and on, for dozens of miles, all with the same low housing density and without an easy way to tell where one town ends and the other begins.
It sounds like a poorly thought-out result of uncontrolled urban sprawl in some mildly developed, emerging country. Except that there are four-line highways going across the whole area, regularly passed through by one of the fanciest, most luxurious and modern cars. A good number of them is powered by electric engines, by the way, and it’s not exactly problematic to find a parking spot where you could charge such a vehicle.

Not to mention that some of them actually drive by themselves…

But you don’t have to be on the lookout for Google’s Priuses in order to see the logos of IT companies. They are literally everywhere. A hip startup might still be in the garage, but they will have billboard next to the freeway. (I’ve seen one from box.net, for example).
Meanwhile, more established companies will have their buildings and campuses pretty much next to each other, often with big swaths of land left around for further expansion. And the most successful ones will make their hometowns’ names known worldwide, with Cupertino (Apple), Mountain View (Google) and Palo Alto (Facebook) standing out as prominent examples.

In all this high-tech and Web-crazed environment, I cannot help but wonder where’s one thing that you would surely expect to see here. Something that the IT crowd babbles about for quite some time now and doesn’t seem to stop anytime soon. That crucial stuff that makes the Internet go round.

Clouds. So far, I haven’t seen any.

Don’t Throw Old Books Away

When moving in to a new flat just few days ago, I had to find a place for all the various IT books I’ve accumulated over the years. It’s not exactly a copious amount, but it was just enough to fill the shelves of quite big bookcase. Shuffling through them, I was surprised to find some really old ones, straight from the long forgotten era of Windows 9x.

Like these.

Many of the technologies they describe are long outdated (the different techniques for rendering 3D effects age particularly fast, for example). None of them seem to be completely phased out – you need to wait few decades for that, not just one – but I doubt they get used very much outside of maintenance of legacy systems. And definitely no one gets excited about, say, Delphi or Visual Basic now. That boat simply doesn’t float anymore, partially because its ocean – desktop platforms in general and Windows in particular – is slowly leaking away for quite some time now.

Does it mean all the books treating about those nigh-ancient subjects are little more than paper waste now? I wouldn’t be so sure. The whole purpose of IT books is far from being an up-to-date reference of anything. Bits over wire travel much faster than letters on paper, after all. It doesn’t happen very often that a developer needs to consult a book regularly, especially during a coding session. Although some timeless classics can be an exception, online documentation or sites like StackOverflow have replaced books for most intents and purposes.

Most – but not all. Some topics are better tackled practically if you first have a bit of theoretical foundation which you then iteratively refine after gaining more experience. You will finally put the book away, of course, but then you can still use it later to quickly revise your knowledge if need be. It doesn’t matter that it was standing dormant for many months or years, as long as it proves useful when there is urgent need to flip through its pages again.

What if you cannot even conceive how and when that prehistorical literature can be of any use whatsoever?… Well, I still wouldn’t be so quick with getting rid of it all. See, IT as a whole has this surprising tendency of going in circles, and periodically regurgitating old ideas into new forms and shapes. Hence the ability to generalize is important for long-term success in this field – at least the kind of success that’s more palpable than 15 minutes of Hacker News fame.

But to see patterns and connections between seemingly unrelated pieces of technology, you need have at least vague mental trace of all of them. And to verify you are not just seeing things, it’s often necessary to go back and read up again.

More often than not, that new thing will turn out to be just an old new thing.

Open Source and the Prisoner’s Dilemma

Some time ago, on one of the forums I got into discussion about merits and motivations of releasing projects and code as open source. Turns out that many people cannot exactly wrap their heads around the concept of giving away your code for free. Even leaving the exact meaning of ‘free’ aside (it applies to both of them), I believe we can observe a kind of cultural gap here. Strangely enough, it’s not even the case of Nerds vs. Rest of the World: the geek community is in itself somewhat divided with respect to this issue.

And that’s OK, in a way. I know it may not be obvious how value can be preserved if we just volunteer our time and skills for open source projects and don’t receive any direct compensation in return. Honestly, I’m still kinda amazed how it all works out, but I have my small theory. It’s somewhat tangential to the typical gift culture explanation, but could also shed some light on companies’ motivations for contributing to OSS.

Long story short, I think the relation dynamics between open source contributors and beneficents can be described in term of the prisoner’s dilemma: a rather classic example from the game theory. Before pursuing the analogy further, let’s have a brief look at this curious puzzle.

Of prisoners and stealing

Like the name suggests, the prisoner’s dilemma can be formulated in terms of jail, prisoners, cooperation and defection. I prefer an alternate setting, though, as it seems to better illustrate the concept and may be easier to understand. You can check the original formulation in Wikipedia, among other sources.

There is small game-theoretical difference between those two scenarios, but it’s largely irrelevant to our discussion.

Consider a two-player game with a potential reward of $100. The money can be taken by one of the players or split evenly among them. There is also a possibility of both players getting nothing. It all depends on how the players themselves decide what to do with the money.

They make independent decisions by choosing one of two options. They can decide to either split the money evenly between both of them, or steal (figuratively) the whole sum for themselves.
What happens next is the result of both decisions, revealed and applied at once:

• If both players choose to split, the reward is indeed split evenly. Each player walks away with $50.
• If both players choose to steal, they walk away with nothing.
• If one players choses to steal while the other decides to split, stealer gets the whole $100. The other player walks away with nothing.

Sounds contrived?… It’s been actually tested in real life (as much as television can be called that), sometimes with dramatic results.

If you’re splitting and you know it…

What is the optimal strategy in a setting like this? If both opponents are known to be rational agents, they should arrive at the same conclusions. Because they cannot know each other’s thoughts, they both may only speculate what – probability that opponent steals – may be.

We know their decisions are independent, though, so shall remain the same regardless of what the player chooses in the end. Hence the expected values of both decisions seem to clearly indicate which one is better:

Looks like we should simply steal and be done with it.

But wait! Since both players are rational agents, they will both arrive at the same conclusion. Moreover, each will know the other thought the same. So they both know that is actually 1. Unfortunately, in this case

and they will both get nothing if they follow this logic.

Pity. Or maybe it’s better to split, then? If they both do just that, each will at least get $50 instead of nothing… Yes, this looks like a much better alternative, especially that we can count on both players to apply this reasoning; they are rational agents, after all. So by this logic, they will both split and everyone will get something in the end…

Well, except that now is 0, so it’s actually better to steal instead. Oh, and both players know that, obviously, and are not happy about it… again. Hence they will rather split, which makes stealing more attractive option, which in turns compels splitting – and so on.

Complexity for the rescue

This example of circular meta-reasoning is unresolvable in two-player case, because a single choice will make or break the system. Fortunately, reality is much more complex, with multiple agents making countless decisions all the time.

Decisions such as, for example, whether to open source this new project, or maybe contribute to some existing one.

Just like with the situation described above, looks like the optimal choice is to “steal”: to draw liberally from the vast expanse of existing open source software while contributing nothing in return. No single such behavior would cause the whole ecosystem to crumble, so the incentive for exploitation is very tangible. Heck, it’s not even obvious what’s the steal here, because what value loss is incurred by “splitters” is not easily recognizable.

Yet it’s obvious why every party cannot apply this strategy, lest the result will be very suboptimal for everyone. Somewhere between those two extremes (everyone “splits” vs everyone “steals”) there might be the point of equilibrium: where stealers can derive maximum utility without irrevocably harming the game’s dynamics. We don’t really know where that point lies, though, so we just choose to play along and split.

Windows 8 Is Coming and I Don’t Care

The upcoming release of Windows 8 is stirring up the tech world, mostly because of some controversial decisions Microsoft has made regarding the new version of their flagship OS.

Were it a few years ago, I would likely participate in various discussions about this, springing up on message boards I typically frequent. Probably I would have also tested the development preview which was made available several months before. Most likely, I would have clear opinion about viability of the new Metro UI, WinRT apps’ platform or the overall push in the general direction of HTML5.

I would… But I do not. Actually, I couldn’t care less about the whole thing.

It’s not only because I touch my Windows installation once in a blue moon, almost exclusively for gaming. No, that’s mostly because it’s 2012, and Windows still doesn’t support some crucial functionality you could expect from modern operating system.

Unfortunately, most users don’t expect it because they never knew any better. Therefore I will try to highlight some features from alternative operating systems (typically Linux or OSX) that I think Windows can be rightfully scorned for lacking.