Tag Archives: programming

Low Fidelity Abstraction

It’s only through abstraction that we’re able to build the complex software systems that we do today. Hiding unimportant details from developers lets us work more efficiently and most importantly it allows us to devote more of our brain to the higher-level problems we’re trying to solve for our users.

As an obvious example, if you’re implementing a simple arithmetic function in assembly language you have to expend a lot of your brain power to track which registers are used for what, how the CPU will schedule the instructions and so on, while in a high level language you just worry about if you’ve picked the right algorithm and let the compiler worry about communicating it correctly and efficiently to the processor.


More abstraction isn’t necessarily good though. If your abstractions hide important details then the cognitive burden on developers is increased (as they keep track of important information not expressed in the API) or their software will be worse (if they ignore those details). This can take many forms, but generally it makes things that can be expensive feel free by overloading programming language constructs. Here are some examples…

Getters and Setters

Getters and setters can implicitly introduce unexpected, important side effects. Code that looks like:

foo.bar = x;
y = foo.baz;

is familiar to programmers. We think we know what it means and it looks cheap. It looks like we’re writing a value to a memory location in the foo structure and reading out of another. In a language that supports getters and setters that may be what’s happening, or much more may be happening under the hood. Some common unexpected things that happen with getters and setters are:

  • unexpected mutation – getting or setting a field changes some other aspect of an object. For example, does setting person.fullName update the person.firstName and person.lastName fields?
  • lack of idempotency – reading the same field of an object repeatedly shouldn’t change its value, but with a getter it can. It’s even often convenient to have a nextId getter than returns an incrementing id or something.
  • lack of symmetry – if you write to a field does the same value come out when you immediately read from it? Some getters or setters clean up data – useful, but unexpected.
  • slow performance – setting a field on a struct is just about the cheapest thing you can do in high level code. Calling a getter or setter can do just about anything. Expensive field validation for setters, expensive computation for getters, and even worse reading from or writing to a database are unexpected yet common.

Getters and setters are really useful to API designers. They allow us to present a simple interface to our consumers but they introduce the risk of hiding details that will impact them or their users.

Automatic Memory Management

Automatic memory management is one of the great step forwards for programmer productivity. Managing memory with malloc and free is difficult to get right, often inefficient (because we err on the side of simple predictability) and the source of many bugs. Automatic memory management introduces its own issues.

It’s not so much that garbage collection is slow, but it makes allocation look free. The more allocation that occurs the more memory is used and the more garbage needs to be collected. The performance price of additional allocations aren’t paid by the code that’s doing the allocations but by the whole application.

APIs’ memory behavior is hidden from callers making it unclear what their cost will be. Worse, in weirder automatic memory management systems like Automatic Reference Counting in modern versions of Objective-C, it’s not clear if APIs will retain objects passed to them or returned from them – often even to the implementers of the API (for example).


It’s appealing to hide inter-process communication and remote procedure calls behind interfaces that look like local method calls. Local method calls are cheap, reliable, don’t leak user data, don’t have semantics that can change over time, and so on. Both IPC and RPC have those issues and can have them to radically different extents.  When we make calling remote services feel the same as calling local methods we remove the chore of using a library but force developers to carry the burden of the subtle but significant differences in our meagre brains.


But I like abstraction. It can be incredibly valuable to hide unimportant details but incredibly costly to hide important ones. In practice, instead of being aware of the costs hidden by an abstraction and taking them into account, developers will ignore them. We’re a lazy people, and easily distracted.

When designing an API step back and think about how developers will use your API. Make cheap things easy but expensive things obviously expensive.

Brief first impressions of Rust

I had a little play with Rust this week. I’d been meaning to for a long time, but the arrival of 1.0 motivated me to spend a few hours playing around with the tools and the tutorial. I haven’t actually written anything myself yet – I’m sure I’ll have some different, perhaps more valid thoughts after I do that. I’ll probably write a web server – that’s been my go-to hello world program for the past 20 years or so.

But just working through the first couple of chapters of the excellent free, online Rust Book I’ve come away with some impressions. For better or worse I’m comparing Rust to Go and to a lesser extent to C and C++.

Something that Go promised was good tooling and strong opinions on source code organization. I like to tooling just fine, but the source code organization bugs me – I don’t like having all my code and the code I depend on under a single directory. Rust’s source code organization is much more appealing to me – a top level directory with a configuration file Cargo.toml and then a src/ directory with my code. Dependencies are downloaded and stashed somewhere (who cares? I don’t).

Even playing around with the simple examples in the book I was missing “go fmt” and “goimports”. I haven’t gone deep enough into Rust to know if there’s an equivalent that everyone else is using or if the community has the (incorrect) opinion that source code formatting is a lifestyle choice. One of the most charming things about Go and Python are standardized source code formatting. In my team at work we’re using clang-format to (mostly) enforce a consistent style to Objective-C.

For the actual code itself there’s a steep learning curve that I’m still at the bottom of, staring up. I know that the whole point of Rust is a type system that is powerful enough to allow programmers to express complexities of thread safety and resource ownership, but that means that it’s really complicated. I like that it’s explicit and designed to be easily parsed. I like that there’s type inference so I don’t have to type the confusing names of these types all the time. I still have a lot of learning to do.

After my brief time with Rust I’m excited that we have a language that is expressive enough to provide memory and concurrency safety while remaining explicit and predictable. I just hope that I and others are able to become comfortable with the syntactic line noise to use it effectively, and that we find the right problems so we can use it appropriately.


Dan McKinley wrote a great blog post about choosing boring technology. It’s worth reading, but the tl;dr is that by adopting new and exciting technology you introduce additional risk and instability, which limits your ability to innovate on your product. I agree with pretty much everything he said.

There’s another dimension to the boredom question though. I think it’s both a cause and effect of what Dan’s talking about. I call it Ian’s Rule Of Bored Programmers:

A smart programmer tasked with solving a simple problem will make the problem more complicated until it becomes interesting.

I’ve observed this time and time again both in companies and in the open source world. To me the classic open source example is the GNU C Library. The team developing the glibc library, led for many years by Ulrich Drepper are intelligent and experienced, but they working on a library that for the most part is wrappers around syscalls, plus printf and malloc implementations. Almost every user of their library is using the Linux kernel. Instead of a simple, stable library we’ve ended up with a complex, always changing source of innovation and bugs.

At companies that only hire experienced programmers I’ve watched experienced developers faced with a simple but important problem add complexity to the solution to keep their attention. In some cases it takes the form of adopting or developing new languages, databases or other technology (like Dan writes about). In other cases developers chose to solve an abstract problem class rather than the specific concrete problem at hand, which ironically often ends being more brittle and inflexible than a solution specific to the problem would have been.

There doesn’t seem to be an obvious solution to this. Sometime simple problems are important enough that we want someone with experience to tackle it. For myself I just try to check in with myself to make sure I’m not adding complexity where it doesn’t belong.