Paying Down Technical Knowledge Debt

October 21, 2016

Programmers frequently encourage one another to "pay down technical debt": spend a little time cleaning up the mess your team made when they implemented a new feature. That way, it'll be easier to implement the next feature – you won't be bogged down by cruft left behind.

An example of technical knowledge debt in the wild.

Recently I've been accruing a lot of what I call "technical knowledge debt". Here are some examples from my open source work on the Swift compiler:

"On Linux, Swift object files are linked in between 'begin' and 'end' object files: /usr/bin/ld swift_begin.o SourceOne.o SourceTwo.o swift_end.o. These begin and end object files aren't used on Darwin. I don't know why this is. When modifying the Swift build system for Android, I sometimes see errors if I make a change that doesn't put these begin and end objects where they're supposed to be. But I'm focused on getting Swift on Android to work, so I'll look into what these begin and end objects are later."
"It seems like some people think it's a good idea to investigate alternatives to using the -Bsymbolic linker flag when linking the Swift runtime. I googled the flag, but a lot of the explanation goes over my head. I should look into it more some day."
"Besides apple/swift, there's also apple/swift-llvm and apple/swift-clang. They include some changes that aren't in LLVM and Clang trunk, which I assume are necessary for Swift to work. What's in the changes? What would need to change in LLVM in order for Swift to work? It'd be cool to find out someday." (UPDATE: I found out.)

I don't really need to know these things in order to contribute to various parts of Swift. In fact, to make progress, I have to accept the unknown. I don't really understand what -Bsymbolic does, but that doesn't prevent me from fixing bugs.

But at some point, too many unknowns cause me to slow down. Sometimes the Swift Android build stops working due to a seemingly unrelated change to another part of apple/swift. If I've taken too much "technical knowledge debt", I sometimes just try completely random things until the build starts working again. "I don't know why, but adding this linker flag seems to fix it!"

Work like that sucks. It's not efficient, nor is it fun.

Technical knowledge debt is like technical debt

If I accrue too much debt, my work slows to a crawl. I have to spend a significant amount of time paying it off.

On the other hand, if I pay it off as I go, individual tasks take longer, but I never come to a complete stop.

Just like technical debt, whether to accrue knowledge debt is a judgement call. "Do I have the time to learn more about -Bsymbolic right now? Or do I need to fix the build as soon as possible?" There's no one decision that will always be correct.

And like technical debt, it's possible to take too little. Spending a year refactoring a codebase is a terrible idea, if my team needed to ship a new feature in the next three weeks. Similarly, spending a week learning how debuggers are implemented is a bad idea, if I needed to fix a bug by the end of the day.

Technical knowledge debt for the beginner iOS developer

As a thought exercise, I imagined what my relationship with technical knowledge debt would look like if I were just learning how to write iOS apps. Of course, I would start with a new project template in Xcode:

/// AppDelegate.swift

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
  // ...
}

"What does @UIApplicationMain mean? I don't really know, but anyway if I hit ⌘R in Xcode my app runs, so whatever."

"I know that UIKit needs my AppDelegate to conform to UIApplicationDelegate in order for my app to do something when it launches. But why does it conform to UIResponder? Oh well, whatever, let's change the background color of the root view controller."

/// ViewController.swift

class ViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()
        view.backgroundColor = UIColor.blue
    }
}

"I know that this view controller is the root view controller, because Main.storyboard says it is. But I don't see any references to Main.storyboard in the code. How does UIKit know to use that storyboard?"

"I know that I could also change the background color in Main.storyboard, instead of doing so in the Swift code. Is there a reason I should do one instead of the other?"

…and so on.

If a beginning iOS developer tried to answer all of their questions along the way, never taking any knowledge debt, they would spend years before they wrote their first line of code. There are so many technologies and abstractions behind an iOS app, it's not realistic to be "debt-free."

Of course, to develop great apps, a beginner iOS developer needs to pay off that debt. They should learn about @UIApplicationMain, in case they wanted to execute code before UIApplicationMain() was called.

Managing technical knowledge debt

I admire Julia Evans, and I think her closing keynote at RustConf shaped my thoughts on technical knowledge debt. Specifically, at 8'37", Julia asks herself "So, what IS concurrency?"

I think it's important that I ask myself questions like these sometimes. Of course, I can't just sit at my desk all day, asking myself "what is a debugger, anyway?" But if I don't ask myself that at some point, I certainly won't be able to help if someone asks me "how can I attach a debugger to my Swift programs running on my Android device?"