Thoughts on "Out of the Tar Pit"

December 31, 2014

Dave Lee recommended I read Out of the Tar Pit, a paper on software complexity. Here are some thoughts I jotted down while reading:

• Page 2: Dijkstra has got to be the most quotable computer scientist. On software complexity: "…we have to keep it crisp…". My new motto.
• Page 5: The critique of testing, that the "huge number of different possible inputs usually rules out the possibility of testing them all, hence the unavoidable concern with testing will always be–have you performed the right tests?", echoes my thoughts since encountering QuickCheck, Fox, and property-based testing. I have not, in recent years, been practicing strictly "test-driven" development–I sometimes write my tests after the implementation. Colleagues have claimed that disciplined test-driven development "eliminates assumptions one makes in writing tests", but I simply do not think that is true. To be clear, however, I do not think that property-based testing is a silver bullet either.
• Page 5: I do, on the other hand, think that unit tests play a large part in mitigating complexity. "Rely on testing at your peril," the authors warn. I would warn against relying too heavily on any one thing. Of course, the authors go on to say that testing is one valuable tool in an engineer's toolbelt.
• Page 5: "In real large systems however, we regularly encounter complexity whose status as 'inherent in the problem' is open to some doubt." I cringe whenever I hear the phrase "most complex app on the App Store". "Most (needlessly) complex", perhaps.
• Page 6: "The severity of the impact of state on testing noted by Brooks is hard to over-emphasise." Have you ever written a unit test for a method that, despite taking no arguments, has tremendous cyclomatic complexity? Then you've experienced this problem first-hand.
• Page 6-7: "The common approach to testing a stateful system (either at the component or system levels) is to start it up such that it is in some kind of 'clean' or 'initial' (albeit mostly hidden) state, perform the desired tests using the test inputs and then rely upon the (often in the case of bugs ill-founded) assumption that the system would perform the same way–regardless of its hidden internal state–every time the test is run with those inputs." No way. You can test stateful systems thoroughly, you just have to write test cases for each potential state. The problem is that, in complex, stateful systems, it is impossible to enumerate all of the different possible states. I suspect the authors are leading up to the big sell: make your functions stateless, or take state as an explicit (rather than implicit) parameter.
• Page 8: State contamination is an interesting concept. Learning Haskell, which makes such contamination extremely explicit, is a fun way to learn more about it.
• Page 8: Ordering is another fun concept that "purely" functional languages like Haskell made me more aware of.
• Page 10: Code volume is poison. Remember that the next time someone tries to convince you to add yet another layer of indirection to your system.
• Page 11: "Complexity breeds complexity." Preach!
• Page 15: Haskell is capable of interacting with the filesystem, which destroys referential transparency. Until now, I've thought that, by isolating and minimizing the parts of the program that do this, I could preserve simplicity in a program. But as the authors state on page 8, if "the procedure in question (which is itself stateless) makes use of any other procedure which is stateful–even indirectly–then all bets are off, our procedure becomes contaminated and we can only understand it in the context of state. If we try to do anything else we will again run the risk of all the classic state-derived problems discussed above. As has been said, the problem with state is that 'when you let the nose of the camel into the tent, the rest of him tends to follow'." How do the authors propose I write programs that, oh, I don't know, actually do something?

I didn't have enough time to wrap my head around the functional relational programming discussed in the latter half of the paper, unfortunately.