Skip to content
Nov 15 2005 / Aaron

Simplicity is not so Simple

So o’er this weekend I got frustrated with the cron-o-meter development and ripped it apart and threw away the SQL database backend. It really appears, on first blush, to be they type of application born to be a classical database app. However, I couldn’t shake the feeling that using a database wasn’t doing anything other than ‘feeling like the proper way to do it’ and beyond warm software engineering fuzzies, was getting in the way more than anything. So I went back to custom file I/O.

The USDA food database in our old SQL DB took up over 30mb, and took several seconds to initialize and load. I replaced it with with 6500 XML files in a ZIP file, plus a simple index file for searching. This custom ‘database’ loads the index in a split-second, and weighs in at only 4.5mb. And it’s all implemented in about two pages of very easy to read Java code. It took weeks to design and implement the SQL database system and write all the support code to integrate it into the program. It took about 2 hours to write the simple custom solution.

In my Comp Sci. degree, the discussions of Software Engineering is always to design things as abstract and generalized as possible. It is stressed over and over again. We never covered the less obvious evils of over-abstracting and over-generalizing. You end up with a lot of complexity in a general system that handles problems you don’t need a solution for. You often end up having to conform your problems into the proper shape to fit the general solver, like round pegs in square holes. For instance, using a relational database metaphor in the program forced a lot of database specific issues and design considerations. With the SQL database gone, things can be written more naturally.

A general solution can’t make domain specific assumptions or optimizations. A specific solution will always beat the pants of off the general one. Darse has a lot of good anecdotes about this in AI. Perhaps some day he’ll regail us with tales of tic-tac-toe, or Sokoban….

As a lazy coder, it’s always so tempting to write generalized solutions to problems. If you can write something once and then you might be able to use it over and over again for many different problems. However, we get into that habit such that we try and generalize everything, often without thinking if it’s truly needed. If writing 3 specific solutions takes 2 hours each, then why spend 10 hours writing the general solution if in all likelyhood, you’ll only ever need to solve 3 or 4 instances of the problem? Well for one, it’s usually a lot more satisfying to write the general solution. It takes creativity writing highly specific solvers can be very creative too, but the redundancies of writing the same type of things over and over can become dull.

I really enjoyed trying to engineer the crap out of cronometer (and much credit is due to Chris, who is far more adept at proper “Software Engineering” than I). But this weekend I reached a turning point where I just really wanted to finish and use the tool, not just tinker with cool libraries and architectures. Now, it may be me that has the problem in feeling hindered by these constraints. But at the end of the day, the productivity of the coder is really all that matters. It doesn’t matter how ‘proper’ something is if it doesn’t actually boost productivity in practice.

I guess I’m starting to more clearly see this issue now in all coding I do, especially at work. Every day I’m making tough decisions on how to approach a problem, and often the question is to take the aesthetic route or the pragmatic route. Today, while looking over our Poker Server code, I really wanted to refactor it, wrap it in pink ribbons and make it pretty. But after discussing it with Scott, I realized that doing so was unlikely to actually help anything, other than make it look more textbook, and would take more development time to produce than it would ever save down the road.

The big realization is that Generality does not equate to Simplicity. Specific solutions can be far simpler to create, understand, maintain, and debug. If something is easier to work with and understand, then it has lowered the complexity of the project. The Simpler something is, the less chances there are that something can go wrong, or that it will be misused. Achieving simplicity in a complicated domain is far harder that achieving complexity. It is a balancing act of making intelligent choices between general and specific solutions on a case-by-base basis.

Amen.

3 Comments

leave a comment
  1. darse / Nov 16 2005

    "Perhaps some day he’ll regail us with tales of tic-tac-toe, or Sokoban…."

    What, and risk getting clickered? Never!

    Your observations remind me of the joke about the engineer’s 4-bit toaster, versus the computer scientist’s generalized breakfast preparation system (running on a parallel shared memory cluster with multi-threaded re-entrant code and Linux operating system…).
    It seems the pupil has become the master…

    ~d

    PS Maximum capability of a generalized SAT solver applied to Sokoban puzzles: two stones, two goal states, two rooms, in a 10×10 space. Time: several hours. Yay.

    Compare that to Andreas’ domain specific solver, finding *optimal* solutions to puzzles with dozens of stones, goals, and rooms; tricky deadlock situations, and having minimum path lengths of over 800 pushes…

    One of these approaches is a little better than the other.

  2. yer papa / Nov 16 2005

    Complexity and simplicity are strange detractors.

    For instance, an entire volume of Bertrand Russel’s & Alfred North Whitehead’s "Principia Mathematica" used pages of symbols to prove that 1+1=2. Complexity and simplicity are strange detractors.

    Just out of curiosity, Aaron, have you ever heard of a mathematician named, Gregory Chaitin?

  3. Chris Hubick / Nov 23 2005

    I refuse to listen to this blasphemy!

    Someday you are going to program yourself right out of a job!

    Boss: "You mean it’s *all* done? already? Well then…"

    😛

Leave a Comment