Be a Better Programmer, Part 5

  • Jan 30, 2015

Inevitably, as a programmer, you're going to run into a situation where you're faced with a seemingly-impossibly-daunting task, whether it's because the task is large or because it's particularly arcane. While sometimes it's just going to be a long slog (some code bases are just monsters), there's usually some underlying conceit that brings everything together.

Figure Out How Things Are Simple

No matter how crazily complex or impossible-seeming a programming task can be, it's useful to remember that it was written by just some humans, and is meant to be understood. At the fundamental level, it's all just shifting bits around, but even above that everything has a fundamental logic that is usually surprisingly straightforward. There are two main types of things that tend to make me initially say "what the F is even going on here?": low-level, C-or-below-style processing and really huge, sprawling Java-style frameworks.

Low-Level Fiddling

Years ago, I found myself in a position where I had to reverse-engineer the Flash Media Server's temporary object-storage format. That's exactly the sort of thing that sounds freaking impossible at first, but actually isn't so bad when you get to it. The important thing to know going in was that the file was most likely going to be a simplest-possible implementation of pouring an in-memory data structure onto the filesystem. Since the Flash server didn't have any reason to encrypt or compress these files, nor were the data structures used particularly complex, there weren't likely to be any crazy tricks involved. Combine that with some logical assumptions about how the data might be stored - probably a record count and delimiters for records, map entries, etc. - and it's mostly a job of persistence, not arcana.

The C API stuff I've been doing lately is another example in this vein. A lot of C-related code looks like madness (and still does to me), but the thing with C is that everything is kind of pointing to a location in memory, which is treated like a huge array. So if you take a look at the struct definition in the linked post, what it really is is a map where, given a starting point in memory, you can expect to see certain numbers and sequences of bytes. Composite Data in particular has another, higher-level conceit: it is an array of records, each of which begins with indicators as to the type of the record and its length, providing the delightful property that you don't even have to know what a record is in order to safely shuttle it around.

Java Framework Skyscrapers

On the other end of the spectrum, you end up with huge, sprawling frameworks of code, for which Java is particularly prone. Take a look at the XPages stack: it's hundreds upon hundreds of classes and interfaces, just stacked in a huge mountain. But, when it comes down to it, it's still just a web server serving HTTP requests. So the server sits there, waiting for a request to come in (whether it's an initial request, a POST back for a partial refresh, or whatever), and routing those requests to the right place. It does the job of looking at the URL and request data and determining that, for example, "GET /foo.nsf/bar.xsp?baz=true" should create an environment with an appropriate "param" object, pick the XPages app module for the "/foo.nsf" database (loading it from scratch if need be), find the class named "xsp.Bar", and tell it to render itself to HTML.

And the same goes for the page structure. An XPage is a tree, and all of the various component classes, state holders, adapters, factories, renderers, and utilities are in service of that. So when you click a button that causes a partial refresh, at one level it's doing a whole bunch of crap about POSTing data and decoding values and whatnot, but the conceptual conceit is that you're sending a "click" message to a "Button" class (not exactly, but close enough). When you step back and look at it like that, the other parts may still be complicated, but you can really see the role they all play.

Conclusion

So the point is that daunting programming tasks aren't usually that hard. There are plenty of things in this vein that I haven't really tried - say, OpenGL game programming - that seem like "geez, how can anyone even do all of that", but which I am confident would make sense given an appropriate introduction. It helps to keep this in mind when you're approaching a new task or are asked to do something very new to you: it was all made by just some human.