My buddy Connell runs an election/campaign advisory firm. He has a database that he uses to track voter profiles, create outreach lists, and the like. To minimize annoying voters — never a good idea when you’re running a campaign — he needs to stay on top of whoever has already voted, so he can take them off the outreach lists (who has voted is a matter of public record; how you vote is not).
Unfortunately, his database/computer guy isn’t available, and Connell wasn’t sure how to import the voting data. Worse yet, he couldn’t find the manual for the software. Between the two of us we came up with a number of different ways to try and work around these problems…but none of them succeeded. So, as always, when subtlety doesn’t work, it was time to turn to brute force :).
The database uses dBase files. These are an ancient, legacy format — so old that it’s almost not even a legacy anymore — first developed by a company called Ashton-Tate back in the early 1980s. I briefly worked for Ashton-Tate (the name was a combination of the founder’s last name, and the name of his parrot) in the late 1980s. In fact, I joined the company the week after they released a long-delayed, and incredibly buggy, version of dBase which ultimately destroyed their franchise. Proving that timing is everything.
The dBase format is so old that few commercial programs can handle it anymore. The few I tried either couldn’t handle the files, or created updated files that Connell’s program couldn’t hack.
So, in the end, I wrote a C# application which read in the existing file, modified the data for those people who had voted, and wrote the results to a new file. Which Connell’s program happily accepted.
Along the way I had to learn the dBase file format. Great Ghu, is it primitive! Even for the 1980s! A text field with a maximum length of 200 characters is always 200 bytes long, regardless of how long the text actually is. That kind of silliness adds up quickly when you have tens of thousands of records…and was particularly problematic back when disk storage was measured in terms of floppy disks.
At one point a version of the program I wrote read in the entire 30 megabyte file before modifying it and writing the revised file to disk. For various reasons, that resulted in the program using up to 500 megabytes of RAM. Not too significant by today’s standards. But the “top of the line” PC I had at Ashton-Tate had, maybe, two megabytes of RAM. And here I was, thirty years later, building a dBase editor, in less than a day from a standing start, that used 250 times as much. My, how times have changed :).
By the way, I mentioned that dBase IV began Ashton-Tate’s implosion because it was so buggy. Here’s a true story I continue to shake my head over: one of the bugs caused your PC to go into a hard crash/locked state — so bad that you had to power cycle it to recover — whenever you pressed “control-break” while dBase IV was executing a long-running process. As ancient computer users will remember, that’s exactly the key combination you’d press, in any program, to interrupt long-running tasks. It was the “go to” action.
The company knew the bug was in the software before they put it out to market…but they released it anyway. Surely one of the dumbest decisions ever made by a software company.