There are days when I really despise my job… I’m currently running on three straight nights of five hour nights of sleep. To say that my writing quality today is going to suffer as a result, to say nothing of everything else, is a gross understatement. You’ve been warned.
For those of you new here, I’m an IT consultant when I’m not “adding value” to company time by blogging; the company I work for handles IT needs for smaller companies that can’t afford a full-time IT staff, or, occasionally, for companies that need a decently trained extra hand around the server room. The job is fairly basic - keep servers and workstations running, install the occasional bit of software, clean off the occasional virus… it’s pretty vanilla and peaceful (if boring) most of the time. Once in a while, though, I really earn my pay, and this week has been one of those weeks. One of my coworkers was expanding a partition on a server when the server decided it would not only drop the partition but take the partitions on the backup drives with it. End result? One down server and no backup.
Great.
To add insult to injury, this particular server was special. It was running Windows Small Business Server, which meant that, not only was it important as a server, it also was responsible for making sure people could log into the system (yep - only domain controller) and also making sure people got their e-mail (Exchange). It also turned out that the server also hosted everybody’s home folders, but we didn’t know that at the time.
Now, a Windows server, by itself, is an interesting thing to restore. At its most basic, a Windows file server is functionally no harder to restore than any other server - install an operating system, grab a backup, restore some files. You may need to add it to the domain so people can access it, and you might need to tweak some file permissions if they didn’t make it across, but none of that is particularly hard or time consuming. A Small Business server, however… well, that’s a special beast, and, at least in my experience, when one of those goes south, you’re looking at up to 48 man hours (note - not two days; we’re talking six eight-hour shifts here) to get everything restored and all the kinks worked out.
What’s the difference, you ask? Of course you don’t, but I feel like ranting.
Conceptually, there are two basic ways to store information on any system. You can either store information on files, reading and writing to that file whenever a change needs to be made, or you can store information in a database. Now, a database consists of a set of files, but databases are special - they’re designed to be easy for a computer to remember. Allow me to illustrate through metaphor:
Let’s pretend for a second that you and I are working on a book. We only have one copy of the book, but both of us want to be able to work on it. We have a couple of methods at our disposal to keep things in sync:
We can hand-deliver changes to each other. This would be analogous to a file-based system - I would make the change I want to make to the book, you would look at the change, then we would get back to work. There are some drawbacks to this, though. First off, what happens if you and I want to make a change at the same time, and the change affects what the other person is doing? Whose changes take priority? Secondly, this can be rather slow - while I’m making a change in the book, you won’t be able to look at the book (seeing as I have it and all), nor will you be able to make any changes yourself. This causes a rather serious bottleneck.
We can make little notes, including the date and time we made the note, then come together periodically and make the changes together. This would be analogous to a database, and it speeds things up dramatically. We could just rely on the date to determine whose changes go first. Then, periodically, we could both get together, figure out which notes go in which order, make the changes we need to make, and go our separate ways.
Now, let’s say the book gets lost. How can we bring it back? Well, in the first model, it’s fairly simple - we could just agree to have a backup of the book that we decide to start working from. Once a day, we run the book through a copier machine or something similar. The worst that happens is we lose our changes for that day. In the second model, yes, we can still do this, but there’s a catch - what about our notes? Have we been backing up our notes as well? If not, we’re going to have a problem - we just lost all of the changes we were going to make to the book!
Let’s complicate things a little further and pretend that there’s someone else working on a different book, but they’re using our book as a source for material. They want to be able to cite our book by page, paragraph, or even sentence, so they want to keep track of what we’re doing while we’re doing it so they can keep their cites up to date. Going back to our models, there are two ways we can allow this:
Let that person take the book, make their cites, then return it to us. Again, this would be analogous to the file-based system, and it would be slow and cumbersome; we would have to wait for the book to come back before we could make any further changes. On the other hand, if that person loses it for whatever reason, we could just grab our backup, get back to work, and let the person know that we restored the book.
Alternatively…
Let that person keep a copy of the book. The copy will be updated each time we make changes in it. This would be analogous to the database model, and, as I’m sure you can imagine, it would be much faster - we could make changes while they’re citing, let the person know we made some changes, they grab a new copy of the book, life is good. Just one problem, though - what happens if we lose our copy of the book? We could restore it from a previous version, but what if our version and the other person’s version don’t match? We would need to keep a record of what version we’re each working from and compare version numbers - if they don’t match, we’re going to need to come up with some way to rectify that. Worse yet, what if they make citations based on a newer version that we no longer have and they throw away the cites to the older version? What if the citations from the newer version reference something that didn’t even exist in the old version? What if we lost the notes that led to the newer version, perhaps due to our last backup being at least two versions back?
Herein lies the trade-off between databases and files: Databases are way faster than files, but can be rather tricky to bring back to life if something happens. Files are way slower than databases, but are much easier to bring back to life if something happens. Now, guess what Windows uses for everything? That’s right - databases. Guess what happens if you don’t have a good backup of some of those databases and they all tie in against each other? That’s right - I get no sleep, and neither do my coworkers. The worst part about it, of course, is that Microsoft likes to use databases in the most convoluted way imaginable so that way they’re nearly impossible to back up correctly. Pretend, for a sec, that you lost your database for Active Directory, which is responsible for all of your user names and passwords. Fortunately, you have a backup. Can you restore it all by itself somewhere else? Of course not - that would actually make sense. Instead, you get to restore it with the rest of something called “System State”, which just happens to include your entire registry, which just happens to include all kinds of various arcane configuration settings for your hardware and any program that’s running on your computer. Guess what happens if something gets screwed up in the registry? That’s right - you don’t get to restore Active Directory, which means you no longer have user accounts.
Nice, eh?
So, that’s been my week. Once this reign of terror ends, I’ll get some good posts up here. In the meantime, regale yourself with the latest on ID requirements to prevent voter fraud at Cardozaisms. It’s not news, it’s Cardoza. Alternatively, you can also get the latest in relating Professor Chaos and politics from Morbo’s good friend, Rachel Lucas.
Hello, Morbo. How’s the family?
Belligerent and filled with numerous pork treats.
Excellent. I’m Rachel, and I’m pro-war and pro-family.
Well, she’s not pro-family, exactly, at least not in the Quantity is King! sort of way, but you get the idea. Or you don’t. I don’t even care anymore. Burn in hell, hippies!