Friday, April 14, 2006

Casual jeans Friday! Casual jeans Friday!
Yes. I'm at work.
As I understand it, in some cultures today is a holiday. I celebrated by wearing jeans every freaking day all week!
Yesterday was amusing. I hope today it resolves itself.
Our story begins last month with a flashy red light and a panicked call from a customer.
Customer: "I've lost the data drive! I've lost the data drive!"
We have a "server" (one of many, actually) that is nothing but a workstation, perhaps beefed up a bit, but just a workstation. Some of these machines do some pretty amazing things for devices with no hot-swappable parts. Some of these machines do these things without my help. This "server" was one of these unmonitored machines.
I logged in and verified that there was no drive available but the system drive and asked the user if he had a back up.

It was like I punched him in the face.

Customer: "Of course I don't! Isn't that what we have you people for?!?!?"
Me: "We do that on most of your servers, but you opted out on this one. I've never even logged in."

He fumed, but asked what we could do.
Honestly, not much. I could hear the arm banging so I could tell it was broken hardware. I asked if we could send it back to the manufacturer.

Customer: "What about the data?"
Me: "They will assume that if it was important there is another copy somewhere."
Customer: "I'll call back."

It was decided that the valuable but sensitive network schematics would need to be salvaged. This requires specific hardware and clean rooms and electrostatic devices and concern, of which I have none. In the end the client company paid an outfit in Delaware $10,000 for a data restore which could have been avoided by tacking on $5 a month in back up fees.
Whew. Disaster averted. For a company with a completely managed network, those diagrams must have been pretty awesome.
So anyway, yesterday the rebuilt and restored drive was delivered. I drew the short straw and went out to install it and add it back to the "server".
This company in Delaware had removed the platters, copied the data bit by bit and replaced an exact copy on a fresh drive.
I installed the hardware, activated the drive as a partition and, at the customer's request, began a manual back up of the data "just in case".
Does everyone maybe see where this went horribly wrong?
I don't know if there were ever "sensitive network schematics" on the drive before the disaster, but everything that survived the restore process was about 120 gigs of MP3s and 100 gigs of pr0n.
Now. It isn't my place to do anything but back up this data once manually, set it to back up regularly from here on out, and get on with my life.
But I was a little offended. First, the guy has a personal server on his corporate network that is hosting pr0n and MP3s.
Second, he loses it and complains to my managers and (though it isn't my fault) everyone comes to bug me about the lost "vital data".
Third, he cons his boss into paying $10k to restore it under false pretenses AND into paying for the data to be regularly backed up.
In the normal course of my job, it would go something like this:

1. Establish the billable work to be done.
2. Do the work.
3. Create a ticket tracking the time so that the customer can be invoiced.

Sometimes I call or email to confirm when I'm finished.

This time I decided to work closely with the IT Director at the client site and with their accounts payable department.
I thoroughly documented the step-by-step process of restoring the data. I included detailed screenshots of the restored data, in full, for reference. You know. In case the data needs to be restored later.

Hopefully everyone will look closely enough to read the file names or at least notice that they are media files.
Every time I think I'm going to run out of stuff to blog about somebody steps up and does something stupid. In thanks, today's entry is dedicated to (name deleted to comply with non-disclosure agreement) at (company name deleted to comply with non-disclosure agreement). You, sir, are an asshat.
Anyway. Plans for today include running through the empty halls (sales staff has the day off) and later on organizing office chair races on the data center floor.

3 comments:

Pamela Moore said...

Office chair races are the best.

I hope Mr. Asshat gets his in the end. That's just not cool to fill up your work computer with porn, then have the company pay to recover it.

He should have more class and use someone else's work computer for porn like the rest of us.

Anonymous said...

4/16/06 - As I said at lunch today, Garrick, you (and your wife) are hysterical!!!

I hope it was the good pr0n and not the crappy amettuer stuff that you had to retrieve.

My work has a firewall, but, ironically, you can still surf for pr0n!! I figure that's done on purpose so that Jimmy, the freeky (not in a good way) IT guy, can surf his pr0n sites all day at his leisure. He is, after all, in charge of the firewall...

Anyway, lovely visiting with you and the MRS. at Fazolis. Have a great week!!! - Frida

Garrick said...

I didn't really have time to measure the quality, but I have heard firewall guys get the best pr0n.
As an update, the user from the story has been removed from the access list (it happened early Friday) and either is unemployed or at least no longer allowed to call in with problems.
Either way, I'm fine with it.

SoccerGrrl1982