System iNetwork

July 1, 2008

Have We Learned Nothing, People?

Recently, I came across a couple of very old articles regarding disaster recovery (DR) plans and their subsequent failings.

Here's just a snippet from an InformationWeek article that ran way back in 2001:

"I once decided to wipe a server clean -- only after ensuring that the administrator had a clean backup. Being thorough, I actually did a test restore of a file that was backed up the night before. So, I was good to go. I reformatted the server and after reconfiguring everything, restored the data -- no problems!!!

But -- the user account that was driving the backup software was not an administrative account (translation: it couldn't "see" everything). So, the backups (for six months worth of stuff) didn't have any data from any user other than the LAN admin's. Everything was gone. . . ."

And here's another tale from an equally old article on SearchStorage.com:

"In the federal government setting, DR planning 'war stories' ranged from the ridiculous to the infuriating. According to a field service engineer at Legato Systems, one post-Sept. 11 implementation of a tape backup solution for a U.S. intelligence agency produced an interesting conundrum.

'Legato's backup software, NetWorker, was part of the solution provided to the agency by an integrator that included a tape library and several other components. Soon after the installation, we were contacted by the customer who was very upset that he couldn't retrieve any data from his backup. For some reason, the software company always gets blamed whenever the system breaks down and that was the case here,' said the engineer.

In order to troubleshoot the problem at the customer facility, one of Legato's field engineers needed to undergo extensive background checks pursuant to receiving a security clearance. 'When our guy was cleared, he arrived at the site and was escorted around at gunpoint as he tried to find the cause of the problem. It turned out to be pretty simple: The client had loaded their entire tape library subsystem with cleaning tapes. Apparently, when a box of cleaning tapes had been received, someone had put 'Top Secret' labels on each cartridge over the label that explicitly stated that the tapes were not to be used for recording data, but just for cleaning drive heads. For a while, the agency probably had the lowest media cost library in the history of tape...as well as the cleanest read write heads in the industry.'"

So, here we are, seven long years later, and I think a lot of folks out there are still at risk for these types of "Hey, you'll never believe how I lost my job" tales.

Luckily, there are a few solutions out there. Coming directly from our readers in response to a recent forum post asking, "Would someone please share methods being used to provide complete system backups for disaster recovery for an IBM AS/400 iSeries 810, preferably using a remote site?" Two suggested solutions for the user included: iTera Vault and MIMIX, both from Vision Solutions.

MIMIX runs on System i AIX LPARs and replicates the production environment to a backup server in realtime to eliminate data loss. Alternatively, iTERA Vault automatically transmits System i transaction and object changes that occur between tape saves to a protected “vault” on a Linux, Windows, UNIX or System i server.

Dragging your feet on finding a reliable DR solution? What are you waiting for, a disaster?

--Erin Bradford, systems management & availability editor

Posted by ebradford at July 1, 2008 12:54 PM