When the Photomonster barfed its BIOS settings

[GilesGuthrie]

So, my Core i7 system had been a faithful friend for two and a half years. One day I switched it on, left the room, then went back to it to the usual Windows 7 logon screen.

I logged on.

For some reason it didn't load the taskbar icons. Then it fronted up one of the default desktop backgrounds. I was confused.

Then I was worried.

No, terrified.

Windows-E. Explorer opened. In "My Computer", drive D was missing.

Drive D is a 4x 1TB RAID 5 array. 2.7TB usable space, 1.7TB used.

Oh, really, no, please, not this...

Reboot. POST. RAID array "Failed".

It turns out that my motherboard had simply "forgotten" all of its settings. The overclock was gone, the drive controller had defaulted back from RAID to IDE, it was a disaster. The reason the system had booted was because Windows is installed on an SSD. The reason I was onto the failure so quickly was that I had redirected the Users hierarchy to the array.

So, a quick bit of auditing...

• All user profiles: gone
• Music: gone
• 40,000 photograph image library: gone
• E-mail, calendar, contacts: gone
• Personal documents: gone
• Archive of downloaded programs and (legal) registration codes: gone

This was not good.

Not only this, but the Ubuntu Linux Samba server that I'd been using as an in-house backup server had puked on installing Ubuntu 11, and so was offline with formatted disks.

Fortunately, I'd been backing up data (lots of data) online to Mozy. The last I checked, 535GB or so. Photos, home videos, documents, that sort of thing. Music was backed up on a NAS in the garage (as well as on some shiny plastic in the loft).

This wasn't as bad as feared, was it?

I reconstituted an old Vista box I had lying around, and installed the Mozy Restore Manager on it. Logged in, and hit the "give me everything" box. Not enough disk space, but I moved all the data off a spare WD 2TB MyBook to my NAS, and pressed that into service.

I tried in vain to recover the RAID array. I used some tools, but none of them could be persuaded to read the config off the RAID member disks.

Format. Reinstall.

It took a month to finish the reinstall, and there's still stuff missing, such as Lightroom Export configs. But nothing I can't recreate. Some old archive data has gone for good, but that's probably not a bad thing. The real ballache was the amount of configuration data held on that failed array.

So, to recap: I'm not a noob dumping all my data on the crappiest external drive I can find. Key data is stored on a redundant drive array. I've got a backup in the house, and a backup in the cloud. My in-house backup data is spread between three separate devices. AND STILL, I was able to lose data. Because my RAID array failed at the time that my in-house backup server was offline, and because I was not backing up EVERYTHING to the cloud, I lost a month of productivity and stress.

The "failed" in-house backup server has now been disposed of in a WEEE-compliant manner and replaced with a Synology NAS box with 2x 3TB drives in RAID1. My computer duplicates itself onto that box nightly. My cloud backup is back in place, and I've moved to a cloud e-mail & documents provider (with local syncing and local backup).

I hope you can learn from my lessons here. Hopefully if you do happen to lose all your data from your primary computer, you'll have enough redundancy in your backups to recover it.

 

[bass264]

I always back up my files to my 2TB raid so I never lose all my files.

 

[Grayfox]

Raid 5?

Then you may be able to recover it.

 

[GilesGuthrie]

Raid 5?

Then you may be able to recover it.

Nope. I ran several tools, and none of them could pick up the RAID signature from the disks. That was the annoying thing: there was nothing wrong with the disks, it was amnesia on the controller, so quite why it wouldn't then read the config back I don't understand.

 

[Azuremen]

I always back up my files to my 2TB raid so I never lose all my files.

FYI, he had RAID. He lost the files. Might want to make a note here.

Thanks for making me paranoid, GG. Might start using that drop box I have a bit more now...

 

[Casio]

Above is an example of why RAID is not (as many people think) a back-up solution.

Though GG just had bad timing...

 

[bergauk]

Above is an example of why RAID is not (as many people think) a back-up solution.

Though GG just had bad timing...

Depends on the RAID type. Mirroring is good, though.

 

[Casio]

Depends on the RAID type.

How does it depend on the RAID type?

 

[bergauk]

...Really? http://en.wikipedia.org/wiki/RAID#Standard_levels

 

[Casio]

So how is RAID backup again?

EDIT: Now I'm at home I'll add more

I'm not just trying to be argumentative for the sake of it. But there is a lot of misinformation around with people thinking they have a 'backup' system because they run RAID1/5/10 whatever (But thanks for the Wiki cause I don't know what RAID is...).

RAID is not a backup system, it's a reliability system. Many people think running RAID1 is 'backup'. If you delete a file does it allow you to get it back? If your drive file system is corrupt can you recover it? No. It covers you against a physical loss of a disk(s), which is redundancy, not backup (Hence the name RAID).

GG had a great backup system. A NAS doing daily backups with RAID1/5 is a backup system. It does allow you do to things like recover files, retain previous versions, etc. The RAID Configuration increases the reliability of the NAS, but is not a backup system in itself.

 

[bergauk]

Is it not technically backing up files if you're running some form of mirroring?

 

[Casio]

Is it not technically backing up files if you're running some form of mirroring?

No. Again with RAID you're getting disk redundancy and increased uptime. The OP a perfect example of how RAID isn't a backup system. Your RAID controller dies and you lose all your data. If your data gets corrupted on one disk it's corrupted on all disks.

 

[Grayfox]

You could try to run a sector by sector imaging program and try to recover the files that way.

Or will it not work this way?

 

[bergauk]

No. Again with RAID you're getting disk redundancy and increased uptime. The OP a perfect example of how RAID isn't a backup system. Your RAID controller dies and you lose all your data. If your data gets corrupted on one disk it's corrupted on all disks.

So in your opinion, how should I back up data? Since apparently having a redundant backup on two or more disc's doesn't qualify as a backup.

 

[Ibonibo]

Did you try recuva or any other recover programs.

As for the backup : i use 3.5 Internal HDD with docking station. Have a 3 TB HDD and every week I do a backup on it of everything I have on the PC. And the important suff gets a second back up on a 1.5Tb HDD just to be sure.
So I have my data on 3 HDD (which are seperated on 2 PC's) which should be relatively safe

Also have around 40000 photos (a lot were scanned, which is a ****load of work), so I understand the feeling : NNOOOOO!!!

 

[GilesGuthrie]

So in your opinion, how should I back up data? Since apparently having a redundant backup on two or more disc's doesn't qualify as a backup.

Casio is right here. Essentially, RAID protects you from disk failure, which is only one cause of loss of data. After erroneous deletion by a user, disk failure is probably the biggest cause of data loss at home and at work.

Think of it this way:
You have a document you've been working on for ages. You accidentally hit Select All, and then Delete. In a panic, you decide to quit your word processor, and then, because you always save your changes, you save the empty document.

With your data stored on a RAID 1 mirror, the empty file is replicated to the second disk faster than your brain computes the error of its ways. This is not backup, because the automatic replication of the file has wiped your "copy".

To recover this, you need the data to have been copied to a separate disk system. That copy needs to be periodic, not instantaneous, because the time lag between the data creation (or deletion) and replication allows the data set to exist in two distinct configurations in segregated systems. And when one of your systems is not correct, the difference between the systems is what saves your data.

To those asking about RAID recovery
I tried a couple of products to try to recover the disks. In each case, the recovery product needs to get a handle on how the disks are striped, before it can start to read the data off. None of them were able to do this. I don't quite understand why, but all of the ones I tried were of the shareware "let me see if I can recover your data before you pay for me" type, and I didn't need to pay for any of them.

I imagine that a specialist data recovery company would have been able to reconstitute the data, but costs for that run to around £2.5K. My house insurance includes provision for data recovery, so I could have claimed back £2K of that, but I decided to go the data recovery route. I have to say, it's been very effective.

 

[bergauk]

RAID essentially DOES back up your data though. Mirroring is a form of backing up, and while yes, it is possible for the array to be destroyed and have both disc's sets of data be ruined. It isn't likely to happen often. Giles had the perfect storm of failure though.



It still doesn't make sense to me why everyone seems to think that what most people would consider to be backing up data isn't backing up data?

What should I "Joe Consumer" do to back up my data? Burn everything to DVD's? Back up to the cloud? Have a second hard drive? RAID mirroring? Write every 0 and 1 down by hand on masses of paper?

What is truly the BEST thing to do? Because I must be horribly wrong, coming from a workplace where RAID was used as a backup as well as telling all of our customers we built computers for that they should consider having a RAID array.

 

[ROAD_DOGG33J]

What is truly the BEST thing to do? Because I must be horribly wrong, coming from a workplace where RAID was used as a backup as well as telling all of our customers we built computers for that they should consider having a RAID array.

If your workplace was set on fire most of that data would be lost or a pain in the ass to recover. So they could have backed up to a different location.

The best thing to do would depend on how important your data is. If the most important things you have are some basic documents and some game saves then maybe some type of RAID is enough or nothing at all.

 

[GilesGuthrie]

From my personal and professional experience, the most common causes of data loss are (in order from most to least):

1. Unintended deletion/overwrite by user who owns the data
2. Unintended deletion/overwrite by other user
3. Unintended deletion/overwrite by system administrator
4. Single disk drive failure
5. RAID controller failure
6. Server room flooded
7. Server stolen
8. Multiple simultaneous disk drive failures
9. Server room fire

Of these, RAID can protect you from 4, and in certain specific conditions, 8. All other data loss scenarios require that the copy of the data be held outside of the disk system on which the "original" (lost) copy was stored. With a disk mirror, if you delete a file, it goes from the "primary", and then that change is replicated to the "mirror" disk almost instantaneously. Certainly we're talking sub-0.5s. The mirror here is not a backup, because it does not facilitate recovery of the data.

What should I "Joe Consumer" do to back up my data? Burn everything to DVD's? Back up to the cloud? Have a second hard drive? RAID mirroring? Write every 0 and 1 down by hand on masses of paper?

What is truly the BEST thing to do? Because I must be horribly wrong, coming from a workplace where RAID was used as a backup as well as telling all of our customers we built computers for that they should consider having a RAID array.

Two options for backing up data to insure against plausible failures:

1. Backup to the cloud, using reputable provider, e.g. Mozy
2. Removable hard disks, updated min.weekly, stored offsite, e.g. relative's house

Realistically, you want a "set-and-forget" approach, whereby a client on your computer silently backs up content for you. It could be to a network-attached-storage device elsewhere in your house (e.g. garage), or to a cloud provider. These frequently come with versioning, Windows shell extensions, and other features to improve the resilience of your data.

If you can get your data offsite, so much the better. It protects you from dramatic domestic events, including fire, flood and burglary. I host my NAS boxes in the garage because while it's not offsite, the garage is double-skinned from the house, and UK building regulations require (I think anyway) a firebreak segregation between garage and living quarters.

The other thing to remember is the value of your data. If you've just got a few documents to keep, you can sign up for SkyDrive or DropBox or something like that and host your documents there. If you've got half a terabyte of data, then the chances are that the value of the data is greater, and you should be taking commensurate actions to protect it.

 

[Azuremen]

It still doesn't make sense to me why everyone seems to think that what most people would consider to be backing up data isn't backing up data?

Off site back-up is really the only proper back-up, as it will protect you from fires and other data destruction as well.

What should I "Joe Consumer" do to back up my data? Burn everything to DVD's? Back up to the cloud? Have a second hard drive? RAID mirroring? Write every 0 and 1 down by hand on masses of paper?

Cloud data. Hell, you can get a decent amount of space for free even.

What is truly the BEST thing to do? Because I must be horribly wrong, coming from a workplace where RAID was used as a backup as well as telling all of our customers we built computers for that they should consider having a RAID array.

As you've been told, several times now, back-up and redundancy are not the same. And as you've been told now, off-site, seperately managed files are the way to. Removable drives or the cloud, as burnt optical media degrades over the course of a decade.

 

[Pupik]

It used to be that I copied a few CD/DVD-ROMs about every three months, or right before a hurricane. And for every new computer, I just use the 250GB portable drive to copy over the photos, documents, and the like...that's on two desktops. My work has been the best thing I've ever done for backing up stuff on a regular basis: I pretty much keep everything on the two work laptops and my travel notebook. After I get home, I copy over a bunch of files to the desktops.

Nothing fancy, but even all that redundancy seems like more of a hassle than transferring things via a 32GB flash drive every two days or so. If I want to share it with the world, it's on my site somewhere...And then I just wait for one of them fail. I don't tinker with the evil innards of these boxes anymore, I just use them as-is.

Curious though, how much "cloud" could I honestly get for free? I pay about 100/year for 50GB of domainname + webspace, which is plenty enough.

 

[Casio]

What is truly the BEST thing to do? Because I must be horribly wrong, coming from a workplace where RAID was used as a backup as well as telling all of our customers we built computers for that they should consider having a RAID array.

In the most basic set-up with 2 disks. Rather than using RAID1 like most people do, it's much better to keep them as separate, independent drives, and do a complete backup weekly or whenever you think. If you do daily work on it then maybe do a daily backup of your work folder and keep a few days worth of versions.

After that it depends how far you want to go and how much money you want to spend. With external drives so cheap it's not a bad idea to backup onto that as well. A NAS would be the next step up. Depending on what you need to backup physical media is also cheap and viable.

 

[NissanSkylineN1]

How about disk images?

 

[Casio]

How about disk images?

Disk images are ok, but have a greater risk for file coruption, if you have one small error in the entire disk image you most likely will not be able to get anything out of it. Also if you say want to just get back a single file, you still have to restore the whole image, which may be quite time consuming (That's if you have enough space for the extracted image).

Images have their place, say for snapshotting clean states (Like just after a configured Windows Install), and obviously in software deployment. In terms of backup they're fine to be worked into a backup system but I wouldn't use them purely as the backup system.

 

[NissanSkylineN1]

But still, isn't that risk/chance pretty low anyways?

 

[Casio]

But still, isn't that risk/chance pretty low anyways?

Well I guess it depends on our critical your data is.

Personally I have a disk image of my OS drive from just after I installed the OS, main software (Office, Web Browsers, Drivers, Etc), so if for whatever reason I need to reinstall the OS I can just image from that nice and easy. If that image gets corrupt it's not that big a deal to me, other than lost time.

On the other hand, I backup my personal files that I actually care about (Pictures, Personal, Work stuff) on my main PC to an external drive connected to that PC, and to a seperate server in my house weekly (Sunday at 11pm), I also keep the last 3 backups worth, so I have a month worth of weekly versions.

I may have a few thousand files, if one gets corrupted during the transfer then it sucks but the others are fine, hypothetically, if I did it as an image if that gets corupted I can't recover any of it.

 

[Small_Fryz]

It always comes down to a few things

Recovery Point Objective (RPO) When do you want to recover from? how many mins/hours/days can you stand to loose? and your Recovery Time Objective (RTO) how quickly do you need to recover the data and be up and running?

The key thing I tell my customers is, what use is a backup if your not able to restore it. If you really want to be safe you need to be backing up multiple versions and having them on different sources. Brads plan above is very solid for home use.

For snapshots to be fully effective you need to have all applications stopped otherwise you will snap a partially complete transaction which can lead to corruption upon restore. Unless your running fancy software with API's in place. Snapshots are generally used to snap the production environment and store it as temporary cache on a SAN while the proper backup software backs it up to the dedicated backup disc / tape pools, which allows the production servers to keep running without any over head of a backup process running.

As for the RAID argument being a backup? your mis-understanding high availability with backups. Not to mention if you delete a file or something gets corrupted its automatically transferred to the other disc in the RAID array. You need to have a separate version saved from yesterday or what ever to recover from. RAID only allows you to keep going if a disc fails, its not a proper form of backup.

Its a tough argument to have when only talking about your own home environment, its a totally different discussion when your talking about a 20million dollar business.

 

[dice1998]

After reading all of this, I shouldn't heavily depend on only my NAS and look at moving my files to the cloud as a secondary backup...I think as a student I should be able to get somewhat of a discount on paid loud storage (which I don't mind doing for peace of mind)

Although I might upgrade my NAS to 2TB for more space



Do you guys here have any NAS recommendations?

I have a Seagate Blackarmor NAS 110 right now and it's been totally reliable, but i'm not really feeling the GUI that it has.. It's not complicated.. I just don't like it for some reason..

Especially globalaccess..

I'm not looking into anything expensive, 2TB is more than i'd need for a while.

 

[Casio]

Do you guys here have any NAS recommendations?

I have a Seagate Blackarmor NAS 110 right now and it's been totally reliable, but i'm not really feeling the GUI that it has.. It's not complicated.. I just don't like it for some reason..

Especially globalaccess..

I'm not looking into anything expensive, 2TB is more than i'd need for a while.

A lot of my friends have QNAPs and swear by them. Look at the 212 or 412 for just standard use like backing up and storing things, if you want to do a lot of things like streaming then it may be wise to start looking at the higher CPU models. I believe Synology make similar gear/quality to QNAP, though I haven't had hands on experience with them.

For a slightly cheaper solutions, with more basic functionality, I've heard decent things about the Netgear ReadyNAS. Alternatively you can make your own NAS.

Options, options, options...

 

[GilesGuthrie]

After reading all of this, I shouldn't heavily depend on only my NAS and look at moving my files to the cloud as a secondary backup...I think as a student I should be able to get somewhat of a discount on paid loud storage (which I don't mind doing for peace of mind)

Although I might upgrade my NAS to 2TB for more space



Do you guys here have any NAS recommendations?

I have a Seagate Blackarmor NAS 110 right now and it's been totally reliable, but i'm not really feeling the GUI that it has.. It's not complicated.. I just don't like it for some reason..

Especially globalaccess..

I'm not looking into anything expensive, 2TB is more than i'd need for a while.

I have two Synology NASs. A 2-bay DS210j and a 4-bay DS411. Great bits of kit.

I also have friends running QNap kit very happily. My experience of failure rates on Buffalo kit is quite high.