Another Back-up Hardware Question

[iSign]

I have wanted to write out this question all day... almost started to at least twice, but knew it would take some time & have been putting it off.

I was reading the "why even bother" thread & saw a lot of the tech talent here on that thread & found myself spilling my guts about this huge area of mass confusion as well as of absolute importance to me and my business. (and to others I'm sure)

I decided to post the question/rant in it's own thread instead:

I have a question for you tech guys!

First some background:

I have two 500GB WD drives for backup.

I keep my current data on a server running Windows Server 2003. That server has 2 drives in it.

The first drive in my server is partitioned into C and D. C is my OS and Apps. and D is where all four of my workstations open data files from, and save data to.

The second drive (E) is where nightly backups of the D drive are written.

I consider the E drive as a backup.

My first WD MyBook is an Archive of 11 years worth of data.

The second WD MyBook is a backup of the Archives on the first one. This is stored offsite (at home)

I used to use 2 simpletech Network Drives. They are only 300 GB, so when I outgrew them, I got the WD 500GB USB Drives.

FIRST QUESTION: I recently erased my 500GB Archive backup, because it came formatted as FAT32, and would not work with my laptop at home (Vista) ...since I'm storing it there anyway, I wanted to be able to access the files there, since I work at home a lot. So, after I reformatted it to NTSF, I needed to copy the 500GB of Archive data back onto it. This data is in subdirectories of around 100 GB each.

To get the data all copied, I processed one folder at a time & that took, on average, about 12 hours each... meaning it took an entire week, run in the evenings... and often continuing the next day.

IS THERE A FASTER WAY TO MOVE THIS MUCH DATA BETWEEN WD MyBook DEVICES?

The reason I ask, is that I want BOTH devices to be Vista compatible, as it is likely that I will eventually have to use Vista at work too. I just don't want it to take a week, if I reformat the other FAT32 one.

SECOND QUESTION:
I sometimes run across files, that I guess are corrupted, because in the process of copying 100GB folders to an archive... I will get error messages which state some problem, reading such and such file... when I click "so what" (or OK, or whatever )... the operation aborts & I could spend an hour figuring out what was copied already & how to select everything not yet copied EXCEPT the offending file... AND THAT'S ON TOP OF THE 4 OR 5 HOURS I ALREADY MAY HAVE LOST, NOT SEEING THE ERROR MESSAGE RIGHT AWAY!!!!

why can't I just tell the computer: COPY THE GOOD FILES... SCREW THE BAD ONES... JUST DON'T STOP UNTILL YOU DO ALL YOU CAN DO!!!

This second issue drives me absolutely nuts, and is exactly why I processed the recent copying of 500GB into several subdirectories at a time. The amazing thing with that process was that this issue NEVER happened once during that procedure.

(hmm...maybe because it's from EXTERNAL to EXTERNAL drive... and the problem I'm ranting about now, is usually when I make my FIRST archive moving data from an INTERNAL drive to an EXTERNAL drive??)

The bummer is that I now want to do another archive of 62GB from my last 3 months work (currently on my D drive on the server)... so I tried to start the copying procedure this morning... but when I checked on it... sure enough, it had been sitting idle for who knows how long... pestering me with some useless error message... which was so frustrating I failed to take note of the offending file before closing the error message & now will not be able to delete it before trying again. :Oops:

PLEASE TELL ME THERE IS A WAY TO AVOID THE PROCESS STOPPING FOR A PROBLEM FILE, INSTEAD OF JUST SKIPPING THE PROBLEM, & GOING ON WITH THE PROCESS WITHOUT ANY INTERVENTION BY ME!

 

[grafixemporium]

Wow. That's quite a time consuming backup plan you have there. I still don't understand why anyone would want to tote an external hard drive around with them back and forth to work. It's important to back up your data, but you have to live your life too!!

I'm not sure if this will help you or not...

USB 2.0 transfers data at a maximum burst rate of 480 megabits per second. Try to understand, that is not a constant rate... that is the maximum rate it can transfer.

A run of the mill hard drive these days running at 7200rpms will transfer data at a burst rate of around 1 gigabit per second... or more. So the nicer stuff running 10k or 15k is much faster... in excess of 1.6 gigabits per second.

You can figure out the math... but USB 2.0 is quite slow compared to any internal hard drive. Therefore, the method you are using on a regular basis to run backups and then restore them onto your laptop at home... is... well... not ideal.

Look into a backup software that will run your backup on a preset schedule. There are programs out there that can do all kinds of backups... including full, backups, incremental backups, differential backups, mirror backups. Each one has its pros and cons. Differential backups only backup the data that has changed since the last full backup. This is a great way to save tons of time.

Whatever the case, a backup should be just that... a backup. It sounds like you are using your backups as working files once you get them home. Then you run into the problem of having different variations of the same files at home and at work.

If you have to bring your work home with you, burn the file(s) you need on a CD or DVD and bring it home. Give your backup drives a rest.

By the way, if you are encountering that many corrupt files every time you are transferring data, something is wrong. One of your drives could have bad sectors. Run a full scandisk with a surface scan on all your hard drives -- which will take quite a while -- this will tell you if any drives are physically damaged. Data that gets placed in a bad sector will be unreadable. If you find bad sectors, replace that drive ASAP. Bad sectors tend to spread like the common cold.

 

[Sign-Man Signs]

We just back up online. We can choose what we want.

 

[iSign]

thanks for the reply Graphixemporium.

I think I may have failed to make my situation clear.

The routine back-ups are nightly automatic, differential backups, done from one internal to another internal hard drive, on my server, by basic backup software included on windows server 2003.

The less ideal part of my current set-up, is about every 4 months or so... I decide to clean out the primary collection of data on the server. This is to "archive it" and free up space for the next 4 months worth of data.
This is when I first move data to an external drive, after which I copy it also to a second external drive.... then, when it exists 2 other places, I delete it off the hard drive on my server. I usually do this selectively, because around 5% of my files are either for jobs in progress, or for reorder jobs that I tend to leave handy for the next reorder.

There really isn't much toting of external drives going on... just 3 times a year to give myself a second (redundant) archive, which is also stored off-site, in case of fire.

I never restore to my laptop. I just figured the archives at home might as well be compatible, in case I have a need for a file some times. I routinely use remote desktop connection to access my office files... so I never "bring work home"... and I never save a variation back to a back-up, or an archive. These drives are for read-only accessing of old files. New versions are saved with all other current data, on the server.

The biggest hassle is that quarterly (or thirdly) archiving of 2 copies of data files off the server, and your posted transfer speeds explain the reason for that very well, but even that is not so bad really... although I'd love to learn a faster way to get data off the internal drives... the real pain was deciding to reformat one of those 500GB drives.

The idea of reformatting the other drive is seeming less and less worthy of my time.

On the matter of corrupt files, I think most of that is the same files giving me grief repeatedly, because of attempts to "move" all my 11 year collection of files & encountering those old corrupt files each time... BUT, there are some new ones, so I will take your advice to heart & run some scans.

Signman, I have 600 gigs of data to keep track of, and the latest 3 months is already over 60 gigs... how long does that take to "just back up onlione"?

 

[Sign-Man Signs]

Our first back up was about 5 hours at 450 gig. We have four main computers that have about the same, though we don't network each. It's on automatic now and is done at night. I've used it once and it was a breeze to recover a lost file. We back up everything on each computer. I am very happy with thier service. I too used a 220 gig external drive but scrapped it after going to Carbonite.
Here's the link.

http://www.carbonite.com

 

[iSign]

DAMMIT!!! there it goes again....

I'm down here at twelve-freaking-thirty, trying to get ahead on a few things.... including making a second copy of this quarterly back up.... and BAM the damn procedure stops to gripe about the file bluegreen_truckflames_layers.psd

FINE!!!! skip the freakin' file!!!!!.... but NO the entire process is aborted, the folder had 8.? gigs... the folder I'm copying to now has 4.? gigs in it.... & I have to manually hunt down the offending file, sort through 8 gigs looking for what freakin nonsensical order the damn back-up was following.... and somehow select the crap that ain't dealt with yet, only to hope & pray murphy's law won't bring the roof down on me again ...WHHYYYYYYYYY????????

THERE HAS ABSOLUTELY GOT TO BE A BETTER WAY!!!!! Computers are smarter then this 99% of the things I do with them... WHY HAS THIS GOT TO BE SO FREAKING DISFUNCTIONAL!!!??*&%#@!

 

[Sign-Man Signs]

Man, I feel your pain. Hang in their brother. Try that Carbonite. It's free for 15 days I think.

 

[iSign]

may have to do that. Thanks

 

[signage]

Doug have you ever looked into Acronis imaging software? It creates an image of your drive that can be accessed through the software and you can get individual files or folder off. I also take less space than the original drive/files you are imaging.

www.acronis.com

 

[choucove]

There are a few causes for why a file would make a backup stop and throw an error like that. First, if it is a Windows component file (and thus in use by Windows) then the backup program would not be able to copy this file. (Try selecting and copying all the files in your My Documents folder and you'll see an error most likely from a few of these Windows components.) Another reason is that the file is in use or open someplace else. This probably isn't the case in your situation. And the third possible reason that I know of would be that the file was corrupted in its original backup.

Like Brian stated above, there are lots of software out there for creating an image of your drive and duplicating it to your other hard drive. The one I'm most familiar with is Norton Ghost, which can be incredibly powerful. It has a built in tool to directly duplicate one drive to the other. I've never had it pop up with an error message of being unable to re-image a file before. It also allows for compression of data to help save a little space.

Norton Ghost version 12 has been kinda hit or miss for me, but I know of several people who use version 10 or 11 without any problems. I have not yet used Acronis, but I have heard it is another good option.

 

[fozzyber]

Go look at a program called second cop, cheap, works very well,
and stable as a table.
And it can only back up files that have changed, AND moves deleted file to a special folder.
http://www.centered.com/
I have used it for 6+ years it backs up about 750 gig nightly.
(but not for Cypress)

 

[David Wright]

I have Mybook also and you might want to read this.
http://blogs.computerworld.com/why_im_done_with_portable_hard_drives

 

[jtrainor56]

I use a program called Smartsync Pro on both my home network and at work if I have a large amount of data to transfer. I never seem to have problems with corrupt files, it copies everything. There is a free 30 day trial... http://www.smartsync.com/

Joe

 

[ChiknNutz]

Mozy is another online storage solution that seems good and affordable. I've been using their free 2GB edition for a while now, just to see how it goes...so far so good.

 

[iSign]

about 750 gig nightly.

thanks for all the tips folks. 2GB doesn't seem to do much for me... but 750 nightly!! ...wow! Is this just a differential backup, weeding through 750GB looking for what has changed... or is this actually making copies of 750 GB?

I have Ghost 10, and use it to ghost my production machines (no data) so I can get them up and running again without reloading any programs, or re installing any drivers, or re establishing any personal settings and accounts, such as email, winfax, and ftp account settings. It has worked great for that (the one time I had to restore a ghost went perfect)

I never thought of it as a method for backing up, or archiving data. Maybe I'll try it!

 

[GK]

thanks for all the tips folks. 2GB doesn't seem to do much for me... but 750 nightly!! ...wow! Is this just a differential backup, weeding through 750GB looking for what has changed... or is this actually making copies of 750 GB?

I have Ghost 10, and use it to ghost my production machines (no data) so I can get them up and running again without reloading any programs, or re installing any drivers, or re establishing any personal settings and accounts, such as email, winfax, and ftp account settings. It has worked great for that (the one time I had to restore a ghost went perfect)

I never thought of it as a method for backing up, or archiving data. Maybe I'll try it!

I would seriously hope it uses some sort of indexing system to let it know what has been modified so you dont need to reupload 750 gigs of unchanged data

 

[iSign]

I assume it is just a differential backup, but if not, my guess (since he said it's NOT at his sign shop) ...if it really copies 750GB, that maybe he works in some industry creating, or processing massive amounts of huge files, like a photo processing, or video editing service or something....

 

[Rhino]

Tell you what...

You fly me down, put me up and feed me for a couple of weeks each quarter, and I'll handle your backups for you..

Deal??

 

[Gene@mpls]

I second Second Copy- have used it for 6-7 years and it is very flexible. Gene

 

[iSign]

nice try Rhino! (but that avatar looks expensive to feed)

ok... just dl'd the second copy trial.

Last night, I gave up running the 62 gig quarterly backup to the WD external USB drive... after 3 or 4 errors spread over 3 or 4 hours...

I decided, in order to get a second 03-09 Archive before clearing the original files off the server, I would drag a copy to the other drive in my server (where nightly backups are written to)
woohoo!! this procedure did the same 62 gigs with no hiccups!! (still wonder why though?)