Suddenly my backup software (Karen's Replicator which I have been using for more than 10 years now) refuses to back up any file which does not already exist on the destination drive.

Any files created since the last time I ran backups are not replicated, giving an error message in the backup log:

"Error: Could Not Open Destination File (Error 3 - The system cannot find the path specified.): E:\Disk1\Users\All Users\Ahead\Nero BackItUp\Cache\NBService.log"

As you can imagine, this compromises somewhat the efficacy of my backups.

I find it suspicious that this behavior began the first time I ran backups after the installation of the latest Windows 10 update, the "Creators" update.

Ideas?

tanstaafl.

Have you asked Karen?

Unfortunately, according to one of the later replies this thread, she is no longer with us.

https://answers.microsoft.com/en-us/wind...21-5f730bb32bb3

Also, according to that thread, Doug isn't the only person having trouble with this program in Win 10.

Doug, it's possible that the issue you're encountering isn't solvable because the program hasn't been updated for at least 6-7 years.

If your method of backing up is to simply copy newer files to another disk drive, I am wondering if another program might be helpful instead. I have been using "Beyond Compare" for this purpose for a long time. I maintain a few file locations for recent copies of all my data files and all my portable programs: extra disk drives plugged into USB, and also a file share on my NAS.

In the past, I had been using a simple batch file running Robocopy to duplicate things to these locations. But I found that Beyond Compare (with a little configuration tweaking) could do the same thing, but with the added benefit of giving me a clear picture of which files were being backed up which were new, which files were changed, and which files needed to be deleted (which show up as orphaned files on the destination folder). Additionally, it gave me a great deal of control over things like which folders got copied (I only copy certain folders, not the whole disk drive), which files to ignore (like mac .DS_Store files), and what comparison methods were used (datestamp fudge factors, ignore case options, byte-by-byte comparsion, etc.).

Though Beyond Compare is meant to be a programmer's comparison tool, with a little tweaking it happens to work great as a manager for a "direct file copy" backup system.

This also allows me to do things like keep the Video and Music folders on my NAS updated with the latest copies of songs and Rifftrax videos from my hard disk, and it's all done under one tabbed interface. Each tab lets me control a completely different set of folder backups, each with its own settings.

Though it requires some tweaking to set up, and some manual intervention for each backup, I like how I can choose what/when to back up and clearly see everything that's being changed.

I feel compelled to offer my usual recommendation of Crashplan. It'll do local backups for free, and offers many advantages over a direct file copy. You'll get version history, file compression, and encryption in case anyone steals that other drive. You don't have to use their cloud backup service at all.

But I also know that people like what they like

Isn't that the truth. I'm pretty much hard-wired into the idea that "It's not what I'm used to, so it can't be any good!"

That said, I am looking into other backup software now. My problem is that most of it is too complex, offering feature after feature that I'll never need.

My backup needs are simple. I keep two hard drives (one internal, the other an external USB drive) which are for backups only. Every few weeks I do a full system differential backup. That's it.

This one seems like it might work for me, and it has an advantage over Karen's Replicator in that it can produce a drive image, a huge timesaver in the event that my SSD ever goes tits-up, which SSD's have been known to do without warning.

I've downloaded it, but not installed it yet. I'll do that, and poke around with it and see if I can live with it.

Karen Wentworth... RIP.

tanstaafl.

I used Karen's Directory Printer a few times for some weird occasions where I needed a text list of all the directories and subdirectories in a folder. Really nice. Sorry to hear of her passing.

I have a program called Agent Ransack that I use sometimes for that purpose.

tanstaafl.

Well, rats. I didn't know. I knew her years ago, she's around my age.

What is the native backup option in Windows 10 like nowadays ?

I know the past versions could be awful*, but surely they've improved by now ?

It isn't a battle we're ever going to win, but the vastly easier/better backup/recovery options on OSX was one of the things that drove me from Windows to the Mac

* I tried to use the Windows backup stuff a few versions back. In theory it could do what I wanted, create an image of the boot drive on a network share. In practice over the 5 years I left it running, it never once managed to complete the backup run

I had a quick look at my Windows 10 machine.

It looks like they gave up on the image based backup. It is still there, called "Backup and Restore (Windows 7)", but I'm guessing that naming means it is going away.

The new stuff is the File History stuff. That takes the "just backup the files as they are" approach, rather than creating images. So it isn't going to help with getting back to a bootable system, but it could work for backing up Doug's data.

I've just turned it on for my machine and pointed it at a network share, I'll see how it goes. This is the only machine of mine that doesn't really get fully backed up, it is basically just my games machine, but even then I have CrashPlan on it backing up 1GB of game save data.

Doug really should give CrashPlan a go.

Oh dear.

The Windows File History feature failed to backup some of my data.

I like the "just backup the files as they are approach", but it can run into trouble on Windows in some cases. In this case it couldn't backup some of my files to the network share, because it hit the annoying Windows maximum file path limit.

There are some files on my Windows 10 machine that are in folders that pushes them close to the Windows MAX_PATH limit. Putting those files/folders onto a file share that starts //nexus/misc/backups/beast pushes them over the limit.

It doesn't help that within that network path that I pointed Windows at, it then adds "/andy/BEAST/Data/C" to the path before finally storing the files.

Of course many people will never run into this limit, but when you do there is no real way of fixing it

For those who are wondering, the party responsible for the crazy deep folder hierarchy is node.js (I know they've done some work to improve the situation recently to help Windows out).

Ahh, good ole MAX_PATH and it's ancient hard coded limit that still complicates things today. Fun part it, it's not 260 characters, it's 260 bytes, so with UTF8 characters in the name, it causes even more fun.

Maybe Windows 11 will get around to fully ridding the world of that silliness. Until then, still can joke about it:

https://twitter.com/SwiftOnSecurity/status/854913152319983617

It is unfathomable that it is still there.

Although it is unclear what the practical limit is on the Mac. It seems the filesystem has no limit, but some code in the OS has a 1024 byte limit, maybe...

I've used Backup Assist successfully for my Windows 2012 box. It seems to just wrap the native backup functionality in a way that makes it easy to use and actually work.

It allows for daily/weekly external hard disk rotation, and scheduling.

Windows 7/8/8.1/10 are supported.

It costs money, but I'm not sure I'd want to compromise on backups anyway.

PAGE_SIZE on Linux (normally 4096 bytes).

*shudder* I hate that MAX_PATH limit.

What I find particularly disturbing, any time I see a program that fails badly when it hits the max path issue, is...

The people who run into max path issues the most are coders. Development file repositories can get surprisingly deep at times, and I have yet to meet a code repository that hasn't hit this issue at least once or twice. This means that, in many cases, the coders themselves who are writing these tools are hitting this problem *as they develop the tool* and they're not *doing anything about it*.

Well, scratch that one from the list. It has an over-simplified user interface that prevents any kind of backup other than a plain-vanilla backup to a single destination defined during program installation and unchangeable after that.

Okay, time to look at CrashPlan, I guess. I had just assumed that CrashPlan was a cloud-only backup system, but Dignan says otherwise so I'll check it out.

tanstaafl.

What you want is Time Machine. Should fit your needs perfectly.

Another vote for Crashplan (paid user here).

You should be able to just setup two destinations for your two drives and you can bring your offsite one in whenever you want. Crashplan should automatically detect it and just back it up without even clicking an icon. Just make sure it always gets the same assigned drive letter. You probably already need that for your current solution.

Yup. I like their confidence that you'll like the local backup so much, you'll opt into their cloud service. I've used them for about 6-7 years now, and I've restored files in the past.

You can even back up to other computers you own for free, or even to friends' computers (your data is encrypted to all destinations).

Crashplan doesn't do system images, but I guess you were using something else for that.


Great suggestion. I do this whenever I set up backup drives for my clients.

Whilst it's absolutely more time consuming, I'd probably prefer a clean install. I don't do a lot of disaster recovery though so one off here and there doesn't irk me much.

I've only done limited Crashplan restores from the cloud and locally but every spot check I've done on total files, checking logs of new files backed up and so has checked out. The process is a bit heavy being Java only and it does need a bit of RAM to run on a NAS with millions of files but it does the job well and "just work" (tm). I have about 1.4TB compressed in their cloud (took 6+ months on <1Mbit upstream connection)

The only real problem I have with CrashPlan is with massive, regularly changing files. Files like 40-80GB virtual machine images.

I was struggling for a long time to get these to backup properly. Despite grinding away for months on end it sometimes just never managed to get these files backed up.

I think I've resolved this now. For some of the VMs I've switched over to the image format where the drive image is split into lots of smaller files. In other cases I've snapshotted the VM, so there is a large unchanging file and a smaller changing one.

It seems it is the combination of a massive file, that is also regularly changing that gives it problems. I assume these VM images get lots of appends at the end, I think every time it started backing up one of these files it spent all its time wading through the start of the file checking to see if it was backed up and never managed to reach the end before switching away to run another backup set.

Despite the silly name, AOMEI Backupper is a program I have used a lot recently with great success. It's very similar to Acronis Backup, back when Acronis was actually useful and not unreliable bloatware

It's free, although there is a paid version that does more. The main difference seems to be that they moved the drive clone function to the paid one a couple of versions ago. You can achieve the same result but it's a two step process now, without some of the bells and/or whistles.

If you want that function you need to find V3.5, but for just doing differential backups, the current free one works well. It can be scheduled, it's easy to set up, so far seems to be reliable to me, and yes, I've tested the backups by restoring them!

Just so you have another data point. I came across it by accident when I was rebuilding my machine after a problem that ultimately turned out to be the fault of the SoftPerfect Ramdisk program, and got very frustrated with how incredibly slow Acronis was.

pca

I've been playing with CrashPlan for a bit now. Its philosophy of operation is quite different from what I have been doing for the last decade and a half, but it looks like it can do what I need if I change the way I think about backups.

What I've been doing is to create near-clones of my C: and D: drives on my E: drive. That is, my root directory on E: shows "Disk 1" and "Disk 2". Disk 1 had the same directory tree and file structure as my C: drive, with the notable exception of the Windows directory and a few other files that wouldn't copy over because of in-use status. Disk 2 was a byte-for-byte reproduction of my D: drive. Similarly, my F: drive was byte-for-byte copied to an external USB drive L:. and just for a belt-and-suspenders mode, finally the C: and D: drives were copied again to an external USB drive M:, making the E: and M: drives identical twins. Finally, a few times a year my E: and F: drives were copied to N: and O: external drives which are kept off-premises at a neighbor's house. Backups were done at my discretion, on my schedule, not automatically. (They could have been automatic, but I'm enough of a control freak that I opted out of that.)

The point of the above confusion is that I had access to backups that could be used as-is in place of the original drives, except there was no backup of my operating system. I plan to address that shortcoming as part of my transition away from Karen's Replicator.

CrashPlan keeps all the backups in a single compressed, encrypted archive file. Well, in my case I decided on no compression and no encryption, but it is still all in a single big file. I'm not entirely sure why, but this idea makes me uncomfortable, sort of an "all my eggs in one basket" feeling. Also, CrashPlan insists on doing the backups on its schedule, not necessarily mine, Its default is to do a differential backup (or would it be incremental?) every 15 minutes in background. This is contrary to my philosophy. I don't view my backups as a tool to recover from a crash or failed hard drive, but as a safety reservoir I can go to when I screw up a file so badly that I need to replace it with a saved copy. If my backup program replicates my screwups every 15 minutes, that is counter-productive!

But... I can (and have) set CrashPlan to do its work just once a day instead of every quarter hour. It would appear that CrashPlan is backing up incrementally, and saving older backup data. I assume that the "Restore" function in CrashPlan is somehow sorting all that out and somehow allowing me to recover the particular version of the file that I want should the need arise, but I haven't played with Restore enough to understand how that will work, just enough to know that without the CrashPlan application up and running I have no backups available.

That leads me to my two main questions about CrashPlan.

1) Assume the worst, my SSD has died, nothing recoverable on it, including my installation of CrashPlan. Now what do I do? At a guess, I would go to the system clone copy I made previously with such impressive foresight and kept on... uhh, kept it on what? A flash drive? One of my internal hard drives? An external USB drive? A 180KB 5-1/4" floppy? (Just kidding...) What do I do to address this problem?

2) I have my power management set to turn off the monitor after one hour of inactivity, and start sleep mode after two hours. What happens if CrashPlan is backing up data when the two hour cutoff comes? I have been caught by this before with other background operations. I understand that after the initial backup archives are created, the subsequent incremental backups will take very little time, and this problem will not arise. But my first backup of my D: drive and in particular my F: drive (~2 TB) are going to take a long time. Do I need to disable my power management for those?

Below are the backup frequency settings I have chosen, but it is quite possible that I totally misunderstand the nature of these backups and have done it all wrong. Given what I have written above (sorry to have made you wade through all of this) are these reasonable settings? And what do you recommend that I do to make and keep a system clone stuck away in a drawer somewhere in the event of a total SSD failure?

tanstaafl.

Yes there are large files but I think they are split to 4GB so not exactly one big file. I had a feeling the blob concept might make you uneasy but for what Crashplan does with history and versions it's really not feasible to have a direct copy of your folder structure.

15 minutes is not counter productive. It keeps *every* copy it takes every 15 minutes so you can restore any of the backups depending on how far you want to go back. That's what the screenshot you took shows by default except you've changed it to daily. As they get older, it goes down the list when pruning the backups i.e. the 15 minute backups only last for the current week. Then it just keeps daily copies from the week before, then weekly copies from the last 3 months and so on..

Your changes will result in smaller size of your backup sets, but will result in less "coverage" and also completely removing a deleted file after a month. e.g. if you edited a file all day and screwed it up late in the afternoon, your whole day's work would be lost as the most recent backup would be up to 24 hours old. If you had 15 minute backups you could go back in 15 minute steps (for the whole week by default). Covering your screwups I think you might want frequent backups and if you want to keep the size down, just set the additional versions as appropriate.

Further to this the 15 minute thing by default only keeps 1 week of 15 minute backups. AFter that it keeps one per day for 3 months, then once per week for the last year and then once a month forever. So it's not keeping 15 minute copies forever.

Note also it will not backup the file unless it changed so every 15 minutes it's likely to only backup a few files. Note also you have to save for it to backup i.e. keeping you spreadsheet open but not saving it for hours won't work.

Also, with your settings, if a file goes missing by being moved or deleted and you realise over a month later, you can't get that file back. That's why that last setting defaults to "never".

It's fine to use them as long as you understand the impact of having less frequent copies of your data. That's basically what those settings do - thin out the older versions so you can have history going a long long way back but not keep every 15 minute copy. Most backup systems have this concept.

You can "force" a backup by pressing play on the backup set (it should keep track of changed files) or doing a file rescan.

So yes when you restore, you can restore a snapshot based on time OR if just one particular file you can browse the tree and underneath each file will be each copy that's been taken and you can pick the one you want.

Your questions
1) If your system dies, you need to get it up and running outside of Crashplan as it doesn't do bare metal recovery as noted. Other tools can do system images allowing a full restore. As I said, I probably end up doing a reinstall anyway to remove some of the cruft I end up with. So recover to a working OS, install Crashplan. I think you might have to login and then under destinations just select the backup location as you did when you started and it should just read the backup set from your drive

2) Yes if your system goes to sleep there's not much Crashplan or any other backup can do about that. It will pickup after resuming though. Only issue might be that noted by Andy above where really huge files don't get a chance to finish so have to keep restarting. I've seen something like that happening but Crashplan backing up to their cloud (i.e. paid service) does appear to be able to resume backups. If the file keeps changing though that might be problem.

I leave machines on all the time so the sleep thing probably never hurts me but I would be confident in just letting Crashplan do its thing. For your first backup if it's terabytes of data then yes I would turn off power saving to get a proper start.

HTH

Ditto on everything that Christian said. I wanted to address one sentence:


A backup can and should be both, and Crashplan is. First of all, a backup very much SHOULD be what you use to recover from a failed/stolen/damaged drive. How else would you do it? But Crashplan gives you that little bit extra of allowing you to restore any earlier version of your files.

I agree that it's a different program than what you're used to, but I strongly suggest that you leave the defaults alone and just go about your day. Let the application take care of it and you'll be protected.

Given the absolute certainty that you know far more of this than I do, I will change back to the default settings.

When I changed those defaults, I didn't fully understand that CrashPlan retained all of those 15-minute changes rather than over-writing. Incremental backup vs Differential.

Thanks for pointing me in the right direction.

tanstaafl.

This is not incremental vs differential really. Basically Crashplan always does incremental backups and when you restore it can create any point in time from that data.

The idea of incremental and differential back ups really applies to when you are using storage like tape or CD where a tape could be lost or destroyed.

Differential has everything since the last full backup so with a full backup gives you everything for that point in time since the last full backup and doesn't rely on intermediate backups.

Incremental only stores what changed since the last backup of any type i.e. you need multiple tapes/CDs to capture a point in time since the last full backup.

Boy, that's for sure.

Perhaps (probably?) I am totally misunderstanding how to set up the backup tasks, but it seems to be very different from what I had thought.

In my old backup system (Karen's Replicator) I backed up my C: drive to E:\Disk 1. My D: Drive to E:\Disk 2. My F: drive to L:\

I can't figure out with CrashPlan how I am supposed to do that. It is almost as though CrashPlan is putting all the available destinations into a pool and letting the files that are being backed up fall wherever they may within that pool.

That doesn't seem reasonable, and no doubt I am just not smart enough to figure out how to keep things organized the way I used to do it, which is the way I want.

I'll keep poking at it and hopefullly will figure it out.

tanstaafl.

If you want to keep it separate because that's how your backup disks are then you can. Add the three backup disks one by one as Folder Destinations so you have three destinations

Then for the source disks create three backup sets for the three source drives (one each) and point each one individually at each folder. I'm not sure if the free version might limit you here, but the paid one allows more than one and just gives them the name of the driver letter. Actually backup sets might be a problem on the free version.

You don't need to do it that way apart from the fact that your backup disks aren't one contiguous disk. If it were one big disk you'd just point all three source disks at one backup destination and it would just put everything together.

Hmmm, I just looked and it looks like the free version can't do 15 minute backups anyway. It says "Once daily" vs "Continuous" and backup sets are out too. That will be an issue for separate source disks to separate backup disks.

https://www.crashplan.com/en-us/features/compare/

Sorry I guess we are so used to the paid version. Normally it wouldn't be an issue if all the disks were combined.

What are the source and destination disk sizes currently?

[Emphasis mine] At the exact moment you posted your reply, I was discovering buried in a submenu of Settings the Backup Sets dialog box.

My CrashPlan (version 4.8.2, Win64) does support backup sets, and I am now configured to do so. I am also configured for 15-minute backup intervals.

However, the Free version I have also includes a 30-day Cloud trial subscription, and it is possible that after 30 days those two features may go away. If that happens, I'll just upgrade to the paid version, no problem there. I installed the Free version because I am absolutely not interested in Cloud backup (at my upload speed it would take literally months to put my 2TB+ of data into the Cloud). Being a grumpy old curmudgeon, I prefer to keep my data confined to my own computer, not spread out over the internet.

OK, it's going on one in the morning now, I think I have things set up to run now: power management off, destination disks formatted (had to get rid of old backups to make room for new ones), CrashPlan configured as best I can figure out how. Sometime tomorrow I'll be able to see if I did it right.

Thanks for looking out for me.

tanstaafl.

My initial seed took something like 6 months solid (1.5 odd TB and <1Mbps ADSL2+). I just created a backup set with important stuff like photos first and got that up. The initial seed was maybe a month. Then I thought why not and added less important things and once they're there, it's done.

Note that if you pay I would recommend you use the Cloud feature for a last resort backup. Just keep your utmost important stuff if you don't want it hogging you upstream. My "master" backup is the cloud but I also just set up a local copy on my MythTV frontend machine so it's easy to get large volumes of data back quickly. The cloud then basically becomes the "house burns down"/"lightning strike"/"natural disaster" backup.

Also, you *can* provide your own encryption key so that Crashplan themselves cannot even decrypt it. The web restore doesn't work then either as a side effect. You just need to keep that key safe as you have the only copy of it.

Thank-you, but I'm not all that interested in your offer to read through those hundreds of medical claims looking for a way to refuse payment in the future. But somebody else may want to, or perhaps have an even more keen interest in using your medical plan for their own ailments.

Cheers

With his current plan of attack, unless you travel down to Mexico you won't be able to.

Oh, I thought the backups were going "to the cloud", where everyone can hack into them. But if instead they're totally local, then great!

A little bit of research shows... sort of.

The free version keeps a limited version of the backup sets, the limitation being that after the trial period is over I cannot modify the settings on the sets, nor can I add new ones. Since the sets I have now defined are exactly the same as I have used since about 2011, I think I can live with that.

Probably the 15-miute backup setting will stay with the existing backup sets.

I didn't. Oh, it ran all right, but I messed up the destination assignments and ran out of disk space on the E: drive. I don't recommend trying to set up CrashPlan at one in the morning when you are really tired...

Now it does seem to be working, but I won't know for sure until tomorrow.

And... CrashPlan really wants me to do a cloud backup, to the point of putting in a non-deletable backup set to send my Desktop to the Cloud. I think I got around that by parameterizing that one set to backup only between 12:00--12:01 AM and un-checking all of the Monday--Sunday allowable days. We'll see...

Now the question is how do I go about making a restorable clone of my C:\ system SSD? I have several suitably-sized hard drives kicking around that I can put into a USB enclosure. Can I clone to that? Would I clone the entire hard drive or just the Windows directory? Finally, what software is recommended for this? Patrick likes the AOMEI software, are there other recommendations?

tanstaafl.

Just delete that bottom backup set if you don't want it from the Settings page -> Backup tab.

If it's the original one it created (I presume based on the name) then I think you can't delete it, you could have just made it one of your backup sets.

To stop it backing up to the cloud, select the backup set (under Settings->Backup), in the destinations panel click "change" and then uncheck Crashplan Central

I see you have no files selected anyway so it won't back anything up but I know it would irk me having the extra backup set there even if it was harmless.

Actually... you can delete that hard-wired Cloud set. I was one keystroke away from deleting it, then decided to leave it in just to keep my future options open. I changed its backup priority to "4" so it is at the bottom of the list and doesn't annoy me too much.

I have one more set to add to my backup list, the backup of my C: and D: drive backups which will go to the M: drive, a 465GB USB external drive all packaged in a nice enclosure. There are 27 GB of truly non-essential data data on the C: and D: drive that I won't include on the M: drive backup, leaving me with 164 GB of headroom for further storage as the D: drive grows.

Now... the Great Cloning Questions.
1) Obviously I need cloning software to clone my C: system drive. Do you have a specific recommendation?
2) Is it better to clone to a USB hard drive?
3) Should I acquire a spare SSD that matches my current 120GB SSD and clone to that? (I can get the same model Kingston SV300S37A/120G from Amazon for $60.)
4) 4How about cloning to a 128GB Flash Drive?
5) Do I clone the entire C: drive or just the operating system files?
6) I have both USB 3.0 and eSATA external HDD docking stations available. Is one preferable to the other?

My data backups are chugging merrily away now, showing 18 hours to go. If I left the computer alone so that CrashPlan could have the full use of the CPU it would take about half that time. Then, I have to add three more backup destinations: my M: drive (described above); my N: drive (off-premises backup for C: & D:); and my (wait for it!) O: drive (off premises backup for my F: (audio books) drive). Once I get those added and actually run, and my system drive cloned, I'll feel fairly secure.

Hmmm... I guess I should add an off-premises clone copy to the list.

tanstaafl.

Consider moving into the world of SSD as primary storage even for large data. 500GB and even 1TB SSD are no longer stupid expensive. Crazy fast and reliability is generally well understood.

I now primarily use SSD for local storage. My backup networked storage drives are not yet SSD. Even those I expect to be SSD next time around.

Backups can complete a lot faster when source and destination are SSD. And no worries about mechanical shock or impact when using or transporting SSD drives.

If you're looking for a clone of your system drive I wouldn't be looking for a single clone unless you want minimum downtime. In that case RAID1 might be better.

I'd be looking for a tool that clones the disk to an image file. You then have to run a tool to restore that image file usually from some kind of recovery disc you make. That way allows you to have multiple system images. Those system images could then just go into your Crashplan backup store to be deduplicated.

Tools like Acronis can do incremental/differential type backups. I wouldn't recommend Acronis though after fighting to get a refund after proving it didn't work properly.

Having a single clone copy to another identical disk provides only one backup in time. That may be acceptable to you though.

Maybe I'm looking at this wrong, but my idea of a clone copy of my system drive was so that in the event of failure, I would not have to go through the pain of reinstalling Windows and all my software. I keep just about all of my actual data on other drives, and even the data on the system drive is backed up by CrashPlan.

I envision recovery as putting in a new SSD, copying my clone over it, and then restoring from CrashPlan to restore any files that are new or changed since the clone was made.

Am I missing something here?

tanstaafl.

Ummm...actually they are. I already have quite serviceable 1TB and 2TB hard drives. Replacement 1TB SSDs seem to run about five to seven times the cost of similar hard drives, so in my system I can keep what I have for $0, or upgrade to SSD's for about $2,000 worth of 1TB SSDs to keep the same capacity.

The only time I do work that requires any hard drive (or SSD) speed is booting up and doing backups. My bootup is SSD, and now that you've convinced me to go to CrashPlan, backups run quietly in background all the time with no noticeable cost to performance in foreground operations.

If I were building a new computer I would certainly keep the SSD for system drive, maybe an SSD for main data drive, but certainly not for all the backups.

And, I am not all that convinced about the reliability of SSDs. Perhaps their failure rate is lower than a HDD, but HDDs usually give plenty of warning before they quit working, while SSDs are known to fail totally with no warning at all.

I know, I am a Luddite at heart, but I think I'll hang onto my mechanical drives a little longer.

tanstaafl.

I guess it depends on how often you're making that system image.

But RAID 1 would basically give you zero downtime, as your other working drive would just...keep working. Then at your leisure you could pop in another SSD when you're ready to rebuild the array (which I assume would take very little time with an SSD array).

If both drives bit the dust, you'd have that clone you made.

You have only one backup copy and no history if you just clone a copy of your system to another drive.

If you use a tool that images the drives you can have multiple aged copies of that system drive. If it's only a small SSD you could keep many on a large mechanical disk. That can very very up to date too - easily daily if you want.

If your system fails, you boot from a pre-prepard recovery CD or USB, pick the image from another disk and restore on to the SSD.

Still happily using Replicator on various PCs (including Win 10 and Win Server 2012), never come across anything it won't copy. If there's an issue it might be to do with rights. Have you looked at permissions for the destination folder?

"Error: Could Not Open Destination File (Error 3 - The system cannot find the path specified."

That doesn't seem to be a permission problem, and Replicator will find/delete/replace existing files if they have been modified since the last backup. It just won't create new ones.

Does your Windows 10 PC have the latest "Creator" update installed? It was my first backup after that update that my Replicator stopped working.

tanstaafl.

How do I find out whether it has the Creator update?

According to winver

It's version 1511 OS Build 10586.633

My winver says: version 1703, OS Build 15063.138.

??

Someone who knows a great deal more about Windows than I do will need to answer this question.

tanstaafl.

Likewise. I'm on 32bit Pro

Per http://tunecomp.net/check-if-windows-10-creators-update-is-installed-or-not/ ...

If Windows 10 Creators Update is installed you should see Version 1703 (OS Build 15063.0) or greater

Per http://pureinfotech.com/check-windows-10-creators-update-installed/ ...

The Windows 10 Creators update has a version number of 1703, and the Anniversary Update is version 1607.

Looks like I'm out of date.

The version number is a date, which helps decode the various major updates to Windows 10. Each of these are essentially doing a whole OS upgrade, Microsoft has really stepped up their installer/upgrade technology to make this pretty painless and hidden behind the scenes. They are pretty close to macOS upgrades these days as far as ease of upgrading, but as Doug discovered, they can still break some software.

1507 = July 2015 (Initial release)
1511 = November 2015 (November update)
1607 = July 2016 (Anniversary update, added the Linux subsystem)
1703 = March 2017 (Creators update)

The month tends to be when the Windows Insiders (opt in beta testers) receive the near final version of the new iteration. Microsoft is aiming for one of these major updates per year. And they get rolled out in batches once released.

Well, CrashPlan is now up and running, backups are complete except for the off-premises backup, and I am even more uncomfortable about CrashPlan's backup philosophy than I was before.

CrashPlan is quite proud of their De-Duplication process, in which each file as it is backed up is compared block by block (not file by file or filename by filename, but block by block) with all previous files that were backed up and if the data is the same, that block is saved as a pointer to the first block. This is a marvelous idea for saving disk space, which now costs four one-thousandths of a penny per megabyte if you buy it in two terabyte chunks.

Great -- they're saving me a tiny fraction of a penny in storage cost at the expense of removing any semblance of redundancy in my backup.

I used to back up all my data, unarchived, uncompressed, unencrypted, instantly and easily retrievable without special software... and then I backed up my backups the same way, and then I backed up those backups for off premises storage. Now all of my backed up data is kept in a single archive file (even if that archive is spread across several backup sets kept on multiple drives, it's one archive) and that's a lot of eggs to keep in a single basket!



At the moment I am less enthusiastic about CrashPlan than most people. Perhaps it is because I have some basic misunderstanding about how it works, but I don't think that is the case. I have spent a fair number of hours puzzling out their most "un-wizard-like" interface (it is astonishingly easy to set your backups up incorrectly) and studying how it works.

After the time and effort invested in the project, I will probably leave it as is, unless something noticeably better* comes to my attention.

*Better defined as more like how I am used to doing it, not necessarily faster, or more secure, or using less disk space, or whatever.

tanstaafl.

To be clear, they're trying to save more than just storage cost, they're also trying to save time and bandwidth in the backup process. At the moment, both of those things are all local storage to you, so they are fast because you have a lot of bandwidth to your local storage. But if you ever used offsite backup, the deduplication feature would improve the speed of the backup process significantly.

In theory, a single compressed/deduplicated backup like theirs should be just as safe as a single set of file-by-file copies like yours. The only difference is that, to extract a single file from the backup, you need their software to do the extracting.

The redundancy question is a different one. Are you saying that the archive file is a single backup that's been split up amongst multiple storage devices? That doesn't sound right. That sounds like it would increase the risk of data loss rather than decrease it. Are you sure that it's not multiple redundant backups (separate individual complete backups) stored in multiple locations?

The backups are *not* spread across more than one disk. Doug has his multiple disks being backed up onto multiple backup disks but is set so it's a 1:1 source:destination.

You cannot have a Crashplan spanning disks natively. It would have to be a RAID or JBOD combining more than 1 disk in to one larger virtual disk for that to occur.

The bolded underlined statement about redundancy is not correct. It can be no less redundant than any previous backup system. Taking backups of backups is not the best backup solution IMO. Keep multiple stages of backups instead.

If you want redundancy in your backups then add multiple destination disks to each backup set. If that's an external device that is removed occasionally, Crashplan will just update the back up set when the device is plugged in.

I suspect part of the discussion would be the difference between mere copies of files, folders, and entire drive contents, and backups.

Multitudes of copies of folders and files are not (in my opinion) the same thing as structured backups, part of a backup system. Copies and recovery of the copied data may not provide detection or correction of bit rot, file copying bit errors, directory/folder errors, etc.

Cascading 'copies of copies' compounds the risk of propagating bad/damaged data. When file corruption is discovered, finding the least old non-corrupted version could become laborious.

During recovery operations using a copies of copies system, the user alone is responsible for identifying what files are supposed to go where, and actually putting them there. It is even possible to inadvertently copy things in the wrong direction, writing bad data over good. There is no software watching over to provide guidance or gaurdrails.

I do not use Crashplan and have not studied the documentation. I would certainly expect that the archive has not only detection of bit damage but can correct and recover from some level of archive file damage and still restore 100% of the data from the backup.

This. This is what makes Crashplan a backup program, and Karen's software is not. It's the same reason I urge my clients not to treat Dropbox as a backup program.

Let me tell you how I would back up your computer if it were mine:

1- install Crashplan
2- choose the drives/data I want backed up
3- choose the location(s) to back up to

There's honestly not much of a reason to get more complicated than that.

This is how I've set up Crashplan for almost a hundred users over the past few years. All of their data is seen by Crashplan. All of their data is stored in at least one other location (some opt out of the cloud and only want one drive). All of their data can (fairly) easily be restored.

At the end of the day, I think you're still trying to force your previous way of doing things onto a new way of doing things, which doesn't usually work well...

But... but... but... it's different from what I'm used to, so it can't be any good!

I have five different CrashPlan archives on three different hard drives. Are you saying that the de-duplicating that CrashPlan does in any archive is independent of the other four archives, even though each archive has exactly the same filename, i.e., 791869747154424520? I guess that must be the case, otherwise my "M:\Disk 01" archive would be essentially empty as it contains an exact copy of my "E:\Disk 1" archive which is an exact copy of my D:\ disk drive. If it were de-duplicated based on what had been backed up to the E: archive, the M: archive wouldn't have anything in it because the entire M: archive is a duplicate of the E: archive.

It is curious that the M: archive is 56GB smaller than the E: archive, so there must have been some de-duplicating going on there.

Given the likely premise that you guys and the millions of other people happily using CrashPlan know more about this than I do, I'll hang in there with CrashPlan, as it does appear that I do indeed have multiple copies of my data saved so if something happens to one of the archives, I have the data in another one. Along with the original data, of course. At least no-one can accuse me of not being serious about backing up my data!

tanstaafl.

You're making my head spin.

Here is what I know: in the 3-step setup I described in my previous post, if step 3 involved choosing two locations, then yes, I would have a full backup set in both locations. If my local drive had failed, I could get all the exact same data from the cloud.

I've sort of lost the thread on how your system has been set up. Sorry about that...

It only deduplicates *within* a backup archive. If you backup the same file to more than one destination you'll have the file in both destinations independent of one another. Each backup destination is a complete independent backup which is the point. If one backup destination depended on another having two would be somewhat useless.

Deduplication mainly is useful for slow bandwidth connections. If you have multiple copies of one file in different folders, it only sends it once. It works locally and only keeps the one copy though. If one of the files changes then it stores new copy in the backup and the old one stays for both locations as it's part of the history. If only part of a big file changes, it only sends the changed bit.

The big number is the GUID of the machine. Every install or machine has a unique number to identify it. So yes the base filename on each of your destinations will be the same.

Stop trying to understand the files, do you backups and do spot checks using the Restore tab to see what you have. Disconnect backup destinations to simulate lost drives.

You lost me with M:\ and E:\ but in theory they should be the same size if they are the same source data. Note that as soon as one destination misses a backup e.g. you unplug for offsite, the size will change of course as one will grow without the other. If you plug it back in it should catch up but will have missed any intermediate changes of course.

O.K., that makes sense. I am starting to understand this now.

I didn't describe very well what I am doing with E:\ and M:\. It is not as complicated as I made it sound.

My files are on my C:\ and my D:\ drives. I back up the C:\ data to a directory (Disk 1) on the E:\ drive; I back up the D:\ data to a second directory on the E:\ Drive (Disk 2).

Then I repeat the process, backing up to Drive M:\. This suggests to me that E:\ and M:\ should be more or less identical. They aren't. I'll keep an eye on them, examine them with the Restore module, and see if I can find out why. Maybe it's something simple like I didn't turn off compression on the M:\ drive or something.

Thank you, Christian, for the clarification. You are pretty good at that!

tanstaafl.

You might be slowing your backups down by turning compression off. CPUs are extremely fast at compressing stuff now and drives are still faster when they have less data to write...

Though if course if your drives are full, with say scans of medical bills, that might not be the case

In my case, probably not. The great majority of the data I am backing up is already compressed: photos, audio, compiled program files and the like.

Compressing those files frequently results in files that are larger than the originals, although not by much. Well, sometimes maybe not, a quick test on four music files totaling 87MB(Beethoven Ninth) showed a compressed file that was about eight tenths of one percent smaller than the original, but it took about 15 seconds for the 7-Zip program to process the file.

Since I am not backing up to the Cloud, file size, storage required, and speed are not concerns. The backup program just ticks away quietly in the background with no noticeable impact on performance, and now I feel better knowing that even if something happens to one of my backup archives, I sill have another copy of it immediately accessible to me.

One reason Cloud backup is not realistic for me is that even in the unlikely event of optimal conditions, assuming I had already put in the several months of 24-hour a day uploading it would require to put my data into the Cloud, in the event of catastrophic failure it would take almost three weeks of continuous downloading to get it back. I wonder whether a lot of people who think they are perfectly safe with a Cloud backup have taken that into account? Although I guess that just getting the critical data back wouldn't take so long, and the pictures and music etc. could come later.

As for medical bills, my life is an open book. If someone wants to know that I had outpatient surgery on my arm to address a calcified tendon, well, you heard it here first I guess. The only people who might be interested in my medical records is my health insurance company, and they already have them.

tanstaafl.

Some cloud backup services will/can FedEx your entire backup to you on a new hard drive. So recovery can be via overnight courier.

Same for sending the original 'seed' backup to them. You create a backup set, courier the hard drive, then increment backups from there via the cloud.

The big win with cloud backup is that the saved data is physically far removed from your locale and within the cloud service is also redundantly protected. And the continuous incremental cloud backup process means the data exposed to actual loss is only the most recent few minutes or hours worth.

It's sad that, in this day and age, Truck Bandwidth is still faster. Sad but true.

And of course, Randall tackles that question quite adequately...

https://what-if.xkcd.com/31/

I was just going to use that station wagon hurtling down a highway quote. Foiled by xkcd.

What If is genius. The book has some fun additional entries as well.

My 6 & 9 year old both enjoy Randall's What If? and Thing Explainer books. Sure, a chunk of it is over their heads (and mine!) but the writing style is humors and ridiculous.

FWIW, "What If" is currently available on Amazon as an Amazon Prime free book. I just downloaded it.

tanstaafl.