Suddenly my backup software (Karen's Replicator which I have been using for more than 10 years now) refuses to back up any file which does not already exist on the destination drive.

Any files created since the last time I ran backups are not replicated, giving an error message in the backup log:

"Error: Could Not Open Destination File (Error 3 - The system cannot find the path specified.): E:\Disk1\Users\All Users\Ahead\Nero BackItUp\Cache\NBService.log"

As you can imagine, this compromises somewhat the efficacy of my backups.

I find it suspicious that this behavior began the first time I ran backups after the installation of the latest Windows 10 update, the "Creators" update.

Ideas?

tanstaafl.

Have you asked Karen?

Unfortunately, according to one of the later replies this thread, she is no longer with us.

https://answers.microsoft.com/en-us/wind...21-5f730bb32bb3

Also, according to that thread, Doug isn't the only person having trouble with this program in Win 10.

Doug, it's possible that the issue you're encountering isn't solvable because the program hasn't been updated for at least 6-7 years.

If your method of backing up is to simply copy newer files to another disk drive, I am wondering if another program might be helpful instead. I have been using "Beyond Compare" for this purpose for a long time. I maintain a few file locations for recent copies of all my data files and all my portable programs: extra disk drives plugged into USB, and also a file share on my NAS.

In the past, I had been using a simple batch file running Robocopy to duplicate things to these locations. But I found that Beyond Compare (with a little configuration tweaking) could do the same thing, but with the added benefit of giving me a clear picture of which files were being backed up which were new, which files were changed, and which files needed to be deleted (which show up as orphaned files on the destination folder). Additionally, it gave me a great deal of control over things like which folders got copied (I only copy certain folders, not the whole disk drive), which files to ignore (like mac .DS_Store files), and what comparison methods were used (datestamp fudge factors, ignore case options, byte-by-byte comparsion, etc.).

Though Beyond Compare is meant to be a programmer's comparison tool, with a little tweaking it happens to work great as a manager for a "direct file copy" backup system.

This also allows me to do things like keep the Video and Music folders on my NAS updated with the latest copies of songs and Rifftrax videos from my hard disk, and it's all done under one tabbed interface. Each tab lets me control a completely different set of folder backups, each with its own settings.

Though it requires some tweaking to set up, and some manual intervention for each backup, I like how I can choose what/when to back up and clearly see everything that's being changed.

I feel compelled to offer my usual recommendation of Crashplan. It'll do local backups for free, and offers many advantages over a direct file copy. You'll get version history, file compression, and encryption in case anyone steals that other drive. You don't have to use their cloud backup service at all.

But I also know that people like what they like

Isn't that the truth. I'm pretty much hard-wired into the idea that "It's not what I'm used to, so it can't be any good!"

That said, I am looking into other backup software now. My problem is that most of it is too complex, offering feature after feature that I'll never need.

My backup needs are simple. I keep two hard drives (one internal, the other an external USB drive) which are for backups only. Every few weeks I do a full system differential backup. That's it.

This one seems like it might work for me, and it has an advantage over Karen's Replicator in that it can produce a drive image, a huge timesaver in the event that my SSD ever goes tits-up, which SSD's have been known to do without warning.

I've downloaded it, but not installed it yet. I'll do that, and poke around with it and see if I can live with it.

Karen Wentworth... RIP.

tanstaafl.

I used Karen's Directory Printer a few times for some weird occasions where I needed a text list of all the directories and subdirectories in a folder. Really nice. Sorry to hear of her passing.

I have a program called Agent Ransack that I use sometimes for that purpose.

tanstaafl.

Well, rats. I didn't know. I knew her years ago, she's around my age.

What is the native backup option in Windows 10 like nowadays ?

I know the past versions could be awful*, but surely they've improved by now ?

It isn't a battle we're ever going to win, but the vastly easier/better backup/recovery options on OSX was one of the things that drove me from Windows to the Mac

* I tried to use the Windows backup stuff a few versions back. In theory it could do what I wanted, create an image of the boot drive on a network share. In practice over the 5 years I left it running, it never once managed to complete the backup run

I had a quick look at my Windows 10 machine.

It looks like they gave up on the image based backup. It is still there, called "Backup and Restore (Windows 7)", but I'm guessing that naming means it is going away.

The new stuff is the File History stuff. That takes the "just backup the files as they are" approach, rather than creating images. So it isn't going to help with getting back to a bootable system, but it could work for backing up Doug's data.

I've just turned it on for my machine and pointed it at a network share, I'll see how it goes. This is the only machine of mine that doesn't really get fully backed up, it is basically just my games machine, but even then I have CrashPlan on it backing up 1GB of game save data.

Doug really should give CrashPlan a go.

Oh dear.

The Windows File History feature failed to backup some of my data.

I like the "just backup the files as they are approach", but it can run into trouble on Windows in some cases. In this case it couldn't backup some of my files to the network share, because it hit the annoying Windows maximum file path limit.

There are some files on my Windows 10 machine that are in folders that pushes them close to the Windows MAX_PATH limit. Putting those files/folders onto a file share that starts //nexus/misc/backups/beast pushes them over the limit.

It doesn't help that within that network path that I pointed Windows at, it then adds "/andy/BEAST/Data/C" to the path before finally storing the files.

Of course many people will never run into this limit, but when you do there is no real way of fixing it

For those who are wondering, the party responsible for the crazy deep folder hierarchy is node.js (I know they've done some work to improve the situation recently to help Windows out).

Ahh, good ole MAX_PATH and it's ancient hard coded limit that still complicates things today. Fun part it, it's not 260 characters, it's 260 bytes, so with UTF8 characters in the name, it causes even more fun.

Maybe Windows 11 will get around to fully ridding the world of that silliness. Until then, still can joke about it:

https://twitter.com/SwiftOnSecurity/status/854913152319983617

It is unfathomable that it is still there.

Although it is unclear what the practical limit is on the Mac. It seems the filesystem has no limit, but some code in the OS has a 1024 byte limit, maybe...

I've used Backup Assist successfully for my Windows 2012 box. It seems to just wrap the native backup functionality in a way that makes it easy to use and actually work.

It allows for daily/weekly external hard disk rotation, and scheduling.

Windows 7/8/8.1/10 are supported.

It costs money, but I'm not sure I'd want to compromise on backups anyway.

PAGE_SIZE on Linux (normally 4096 bytes).

*shudder* I hate that MAX_PATH limit.

What I find particularly disturbing, any time I see a program that fails badly when it hits the max path issue, is...

The people who run into max path issues the most are coders. Development file repositories can get surprisingly deep at times, and I have yet to meet a code repository that hasn't hit this issue at least once or twice. This means that, in many cases, the coders themselves who are writing these tools are hitting this problem *as they develop the tool* and they're not *doing anything about it*.

Well, scratch that one from the list. It has an over-simplified user interface that prevents any kind of backup other than a plain-vanilla backup to a single destination defined during program installation and unchangeable after that.

Okay, time to look at CrashPlan, I guess. I had just assumed that CrashPlan was a cloud-only backup system, but Dignan says otherwise so I'll check it out.

tanstaafl.

What you want is Time Machine. Should fit your needs perfectly.

Another vote for Crashplan (paid user here).

You should be able to just setup two destinations for your two drives and you can bring your offsite one in whenever you want. Crashplan should automatically detect it and just back it up without even clicking an icon. Just make sure it always gets the same assigned drive letter. You probably already need that for your current solution.

Yup. I like their confidence that you'll like the local backup so much, you'll opt into their cloud service. I've used them for about 6-7 years now, and I've restored files in the past.

You can even back up to other computers you own for free, or even to friends' computers (your data is encrypted to all destinations).

Crashplan doesn't do system images, but I guess you were using something else for that.


Great suggestion. I do this whenever I set up backup drives for my clients.

Whilst it's absolutely more time consuming, I'd probably prefer a clean install. I don't do a lot of disaster recovery though so one off here and there doesn't irk me much.

I've only done limited Crashplan restores from the cloud and locally but every spot check I've done on total files, checking logs of new files backed up and so has checked out. The process is a bit heavy being Java only and it does need a bit of RAM to run on a NAS with millions of files but it does the job well and "just work" (tm). I have about 1.4TB compressed in their cloud (took 6+ months on <1Mbit upstream connection)

The only real problem I have with CrashPlan is with massive, regularly changing files. Files like 40-80GB virtual machine images.

I was struggling for a long time to get these to backup properly. Despite grinding away for months on end it sometimes just never managed to get these files backed up.

I think I've resolved this now. For some of the VMs I've switched over to the image format where the drive image is split into lots of smaller files. In other cases I've snapshotted the VM, so there is a large unchanging file and a smaller changing one.

It seems it is the combination of a massive file, that is also regularly changing that gives it problems. I assume these VM images get lots of appends at the end, I think every time it started backing up one of these files it spent all its time wading through the start of the file checking to see if it was backed up and never managed to reach the end before switching away to run another backup set.

Despite the silly name, AOMEI Backupper is a program I have used a lot recently with great success. It's very similar to Acronis Backup, back when Acronis was actually useful and not unreliable bloatware

It's free, although there is a paid version that does more. The main difference seems to be that they moved the drive clone function to the paid one a couple of versions ago. You can achieve the same result but it's a two step process now, without some of the bells and/or whistles.

If you want that function you need to find V3.5, but for just doing differential backups, the current free one works well. It can be scheduled, it's easy to set up, so far seems to be reliable to me, and yes, I've tested the backups by restoring them!

Just so you have another data point. I came across it by accident when I was rebuilding my machine after a problem that ultimately turned out to be the fault of the SoftPerfect Ramdisk program, and got very frustrated with how incredibly slow Acronis was.

pca

I've been playing with CrashPlan for a bit now. Its philosophy of operation is quite different from what I have been doing for the last decade and a half, but it looks like it can do what I need if I change the way I think about backups.

What I've been doing is to create near-clones of my C: and D: drives on my E: drive. That is, my root directory on E: shows "Disk 1" and "Disk 2". Disk 1 had the same directory tree and file structure as my C: drive, with the notable exception of the Windows directory and a few other files that wouldn't copy over because of in-use status. Disk 2 was a byte-for-byte reproduction of my D: drive. Similarly, my F: drive was byte-for-byte copied to an external USB drive L:. and just for a belt-and-suspenders mode, finally the C: and D: drives were copied again to an external USB drive M:, making the E: and M: drives identical twins. Finally, a few times a year my E: and F: drives were copied to N: and O: external drives which are kept off-premises at a neighbor's house. Backups were done at my discretion, on my schedule, not automatically. (They could have been automatic, but I'm enough of a control freak that I opted out of that.)

The point of the above confusion is that I had access to backups that could be used as-is in place of the original drives, except there was no backup of my operating system. I plan to address that shortcoming as part of my transition away from Karen's Replicator.

CrashPlan keeps all the backups in a single compressed, encrypted archive file. Well, in my case I decided on no compression and no encryption, but it is still all in a single big file. I'm not entirely sure why, but this idea makes me uncomfortable, sort of an "all my eggs in one basket" feeling. Also, CrashPlan insists on doing the backups on its schedule, not necessarily mine, Its default is to do a differential backup (or would it be incremental?) every 15 minutes in background. This is contrary to my philosophy. I don't view my backups as a tool to recover from a crash or failed hard drive, but as a safety reservoir I can go to when I screw up a file so badly that I need to replace it with a saved copy. If my backup program replicates my screwups every 15 minutes, that is counter-productive!

But... I can (and have) set CrashPlan to do its work just once a day instead of every quarter hour. It would appear that CrashPlan is backing up incrementally, and saving older backup data. I assume that the "Restore" function in CrashPlan is somehow sorting all that out and somehow allowing me to recover the particular version of the file that I want should the need arise, but I haven't played with Restore enough to understand how that will work, just enough to know that without the CrashPlan application up and running I have no backups available.

That leads me to my two main questions about CrashPlan.

1) Assume the worst, my SSD has died, nothing recoverable on it, including my installation of CrashPlan. Now what do I do? At a guess, I would go to the system clone copy I made previously with such impressive foresight and kept on... uhh, kept it on what? A flash drive? One of my internal hard drives? An external USB drive? A 180KB 5-1/4" floppy? (Just kidding...) What do I do to address this problem?

2) I have my power management set to turn off the monitor after one hour of inactivity, and start sleep mode after two hours. What happens if CrashPlan is backing up data when the two hour cutoff comes? I have been caught by this before with other background operations. I understand that after the initial backup archives are created, the subsequent incremental backups will take very little time, and this problem will not arise. But my first backup of my D: drive and in particular my F: drive (~2 TB) are going to take a long time. Do I need to disable my power management for those?

Below are the backup frequency settings I have chosen, but it is quite possible that I totally misunderstand the nature of these backups and have done it all wrong. Given what I have written above (sorry to have made you wade through all of this) are these reasonable settings? And what do you recommend that I do to make and keep a system clone stuck away in a drawer somewhere in the event of a total SSD failure?

tanstaafl.

Yes there are large files but I think they are split to 4GB so not exactly one big file. I had a feeling the blob concept might make you uneasy but for what Crashplan does with history and versions it's really not feasible to have a direct copy of your folder structure.

15 minutes is not counter productive. It keeps *every* copy it takes every 15 minutes so you can restore any of the backups depending on how far you want to go back. That's what the screenshot you took shows by default except you've changed it to daily. As they get older, it goes down the list when pruning the backups i.e. the 15 minute backups only last for the current week. Then it just keeps daily copies from the week before, then weekly copies from the last 3 months and so on..

Your changes will result in smaller size of your backup sets, but will result in less "coverage" and also completely removing a deleted file after a month. e.g. if you edited a file all day and screwed it up late in the afternoon, your whole day's work would be lost as the most recent backup would be up to 24 hours old. If you had 15 minute backups you could go back in 15 minute steps (for the whole week by default). Covering your screwups I think you might want frequent backups and if you want to keep the size down, just set the additional versions as appropriate.

Further to this the 15 minute thing by default only keeps 1 week of 15 minute backups. AFter that it keeps one per day for 3 months, then once per week for the last year and then once a month forever. So it's not keeping 15 minute copies forever.

Note also it will not backup the file unless it changed so every 15 minutes it's likely to only backup a few files. Note also you have to save for it to backup i.e. keeping you spreadsheet open but not saving it for hours won't work.

Also, with your settings, if a file goes missing by being moved or deleted and you realise over a month later, you can't get that file back. That's why that last setting defaults to "never".

It's fine to use them as long as you understand the impact of having less frequent copies of your data. That's basically what those settings do - thin out the older versions so you can have history going a long long way back but not keep every 15 minute copy. Most backup systems have this concept.

You can "force" a backup by pressing play on the backup set (it should keep track of changed files) or doing a file rescan.

So yes when you restore, you can restore a snapshot based on time OR if just one particular file you can browse the tree and underneath each file will be each copy that's been taken and you can pick the one you want.

Your questions
1) If your system dies, you need to get it up and running outside of Crashplan as it doesn't do bare metal recovery as noted. Other tools can do system images allowing a full restore. As I said, I probably end up doing a reinstall anyway to remove some of the cruft I end up with. So recover to a working OS, install Crashplan. I think you might have to login and then under destinations just select the backup location as you did when you started and it should just read the backup set from your drive

2) Yes if your system goes to sleep there's not much Crashplan or any other backup can do about that. It will pickup after resuming though. Only issue might be that noted by Andy above where really huge files don't get a chance to finish so have to keep restarting. I've seen something like that happening but Crashplan backing up to their cloud (i.e. paid service) does appear to be able to resume backups. If the file keeps changing though that might be problem.

I leave machines on all the time so the sleep thing probably never hurts me but I would be confident in just letting Crashplan do its thing. For your first backup if it's terabytes of data then yes I would turn off power saving to get a proper start.

HTH

Ditto on everything that Christian said. I wanted to address one sentence:


A backup can and should be both, and Crashplan is. First of all, a backup very much SHOULD be what you use to recover from a failed/stolen/damaged drive. How else would you do it? But Crashplan gives you that little bit extra of allowing you to restore any earlier version of your files.

I agree that it's a different program than what you're used to, but I strongly suggest that you leave the defaults alone and just go about your day. Let the application take care of it and you'll be protected.

Given the absolute certainty that you know far more of this than I do, I will change back to the default settings.

When I changed those defaults, I didn't fully understand that CrashPlan retained all of those 15-minute changes rather than over-writing. Incremental backup vs Differential.

Thanks for pointing me in the right direction.

tanstaafl.