I have finished my scanning project, all 165,873 pages of it.

I will be giving the files to my client on a portable USB hard drive, and will encourage her to copy the hard drive to her own computer (which she hopefully backs up occasionally) and then put the portable drive in a safe deposit box or someplace off site.

Additionally, I have burned the data onto a set of DVD-R discs to be kept in reserve as a last-ditch, desperation recovery option.

How long will these DVD-Rs retain their data integrity?

How long will the external USB hard drive retain data integrity?

Assume that both will be kept in reasonable storage conditions.

tanstaafl.

If they're high-quality DVD-Rs, maybe ten years. If they're not, maybe two or three years.

Hard drives: probably about ten years.

Flash drives: probably about ten years.

Magnetic tape: thirty years.

My general rule of thumb is that if it isn't spinning on two platters, it's as good as gone. Optical media is a grain of sand away from being useless. Hard drives a drop away. Tape is probably better, but expensive and difficult to find the necessary drive after x number of years and certainly not 100%.

If you don't know when copy A goes bad, copy B will likely go bad shortly thereafter.

Like the monks of old, I find I keep copying my data forward onto newer and newer drives. I just hope I'm not silently introducing errors.

Yeah, but if worse comes to worst, the data is still the most likely to exist on the tape, and you should always be able to find some way to read the data off.

If you want to be super paranoid, save it on as many different types of media as possible. Each has its own strengths and weaknesses. For example, a flash drive might survive a flood that magnetic tape might not. And instead of just saving individual files, save files plus error correction files, using something like parchive2.

Also, as Matthew says, verify the data occasionally. If you don't know that one of the copies has gone bad, you don't know that all of them have gone bad.

And don't rely on non-open software. It might be hard in fifteen years to find the right tape drive, but it's going to be even harder to figure out how to read that undocumented file format.

Don't use USB for the copying forward process, since it completely lacks data integrity checks.

eSATA, or possibly even Firewire, is a better choice from that perspective (says he who uses USB for nearly everything..).

There was an IEEE paper back in 2006 that estimated Blu-Rays as having a life expectancy over 50 years in the Amazon, 200 years in Montreal, 500 years in Osaka, 1000 years in a typical office environment, and 2000 years in Montreal based on temperature and humidity. Whether it is a normal Blu-Ray or a special archival one, I don't know. The title is "Over 500 Years Lifetime Dual-layer Blu-ray Disc Recordable based on Te-O-Pd Recording Material" by Naoyasu Miyagawa, Hideki Kitaura, Katsuyuki Takahashi, Yukako Doi, Haruhiko Habuta, Shigeru Furumiya, Ken’ichi Nishiuchi and Noboru Yamada.

I'm not sure if it is just a marketing piece or what, but it was presented at the Optical Data Storage Topical Meeting in 2006.

I'd have to ask 'How common is it that "Te-O-Pd Recording Material" is used the Blu-Ray's commonly found on sale?'

That is why I said I didn't know if it was a marketing piece. The paper sounds like they make the coating/whatever (they being Matshita, it looked like the authors worked there). Even if it isn't common, that wouldn't be a bad archival medium, as long as you kept up with technology when Blu-Ray started getting phased out.