Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Page 1 of 5 1 2 3 4 5 >
Topic Options
#296657 - 10/04/2007 20:45 Hardware RAID Advice
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
After much messing about with software RAID 5 with FreeNAS I have given up. I'm not finding it's a solution I feel my data is safe with. This could be down to the fact I am still a total Linux/BSD noob, but FreeNAS also seems to be locking up my system quite a bit so I don't think it's all my fault

So I am looking for a hardware solution, I am using IDE drives (3 Seagate 250Gb and 1 x WD 250Gb RE) the rest of the setup is pretty bog standard. I think I may use clarkconnect for the NAS part as I already run it as my firewall/router etc...

Any advice? I want it to be cheap as the system is manily made up of parts I already had and the idea of the project is to be low cost but highly stable. I'm not that botherd about speed but so far I have found a Socket 478 P4 2.8Ghz, 1Gb RAM, random MB (think it's an ex-HP MSI OEM from a system that was split at work) and a dodgy D-Link Gigabit NIC.

I could splash out on a LSI MegaRAID i4 they seem to be pretty reasonable on eBay.

Cheers

Cris.

Top
#296658 - 10/04/2007 21:35 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
The 3Ware cards are the most highly recommended for this.

-ml

Top
#296659 - 10/04/2007 21:48 Re: Hardware RAID Advice [Re: Cris]
Attack
addict

Registered: 01/03/2002
Posts: 599
Loc: Florida
I'm currently using Arch Linux and software raid on an AMD X2 with 5 320GB Seagate drives. One of my drives failed Friday, the replacement should be here today. Other than the drive failure I didn't have any problems with it. The next raid device I setup might be another linux box or this.
_________________________
Chad

Top
#296660 - 10/04/2007 21:53 Re: Hardware RAID Advice [Re: mlord]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Hmmmmm why is it that the best is almost always the most expensive.

Even eBay doesn't throw up many bargains there.

Cheers

Cris.

Top
#296661 - 10/04/2007 22:07 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
I don't know which specific Linux kernel FreeNAS is using, but it doesn't really matter: they all suck big time in fault-handling. The upcoming 2.6.21 kernel is the first one I'd really trust to do most things correctly, but even with that I have patches (my own) to fix obvious dead-in-the-head design relics in the SCSI layer (which is normally used for SATA drives).

So.. it's good, but about to get real for the first time. Hopefully by 2.6.23 or so, it should really kick saa.

EDIT: ahh.. FreeNAS is BSD -based?? Well.. that explains things a bit better..

Cheers


Edited by mlord (10/04/2007 22:08)

Top
#296662 - 10/04/2007 22:12 Re: Hardware RAID Advice [Re: Cris]
Robotic
pooh-bah

Registered: 06/04/2005
Posts: 2026
Loc: Seattle transplant
Quote:
After much messing about with software RAID 5 with FreeNAS I have given up. I'm not finding it's a solution I feel my data is safe with. This could be down to the fact I am still a total Linux/BSD noob, but FreeNAS also seems to be locking up my system quite a bit so I don't think it's all my fault

I'm curious if it's the lockups or your desire for 'safety' that has steered you away from software RAID5.
If it's the lockups, the specs you quote are WAY above what I built for my NAS box, so something's wrong with your software selection or maybe execution. (not pointing fingers or laughing- I'm seriously interested)
If it's 'data safety', I'm wondering what specific points have you found to convince you hardware RAID is the way to go.

I couldn't have built my Debian NAS box without the help of the people on this forum- I'm a linux-noob. I'm very happy with how that box works, by the way. It just works and I don't need to know how or why.

Just wondering about the back-story.
_________________________
10101311 (20GB- backup empeg)
10101466 (2x60GB, Eutronix/GreenLights Blue) (Stolen!)

Top
#296663 - 11/04/2007 04:01 Re: Hardware RAID Advice [Re: Robotic]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
I'm curious if it's the lockups or your desire for 'safety' that has steered you away from software RAID5.


I have software RAID working on my firewall (also webserver etc...) and before putting anything really important on it I went through a series of tests to ensure that if a drive fails I know what to do and know it works. I did suffer a HD failure on that system and I didn't loose any of my data, although it wasn't as smooth as I would have liked.

So far I haven't been able to do the same with RAID 5 and FreeNAS. In fact at the moment I can't even get the thing to mount! So I think it's a matter of trust. I figure a hardware solution will be more able to recover from a drive failure. So I guess it's data security not the lockups.

Quote:
If it's the lockups, the specs you quote are WAY above what I built for my NAS box, so something's wrong with your software selection or maybe execution.


Yea, the specs are a little high, but it's what I have to work with. I am moving towards a solution that takes the pressure off my main clarkconnect server, as I would like to run that mail server on that soon. I think I have already decided to bin FreeNAS and stick with clarkconnect, I know that works and is stable.

Quote:
If it's 'data safety', I'm wondering what specific points have you found to convince you hardware RAID is the way to go.


I read this article. Saw the interface for the BIOS and thought "That's what I need!". If I can find a cheap enough card I think my data will be safer with a hardware solution. Am I wrong?

Cheers

Cris.

Top
#296664 - 11/04/2007 10:24 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
I figure a hardware solution will be more able to recover from a drive failure.


Just remember that "hardware solutions" are really just more software in a different, less-accessible, less-hackable, place. When drives go bad with a "hardware solution", it's more likely to "cope" without intervention, but there's more chance of losing everything too.

I have several *very* high-end "hardware" RAID boxes here, and all of them use Linux software RAID internally.

Cheers

Top
#296665 - 11/04/2007 12:57 Re: Hardware RAID Advice [Re: mlord]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
but there's more chance of losing everything too.


How come?

I think for me there is no chance of getting data back from a Linux RAID 5 as I just don't know what I am doing. Is there a distribution that would offer me a better interface so I don't need to get too tehnical if the worst happens.

Quote:
I have several *very* high-end "hardware" RAID boxes here, and all of them use Linux software RAID internally.


I am very happy with my clarkconnect software RAID, it worked very well, it's RAID 5 that seems to be the problem, but I really want to give it a go as I want the max space out of the drives I have (read : need space!).

Cheers

Cris.

Top
#296666 - 11/04/2007 13:10 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Quote:
but there's more chance of losing everything too.


How come?


Because many of them use some kind of proprietary headers on the drives, and when the "hardware" part dies, there's no easy way to get software that can understand the quirky layouts.

Cheers

Top
#296667 - 11/04/2007 14:06 Re: Hardware RAID Advice [Re: mlord]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Oh yea, big problem there, esp if I use a 2nd hand eBay card.

Well, now I am totally confused. I don't want to spend much more money on this, can anyone recommend a Linux disro I may get on with ???

Cheers

Cris.

Top
#296668 - 11/04/2007 14:31 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Oh yea, big problem there, esp if I use a 2nd hand eBay card.

Well, now I am totally confused. I don't want to spend much more money on this, can anyone recommend a Linux disro I may get on with ???

Cheers

Cris.


Well, totally sight-unseen, I would recommend Ubuntu Server. The new "Fiesty" is due for final (initial) release later this month, and would probably do an excellent job for you.

(I don't currently have any RAIDs set up around here; the big iron bits I just do manually, because I'm working at a layer or two beneath the RAID s/w. In other words, I'm not much use to you unless you can pop over with your system for an evening. )

Top
#296669 - 11/04/2007 14:35 Re: Hardware RAID Advice [Re: mlord]
RobotCaleb
pooh-bah

Registered: 15/01/2002
Posts: 1866
Loc: Austin
Out of curiosity, what would you do if he showed up with his box?

Top
#296670 - 11/04/2007 14:38 Re: Hardware RAID Advice [Re: mlord]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Mmm.. when using the regular (non "server") Ubuntu, the RAID installers are on the "alternate installation" disc images, rather than the default "live" CD images.

Dunno about the "server" edition, but if you go to download a copy, it sounds like the "alternate install" CD is the one you want for RAID.

-ml

Top
#296671 - 11/04/2007 14:42 Re: Hardware RAID Advice [Re: mlord]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
I'm not much use to you unless you can pop over with your system for an evening.


Great! I'll be on the 20:00, see you in 12 hours!

Now do you think I can check in a 4U Rackmount case as hand baggage....

On the more serious note, I will give your suggestion a try and see how I get on.

Cheers

Cris.

Top
#296672 - 11/04/2007 14:46 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
I will give your suggestion a try and see how I get on.



Ack. I'm pulling down the current daily build of "Fiesty Server" to try on one of the racks here later today or tomorrow. If I remember, I'll report back on how it looks for RAID stuff.

Cheers

Top
#296673 - 11/04/2007 16:42 Re: Hardware RAID Advice [Re: Cris]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
Personally, I would suggest running Solaris on it. Two of my biggest problems with Linux are that it regularly completely revamps large subsystems and its documentation is for shit. Both of these problems have a huge impact on RAID.

While Solaris has overhauled its RAID subsystem from time to time (twice in the last ten years, I think), the user interface has remained largely the same. In addition, its documentation is very complete and, I think, understandable.

It may be important to point out that Solaris is free of cost these days, and has been for many years. It is possible that you might have a hardware incompatibility, though. I haven't kept up to date on Solaris x86's hardware compatibility.


Edited by wfaulk (11/04/2007 18:02)
_________________________
Bitt Faulk

Top
#296674 - 11/04/2007 16:57 Re: Hardware RAID Advice [Re: wfaulk]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Two of my biggest problems with (???) are that ...

Missing noun, there.

Quote:
... it regularly completely revamps large subsystems

Ah.. MS-Windows. Okay.

Quote:
.. and its documentation is for shit.

Yup, must be MSWin.

It certainly couldn't be Linux on that first point, at least not for RAID stuff -- the interface has been stable for much of the past decade. The second point could apply to almost anything, though.

Cheers


Edited by mlord (11/04/2007 16:58)

Top
#296675 - 11/04/2007 18:07 Re: Hardware RAID Advice [Re: mlord]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
I will never say that Windows is better than Linux, but that doesn't mean that Linux doesn't change stuff out with little ramification. They might decide to swap out the RAID system tomorrow with a completely new interface. Just because they haven't yet doesn't mean they won't, and their track record doesn't inspire confidence.

That doesn't mean that any of the things they've swapped out or the things they've replaced them with are bad, just that they constantly change, and if a concern is the ability to reproduce, that might be a significant legitimate concern.
_________________________
Bitt Faulk

Top
#296676 - 11/04/2007 18:50 Re: Hardware RAID Advice [Re: Cris]
Neutrino
addict

Registered: 23/01/2002
Posts: 506
Loc: The Great Pacific NorthWest
Cris;

One of the questions that I don't believe has been asked yet is why? Why do you want to use raid5? I currently have both the Promise SX6000 and a 3ware RAID 5 controller sitting on the bench. I don't think I'll every go that route again. The cost of harddrives has drop so much that I no longer feel the need to use something as complex as raid 5 for a hardware backup solution. 320gb drives warranted for 5 years are available for $80.00 USD. Why not just go RAID1 or use a large usb drive to backup your data?

Through the course of my usage of the above two controllers I had a few problems. The promise card declared non-existent simultaneous multiple hard drive failures. Not a good thing. Deleting and recreating the array without initialization saved the day but the pucker factor was huge. The 3ware would rebuild the raid1 array I had on it from time to time. I had both a raid5 and a raid1 running on this controller.

At any rate in todays world with the low cost of storage is this really the best solution?

Top
#296677 - 11/04/2007 18:55 Re: Hardware RAID Advice [Re: wfaulk]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
that doesn't mean that Linux doesn't change stuff out with little ramification.


Huh? Linux is an operating system kernel. It didn't exist in 1991. By 1992 it was a complete OS. By 1999 many of the major subsystems (network, RAID, various filesystems, etc..) had been replaced with better implementations.

For the past decade (give or take a year), *none* of this has had any significant external regressions. Lots of new stuff, for sure, but the old stuff is still there and still works. I still do my company invoices on a WordPerfect binary from the mid-1990s.

We actually have a very firm rule requiring no userspace impact when kernel internals are rearranged.
Quote:

They might decide to swap out the RAID system tomorrow with a completely new interface

Not true. Sure, "they" might *add* another RAID implementation, but the one that's there now will also have to stay put for a very long time -- we require it.

Okay, so now look beyond the kernel itself to the whole system. Redhat is the longest lived example, and they've tried very hard to remain more-or-less "the same" in clients eyes. Which is why I dislike them, of course: their distro is still living in the late 1990s, while newer ones (like Ubuntu) with no legacy-base have been free to build in mid-2000s code.

Cheers

Top
#296678 - 11/04/2007 19:12 Re: Hardware RAID Advice [Re: Neutrino]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
Why not just go RAID1 or use a large usb drive to backup your data?


All good points! The basic idea was to create something cheap (ie using as many parts as I already have) and something that offers as near to 100% up time of data 24/7 as possible with DIY kit.

I am very very lazy when it comes to backups, I want a NAS to sit on my network to take regular automated backups from my 4 main systems in the house. A USB drive is out of the question. The HD's in this system have been taken from various other ideas for back up involving USB solutions.

As for RAID 5, I guess it looked really good on the box in effect giving me 50% extra space from 4 drives. I thought it would be easy, as I have found other RAID solutions in the past. I am starting to wonder if I should start looking at the other solutions again, but it has to sit on the network.

FreeNAS has been a bit of a disapointment, it looks really good, says it does everything I want it to, but it's not very stable yet. I think I will go with Mark's suggestion as this was also used in an article I linked to further up in the thread. I also may try clarkconnect as I am confident with that.

One thing I really wanted was a system that would boot from a USB stick. FreeNAS does this really well. Power consumption is quite important for me, and the idea of not having HD's spun up until really needed sounds like a good idea to me. Heat is an ongoing issue with my under stairs server room, which seems to be stable at the moment, but I think 5 HD's spun up will tip that balance

Cheers

Cris.

Top
#296679 - 11/04/2007 19:21 Re: Hardware RAID Advice [Re: Cris]
andym
carpal tunnel

Registered: 17/01/2002
Posts: 3996
Loc: Manchester UK
Quote:
FreeNAS has been a bit of a disapointment, it looks really good, says it does everything I want it to, but it's not very stable yet.


There's a couple of people I know that use it without any problems. Could there be a hardware problem? Their hardware is a little more antiquated than yours but other than that it seems pretty solid.
_________________________
Cheers,

Andy M

Top
#296680 - 11/04/2007 19:30 Re: Hardware RAID Advice [Re: andym]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
Could there be a hardware problem?


I don't think so.

The lock ups are pretty consistant. For example, once the RAID 5 is complete, it locks when you hit the format button, every time! I have tried different RAM, but when I pulled them out I noticed the originals were Crucial, and that this MD and CPU were my CCTV server for about a year, in this case. Never crashed once, which is pretty amazing for XP I think.

I can bring it over on Sunday if you want to take a look for me

Cheers

Cris.

Top
#296681 - 11/04/2007 19:55 Re: Hardware RAID Advice [Re: mlord]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
Well, my usual example is the NAT/FW code. How many times has that been swapped out over the years? Three? Each incompatible with the previous.

How many attempts at a /dev filesystem have there been? How often did that break existing things? Glibc incompatibilites have abounded. Gcc problems. The list goes on and on.

And I intentionally use "they", specifically because it's not a single entity. And that lack of a single entity is the cause of many of these problems. It's not as if it doesn't create some good things, too. There's a lot to be said for a lack of monopoly (though RedHat had something of a de facto monopoly for many years), but it also results in incompatibilities, reduplication of effort, etc.

And, yes, I recognize that "Linux" only really applies to the kernel. You know what I mean; don't be disingenuous. If there was a better term for what I was talking about, I'd have used it.
_________________________
Bitt Faulk

Top
#296682 - 11/04/2007 20:07 Re: Hardware RAID Advice [Re: Cris]
pim
addict

Registered: 14/11/2000
Posts: 474
Loc: The Hague, the Netherlands
Here's my pro Linux software RAID strory:

All my tunes (flac & mp3) are on a RAID-5 array of 6 250 GB drives. When one of the drives died, I replaced it with a 500 GB drive lying around. The broken drive was under warranty, and a replacement drive arrived two weeks later. I forced the 500 GB drive to failed state and inserted the replacement drive, assuming I could just rebuild the array. Then Murphy hit me: a second drive from my array made horrible noise when I applied power. It was dead. Now two drives were faulty and my array would not start, even after switching back to the 500 GB drive. The 500 GB was non-fresh and would not be considered as part of the array. Luckily mdadm had a --force option that would bring back the array to life. All my data was there and intact. I wonder whether any hardware RAID solution would save me in this situation.

Here's my con hardware RAID story:

I once lost a lot of data on an IBM ServeRAID array, where the RAID controller would give an I/O error on a sector of the virtual disk, where every drive that built the array was just fine. Recreating the array solved the condition, but the data was lost.

Pim

Top
#296683 - 11/04/2007 20:09 Re: Hardware RAID Advice [Re: wfaulk]
pim
addict

Registered: 14/11/2000
Posts: 474
Loc: The Hague, the Netherlands
Quote:
f there was a better term for what I was talking about, I'd have used it.


GNU/Linux ?

Pim

Top
#296684 - 11/04/2007 20:23 Re: Hardware RAID Advice [Re: Cris]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
I set up this page:
http://linux-raid.osdl.org/index.php/Main_Page
It was a while ago and I've not finished <blush>

I got permission to copy and modify the linux-RAID FAQ and I hang out on the linux-raid list a bit too.

I run a 1.2TB RAID5 for my myth box and a 1.2Tb RAID6 for mrs lbt's video editing habit.
Our important stuff is striped on raid0 (and then backed up using snapshots).

I have replaced 4 hard disks showing SMART failures since xmas and have another 3 to return. Currently my RAID6 is 'degraded' (ie has a dead disk) but since it can survive 2 dead disks it would still run fine if another disk died before tomorrow.

Oh, tonight I finally got around to setting up wake-on-lan too.

So I guess if you have any issues.... I've got some hands on experience and I'd be happy to help
_________________________
LittleBlueThing Running twin 30's

Top
#296685 - 11/04/2007 20:41 Re: Hardware RAID Advice [Re: wfaulk]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Well, my usual example is the NAT/FW code. How many times has that been swapped out over the years? Three? Each incompatible with the previous.

And very heavy overlap among them. For several years the kernel could still be used with any of the three methods. Gradually, the first one (ipfwadm) was dropped, then the second (ipchains) got dropped quite recently after years of "depreciation". The current userspace interface, "iptables", has been in place for about 9-10 years now. *That's* stability in this world.
Quote:

How many attempts at a /dev filesystem have there been?

The original "static /dev" from 1991 is still supported and works fine. Along the way, we picked up "devfs" as a clearly marked "EXPERIMENTAL" method, and it was carried for many years. Only a month or two ago did it finally get dropped from the bleeding edge kernels. Over the past two years, various new features in the kernel have enabled proper hotplug support, and a userspace "udev" device manager app has grown up to use those features. But again, the original 1991 method still works just fine.

Quote:
How often did that break existing things?

Zero, or near to it, unless one deliberatly switched to a different method.
Installing an entire new system, eg. Ubuntu, of course will give the new methods as default. As it should.

Quote:
Glibc incompatibilites have abounded.

That's a BIG stretch. More correctly, all modern OSs abandoned the original UNIX executable format "a.out" in the early 1990s, in favour of the newer ELF format. This also meant a new set of ELF libraries, separate from the a.out ones. Both can/do coexist. My current bleeding edge Linux kernel *still* can run both kinds of binaries. Very VERY backward compatible.

About 10 years ago, GNU libc (glibc) had a major revision, again producing a new set of library interfaces that are quite different from before. So again, a new set of libs and a new program loader (ld.so.2) got installed in parallel with the old. My old Wordperfect program still uses the old libs, on the latest Kubuntu systems. Fantastic backward compatibility there.

Quote:
Gcc problems.

That's not Linux. And it's not a problem anyway. Just continue to use whatever old version you like. Until very recently, kernel developers were all still using the ancient gcc-2.95 to build kernels. I think it still works today, though we've moved on.

Quote:
The list goes on and on.

It certainly does! Stellar backward compatibility, no major upheavals in established distros. The hot new ones like (K)Ubuntu of course install only the latest stuff by default, but the older stuff is there too.

One can definitely fault a Linux distro on many points, but not this one.

Been there, done that, still doing it.

-ml

Top
#296686 - 12/04/2007 00:31 Re: Hardware RAID Advice [Re: pim]
drakino
carpal tunnel

Registered: 08/06/1999
Posts: 7868
mdadm --force is a scary thing to use late at night trying to recover the array. For those that have never seen it, --force is buried in the manual under a ton of warnings, then when run, the binary also warns you again in a full screen of text and a second switch you have to use to then really confirm. I had to go through it back when I had one drive fail, and another was on it's way out. It let me get most of the data off the array, with only the files at the bad points being lost. I think it ended up being one CDs worth of MP3s that I had to rerip from that.

In fact, if I remember, thats when I posted here asking about having surface scans done from time to time to catch errors quicker. Double failures are the worst with RAIDs, hardware or software. I got spoiled by the Compaq RAID hardware always checking disks when they could, as it often rooted out failures way before the drive completely pitched over dead.

I've had good experiences with hardware RAID, but the type of solutions I worked with in my past job are likely way outside the price range of home use. One benefit I really remember that usually is only in the realm of software RAID is the portability of the data. A 1995 Smart Array controller could die in some server, and even if a like for like replacement wasn't available, the drives could be plugged into a 2007 Smart array (likely 7th or 8th generation by now) and still be able to read the data. A few newer features like full array expansion and such might not be available, but the data would still be in tact and could be moved off to a newer system. Sadly similar compatibility stories are rare on the hardware side.

Top
#296687 - 12/04/2007 06:34 Re: Hardware RAID Advice [Re: mlord]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
I love GNU with linux

Of course, I'm quite familiar with it (and admit to having occasional minor quarrels!)
Probably a lot like people familiar with another OS.

Quote:
Quote:
How many attempts at a /dev filesystem have there been?

The original "static /dev" from 1991 is still supported and works fine. Over the past two years, various new features in the kernel have enabled proper hotplug support, and a userspace "udev" device manager app has grown up to use those features. But again, the original 1991 method still works just fine.
Quote:
How often did that break existing things?

Zero, or near to it, unless one deliberatly switched to a different method.
Installing an entire new system, eg. Ubuntu, of course will give the new methods as default. As it should.


As a matter of fact, because I installed my machines a *looong* time ago and just slowly upgraded, swapping out graphics, RAM, CPUs, motherboards, hard disks etc over time (no 'activation' here) I only moved to udev last week.
I think I wanted better device naming for my USB memory stick.
Everything else just worked the same
I just typed:
apt-get install udev
and there it was (no reboot FWIW, just a completely different way of accessing hardware online at once).
This is under Debian though - none of this flash Ubuntu nonsense

Quote:
Quote:
The list goes on and on.

It certainly does! Stellar backward compatibility, no major upheavals in established distros. The hot new ones like (K)Ubuntu of course install only the latest stuff by default, but the older stuff is there too.



I installed Redhat 4.1 (I think) on an AMD something machine a while back (1997). Then I upgraded to Redhat 7.2 when it came out. in 2002-ish Then, since it worked, I left it serving my emails. I upgraded the kernel along the way for various reasons - it became a firewall so I suspect I did the ipxxxx things. I'm sure a lot of the code was the same as I ran in '96 - it certainly was running the same 1992 1Gb SCSI disks I liberated from the skip at SGI when I retired it. It's still in the spare room for sentimental reasons.

Also, just before I retired it, since it had an ISA motherboard I dug out an old 10Mb hard disk that I used in college on an Amstrad 2086 in about 1992-ish. It was MFM and used RLE (I think). Compiled the kernel, plugged it in and read my data.

Couldn't run the turbo-pascal - though I daresay I could have loaded an emulator and installed something...

And, as for the gcc thing - last week was yet another first for me
My first empeg kernel compile on my 2.6.18.3 (old, I know) *Intel based* kernel.
I just downloaded the 2.95 arm cross compile toolchain built on god-knows what system back in the mists of time and ran it and got a new kernel.

I wonder if the same proprietary toolchain used in Cambridge back in the day would run under Vista?
I can't remember why the player code is now totally dead but isn't it because the toolchain won't run? (really, I don't know but I thought that was the reason).

Feel free to moan about suspend/hibernate though
Or 3D-acceleration on graphics cards.
or many many other things - what's life without tribes?
_________________________
LittleBlueThing Running twin 30's

Top
#296688 - 12/04/2007 06:53 Re: Hardware RAID Advice [Re: drakino]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
Quote:
mdadm --force is a scary thing to use late at night trying to recover the array.


Yep - been there...

One of my motivations for the wiki was to try and gather best-practices to explain what you should do in this situation.

Oh, and 'this situation" is usually a catastrophic failure where you've lost multiple drives at once or in a cascade. Your array is dead, dead, dead. You have blown the redundancy you designed in (ie you lost 3+ drives in a RAID6 array).

You have however done data-recovery using ddrescue and friends and there is a chance that you can recover something - try *that* on a hardware raid

I think I'm right in saying that if you use --force it will recreate your superblocks and not touch your data.
If you get it wrong you re-run it with different parameters until it works.

**BUT**
If, for some reason the kernel thinks that your array needs 'synchronising' then it will start writing all over your data and 'goodnight Vienna'.
So it's a good idea to use --force with the 'missing' keyword to assemble the array in 'degraded' mode. That way you can try a read-only mount and see if there is anything resembling a filesystem there...

This kind of stuff is what I want to put up on the wiki after running it by the list.

The linux-raid list (http://vger.kernel.org/majordomo-info.html) is a very friendly and helpful place BTW - far less busy than lkml and much more newbie-friendly - do pop in if you have any raid or recovery questions...


PS My favourite HW raid issues: when your raid card goes titsup - how much are they? Do they even make those Adaptec raid cards you bought 4 years ago? Are the new ones compatible? What happens if you can't get a disk that's *exactly* the same anymore?
_________________________
LittleBlueThing Running twin 30's

Top
#296689 - 12/04/2007 07:05 Re: Hardware RAID Advice [Re: LittleBlueThing]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
PS My favourite HW raid issues: when your raid card goes titsup - how much are they? Do they even make those Adaptec raid cards you bought 4 years ago? Are the new ones compatible? What happens if you can't get a disk that's *exactly* the same anymore?


So it looks like my decision is quite a simple one....

Option 1 - Linux Software RAID 5: Need to learn how to use Linux at a level that allows me to recover if the worst happens, and also learn how to use Linux to actually get the thing going in the first place

Option 2 - Hardware RAID 5: Easier to set up and maintain, but if I get a card failure I could loose everything, but the data is still on the drives. A possible solution to this problem could be to buy 2 identical cards from eBay if I can find them cheap enough, so in that event all I do is swap to my back up card and away I go?

I would like to try Hardware RAID and FreeNAS as I still get to boot from USB Stick then. I think I will hunt down an old card and until I find one read your Wiki (looks great BTW!) and give software a go.

Lots of good advice and pointers here, thanks all for your help, keep 'em coming...

Cheers

Cris.

Top
#296690 - 12/04/2007 08:06 Re: Hardware RAID Advice [Re: Cris]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
Maybe look at : http://www.openfiler.com/

It's a CentOS (based on the source RedHat releases) solution but it may be too techy - not sure

If you are going for a linux based solution then I'm honestly not sure that hardware based raid is easier to setup though - anyhow, yell (or PM) if you need anything.
_________________________
LittleBlueThing Running twin 30's

Top
#296691 - 12/04/2007 08:08 Re: Hardware RAID Advice [Re: LittleBlueThing]
andy
carpal tunnel

Registered: 10/06/1999
Posts: 5916
Loc: Wivenhoe, Essex, UK
Quote:
I wonder if the same proprietary toolchain used in Cambridge back in the day would run under Vista?
I can't remember why the player code is now totally dead but isn't it because the toolchain won't run? (really, I don't know but I thought that was the reason).


I could be wrong, but as far as I know there was no such proprietary toolchain. The empeg player code is C/C++ code that runs on Linux, I have always assumed that it was also built on Linux using the standard toolchain.
_________________________
Remind me to change my signature to something more interesting someday

Top
#296692 - 12/04/2007 08:52 Re: Hardware RAID Advice [Re: andy]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
Quote:
I could be wrong, but as far as I know there was no such proprietary toolchain. The empeg player code is C/C++ code that runs on Linux, I have always assumed that it was also built on Linux using the standard toolchain.


It's a gcc 2.95 (I think) crosscompiler, running on linux-i386, targetting ARM.
The Windows components were compiled with Visual C++ 6, which does still run on Vista, so ner.
_________________________
-- roger

Top
#296693 - 12/04/2007 11:05 Re: Hardware RAID Advice [Re: Roger]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
Quote:
compiled with Visual C++ 6, which does still run on Vista, so ner.

to you too

(at last, a proper debate!)
_________________________
LittleBlueThing Running twin 30's

Top
#296694 - 12/04/2007 13:45 Re: Hardware RAID Advice [Re: mlord]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Quote:
I will give your suggestion a try and see how I get on.



Ack. I'm pulling down the current daily build of "Fiesty Server" to try on one of the racks here later today or tomorrow. If I remember, I'll report back on how it looks for RAID stuff.

Cheers

I'm building the Ubuntu Feisty system right now using yesterday's image of the "alternate" installer CD. So far, so good. I found this set of instructions that I'm using as a loose guide for this:

http://www.debian-administration.org/articles/512

There are a few things I'm trying differently, and we'll see how those work out.

My oh my, it certainly does take some time to construct a 3TB RAID5 array from scratch..

Top
#296695 - 12/04/2007 14:34 Re: Hardware RAID Advice [Re: mlord]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
Quote:

My oh my, it certainly does take some time to construct a 3TB RAID5 array from scratch..

Seriously?
Or do you mean the whole system rather than just the array?

IIRC my 1Tb arrays took seconds to create and more seconds to build a filesystem (using XFS though).

It does take a while for the background sync to finish - but you can use the array as normal in the meantime.
_________________________
LittleBlueThing Running twin 30's

Top
#296696 - 12/04/2007 15:21 Re: Hardware RAID Advice [Re: LittleBlueThing]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Quote:

My oh my, it certainly does take some time to construct a 3TB RAID5 array from scratch..

Seriously?
Or do you mean the whole system rather than just the array?



Just the array -- it does it in the background, of course, but the initial sync involves rewriting the entire contents of the entire space of the entire array. That takes a while with 3TB of disk space.

But I'm redoing it again now -- my experiment didn't pan out, and smaller physical RAID partitions will make testing failure modes a lot less time consuming later on.

So this time, the RAID5 will be only 120GB instead of 3TB.
Quote:

It does take a while for the background sync to finish - but you can use the array as normal in the meantime.

Exactly!

Top
#296697 - 12/04/2007 15:27 Re: Hardware RAID Advice [Re: mlord]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4180
Loc: Cambridge, England
Quote:
Just the array -- it does it in the background, of course, but the initial sync involves rewriting the entire contents of the entire space of the entire array. That takes a while with 3TB of disk space.

Why, though? Can't it just treat the sector contents as undefined until written to? Why isn't setup of an empty array instantaneous?

Peter

Top
#296698 - 12/04/2007 15:49 Re: Hardware RAID Advice [Re: mlord]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
Quote:

But I'm redoing it again now -- my experiment didn't pan out, and smaller physical RAID partitions will make testing failure modes a lot less time consuming later on.

So this time, the RAID5 will be only 120GB instead of 3TB.


don't forget bitmaps.

Resyncing a dirty Tb array takes seconds
_________________________
LittleBlueThing Running twin 30's

Top
#296699 - 12/04/2007 15:54 Re: Hardware RAID Advice [Re: peter]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
Quote:

Why, though? Can't it just treat the sector contents as undefined until written to? Why isn't setup of an empty array instantaneous?


(following is edited from a message from Neil Brown - this comes up in the list a fair bit).

There is an --assume-clean option which is safe for raid1 and raid6 but for raid5 it is NOT safe to skip the initial sync. It is possible for all updates to be "read-modify-write" updates which assume the parity is correct. If it is wrong, it stays wrong. Then when you lose a drive, the parity blocks are wrong so the data you recover using them is wrong.
_________________________
LittleBlueThing Running twin 30's

Top
#296700 - 12/04/2007 16:17 Re: Hardware RAID Advice [Re: LittleBlueThing]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
There's certainly a reasonable way to implement this, which would involve some sort of bitmap of written-to and not-written-to blocks, but, as I recall, Linux RAID doesn't keep any metadata on disk, so there's not any way to record this metadata. (Correct me if I'm wrong about that.) Not that all (or even any, necessarily) on-disk metadata RAID systems implement this. Recurring storage and the need to journal that data probably outweighs the cost of a one-time setup.
_________________________
Bitt Faulk

Top
#296701 - 12/04/2007 16:45 Re: Hardware RAID Advice [Re: wfaulk]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
There is now a bitmap mechanism recording dirty blocks: http://linux-raid.osdl.org/index.php/Bitmap
(Now I know how Tony feels )

Could you develop it to allow -assume-clean? Probably. Is it worth it? As you say, probably not


Edited by LittleBlueThing (12/04/2007 16:46)
_________________________
LittleBlueThing Running twin 30's

Top
#296702 - 12/04/2007 19:48 Re: Hardware RAID Advice [Re: LittleBlueThing]
drakino
carpal tunnel

Registered: 08/06/1999
Posts: 7868
Quote:
There is an --assume-clean option which is safe for raid1 and raid6 but for raid5 it is NOT safe to skip the initial sync.


Why would this be safe for raid 6, but not raid 5? Both use distributed parity for the redundancy. Raid 6 simply has a different algorithm in use across the drives to ensure any 2 disks can fail, by using 2 disks worth of space for the parity.

Top
#296703 - 12/04/2007 20:16 Re: Hardware RAID Advice [Re: drakino]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
My guess would be that the implementation of Linux's RAID6 is such that it requires a complete recalculation of the parity blocks with each write. Sounds like a massive write-performance hit (due to the requirement of needing to read that block on each disk) to me. But since RAID6 can be implemented in any way (the specification requires only that it be able to survive two simultaneous disk failures) and I don't feel like crawling through source code, I could easily be guessing wrongly.


Edited by wfaulk (12/04/2007 20:18)
_________________________
Bitt Faulk

Top
#296704 - 12/04/2007 20:48 Re: Hardware RAID Advice [Re: wfaulk]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
From the author again:
Quote:
For raid6 it is also safe to not sync first, though with the same caveat as raid1. Raid6 always updates parity by reading all blocks in the stripe that aren't known and calculating P and Q. So the first write to a stripe will make P and Q correct for that stripe. This is current behaviour. I don't think I can guarantee it will never changed.


I suppose 'that aren't known' matters. Will you often be writing data that you've just read (so blocks are in cache) or writing enough that you know all the blocks anyway.

Also see: http://www.kernel.org/pub/linux/kernel/people/hpa/raid6.pdf
_________________________
LittleBlueThing Running twin 30's

Top
#296705 - 13/04/2007 06:26 Re: Hardware RAID Advice [Re: peter]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
Quote:
Why, though? Can't it just treat the sector contents as undefined until written to? Why isn't setup of an empty array instantaneous?


Yeah, I wondered that when it took a couple of hours for my 3ware card to build a new 1.3Tb (4x500GB) RAID5 array...
_________________________
-- roger

Top
#296706 - 13/04/2007 07:03 Re: Hardware RAID Advice [Re: Roger]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
Just because I'm interested.... could you use the array whilst this was going on?
_________________________
LittleBlueThing Running twin 30's

Top
#296707 - 13/04/2007 07:57 Re: Hardware RAID Advice [Re: LittleBlueThing]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
Quote:
Just because I'm interested.... could you use the array whilst this was going on?


No. The 3ware BIOS configuration screen wouldn't let me exit until it was finished. This is an 8500-series card. Maybe the newer/higher end ones will...
_________________________
-- roger

Top
#296708 - 13/04/2007 08:47 Re: Hardware RAID Advice [Re: mlord]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
But I'm redoing it again now -- my experiment didn't pan out


Well that makes me feel a whole lot better

If you have problems it must be nearly impossible for me to do it!

Amazingly I got a knock on my door about an hour ago, it was the postman. I bid on one of the LSI cards the otherday and it's turned up already! £35 (US$65 ?) no bad!

Within the hour I have FreeNAS running, RAID5 working and the box sat on my network taking files. Certainly pain free to this point. I haven't however figured out how to tell the array is healthy to start with, and of course haven't tested it yet for drive failure/recovery.

On thing I have noticed is speed, so far this seems to be much slower than the software solution, but maybe this is because the array is still building in the background, the drive are certainly running, or maybe the card won't let them power down?

I still want to try the software approach due to some of the arguments put forward in the thread, I just need a slightly less painful experience.

Cheers

Cris.

Top
#296709 - 13/04/2007 09:54 Re: Hardware RAID Advice [Re: Cris]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
and of course haven't tested it yet for drive failure/recovery


Seems to handle it, remove a drive, alarm sounded, checked BIOS, added drive back in, auto reuilding now. I can still access all the stuff on the drive while this is happening.

I don't think this card supports any form of power saving, this may in itself rule this out as an option, how erver well it seems to be working at the moment.

Cheers

Cris.

Top
#296710 - 13/04/2007 12:04 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Quote:
But I'm redoing it again now -- my experiment didn't pan out


Well that makes me feel a whole lot better


Heh.. I just didn't follow the directions given -- tried something different (yet similar..) and it failed to boot after the install. Got distracted with another project afterwards, so I'll try it again today sometime.

Cheers

Top
#296711 - 13/04/2007 14:02 Re: Hardware RAID Advice [Re: mlord]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Quote:
Quote:
But I'm redoing it again now -- my experiment didn't pan out


Well that makes me feel a whole lot better


Heh.. I just didn't follow the directions given -- tried something different (yet similar..) and it failed to boot after the install. Got distracted with another project afterwards, so I'll try it again today sometime.



It works just fine (install/booting, that is). I "got lucky" with the cdimage I pulled down on Wednesday -- it had a broken initramfs-tools package that screwed the reboot stuff. That got fixed yesterday, and the new packages now "just work" as they should.

This is all still pre-release, for another week.
I'll try pulling/reinserting drives next.

Cheers

Top
#296712 - 13/04/2007 14:39 Re: Hardware RAID Advice [Re: mlord]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
I'll try pulling/reinserting drives next.


Manual intervention is required here. The raid tools, specifically mdadm, are crap. But they do work, so one just has to write the magic into a recipe script and then never worry about it again.

I pulled one of the four SATA drives, which had partitions that were members of the RAID1 (root filesystem) and RAID5 (/usr/local filesystem) arrays. There was a pause, while the low level libata tried to recover the failed drive, and then the system just continued working afterward.

But reinserting the drive is a current known hassle. It gets seen as an entirely different device than the one that was removed. So manual intervention is required to delete the old unit from the RAIDs, and then add the new one.

The drive I pulled was /dev/sdb, and then became /dev/sde on reinsertion.
So I had to do this to get it back into the arrays without a reboot:

mdadm /dev/md0 -f /dev/sdb1 -r /dev/sdb1 -a /dev/sde1
mdadm /dev/md1 -f /dev/sdb2 -r /dev/sdb2 -a /dev/sde2

That could easily be scripted.

But there was another wrinkle: after the drive is pulled, /dev/sdb no longer exists.
Well, sort of. It''s "half there", but udevd nuked the /dev/ entries for it.
This prevents actually issuing the above commands.

So the full sequence on a udev system is this:
---------------------------------
(1) removal:
## yank the drive, wait about 30 seconds for the fault-recovery to give up, then:

mdadm /dev/md0 -f /dev/.static/dev/sdb1 -r /dev/.static/dev/sdb1
mdadm /dev/md1 -f /dev/.static/dev/sdb2 -r /dev/.static/dev/sdb2

## reinsert the drive, wait about 30 seconds again, then:

mdadm /dev/md0 -a /dev/sdb1
mdadm /dev/md1 -a /dev/sdb2

---------------------------------
Note that with this method, the new drive gets the same name as it previously had.
But with USB discs coming and going, there's no guarantees, unfortunately.

Also, if the inserted drive is a new one, then it will need to first be partititioned correctly before doing those last two commands. The simple way, given identical drives, is to do this just before those last two commands:

dd if=/dev/sda of=/dev/sdb bs=512 count=1
hdparm -z /dev/sdb

Definitely in need of a higher level tool to simplify this.
Mind you, a simple bash script (20-50 lines?) could totally automate it all.

Cheers

Top
#296713 - 14/04/2007 05:05 Re: Hardware RAID Advice [Re: mlord]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Wow! Thanks Mark!

Well my hardware RAID recovery didn't go quite how I wanted. I didn't loose my data in the event of loosing a drive, which I did with my previous software RAID attempts. But, I couldn't get the thing to rebuild.

I can really see the disadvantage of not having any tools to monitor the array. I think that the drive would begin to recover, but when I re-booted it started over. I couldn't see a way of checking, even from the command line in freenas, what stage the rebuild was at. The only place tht told me that was the RAID's BIOS.

So I am still torn between both solutions. I like both for different reasons. I do have enough spare HD's to run 4 x 250Gb and 3 x 120Gb arrays so maybe I'll run both side by side and directly compare how I cope with recovery etc....

I could of course just drop back to a simple mirror and forget RAID5, but I really want that extra 250Gb.

Cheers

Cris.

Top
#296714 - 14/04/2007 09:44 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:

I could of course just drop back to a simple mirror and forget RAID5, but I really want that extra 250Gb.



One other thing I noticed, is that the RAID5 had substantially greater throughput on large sequential reads (like copying big files around -- .flac or .mpeg2 for instance.

On RAID1 (Linux soft) the throughput was the same as for a single drive on reads. For RAID5 it was more like a RAID0 -- 3 drives in parallel. 71Mbytes/sec vs. 210Mbytes/sec.

Cheers

Top
#296715 - 14/04/2007 09:52 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:

I could of course just drop back to a simple mirror and forget RAID5, but I really want that extra 250Gb.


Yeah, I know the feeling.

A *lot* of people seem to prefer RAID 1+0 with a four drive setup.
That's two pairs of RAID1, which are then combined into a single virtual drive
with RAID0 striping across the two pairs. Or the other way around (0+1)

Very reliable, very fast -- more so than just two pairs of RAID1 would be.
But still no extra 250GB.

Cheers

Top
#296716 - 14/04/2007 09:57 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Wow! Thanks Mark!

Well, on a related note: the Feisty bug database is presently littered with RAID roadkill. Lots of people seem to have seen similar installer issues to what I ran into, and my fix for it (intervene in the boot process to get it up, and then just update the packages to repair it all) doesn't work for most of them.

So there are/were still other issues. I think most problems were with PATA (IDE) drives rather than pure SATA configurations -- the kernel currently has two sets of device drivers (backward compatibility overlap again, Bitt) for PATA, and Ubuntu apparently has some kinks to work out in their driver selection logic.

I'll pull another image down later today, and see if things are improving or not -- and if not, then perhaps fix it for them.

Cheers

Top
#296717 - 14/04/2007 12:48 Re: Hardware RAID Advice [Re: mlord]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
Quote:
A *lot* of people seem to prefer RAID 1+0 .... Or the other way around (0+1)

Technically, RAID 1+0 (more commonly referred to as "RAID 10"), is superior to RAID 0+1 due to the number of drives that can be lost and the RAID still remain active, but, in the case of just four drives total (which is the minimum setup for these types of RAID), it degenerates to being the same.
_________________________
Bitt Faulk

Top
#296718 - 14/04/2007 14:13 Re: Hardware RAID Advice [Re: mlord]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Quote:
Wow! Thanks Mark!

Well, on a related note: the Feisty bug database is presently littered with RAID roadkill. Lots of people seem to have seen similar installer issues to what I ran into, and my fix for it (intervene in the boot process to get it up, and then just update the packages to repair it all) doesn't work for most of them.


Okay, the latest word on this is that those fixes have now made it into the installer disks as of 14-April-2007, and RAID installs now seem to be working for everyone.

Woo-Hoo!

Top
#296719 - 14/04/2007 14:37 Re: Hardware RAID Advice [Re: mlord]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
But still no extra 250GB


In an ideal world I would just go out any buy 4 x 500Gb SATA drives, a local supplier has Seagate drives at under £70 now. With this solution 1Tb would be more than enough so RAID10 would be the perfect solution.

Although speed isn't a major issue at present, I'm running and ever increasing Gigabit network at home. At the moment it's just the Macbook and one of the servers as my switch only has 2 Gigabit ports. I have noticed that even though the spec of the new NAS box is much higher than my current clarkconnect firewall, the CC box transfers are much quicker. This could be down to the intel gigabit nic I suppose, the one in the NAS is a £15 D-Link (nasty!).

If only I had put a bet on Ascot I could be straight down to pick up my SATA drives

Cheers

Cris.

Top
#296720 - 14/04/2007 14:40 Re: Hardware RAID Advice [Re: mlord]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
Okay, the latest word on this is that those fixes have now made it into the installer disks as of 14-April-2007, and RAID installs now seem to be working for everyone.


With the final release so imminent I may wait for that.

Am I right in assuming I can't boot from RAID5? Or could I do a complex setup of striping and mirroring a 1Gb partition and then RAID5 the rest? Putting a 5th drive in the system isn't something I really want to do at this stage if I can help it.

Cheers

Cris.

Top
#296721 - 14/04/2007 14:57 Re: Hardware RAID Advice [Re: Cris]
drakino
carpal tunnel

Registered: 08/06/1999
Posts: 7868
Oh, one tip if you go software side. Don't RAID the hard disk. Create partitions on the disks and RAID those, making the partition a bit smaller then the hard disk. That way, if one of your Brand X 120gig drives fails, and only a Brand Y 120g can be found later. There is a possibility Brand Y 120 gig will be a smidge smaller then Brand X, and by using a smaller setup of space on the old disk, the new one should still work into the array.

Quote:
I do have enough spare HD's to run 4 x 250Gb and 3 x 120Gb arrays


Another advantage to using partitions would be for this setup. Instead of making two separate arrays with two separate disk groups, you could make 2 arrays across the entire 7 disks. IE, 120gb in use on all 7 disks for a 720gig usable RAID chunk, then with the 130 gigs left on the 250s, turn that into a 390 gig usable array.

I'm currently doing something similar in my server. It has 3 160 drives, 1 200, and 1 500. All 5 are in the 160 gig per disk RAID. The 200 is in there because I couldn't find another 160 to replace the second failure in the array, and the spare 40gigs of space ends up being used as a quick and easy place to store a few backups of /etc and other critical areas of the root drive to allow me to restore to an earlier time. The 500 uses the spare space it has as my "unraid" share. This gets used by my laptop and Mac Mini as a backup space. I'm tempted to go grab another 2 500 gig drives and expand that unprotected space into a proper raid 5, along with expanding the 160 gig per disk raid area I have.

Top
#296722 - 14/04/2007 16:02 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:

Am I right in assuming I can't boot from RAID5? Or could I do a complex setup of striping and mirroring a 1Gb partition and then RAID5 the rest? Putting a 5th drive in the system isn't something I really want to do at this stage if I can help it.


I'm not sure on that one. GRUB cannot boot from RAID5 yet; LIlO might be able to. But neither of them would be able to handle failover situations, so, *no*.

The standard way is to have a smallish RAID1 partition for /boot (or the rootfs), and a RAID5 partition for the rest. On each drive. Just like in the article I linked earlier. Plus a few extra GRUBbies so that grub will auto-failover among the available RAID1 partitions. I haven't set that up yet, but will give it a go later. Maybe tomorrow, as The Game will be on in an hour here, and I'd like to watch it (almost) live..

Cheers

Top
#296723 - 14/04/2007 16:07 Re: Hardware RAID Advice [Re: mlord]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:

The standard way is to have a smallish RAID1 partition for /boot (or the rootfs), and a RAID5 partition for the rest. On each drive. Just like in the article I linked earlier.


That article also suggested going with a rather complex LVM setup on top of the RAID5, and then a complex partitioning setup on top of that. This also implies having the swap partition there, on the LVM on the RAID5. One plus with this, is that then swap won't go bonkers if a disk fails, thus avoiding another source of system crashes.

EDIT: LVM (Logical Volume Manager) is just a way of grouping things together into a new virtual "block device (disk)", which can then be partitioned etc.. just like a real drive (and more..). The reason you need it here, is that the RAID drivers in Linux don't currently support being partitioned themselves. So by assigning LVM to manage the entire RAID, one can then partition the LVM instead of the lower-level RAID, and it works. Hack. /EDIT.

Personally, I'd partition each drive identically: smallish RAID1 partition, largish RAID5 partition. Put /boot on the RAID1, use LVM on the RAID5. Then partition the LVM with a smallish swap area, a pair of small/medium O/S partitions, and a massive data partition.

With the two O/S partitions, you can experiment with a newer O/S version in the future, and yet still have your original working O/S version intact until all is well with the new one.

Cheers


Edited by mlord (14/04/2007 16:10)

Top
#296724 - 14/04/2007 16:14 Re: Hardware RAID Advice [Re: mlord]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:

With the two O/S partitions, you can experiment with a newer O/S version in the future, and yet still have your original working O/S version intact until all is well with the new one.


Note that an advanced user would likely have just a single O/S partition, and would not immediately allocate the rest of the LVM to the massive data area.

Instead, they would allocate from the LVM to the data area in large chunks, as the need grows. And they'd use the nifty "LVM snapshot" capability to "fork" the O/S partition when experimenting with upgrades.

Very very fancy and useful, but maybe a bit much for starting out.

Cheers


Edited by mlord (14/04/2007 16:15)

Top
#296725 - 14/04/2007 17:41 Re: Hardware RAID Advice [Re: drakino]
andy
carpal tunnel

Registered: 10/06/1999
Posts: 5916
Loc: Wivenhoe, Essex, UK
Quote:
Oh, one tip if you go software side. Don't RAID the hard disk. Create partitions on the disks and RAID those, making the partition a bit smaller then the hard disk. That way, if one of your Brand X 120gig drives fails, and only a Brand Y 120g can be found later. There is a possibility Brand Y 120 gig will be a smidge smaller then Brand X, and by using a smaller setup of space on the old disk, the new one should still work into the array.



The same thing can apply to disks even of the same model number, I had an expensive mistake where my 18GB IBM drives were a few hundreds of kilo bytes smaller than the existing disks with the same model number, making them useless as replacements in my RAID5 array.
_________________________
Remind me to change my signature to something more interesting someday

Top
#296726 - 15/04/2007 18:47 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Mmmm. I stumbled across the www.cooldrives.com site today. Wow, what a selection of interfaces, enclosures, etc. for SATA / IDE stuff.

Cheers

Top
#296727 - 15/04/2007 22:51 Re: Hardware RAID Advice [Re: Cris]
Seth
new poster

Registered: 15/04/2007
Posts: 15
Cris,
I too got the same LSI Megaraid card--couple days ago too. Having similar issues. I figured I'd explore recovery scenarios before I have to use them for real in the future. I too had trouble getting the array to rebuild, even in Windows with Power Console Plus. The best it ever did was rebuild to 50% overnight and freeze at 50%. Other times it just failed, or went to about 28% and then started over. I even tried rebuilding just within the WebBIOS utility and that failed.

Oh, yeah, and to clarify, all I did was take one of the working drives, "fail" it, and then attempt to make it rebuild. Same drive that was just working, so that eliminates all those "sizing" issues and other random gotchas. No reason it shouldn't rebuild.

It is SLOOOOOOOOW rebuilding even when making progress. An hour to rebuild about 5% or so--I have brand new 300GB drives. I also tried adjusting the rebuild speed. I can handle it taking a half day to rebuild as long as it actually rebuilds to 100%. But 3 consecutive failures is not acceptable.

Is this the problem you were having?

For me at this point, I'm saying no to MegaRAID. I starting to think mlord is right. Software RAID may be the way to go. I've always been under the impression that a hardware solution was better, or at least easier. But I'm rethinking that opinion now. The i4 card was made before all the gigantic and faster 7200rpm drives, so I'm guessing that has something to do with it. I also don't like the lie in the product datasheet on LSI's site. It says auto-resume rebuild upon reboot. That does NOT happen, and if you search their knowledge base, it actually says that the card restarts the rebuild process FROM THE BEGINNING upon reboot. That is just a simple lie. I suppose the "hot swap" claim is probably also a lie at this point.

I'm curious what others think of this card or other similar cards. Should I just throw it in the garbage?

Top
#296728 - 15/04/2007 23:38 Re: Hardware RAID Advice [Re: Seth]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
The thing to remember here is, RAID is complicated enough that *nobody* really does it in hardware. It's all software (or flash-based software a.k.a. "firmware").

So.. would you rather trust your data and time to a single factory burn of the software, or rely on something that can-be / is bug-fixed as the need arises, a.k.a. "software RAID". And if the O/S does the (software) RAID, then it can also (in theory) do a superior job of scheduling the I/O to each individual drive (= faster I/O).

The answer may also depend upon one's choice of O/S.

Cheers

Top
#296729 - 16/04/2007 01:48 Re: Hardware RAID Advice [Re: mlord]
Seth
new poster

Registered: 15/04/2007
Posts: 15
My problem is that I'm not a Linux god. So without rehashing the debate about backwards compatibility, I'll just say that suppose there are some new updates to the RAID implementation and I end up screwing up my data. Maybe I just don't choose the right compile flags to ensure compatibility or something like that.

What I am saying is that I don't quite trust myself with software ;-).

Also what if I have a catastrophic hardware failure (fried motherboard or something) and have to rebuild everything. I KNOW that it is possible to rebuild everything to get the data off the drives, but maybe I'm not capable of doing it without making a mistake somewhere along the way.

I'm not saying I'm an idiot, just that I've come to understand that I figure things out after stubbing my toe a couple times. I don't want to stub my toe at a critical time in the recovery process.

Not disagreeing with you at all, I just have a few concerns given my ability (or lack of) to fly by the seat of my pants without making any mistakes.

Top
#296730 - 16/04/2007 04:18 Re: Hardware RAID Advice [Re: Seth]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
My problem is that I'm not a Linux god.


Me too!

I have only had the chance to test the MegaRAID i4 once so far. What I did notice is that if you reboot it does start the rebuild right from the start again. ie if you were at 5% and need to reboot your machine for some reason, when it all comes back up the MegaRAID starts again from 0%, but I did find it started again automatically.

I'm really glad I hve given "hardware" RAID a try. I can now really see Mark's point, you do stand more of a chance of getting your data back with software RAID if you know what you are doing.

My problem is I don't know what I am doing, yet

The other problem with the MegaRAID i4 is that it keeps the disc spun up 100% I left the system on over the weekend while I was away from home, nobody as been using the NAS and the drives are a full temp. For this reason alone the i4 is going back on eBay. This can't be good for the type of drive I am using here!

Cheers

Cris.

Top
#296731 - 16/04/2007 11:31 Re: Hardware RAID Advice [Re: Seth]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
suppose there are some new updates to the RAID implementation and I end up screwing up my data. Maybe I just don't choose the right compile flags to ensure compatibility or something like that.

Nope. No need for the FUD. That cannot happen. To screw your data, you have to work much harder than that, and issue commands that do it on purpose, and bypass the warnings that most of them give you.

Quote:
Also what if I have a catastrophic hardware failure (fried motherboard or something) and have to rebuild everything. I KNOW that it is possible to rebuild everything to get the data off the drives, but maybe I'm not capable of doing it without making a mistake somewhere along the way.

Nothing to do with Linux there -- same issues regardless.

Backups are safer than RAID.

Cheers

Top
#296732 - 16/04/2007 15:01 Re: Hardware RAID Advice [Re: mlord]
Seth
new poster

Registered: 15/04/2007
Posts: 15
I'm not throwing out FUD, and my comments about rebuilding aren't aimed at Linux. I'm talking about my own comfort level based on what I know and what I don't know. In this case, there is a lot I don't know, and freely admit it.

I use Linux enough to be able to get around. I don't know it forwards and backwards in my sleep. I'm just saying I'm thinking through scenarios, and suppose I have to rebuild the array on a different box. I know it is possible, and I can probably figure out how to do it. But I don't know the process (today), and don't know what risks there might be. I see posts about using the force flag and various "pucker" factors in this thread and hence some of my fears. I figure plugging the array into the new box would require forcing it online.

And the RAID actually is my backup, so if the whole thing goes belly up, I'm actually most likely fine.

Top
#296733 - 16/04/2007 15:37 Re: Hardware RAID Advice [Re: Cris]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
Quote:
Quote:
My problem is that I'm not a Linux god.


I'm really glad I hve given "hardware" RAID a try. I can now really see Mark's point, you do stand more of a chance of getting your data back with software RAID if you know what you are doing.

My problem is I don't know what I am doing, yet


That's the thing about 'community'

The linux-raid list has regular 'help' emails and they almost all get sorted - I think the only caveat is "don't mess if you don't know what you're doing - just ask".

Also, you mentioned wanting to power down the disks in your raid. I'm not sure how well that works 'out of the box' under linux. You may need to do some fiddling.
_________________________
LittleBlueThing Running twin 30's

Top
#296734 - 16/04/2007 15:44 Re: Hardware RAID Advice [Re: Seth]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
Quote:
I'm not throwing out FUD

No - I think Mark was heading it off before it became fear etc ie FUD.


Quote:
I see posts about using the force flag and various "pucker" factors in this thread and hence some of my fears. I figure plugging the array into the new box would require forcing it online.

That's the problem with being fully honest - I did try to emphasise that the --force stuff was to do recovery in situations where you were already royally FUBARed.

Let me put it this way, if and only if your system burns to the ground and is eaten by ants then linux raid *may* need to use the --force command Better?

Moving to another machine - business as usual.
Disk failure - business as usual.
Switch from RedHat to SUSE to Debian - business as usual.
Upgrade kernel - business as usual.
etc etc

One of my biggest problems with linux RAID way back when was when a disk failed and I didn't notice for months - the thing just kept going.
_________________________
LittleBlueThing Running twin 30's

Top
#296735 - 16/04/2007 16:11 Re: Hardware RAID Advice [Re: LittleBlueThing]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
Also, you mentioned wanting to power down the disks in your raid. I'm not sure how well that works 'out of the box' under linux. You may need to do some fiddling.


In FreeNAS it was one feature that worked really well, it must cache part/all of the file table (or whatever?) as you could pretty much browse the NAS and the drives spun up when a file was requested, but I never did get to the point of mave many files on it.

The only slight problem was that the WD drive sins up much slower than the seagate driver, that wa causing some retry errors in the log files, but from my point of view it worked pretty well.

I think the Software RAID argument is winning the battle at the moment, like you said if I get stuck all I need to do is ask

Cheers

Cris.

Top
#296736 - 16/04/2007 17:11 Re: Hardware RAID Advice [Re: Seth]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
I'm not throwing out FUD

Just trying to prevent it from being interpreted as such when somebody stumbles over this thread via Google.

You actually sound to me like a pretty competent user of this stuff -- more power to ya!

Cheers

Top
#296737 - 17/04/2007 03:07 Re: Hardware RAID Advice [Re: mlord]
Neutrino
addict

Registered: 23/01/2002
Posts: 506
Loc: The Great Pacific NorthWest
This has been an interesting thread. As I mentioned at the beginning of it I am no longer interested in going with RAID5. I personally believe that with the huge decrease in the cost of storage that it has lost it’s luster as a viable choice for the home user. I have no desire to put all of my eggs in one basket. I have been down that road where I had a terabyte of data all in a RAID5 configuration and if anything happens, I am hosed.

Considering Linux as a software RAID solution, I wonder how many people in the world have intimate knowledge of Linux on a par with some of the contributors of this forum? I’ll bet not many. This is the reason why software RAID5 is not an option for me. Some of you make it sound so easy just to do whatever needs to be done to fix whatever problems might occur. For me this is an impossibility, heck, I was absolutely overjoyed when I managed to get Ubuntu to run on my Intel box equipped with a 8800 graphics cards. I think this was a “let’s celebrate and go get Sushi, my treat” night.

I would also like to use a NAS so as not to have to have a specific computer running in order to see the data that I want to share across my network. I also don’t want to spend a lot of money finding a solution. I have 6 300GB PATA drives just sitting around that came out of the RAID5. Just for fun I have chosen to play with this –
http://www.newegg.com/Product/Product.aspx?Item=N82E16833124036
along with these to put my drives in - http://www.newegg.com/Product/Product.aspx?Item=N82E16817716034
There are a ton of firmware hacks that will allow you to run multiple drives off of the USB ports or RAID1 if you want. There is also a hardware hack that will let you double the speed of the NAS just by removing a diode.

A raid system is not the answer to total data protection. Corrupted data is just as corrupted regardless of the number of copies you have. To truly protect your data I believe you must do periodic backups to devices that are generally kept offline. I think this is the only way that you can be somewhat sure that your data is safe. Periodic backups, there is no substitute.

I really enjoy this forum; the intellectual power here is truly amazing. You guys rock.
_________________________
No matter where you might be, there you are.

Top
#296738 - 17/04/2007 03:46 Re: Hardware RAID Advice [Re: Neutrino]
Cris
pooh-bah

Registered: 06/02/2002
Posts: 1904
Loc: Leeds, UK
Quote:
Periodic backups, there is no substitute.


But what too ???

The task of backing up 700Gb to DVD doesn't really appeal, and thenttrying to find what I want would be almost impossible.

My RAID NAS will be my backup device, all the information will be on other machines as well, it will just be a nice big pool of space to run the automatic backups to.

I think the cost of any other solution, in both money and time, would be to great for me.

Cheers

Cris.

Top
#296739 - 17/04/2007 04:33 Re: Hardware RAID Advice [Re: Cris]
Neutrino
addict

Registered: 23/01/2002
Posts: 506
Loc: The Great Pacific NorthWest
True, backing up 700GB IS a hassle but how much of this really changes? If you are talking media than I would expect that most of this is static data. This is the case in my situation. 1TB of movies and audio with the occasional movie or itunes download added. After the initial backup of the data I need to only add to it whatever I add to my collect. This can be done periodically. To what, would be off line drives that I only plug in to add data to. I believe that any backup scheme that is constantly online and powered up has the risk failing. Yes, it is true that there is a chance that the drives I keep offline might someday not power up when I need them but the amount of time that they are online is so small that I doubt that will happen. As far as cost is concerned what does a TB cost these days, $250.00 USD? (Based on 3 320GB drives at $80.00 USD each)

Also I didn't realize that you where planning on using your raid as a backup storage for data that is kept online in another location. When I was using RAID5 it was the total repository of all of my media. There was no other backup. At the time I considered RAID5 to be sufficient unto itself. This was a mistake. How are you planning on monitoring the status of your drives? I used the Promise Superswap 1000 drive bays that gave constant condition status via LED’s. Even with this I lost my data once. You think it’s a hassle copying 700GB? How about re-ripping a TB? I guess I have been burned too many times. What I purpose is probably extreme.
_________________________
No matter where you might be, there you are.

Top
#296740 - 17/04/2007 11:56 Re: Hardware RAID Advice [Re: Neutrino]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:

Considering Linux as a software RAID solution, I wonder how many people in the world have intimate knowledge of Linux on a par with some of the contributors of this forum? I’ll bet not many. This is the reason why software RAID5 is not an option for me. Some of you make it sound so easy just to do whatever needs to be done to fix whatever problems might occur.

Again, the issues with administrating a RAID array are not unique to Linux. RAID5 failures will likely be just as complex to recover from regardless of the underlying O/S, or whether it's "hardware" or "software" RAID.

Just trying to keep things clear here for posterity (aka. "google").

Top
#296741 - 17/04/2007 12:23 Re: Hardware RAID Advice [Re: mlord]
Neutrino
addict

Registered: 23/01/2002
Posts: 506
Loc: The Great Pacific NorthWest
Quote:
Again, the issues with administrating a RAID array are not unique to Linux. RAID5 failures will likely be just as complex to recover from regardless of the underlying O/S, or whether it's "hardware" or "software" RAID.

Just trying to keep things clear here for posterity (aka. "google").


I completely agree, I did lose a TB of RAID5 data on a Windows box. In fact it sounds to me like recovery might actually be easier under Linux IF you possess the knowledge to do what must be done to recover it.
_________________________
No matter where you might be, there you are.

Top
#296742 - 17/04/2007 12:37 Re: Hardware RAID Advice [Re: Neutrino]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
Quote:

When I was using RAID5 it was the total repository of all of my media. There was no other backup. At the time I considered RAID5 to be sufficient unto itself. This was a mistake. How are you planning on monitoring the status of your drives? I used the Promise Superswap 1000 drive bays that gave constant condition status via LED’s. Even with this I lost my data once. You think it’s a hassle copying 700GB? How about re-ripping a TB? I guess I have been burned too many times. What I purpose is probably extreme.

There's a lot being said here; some is linux specific but most, I think, is generic.

Clearly if you are experienced in one OS and move to another then there's a learning curve - that said:

Setting up:
Linux RAID is simple and generally just works.
BIOS or windows RAID is probably a touch simpler and generally just works.
Not much in it.

Monitoring:
Linux RAID is simple and generally just works via email. [My machines talk to me if a disk fails.]
BIOS or windows RAID may use email, LEDs, beep or just notify you on reboot (not good for a server) .
Not much in it.

Disaster:
For an average user it is feasible to recover a Linux RAID systems when things go badly wrong (and I'm talking about massive hardware failure here!) but it needs to be dug into very deeply and can become very complex indeed. [However a few hours of hair pulling may be a lot easier than a 1Tb restore!]

For an average user if you have an unexpected hardware failure with windows or BIOS RAID then you are screwed. You would need to spend real cash (many hundreds of $) with a forensic data recovery firm to stand a chance of getting your data back - or just start restoring.

Huge difference.


Another problem area is how people thing about risk, backups and resilience.

In other words : do I bother with RAID or do I just have backups.

It's complex; but in general RAID reduces the likelihood of needing to restore from backups - Neutrino you are 100% spot on when you say it is not sufficient to replace backups. However it does almost eliminate the need to restore from backups for disk failures.

That's worth restating:
* I'll (almost) never need to restore from backup due to disk failure.

Here, I think the problem you will face Neutrino is that you are almost *certain* to lose all your data and have to restore from backup. (I assume you have 1Tb of data on 3x320Gb disks = no resilience). Your system is 3 times more fragile than a single disk system. Assuming drives are 95% likely to die in 6 years (even spread) then your system is 95% likely to die within 2 years - I hope you have an OCD tendency to backup!



FWIW I have multiple data stores.
One is for music and films that I have recorded or ripped. The backup is the original media. When (not if) a drive fails, I just replace it. I doubt I will ever have to re-rip. If I didn't have a resilient setup then I'd have to re-rip (or restore) every time a drive failed.

Another store is a simple striped array.
I hold important data on there and it backs up incrementally onto a separate striped array using snapshots.
Having the backup online all the time means I can monitor the health and replace drives.


I hope this comes across in the spirit it's intended

A few years ago I had a disk failure on the disk with all my photos on it.
I bought a replacement hard disk and did a disk copy from my backup to my new disk.
Sadly I swapped the device names and overwrote my backup with a blank disk. I felt a horrible cold chill and the hairs on the back of my neck stood up as I realised that I'd lost years worth of photos.
Luckily I was a sysadmin at the time and by 'habit', as soon as the disk failed I made a copy of the backup to some free space. It took me 10 seconds to remember that I had the copy - they were not pleasant seconds.
I did however just do a quick 'copy' and the timestamps on all those files show the date and time of the near-disaster.

So the reason I bother with this RAID stuff is because I really worry about the 'normal' people who are going to lose their digital memories and I try to help out
I would like to see Linux RAID get a nicer frontend (for home use) and better disaster recovery etc - it will happen.
_________________________
LittleBlueThing Running twin 30's

Top
#296743 - 17/04/2007 13:10 Re: Hardware RAID Advice [Re: LittleBlueThing]
matthew_k
pooh-bah

Registered: 12/02/2002
Posts: 2298
Loc: Berkeley, California
Quote:

So the reason I bother with this RAID stuff is because I really worry about the 'normal' people who are going to lose their digital memories and I try to help out

And I was agreeing with your post until I read that. And it came right after an example of PEBKAC which always shows the true weakness of raid.

It doesn't matter how reliable your raid array is if you press the delete key.

I've said it before here. I've said it at work (though my boss doesn't really take it to heart). I've told family (who give me funny looks when I mention "parity"). Raid is not a backup solution. Raid is a high availability solution.

Reriping from original media is a backup plan, and perfectly viable. You can't do that with digital memories. If it's worth saving, it's worth saving twice.

I'll get off my soap box now. But I won't hesitate to get back on it if someone starts implying raid is a backup solution.

Matthew
PS: Perhaps I'm reading your post wrong, and you mean preserving memories is the reason you help other people with their RAID recovery. I still stand by my rant about backups, even if it doesn't apply to you.

Top
#296744 - 17/04/2007 13:29 Re: Hardware RAID Advice [Re: matthew_k]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
We agree...

Quote:
Quote:

So the reason I bother with this RAID stuff is because I really worry about the 'normal' people who are going to lose their digital memories and I try to help out

And I was agreeing with your post until I read that.
...
I'll get off my soap box now. But I won't hesitate to get back on it if someone starts implying raid is a backup solution.

Matthew
PS: Perhaps I'm reading your post wrong, and you mean preserving memories is the reason you help other people with their RAID recovery. I still stand by my rant about backups, even if it doesn't apply to you.


You did misunderstand - but it was a long post and they're not always the clearest things to read or write.

For "RAID stuff" substitute "RAID and backup stuff".

RAID is bloody useful but it is not a backup solution.

PS Somehow it has more of an "Oh f*ck, did I really do that" feeling than "Kodak moments"
_________________________
LittleBlueThing Running twin 30's

Top
#296745 - 18/04/2007 18:56 Re: Hardware RAID Advice [Re: Cris]
Robotic
pooh-bah

Registered: 06/04/2005
Posts: 2026
Loc: Seattle transplant
I've been following this thread's progression with great interest and I can really empathize with Cris and the others. I am also a Linux challenged individual.
With the help of some members on the board I built a NAS out of an old computer of mine. (the thread is around somewhere... ah- here it is.)
It works great (thanks to all!) but this thread has me thinking I need to further my understanding of the device.
For example- I have no idea how to check the health of the unit or recover from a failed drive. I'll certainly have to learn about that soon.

Anyway, I just saw the For Sale posts of Neutrino's and wondered if the following scenario is a possibility under Linux:
Use an IDE expansion PCI card (I'm thinking most often such cards are sold as 'raid controllers', but I haven't investigated this at all, so I may be going in the wrong direction) to allow more drives to be connected to the motherboard. Instead of combining the drives as under 'hardware raid', just leave them as individual drives and let the linux raid use them. You could boot off of one of the motherboard's IDE headers and if *that* drive went south, then reinstalling linux shouldn't be a big deal.

What I'm thinking about is this- in my homebrew NAS, the three drives are partitioned such that the linux install is strewn over all of the units (I'm not even sure exactly how it was set up, so forgive my lack of detail and understanding). It seems that any one failed drive requires that the linux install be recreated before the raid5 array could be rebuilt from the remaining drives.
It just seems that if I want to protect myself against failed drives, then treating each drive as a single partition and single element in the array should make things 'easier'. Sounds less complex to me, but then again I don't know what I'm talking about and that's why I'm asking...
Am I off base?

My kit is soo old that my motherboard won't boot from PCI or USB. That's why I was thinking I could expand the capacity for drives with a card.
Hmmm- maybe if I want to localize my linux install but keep my hardware setup as-is I could boot and run from a purpose-built live CD. ?? again, I'm going off on tangents that I'm not sure are really needed.

Hey, if you got this far I really want to thank you for reading! LOL
_________________________
10101311 (20GB- backup empeg)
10101466 (2x60GB, Eutronix/GreenLights Blue) (Stolen!)

Top
#296746 - 18/04/2007 19:02 Re: Hardware RAID Advice [Re: Robotic]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
For a system like that, I would do mostly what you've already figured. Boot from the onboard IDE with a dedicated drive (or compactflash card or DiskOnChip module etc..), and then use the PCI SATA controller to implement a RAID array in software.

The choice would likely fall down to RAID1 (best redundancy, simplest recovery, lowest capacity) and RAID5 (much faster reads, higher capacity, most complex recovery [but not *that* complex!]).

If the onboard IDE is a real drive, then I'd install the O/S there. If the onboard IDE device is a flash thingie, then I'd likely just put the /boot partition there, and everything else onto the RAID array. In fact, the latter is what I often recommend.

Cheers

Top
#296747 - 18/04/2007 20:45 Re: Hardware RAID Advice [Re: Robotic]
Neutrino
addict

Registered: 23/01/2002
Posts: 506
Loc: The Great Pacific NorthWest
Prices are negotiable!
_________________________
No matter where you might be, there you are.

Top
#296748 - 18/04/2007 20:55 Re: Hardware RAID Advice [Re: Neutrino]
Neutrino
addict

Registered: 23/01/2002
Posts: 506
Loc: The Great Pacific NorthWest
Last night I was playing around and downloaded FreeNAS. I have an extra PC that is really quite an overkill for this but I used it anyway just to take a look at it. FreeNAS is a cinch to setup. Took maybe 30 minutes start to finish to have a NAS on the network. I only hooked up 1 drive just for testing and it went without a hitch. It took 4 minutes and 39 seconds to transfer a 1GB file across the LAN to the NAS. Network utilization ran from 15 to 65%. Not too bad really. I can see why this can be a popular alternative. I should have my Slug tomorrow. I'll set that up and see how it compares. It will provide a much less energy hungry solution but I have no idea what the speeds are going to be like. With a 266mhz clock I have my doubts.
_________________________
No matter where you might be, there you are.

Top
#296749 - 18/04/2007 21:24 Re: Hardware RAID Advice [Re: mlord]
Robotic
pooh-bah

Registered: 06/04/2005
Posts: 2026
Loc: Seattle transplant
Quote:
If the onboard IDE is a real drive, then I'd install the O/S there. If the onboard IDE device is a flash thingie, then I'd likely just put the /boot partition there, and everything else onto the RAID array. In fact, the latter is what I often recommend.

Cheers

Thanks, Mark!

What if I install Linux on an old 1GB microdrive through those CF/IDE adaptors like in the CF empeg thread!
(Just kidding- I'd expect performance would suffer. Meanwhile, I've got a couple of useless microdrives hanging around.)
_________________________
10101311 (20GB- backup empeg)
10101466 (2x60GB, Eutronix/GreenLights Blue) (Stolen!)

Top
#296750 - 18/04/2007 21:34 Re: Hardware RAID Advice [Re: Robotic]
matthew_k
pooh-bah

Registered: 12/02/2002
Posts: 2298
Loc: Berkeley, California
Quote:
Meanwhile, I've got a couple of useless microdrives hanging around.)

Open them up and frame them... I've never seen the inside of one, but I've always considered them an amazing technical accomplishment.

Matthew

Top
#296751 - 18/04/2007 22:45 Re: Hardware RAID Advice [Re: Robotic]
drakino
carpal tunnel

Registered: 08/06/1999
Posts: 7868
Quote:
Use an IDE expansion PCI card (I'm thinking most often such cards are sold as 'raid controllers', but I haven't investigated this at all, so I may be going in the wrong direction) to allow more drives to be connected to the motherboard.


I'm using two different PCI IDE controllers in my server currently, one that only does normal drive access, and one that offers some sort of raid protection. The RAID card has a mini bios setup program to setup the RAID, and if you never go in there and set it up, it just runs as a normal IDE controller. Linux sees both controllers the same way, and just accesses the disks directly.

Top
#296752 - 18/04/2007 22:59 Re: Hardware RAID Advice [Re: drakino]
Neutrino
addict

Registered: 23/01/2002
Posts: 506
Loc: The Great Pacific NorthWest
I'm using a Promise Ultra/133 2 channel ide / pci card to access more PATA drives. This can be had cheap! One of these days all my PATA drives (7) will have died and I will move on....
_________________________
No matter where you might be, there you are.

Top
#296753 - 18/04/2007 23:04 Re: Hardware RAID Advice [Re: Neutrino]
Neutrino
addict

Registered: 23/01/2002
Posts: 506
Loc: The Great Pacific NorthWest
OK, I added a second drive to the FreeNAS sytem and made a RAID1 system. One drive is connected via the secondary channel of IDE1 and the other is through a USB port. Not a conventional construct but it seems to work. Transfer time for a 1GB file has increased to 5:36. Interesting....
_________________________
No matter where you might be, there you are.

Top
#296754 - 19/04/2007 00:52 Re: Hardware RAID Advice [Re: matthew_k]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Quote:
Meanwhile, I've got a couple of useless microdrives hanging around.)

Open them up and frame them... I've never seen the inside of one, but I've always considered them an amazing technical accomplishment.



Presenting.. a GeoCaching "Travel Bug" in my honour: the actual item that was inside my camera at the 2003 Empeg Owners' Meet:



I tore the cover off, and later left it as loot in a geocache. Another local cacher retrieved it, and converted it into a "travel bug" named "Mlord's Micro Drive" (TBAF86).

Cheers


Attachments
297879-md.jpg (161 downloads)



Edited by mlord (19/04/2007 00:57)

Top
#296755 - 19/04/2007 00:56 Re: Hardware RAID Advice [Re: Robotic]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:

What if I install Linux on an old 1GB microdrive through those CF/IDE adaptors like in the CF empeg thread!
(Just kidding- I'd expect performance would suffer. Meanwhile, I've got a couple of useless microdrives hanging around.)


Heh.. they'd be just fine for the /boot drive, but I didn't find them terribly reliable -- my original died after a small jolt, as did it's replacement.

Still, nifty little things. Would you sell me one, and if so, for how much? (I miss my old ones).
My email address is in my BBS profile.

Cheers!

Top
#296756 - 19/04/2007 04:45 Re: Hardware RAID Advice [Re: mlord]
Robotic
pooh-bah

Registered: 06/04/2005
Posts: 2026
Loc: Seattle transplant
Quote:
Still, nifty little things. Would you sell me one, and if so, for how much? (I miss my old ones).

You've got mail!
_________________________
10101311 (20GB- backup empeg)
10101466 (2x60GB, Eutronix/GreenLights Blue) (Stolen!)

Top
#296757 - 19/04/2007 06:19 Re: Hardware RAID Advice [Re: Seth]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
I forgot to comment on this :
Quote:
I'm curious what others think of this card or other similar cards. Should I just throw it in the garbage?

I wouldn't use their RAID capability - but if you have one then I'd use it as a multi-drive controller card.

Actually, you may or may not know that many cheap RAID cards are just simple multi-channel controllers - as this tongue-in-cheek FAQ says...
_________________________
LittleBlueThing Running twin 30's

Top
#296758 - 19/04/2007 07:21 Re: Hardware RAID Advice [Re: LittleBlueThing]
tman
carpal tunnel

Registered: 24/12/2001
Posts: 5528
Quote:
Actually, you may or may not know that many cheap RAID cards are just simple multi-channel controllers - as this tongue-in-cheek FAQ says...

The Promise PATA "RAID" cards were actually their normal PATA controllers but with a different firmware and a resistor moved to change the chip ID. If you changed the resistor then you could reflash it with the RAID firmware. Not really that useful tho...

Top
#296759 - 19/04/2007 08:12 Re: Hardware RAID Advice [Re: LittleBlueThing]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
Quote:
many cheap RAID cards are just simple multi-channel controllers


Yeah, it's often all in the drivers, with maybe some acceleration features on the card.

It's usually better to use the card in JBOD mode (just a bunch of disks), and rely on the OS's software RAID. On Linux, anyway -- I've never tried Windows' software RAID.
_________________________
-- roger

Top
#296760 - 19/04/2007 21:16 Re: Hardware RAID Advice [Re: Roger]
Neutrino
addict

Registered: 23/01/2002
Posts: 506
Loc: The Great Pacific NorthWest
I got "the slug" today. You gotta love newegg. Great service, fast shipping. Anyway, it won hands down on speed against FreeNas on my simple test of writing a 1gb file to it. I was able to write the same file to the slug that I did to the Freenas box in just 3:23. This is a full minute faster. It's as cute as a bug too. Pretty cool for 80 bucks.
_________________________
No matter where you might be, there you are.

Top
#296761 - 19/04/2007 23:26 Re: Hardware RAID Advice [Re: Neutrino]
Folsom
member

Registered: 12/08/2001
Posts: 175
Loc: Atlanta
I don't suppose you have a Kill-A-Watt or something similar to measure the power drain of the slug, do you? Can you tell me what the power supply is rated to get an upper bound?

I would like to get a NAS but I want low power. I was going to look at a DNS-323, but I would prefer something that took external USB drives rather than internal drives.

Top
#296762 - 20/04/2007 04:22 Re: Hardware RAID Advice [Re: Folsom]
Neutrino
addict

Registered: 23/01/2002
Posts: 506
Loc: The Great Pacific NorthWest
It uses the same 5vdc @ 2amp powersupply as most everything else I've bought from Linksys lately so it must not draw more than 10watts. I would expect half that is more like it. It runs pretty much stone cold.
_________________________
No matter where you might be, there you are.

Top
#296763 - 20/04/2007 04:27 Re: Hardware RAID Advice [Re: Neutrino]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
Quote:
along with these to put my drives in - http://www.newegg.com/Product/Product.aspx?Item=N82E16817716034


Hmm. That looks like a rebranded IcyBox (or vice versa).

I've got a couple of those, and I found them a bit flimsy. I've also got a couple of Enermax external enclosures, which are a bit more solid.
_________________________
-- roger

Top
#296764 - 25/04/2007 01:53 Re: Hardware RAID Advice [Re: LittleBlueThing]
Seth
new poster

Registered: 15/04/2007
Posts: 15
Quote:
I wouldn't use their RAID capability - but if you have one then I'd use it as a multi-drive controller card.

Problem is that I cannot seem to get that card to present an individual disk as just a single volume. The interface isn't that intuitive, but I've tried just about everything and it seems that I have to create some sort of array between 2 or more disks before I can actually get to the step of creating a volume. If I don't create a volume, then the drive is essentially invisible to the system. Otherwise I DID think of doing that, I just haven't figured it out with that card yet.

Top
#296765 - 25/04/2007 02:01 Re: Hardware RAID Advice [Re: Seth]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
The MegaRAID® i4 IDE RAID Controller Configuration Software Guide has a section on "Using a pre-loaded IDE drive as-is". That's probably what you want.
_________________________
Bitt Faulk

Top
#296766 - 07/05/2007 19:31 Re: Hardware RAID Advice [Re: wfaulk]
Seth
new poster

Registered: 15/04/2007
Posts: 15
Looks like I was trying the impossible. I had a RAID 5 array defined as the first drive and was trying to make a fourth drive a single drive. According to that I have to do the opposite....

Thanks for pointing that out.

Top
#296767 - 04/06/2007 19:06 Re: Hardware RAID Advice [Re: Cris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:

I can really see the disadvantage of not having any tools to monitor the array. I think that the drive would begin to recover, but when I re-booted it started over. I couldn't see a way of checking, even from the command line in freenas, what stage the rebuild was at. The only place tht told me that was the RAID's BIOS.



cat /proc/mdstat

or even:

watch -n1 cat /proc/mdstat


Edited by mlord (04/06/2007 19:06)

Top
#296768 - 20/06/2007 02:39 Re: Hardware RAID Advice [Re: mlord]
Robotic
pooh-bah

Registered: 06/04/2005
Posts: 2026
Loc: Seattle transplant
Quote:
...nifty little things...

Quote:
2 Pack of Hitachi 4GB Microdrive for $24.99 Shipped

on woot today- missed it!
_________________________
10101311 (20GB- backup empeg)
10101466 (2x60GB, Eutronix/GreenLights Blue) (Stolen!)

Top
#296769 - 24/08/2007 10:48 Re: Hardware RAID Advice [Re: mlord]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
I found this set of instructions that I'm using as a loose guide for this:

http://www.debian-administration.org/articles/512


Somebody PM'd me this week, asking why that article designated one "drive" as a "hot spare" in a RAID1 array.

I very much prefer such questions to be asked in the forums/threads, like now, rather than in PM -- because this way everyone gets to see the question and contribute to the answers, and it's all nice and searchable afterward.

So, back to the question: why a "hot spare" for RAID1? Well, normally that would not make much sense to most people, as the entire concept of RAID1 is a bunch of "hot spares" kept up-to-date in parallel. But in this case, the RAID1 was just a small partition of each drive, and the bulk of the space was in a RAID5 array. A "hot spare" makes good sense for RAID5, and so that's why they did it that way.

The idea is that when a RAID5 drive fails, the system suddenly has no redundancy, so it becomes imperative to configure/update a new drive immediately --> the hot spare.

And since that article was treating all drives more or less the same, they decided to "hot spare" the RAID1 partition of that same drive for consistency. So that the drive could be removed/replaced/whatever at any time without impacting either of the two RAIDs.

There's also a typo in that article, where in the section dealing with RAID5 they accidently use "RAID1" in one of the instructions:

    Now you will want to choose Create MD device again.

    * Multidisk device type: RAID5
    * Number of active devices for the RAID5 array: 3
    * Number of spare devices for the Raid1 array: 1

Cheers


Edited by mlord (24/08/2007 10:54)

Top
#296770 - 24/08/2007 14:26 Re: Hardware RAID Advice [Re: mlord]
andy
carpal tunnel

Registered: 10/06/1999
Posts: 5916
Loc: Wivenhoe, Essex, UK
Surely a hotspare for RAID1 has more uses that just that. If if you have three disk and use all of them in your RAID1 array then the OS has to write to all three disks all the time.

If you make the third disk a hotspare instead then you won't have to take the overhead of writing to the third disk normally, but if one of the other disk fails a rebuild to the hotspare will start without any intervention.
_________________________
Remind me to change my signature to something more interesting someday

Top
#296771 - 24/08/2007 14:35 Re: Hardware RAID Advice [Re: andy]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
... and you can have a hot spare shared between multiple arrays.
_________________________
LittleBlueThing Running twin 30's

Top
#296772 - 24/08/2007 14:52 Re: Hardware RAID Advice [Re: LittleBlueThing]
andy
carpal tunnel

Registered: 10/06/1999
Posts: 5916
Loc: Wivenhoe, Essex, UK
Does that work in the same way on Linux software RAID as it does on hardware RAID, i.e. can does the hot spare have to have the right partitions it place or can it be a unpartitioned disk ?

(my RAID1 Debian box doesn't have any hotspares yet)
_________________________
Remind me to change my signature to something more interesting someday

Top
#296773 - 25/08/2007 23:15 Re: Hardware RAID Advice [Re: andy]
Seth
new poster

Registered: 15/04/2007
Posts: 15
I was the offender..... :0 Sorry about that.

I think the thing that I didn't understand was what the config was actually doing in an operational sense. Andy sort of touched on it here.

The article had three drives in the RAID1 array. Since RAID1 is mirroring, I've never seen 3 drives in a mirrored array. So I was wondering if one drive is actually mirrored to the other 2 in the array.

Then on top of that, the RAID1 array had a spare drive..... So three active, 1 spare. I wasn't sure what that meant it was doing.

So it sounds like this...someone correct me if I am wrong:

- All writes to the boot partition are mirrored to two other drives.
- If any one of the three drives fails, the fourth will automatically join the array so that the first partition is still mirrored to two others.
- If the first drive fails, then the system would boot off drive two which would then be mirrored to drive three and four.

As an aside, the main thing that I didn't like about the article was that it presumed with just 4 drives that you want a hot spare. Well, if I wanted half my total drive capacity, I'd just use RAID1 in the first place. I'm using RAID5 to get more storage space and have it be resilient. I have a spare sitting on my shelf and am not too lazy to plug it in if I need it :-).

Which brings me to another question. Anyone know of any drive monitoring software that can detect drive failures and send an email or an alert of some sort. For my needs, that is sufficient...I don't need a hot spare if I can get notification when there is a problem.

Top
#296774 - 25/08/2007 23:21 Re: Hardware RAID Advice [Re: Seth]
Seth
new poster

Registered: 15/04/2007
Posts: 15
Forgot the other question I had. Any recommendations on encrypted file systems?

Top
#296775 - 26/08/2007 07:29 Re: Hardware RAID Advice [Re: Seth]
andy
carpal tunnel

Registered: 10/06/1999
Posts: 5916
Loc: Wivenhoe, Essex, UK
Quote:

Which brings me to another question. Anyone know of any drive monitoring software that can detect drive failures and send an email or an alert of some sort.


"aptitude install smartmontools" though you need a relatively recent setup for that to work with RAID (i.e. it doesn't work on Debian Sarge, but it does on Etch).
_________________________
Remind me to change my signature to something more interesting someday

Top
#296776 - 26/08/2007 15:10 Re: Hardware RAID Advice [Re: andy]
LittleBlueThing
addict

Registered: 11/01/2002
Posts: 612
Loc: Reading, UK
and, of course, mdadm runs in monitoring mode and sends emails like these:

Code:
This is an automatically generated mail message from mdadm
running on haze

A DegradedArray event had been detected on md device /dev/md0.

Faithfully yours, etc.

P.S. The /proc/mdstat file currently contains the following:

Personalities : [raid1] [raid6] [raid5] [raid4]
md1 : active raid5 sdd1[0] sda1[2] sdc1[1]
390716672 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md0 : active raid1 sdc2[1]
3783232 blocks [2/1] [_U]

unused devices: <none>

_________________________
LittleBlueThing Running twin 30's

Top
#296777 - 26/08/2007 18:06 Re: Hardware RAID Advice [Re: Seth]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:

The article had three drives in the RAID1 array. Since RAID1 is mirroring, I've never seen 3 drives in a mirrored array. So I was wondering if one drive is actually mirrored to the other 2 in the array.


RAID1 arrays of four drives are actually quite common in the commercial systems I work on here. These people are really *paranoid* about their data, and don't want it ever to be exposed with no hot backup online. With only 2-drive RAID1, when one drive fails they would then be without a live backup until a new drive got "built" back into the RAID1. With three drives, no such problem. With four drives, even safer.

Quote:
So it sounds like this...someone correct me if I am wrong:

- All writes to the boot partition are mirrored to two other drives.
- If any one of the three drives fails, the fourth will automatically join the array so that the first partition is still mirrored to two others.
- If the first drive fails, then the system would boot off drive two which would then be mirrored to drive three and four.


Yup, that's the idea. Many modern BIOSs will automatically do that last step (boot from first available drive) in particular.

-ml

Top
#296778 - 26/08/2007 20:40 Re: Hardware RAID Advice [Re: mlord]
andy
carpal tunnel

Registered: 10/06/1999
Posts: 5916
Loc: Wivenhoe, Essex, UK
Quote:

Yup, that's the idea. Many modern BIOSs will automatically do that last step (boot from first available drive) in particular.



Does anyone have any idea on how I can actually test that step in my Linux software RAID setup. I have tested disconnecting the primary drive, but that doesn't seem to be testing quite the same case of the actual failure of one drives to spin up. I can't work out how to simulate that case though.
_________________________
Remind me to change my signature to something more interesting someday

Top
#296779 - 26/08/2007 21:24 Re: Hardware RAID Advice [Re: andy]
Seth
new poster

Registered: 15/04/2007
Posts: 15
How about just disconnect the power. I know, that isn't quite the same thing, but I've struggled with that same question myself--how to test real life failure scenarios.

Top
#296780 - 26/08/2007 22:25 Re: Hardware RAID Advice [Re: Seth]
andy
carpal tunnel

Registered: 10/06/1999
Posts: 5916
Loc: Wivenhoe, Essex, UK
Disconnecting the power to the drive isn't the same thing at all, it is no different to disconnecting the SATA cable, which I have tested.
_________________________
Remind me to change my signature to something more interesting someday

Top
#296781 - 27/08/2007 00:13 Re: Hardware RAID Advice [Re: andy]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
Quote:
Disconnecting the power to the drive isn't the same thing at all, it is no different to disconnecting the SATA cable, which I have tested.


There's no good way to test every possibility. Fortunately, with most drive (it seems), failure to spin-up means the drive will also fail to complete the "IDENTIFY" handshake, and a good BIOS will then assume the drive is not present and try the next.

But there's always a possible pissy spot there, and without a failsafe BIOS (source code generally required to verify that..), one never knows..

Cheers

Top
#296782 - 31/08/2007 04:36 Re: Hardware RAID Advice [Re: andy]
Seth
new poster

Registered: 15/04/2007
Posts: 15
Quote:
Disconnecting the power to the drive isn't the same thing at all

That is why I said it wasn't really the same thing, but how else are you going to test that without hacking a BIOS to control how a drive initializes at boot?

Top
#296783 - 31/08/2007 06:23 Re: Hardware RAID Advice [Re: Seth]
andy
carpal tunnel

Registered: 10/06/1999
Posts: 5916
Loc: Wivenhoe, Essex, UK
Quote:
but how else are you going to test that without hacking a BIOS to control how a drive initializes at boot?


I don't know, that was the whole point of asking if it was possible and if so how !
_________________________
Remind me to change my signature to something more interesting someday

Top
#296784 - 31/08/2007 10:06 Re: Hardware RAID Advice [Re: andy]
tman
carpal tunnel

Registered: 24/12/2001
Posts: 5528
Quote:
Quote:
but how else are you going to test that without hacking a BIOS to control how a drive initializes at boot?


I don't know, that was the whole point of asking if it was possible and if so how !

I'd say a rubber mallet would work...

Top
#296785 - 31/08/2007 12:17 Re: Hardware RAID Advice [Re: tman]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
Quote:
I'd say a rubber mallet would work...


Angle-grinder? Through the platters?
_________________________
-- roger

Top
#296786 - 26/10/2007 02:52 Re: Hardware RAID Advice [Re: andy]
Seth
new poster

Registered: 15/04/2007
Posts: 15
Quote:
I don't know, that was the whole point of asking if it was possible and if so how !


I had some unrelated issues on another system here at home which brought this thread back to mind. You were asking about different boot scenarios and how the BIOS will handle them. Without going into the details of my issues here, let me say that I think there is no way to deal with every possible contingency to ensure the system boots properly.

Here are a couple situations I am quite confident will prevent your system from booting despite all the efforts to make everything redundant:
1) Drive spins up and identifies itself to the BIOS but is not functional, or at least not really functional enough to boot from
2) Drive is the master (or is on a SCSI bus) and is malfunctioning to the point of making the channel/bus unuseable. I had this happen where one drive (the second on the bus) went bonkers (not even a real crash), and prevented the other two drives from [effectively] communicating with the controller. And when this happened, half of the boot attempts hung without even giving a "non-system disk" error.
3) Any other type of failure that allows the BIOS to see the presence of the disk, but disk is not accessible.

All of those will likely give you one of those "non-system disk or disk error" messages on boot up. The best failure scenario is where the drive is completely dead or non readable by the BIOS, in which case it most likely will move onto the next device. As I understand it, since drives became auto-detectable by the BIOS years ago, this mechanism actually works by storing the parameters in a sector on the disk, and the BIOS reads that. So if the drive really fails to spin up, the BIOS can't possibly identify it.

It occurred to me that one possible way of testing how your BIOS reacts to this type of scenario where the device exists, has power, but is "not spinning up" is to test something similar but different. If you had two drives on different IDE channels, you could deliberately misconfigure the first to be a slave instead of master on its channel, or something of that nature. If that doesn't work, maybe there is something else along those lines you could do to essentially accomplish the same thing. Then you would see what the BIOS does in that circumstance, i.e. move onto the next drive or just choke.

Just some thoughts I wanted to pass along.

BTW, for those interested, I continue to be more and more a fan of this software RAID approach using a generic Linux distro. I was enamored first by the hardware RAID, then by the specialty NAS OS (with software RAID) approaches, for different reasons. I thought it nice to have a good customized NAS GUI with slick performance graphs and so forth, but in the end none of those options stacked up to just good ol Linux. Far more robust and resilient than anything else I have experimented with. Alas, I have to do some of the customization work and configuration, but in the end I have what I need and it works like I want it to.

Top
Page 1 of 5 1 2 3 4 5 >