Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Page 2 of 2 < 1 2
Topic Options
#356702 - 06/12/2012 19:40 Re: FlexRaid [Re: BartDG]
Taym
carpal tunnel

Registered: 18/06/2001
Posts: 2504
Loc: Roma, Italy
Originally Posted By: Archeon
Originally Posted By: Taym
It allows redundancy and volumes across physical HDD at the file level, and supposedly HDD removed from a Storage Space pool are readable independently.

This is not true, AND it's the big difference between the Drive Extender of WHS and Storage Spaces.

If that is the case, that's a big step back from DE. However, I did not read anything clear on this in their very documentation (hence my "supposedly"). Have you actually tested this? My guess is that a drive is not immediately readable but it becomes so after few clicks in Disk Manager. Just a guess.

Quote:
AFAIK, there is also no redundancy with Storage Spaces, only drive pooling, making this even more dangerous.

There actually is. It can be configured up to double drive redundancy. This is clearly stated by MS. I think SS would not even make sense without redundancy.
_________________________
= Taym =
MK2a #040103216 * 100Gb *All/Colors* Radio * 3.0a11 * Hijack = taympeg

Top
#356704 - 06/12/2012 19:46 Re: FlexRaid [Re: Taym]
BartDG
carpal tunnel

Registered: 20/05/2001
Posts: 2616
Loc: Bruges, Belgium
Originally Posted By: Taym
Originally Posted By: mlord
RAID arrays don't deal well with power failures, and take forever to rebuild/self-check


Oh, ok, sure. I did miss the point of the thread. smile
I also agree that RAID-5 id definitely not a good solution for home, and I see why looking into something better.

RAID5 is indeed too dangerous to use, for the reason Tom has explained. That's why a lot of 'hardcore' home users started using RAID6, where two drives may fail. This is course doesn't help with the slow rebuild times and scrubbing etc... It also means you'll need at least 4 drives and a lot of drive space simply for parity calculations so you won't have that much free space left. (if you only use 4 disks, you'll loose half of your disk capacity, which makes it rather costly)

If I was to use a 'regular' RAID system, I'd use ZFS (which isn't 'regular RAID' in any sense, but you probably get my drift). ZFS allows for up to triple parity, but here also double is more than sufficient. The big plus about ZFS is that it's self-healing, is a copy-on-write filesystem and because of this can use snapshots which are very handy.

The only reason I didn't start using ZFS is because, as I understand it, it's not possible to add disk space to a created RAIDZ array in a drive pool. It is possible though to add disks to the drive pool and to create additional RAIDZ arrays, but that's not exactly what I want. Like most users, I simply want to see one big volume, containing all my disk space. The big plus of ZFS is it's robustness and it's speed.

But of course there is a downside to ZFS as well, it being the demands of ZFS are pretty high if you want to use it to the best of it's capabilities, especially the memory demands (and this is especially true if you want to use de-duplication). On the other hand, it allows you to use an SSD cache drive which speeds things up considerably. But on the other side, I use a gigabit network at home. The theoretical maximum speeds is 125 MB/s (if I don't use any link aggregation, which I could do but don't). So if the server it capable of providing that speed, it's sufficient, because I don't think I'll benefit much from having SSD-type speeds available if the network can't keep up with it.

So all this has led me back to unRAID. But now I start reading stories again of 50 hours of 'preclearing' drives (making new drives ready for use with unRAID - a mandatory part of the procedure) before you can start to use it, or days to calculate the first parity calculation (so it takes days before you are somewhat protected). Also, the speed of unRAID isn't all that good. I read about speeds of 40-50 MB/s which I find not very impressive.

In other words - I'm still not quite sure which route I'll take. My demands are rather simple though: I want a NAS system which will provide me with speedy file transfers (+100 MB/s), drive pooling (with easy expansion when full) and some form of protection (don't worry, I'll still keep a separate backup as well). As far as I can tell, there is no do-it-yourself software available yet which gives me all those options. Commercial NAS'es like Synology also are not an option, both because of their price and because they still use old RAID technology and no drive pooling.
_________________________
Riocar 80gig S/N : 010101580 red
Riocar 80gig (010102106) - backup

Top
#356705 - 06/12/2012 20:03 Re: FlexRaid [Re: Taym]
BartDG
carpal tunnel

Registered: 20/05/2001
Posts: 2616
Loc: Bruges, Belgium
Originally Posted By: Taym

If that is the case, that's a big step back from DE. However, I did not read anything clear on this in their very documentation (hence my "supposedly"). Have you actually tested this? My guess is that a drive is not immediately readable but it becomes so after few clicks in Disk Manager. Just a guess.

No, not tested it myself because I don't want to be the guinea pig. But you can easily find horror stories about these new SS, just Google for them. It seems SS is still very young technology which may get better with new revisions, but for now, it's probably not a good idea to use.

Originally Posted By: Taym

Quote:
AFAIK, there is also no redundancy with Storage Spaces, only drive pooling, making this even more dangerous.

There actually is. It can be configured up to double drive redundancy. This is clearly stated by MS. I think SS would not even make sense without redundancy.

You're right, I take that back. There is redundancy, like there was with Drive Extender. You can choose to mirror the data or use parity. But unlike with Drive Extender, it's no longer possible to mirror only one specific folder. This was very handy: stuff that was important you mirrored, less important stuff you didn't. The benefit of this was clear: you saved disk space. Since you cannot do this with SS any more, you'll need a lot more disk space for redundancy. Of course, this all is of little help when the entire SS goes bad. This is known to happen now and then, so much even that I wouldn't risk using it for now, not until there is an option to actually recover data from a failed SS.
_________________________
Riocar 80gig S/N : 010101580 red
Riocar 80gig (010102106) - backup

Top
#356706 - 06/12/2012 20:42 Re: FlexRaid [Re: BartDG]
Taym
carpal tunnel

Registered: 18/06/2001
Posts: 2504
Loc: Roma, Italy
How big a volume would you like to get, to begin with?


Edit:
Horror Stories: I've been looking around the web, but all I read was pretty trivial mistakes, I think. I am not sure it is a too young technology at its core features (reliability above all). It rather seems to be so in terms of GUI and user experience. But again, these are just words, I have no direct experience with SS. Two or three old USB disks should be enough to answer few basic questions, though. I'll post what I find out.


Edited by Taym (06/12/2012 20:48)
_________________________
= Taym =
MK2a #040103216 * 100Gb *All/Colors* Radio * 3.0a11 * Hijack = taympeg

Top
#356712 - 07/12/2012 01:04 Re: FlexRaid [Re: BartDG]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14496
Loc: Canada
Originally Posted By: Archeon
RAID5 is indeed too dangerous to use, for the reason Tom has explained. That's why a lot of 'hardcore' home users started using RAID6, where two drives may fail. This is course doesn't help with the slow rebuild times and scrubbing etc... It also means you'll need at least 4 drives and a lot of drive space simply for parity calculations so you won't have that much free space left. (if you only use 4 disks, you'll loose half of your disk capacity, which makes it rather costly)

The thing about "4 drives" is that this is often the limit for an enclosure or chassis. Sure, there are larger (and smaller) units, but four is pretty common.

So with four drives, just use two of them straight-up (or with mhddfs), and then use the other two as 100% backup copies.

So two data drives, and a full backup. Obviously better than RAID6 w/o a backup, and better than RAID1, because errors/deletions don't get mirrored until you tell them to get mirrored.

Top
#356713 - 07/12/2012 02:36 Re: FlexRaid [Re: mlord]
tanstaafl.
carpal tunnel

Registered: 08/07/1999
Posts: 5549
Loc: Ajijic, Mexico
Originally Posted By: mlord
So with four drives, just use two of them straight-up (or with mhddfs), and then use the other two as 100% backup copies.
Which is exactly what I'm doing, except it's five drives: three and two.

C:> and D:> backed up to an internal E:> drive; and F:> backed up to an external L:>.

G:>, H:>, and I:> are used for temporary USB plugins (Ipod, flash drive, camera, whatever) and J:> and K:> are external backups saved off-premises.

It works for me. YMMV.

tanstaafl.
_________________________
"There Ain't No Such Thing As A Free Lunch"

Top
#356721 - 07/12/2012 13:33 Re: FlexRaid [Re: tanstaafl.]
Taym
carpal tunnel

Registered: 18/06/2001
Posts: 2504
Loc: Roma, Italy
I have had a server always on at home since the days of Fidonet BBSs (I used to run one), and so I am used to have a machine always on in my home that I can rely on.
Currently, my home server hosts few websites, my data, my gf's data, my picture and music collection, and more stuff.
Since I don't keep (yet) a video collection in it, 2TB seem to be enough so far, even though my recent interest in photography is (not so) slowly pushing me to considering an expansion.

In any case, I have a RAID-1 storage system including system and data volumes.

In addition to that, I have three more "on-line" 2TB disks that I use for automatic daily, weekly, monthly backups, which, over time, I found are more than enough to address any deleted-by-mistake issue.

And finally, I have two "off line" USB disks I use for manual backup.

This has worked very well so far. I keep considering an on-line backup but I never got myself to getting the whole system setup, even though I like Crashplan. My bad.

Now, besides the obvious advantage of eliminating the down time in case of a system disk failure and consequent restore from backup (I never gave up the mentality of having the home server up and running at all times, even though in these days that is not as important to me as it used to be), the RAID-1 allows me to expand my drives quickly and with just 2 reboots (as I don't have hot plug storage in my case). If it wasn't for that, I would probably reconsider a simpler model.
On the other hand, my home server also serves a purpose of testing, experimenting, and fun, so some "unnecessary" redundancy is welcome anyway.

In fact, as Mark here recommends a good 3TB disk, I'll probably get two. Also because people in here convinced me that RAW pictures are obviously a good thing to keep. smile
_________________________
= Taym =
MK2a #040103216 * 100Gb *All/Colors* Radio * 3.0a11 * Hijack = taympeg

Top
#356724 - 07/12/2012 18:58 Re: FlexRaid [Re: Taym]
canuckInOR
carpal tunnel

Registered: 13/02/2002
Posts: 3212
Loc: Portland, OR
Originally Posted By: Taym
Also because people in here convinced me that RAW pictures are obviously a good thing to keep. smile
Unless the picture sucks. No point in keeping the RAW file for a blurry picture. smile

Top
#356725 - 07/12/2012 19:54 Re: FlexRaid [Re: canuckInOR]
tanstaafl.
carpal tunnel

Registered: 08/07/1999
Posts: 5549
Loc: Ajijic, Mexico
Originally Posted By: canuckInOR
Unless the picture sucks. No point in keeping the RAW file for a blurry picture.
Oh, surely you're mistaken.

Just last week on one of the CSI television shows, I saw them digitally enhance a low-resolution picture from an ATM security camera so that they could read the license plate on the car of a fleeing felon that was six blocks away. In the dark. During a rainstorm. At 80 miles an hour.

Surely you could accomplish the same thing with Lightroom or Photoshop.

Couldn't you?

smile

tanstaafl.
_________________________
"There Ain't No Such Thing As A Free Lunch"

Top
#356726 - 07/12/2012 21:12 Re: FlexRaid [Re: canuckInOR]
Taym
carpal tunnel

Registered: 18/06/2001
Posts: 2504
Loc: Roma, Italy
Originally Posted By: canuckInOR
Originally Posted By: Taym
Also because people in here convinced me that RAW pictures are obviously a good thing to keep. smile
Unless the picture sucks. No point in keeping the RAW file for a blurry picture. smile

smile
Actually... Time, and good algorithms,may prove you wrong sooner than later! smile
I read something about it on dpreview a while back, and Google seems to return some software titles, already available. Admittedly, I am not so sure a RAW file is needed for that.
_________________________
= Taym =
MK2a #040103216 * 100Gb *All/Colors* Radio * 3.0a11 * Hijack = taympeg

Top
#356731 - 08/12/2012 14:56 Re: FlexRaid [Re: Taym]
RobotCaleb
pooh-bah

Registered: 15/01/2002
Posts: 1866
Loc: Austin

Top
#356758 - 10/12/2012 18:14 Re: FlexRaid [Re: Taym]
canuckInOR
carpal tunnel

Registered: 13/02/2002
Posts: 3212
Loc: Portland, OR
Originally Posted By: Taym
Originally Posted By: canuckInOR
Originally Posted By: Taym
Also because people in here convinced me that RAW pictures are obviously a good thing to keep. smile
Unless the picture sucks. No point in keeping the RAW file for a blurry picture. smile

smile
Actually... Time, and good algorithms,may prove you wrong sooner than later! smile
I read something about it on dpreview a while back, and Google seems to return some software titles, already available. Admittedly, I am not so sure a RAW file is needed for that.

True enough -- de-blur is a pretty good PhD topic, and I have seen some pretty decent results from some of those systems. If your blurry shot is the only shot you've got, it might be worth keeping.

Top
Page 2 of 2 < 1 2