When I started using zfs a while back, I got the impression that setting my home server up with mirror sets rather than some kind of zraid would offer the most reliable setup for my data. My data is just what you''d expect on a home lan... no real commercial value involved. I''ve since created 2 zpools beyond rpool each with a single mirror set. I happened to notice someones'' config posted here recently where a single zpool was made up of several mirror sets. From: Andreas H?schler <ahoesch at smartsoft.de> Subject: Replacing disk in zfs pool Newsgroups: gmane.os.solaris.opensolaris.zfs To: zfs-discuss at opensolaris.org Date: Fri, 9 Apr 2010 10:58:16 +0200 Message-ID: <099E714D-43B6-11DF-83FB-000393CA0072 at smartsoft.de> I hadn''t even thought of such a setup, but wonder now if that would have been a better way to go. My needs are small, and the zfs server acts mostly as NAS for home lan. I''ve been thinking the mirrors on the zfs server were the final stopping place for my backups. I''m thinking the mirrors are reliable enough that I don''t do even more backups of the backup zpools. I mean other than auto snapshots. I''m thinking a crippled mirror can be recovered rather than needing a backup of it. And that short of 2 mirrored disks dieing at the same time. I''m in pretty good shape. Am I way wrong on this, and further I''m curious if it would make more versatile use of the space if I were to put the mirrored pairs into one big pool containing 3 mirrored pairs (6 discs) So where they had been separate pools, where one might fill up while another stayed fairly empty, if they were all in a single pool none would fill up until they all filled up.
On Fri, April 9, 2010 14:38, Harry Putnam wrote:> I happened to notice someones'' config posted here recently where a > single zpool was made up of several mirror sets. > > From: Andreas H??schler <ahoesch at smartsoft.de> > Subject: Replacing disk in zfs pool > Newsgroups: gmane.os.solaris.opensolaris.zfs > To: zfs-discuss at opensolaris.org > Date: Fri, 9 Apr 2010 10:58:16 +0200 > Message-ID: <099E714D-43B6-11DF-83FB-000393CA0072 at smartsoft.de> > > I hadn''t even thought of such a setup, but wonder now if that would > have been a better way to go.Probably; unless you need different performance out of the two, or something.> My needs are small, and the zfs server acts mostly as NAS for home > lan.That''s the job mine does; keeping all those photos, and a little music.> I''ve been thinking the mirrors on the zfs server were the final > stopping place for my backups. I''m thinking the mirrors are reliable > enough that I don''t do even more backups of the backup zpools. > > I mean other than auto snapshots. > > I''m thinking a crippled mirror can be recovered rather than needing a > backup of it. And that short of 2 mirrored disks dieing at the same > time. I''m in pretty good shape. > > Am I way wrong on this, and further I''m curious if it would make more > versatile use of the space if I were to put the mirrored pairs into > one big pool containing 3 mirrored pairs (6 discs)Well, my own thinking doesn''t consider that adequate for my own data; which is not identical to thinking you''re actually "wrong", of course. Issues I see include: Flood, fire, foes, bugs, user error. "rm -rf /" will destroy your data just as well on the mirror as on a single disk, as will hacker breakins. OS and driver bugs can corrupt both sides of the mirror. And burning your house down, or flooding it perhaps (depending on where your server is; mine''s in the basement, so if we flood, it gets wet), will destroy your data. I make and keep off-site backups, formerly on optical media, moving towards external disk drives.> So where they had been separate pools, where one might fill up while > another stayed fairly empty, if they were all in a single pool none > would fill up until they all filled up.Yes, that''s the advantage. I''m running three mirror vdevs in one data pool. -- David Dyer-Bennet, dd-b at dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info
Mirrored sets do protect against disk failure, but most of the time you''ll find proper backups are better as most issues are more on the order of "oops" than "blowed up sir". Perhaps mirrored sets with daily snapshots and a knowedge of how to mount snapshots as clones so that you can pull a copy of that file you deleted 3 days ago. :) If your especially paranoid a 3 way mirror set with copies set to 2. =) -- This message posted from opensolaris.org
On Fri, 9 Apr 2010, Harry Putnam wrote:> > Am I way wrong on this, and further I''m curious if it would make more > versatile use of the space if I were to put the mirrored pairs into > one big pool containing 3 mirrored pairs (6 discs)Besides more versatile use of the space, you would get 3X the performance. Luckily, since you are using mirrors, you can easily migrate disks from your existing extra pools to the coalesced pool. Just make sure to scrub first in order to have confidence that there won''t be data loss. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
"David Dyer-Bennet" <dd-b at dd-b.net> writes: [...]>> Am I way wrong on this, and further I''m curious if it would make more >> versatile use of the space if I were to put the mirrored pairs into >> one big pool containing 3 mirrored pairs (6 discs) > > Well, my own thinking doesn''t consider that adequate for my own data; > which is not identical to thinking you''re actually "wrong", of course. > > Issues I see include: Flood, fire, foes, bugs, user error. "rm -rf /" > will destroy your data just as well on the mirror as on a single disk, as > will hacker breakins. OS and driver bugs can corrupt both sides of the > mirror. And burning your house down, or flooding it perhaps (depending on > where your server is; mine''s in the basement, so if we flood, it gets > wet), will destroy your data.Yeah there is all that, but in my case the data is also on the other machines in bits and pieces over several machines. Is yours only on the zfs server? An example here might be my photo/music collection. It resides on a windows XP pro machine where I have the tools I use to tinker with it. I back it up to the zfs server, but what is on the server is always a bit older (between backups) than the current version on the windows machine. So if the zfs server were to be beamed up to another solar system, I''d still have the latest greatest version on the windows machine. Anyway losing my entire house and several machines would leave me with much bigger problems than losing my photo/music collection. The shelter I''d be living in wouldn''t have room for several machines. Nor would I have money to spend on such luxuries.> I make and keep off-site backups, formerly on optical media, moving > towards external disk drives.I''d be interested to hear about that. If you think its OT here feel free to write me direct (reader AT newsguy DOT com) I have something like a terabyte of data on the server. Man I''d really hate to try to back that up to optical media. Even to external hard drive would be a major time sync. Just backing up the 80 or so GB of Photos/music to optical medial is a nasty undertaking. I quit doing that when it grew past 15 gb or so. [...] thanks for the other (snipped) input.
Bob Friesenhahn <bfriesen at simple.dallas.tx.us> writes:> On Fri, 9 Apr 2010, Harry Putnam wrote: >> >> Am I way wrong on this, and further I''m curious if it would make more >> versatile use of the space if I were to put the mirrored pairs into >> one big pool containing 3 mirrored pairs (6 discs) > > Besides more versatile use of the space, you would get 3X the > performance.That speed up surprises me. Can you explain briefly how that works? If that is a bit much to ask here, maybe a pointer to specific documentation?> Luckily, since you are using mirrors, you can easily migrate disks > from your existing extra pools to the coalesced pool. Just make sure > to scrub first in order to have confidence that there won''t be data > loss.Would that be in pairs, or can you show a generalized outline of how that would be done. Again, a documentation pointer would be good too. Is it wise to have rpool as the migration destination, making it the only pool?
Richard Jahnel <richard at ellipseinc.com> writes: [...]> Perhaps mirrored sets with daily snapshots and a knowedge of how to > mount snapshots as clones so that you can pull a copy of that file > you deleted 3 days ago. :)I''ve been doing that with the default auto snapshot setup, but hadn''t noticed a need to mount a snapshot as a clone in order to be able to pull a file. I''ve just sought out most recent snapshot with the file and copied it. I''m not sure now if I''ve retrieved lost files, or just got an older version that way... but I think both. Can you explain a bit why you need to mount a snapshot as clone?
On Sat, 10 Apr 2010, Harry Putnam wrote:>>> Am I way wrong on this, and further I''m curious if it would make more >>> versatile use of the space if I were to put the mirrored pairs into >>> one big pool containing 3 mirrored pairs (6 discs) >> >> Besides more versatile use of the space, you would get 3X the >> performance. > > That speed up surprises me. Can you explain briefly how that works?It is quite simple. With three sets of mirrors in the pool, the data is distributed across the three mirrors. There is 3X the hardware available for each write. There is more than 3X the hardware for each read since either side of a mirror may be used to satisfy a read.> If that is a bit much to ask here, maybe a pointer to specific > documentation?That would be too much to ask since it is clear that you did not spend more than a few minutes reading the documentation. Spend some more minutes. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Bob Friesenhahn <bfriesen at simple.dallas.tx.us> writes:> On Sat, 10 Apr 2010, Harry Putnam wrote: > >>>> Am I way wrong on this, and further I''m curious if it would make more >>>> versatile use of the space if I were to put the mirrored pairs into >>>> one big pool containing 3 mirrored pairs (6 discs) >>> >>> Besides more versatile use of the space, you would get 3X the >>> performance. >> >> That speed up surprises me. Can you explain briefly how that works? > > It is quite simple. With three sets of mirrors in the pool, the data > is distributed across the three mirrors. There is 3X the hardware > available for each write. There is more than 3X the hardware for each > read since either side of a mirror may be used to satisfy a read.Thanks for your comments. Its always so much easier when someone explains in plain english.>> If that is a bit much to ask here, maybe a pointer to specific >> documentation? > > That would be too much to ask since it is clear that you did not spend > more than a few minutes reading the documentation. Spend some more > minutes.Well now, that would only be true if you meant very recently. In fact I have spent quite hefty amounts of times reading zfs and opensolaris documentation. Including hefty tracts of the `Bible'' (that isn''t even close) book. The trouble is that I''m understand about 1/10 of it, and that 1/10 soon departs my pea brain when I don''t use it daily. You may be used to dealing with folks who have a basic understanding of OSs'' and programming, rc files and etc. Probably some amount of formal higher education too. I''ve come on that kind of info in a very haphazard, hard scrabble way. My education stopped in 9th grade, I went to work at that point, out in the west of our country. Started industrial work a few yrs later (1965) and eventually became a field construction boilermaker and worked around many of the midwest, western and west coast states on refineries, powerplants, steelmills, and other big industrial plants (now retired). None of that was very conducive to the finer points of Operating systems and programming or admin chores. So whatever I''ve learned about that kind of stuff in the last 10 yrs or so has gaping holes that you could drive 18 wheelers through. I''ve never found that reading documentation I barely understand is a very good way of actually learning something. On the other hand, hearing about it from current practitioners and heavy experimentation is (for me) the best way to learn about something. With some of that going, then the documentation may start to be a lot more meaningful, once I have something to hang it on. Egad... sorry about the rant but sometimes it seems to just need to be said. Thanks again for what you have contributed, not just to this thread but your many hundreds, maybe thousands of messages of help to others as well.