Hi everyone. I''ve got a couple ideas for good zfs GSoC projects, but wanted to stir some interest. Anyone interested to help mentor? The deadline is around the corner so if planning hasn''t happened yet it should start soon. If there is interest who would the org administrator be? Thanks ./Christopher
C. Bergstr?m wrote:> > Hi everyone. > > I''ve got a couple ideas for good zfs GSoC projects, but wanted to stir > some interest. Anyone interested to help mentor? The deadline is > around the corner so if planning hasn''t happened yet it should start > soon. If there is interest who would the org administrator be?I might be interested in mentoring. I''ve done GSoC mentoring in the past. -- Darren J Moffat
Care to share any of those in advance? It might be cool to see input from listees and generally get some wheels turning... On Wed, Feb 25, 2009 at 4:39 AM, "C. Bergstr?m" <cbergstrom at netsyncro.com> wrote:> > Hi everyone. > > I''ve got a couple ideas for good zfs GSoC projects, but wanted to stir some > interest. ?Anyone interested to help mentor? ?The deadline is around the > corner so if planning hasn''t happened yet it should start soon. ?If there is > interest who would the org administrator be? > > Thanks > > ./Christopher > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Blake wrote:> Care to share any of those in advance? It might be cool to see input > from listees and generally get some wheels turning... >raidz boot support in grub 2 is pretty high on my list to be honest.. Which brings up another question of where is the raidz stuff mostly? usr/src/uts/common/fs/zfs/vdev_raidz.c ? Any high level summary, docs or blog entries of what the process would look like for a raidz boot support is also appreciated. Cheers, ./Christopher
RaidZ grow support On Fri, Feb 27, 2009 at 11:23 PM, "C. Bergstr?m" <cbergstrom at netsyncro.com>wrote:> Blake wrote: > >> Care to share any of those in advance? It might be cool to see input >> from listees and generally get some wheels turning... >> >> > raidz boot support in grub 2 is pretty high on my list to be honest.. > > Which brings up another question of where is the raidz stuff mostly? > > usr/src/uts/common/fs/zfs/vdev_raidz.c ? > > Any high level summary, docs or blog entries of what the process would look > like for a raidz boot support is also appreciated. > > > Cheers, > > > ./Christopher > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090228/9a4821a4/attachment.html>
On Feb 27, 2009, at 18:23, C. Bergstr?m wrote:> Blake wrote: >> Care to share any of those in advance? It might be cool to see input >> from listees and generally get some wheels turning... >> > raidz boot support in grub 2 is pretty high on my list to be honest.. > > Which brings up another question of where is the raidz stuff mostly? > > usr/src/uts/common/fs/zfs/vdev_raidz.c ? > > Any high level summary, docs or blog entries of what the process > would look like for a raidz boot support is also appreciated.Given the threads that have appeared on this list lately, how about codifying / standardizing the output of "zfs send" so that it can be backed up to tape? :)
David Magda wrote:> > On Feb 27, 2009, at 18:23, C. Bergstr?m wrote: > >> Blake wrote: >>> Care to share any of those in advance? It might be cool to see input >>> from listees and generally get some wheels turning... >>> >> raidz boot support in grub 2 is pretty high on my list to be honest.. >> >> Which brings up another question of where is the raidz stuff mostly? >> >> usr/src/uts/common/fs/zfs/vdev_raidz.c ? >> >> Any high level summary, docs or blog entries of what the process >> would look like for a raidz boot support is also appreciated. > > Given the threads that have appeared on this list lately, how about > codifying / standardizing the output of "zfs send" so that it can be > backed up to tape? :)It wouldn''t help. zfs send is a data stream which contains parts of files, not files (in the usual sense), so there is no real way to take a send stream and extract a file, other than by doing a receive. At the risk of repeating the Best Practices Guide (again): The zfs send and receive commands do not provide an enterprise-level backup solution. -- richard
zfs send is great for moving a filesystem with lots of tiny files, since it just handles the blocks :) I''d like to see: pool-shrinking (and an option to shrink disk A when i want disk B to become a mirror, but A is a few blocks bigger) install to mirror from the liveCD gui zfs recovery tools (sometimes bad things happen) automated installgrub when mirroring an rpool On Fri, Feb 27, 2009 at 8:02 PM, Richard Elling <richard.elling at gmail.com> wrote:> David Magda wrote: >> >> On Feb 27, 2009, at 18:23, C. Bergstr?m wrote: >> >>> Blake wrote: >>>> >>>> Care to share any of those in advance? ?It might be cool to see input >>>> from listees and generally get some wheels turning... >>>> >>> raidz boot support in grub 2 is pretty high on my list to be honest.. >>> >>> Which brings up another question of where is the raidz stuff mostly? >>> >>> usr/src/uts/common/fs/zfs/vdev_raidz.c ? >>> >>> Any high level summary, docs or blog entries of what the process would >>> look like for a raidz boot support is also appreciated. >> >> Given the threads that have appeared on this list lately, how about >> codifying / standardizing the output of "zfs send" so that it can be backed >> up to tape? :) > > It wouldn''t help. ?zfs send is a data stream which contains parts of files, > not files (in the usual sense), so there is no real way to take a send > stream and extract a file, other than by doing a receive. > > At the risk of repeating the Best Practices Guide (again): > The zfs send and receive commands do not provide an enterprise-level backup > solution. > -- richard > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Gnome GUI for desktop ZFS administration On Fri, Feb 27, 2009 at 9:13 PM, Blake <blake.irvin at gmail.com> wrote:> zfs send is great for moving a filesystem with lots of tiny files, > since it just handles the blocks :) > > > > I''d like to see: > > pool-shrinking (and an option to shrink disk A when i want disk B to > become a mirror, but A is a few blocks bigger) > > install to mirror from the liveCD gui > > zfs recovery tools (sometimes bad things happen) > > automated installgrub when mirroring an rpool > > > > > On Fri, Feb 27, 2009 at 8:02 PM, Richard Elling > <richard.elling at gmail.com> wrote: >> David Magda wrote: >>> >>> On Feb 27, 2009, at 18:23, C. Bergstr?m wrote: >>> >>>> Blake wrote: >>>>> >>>>> Care to share any of those in advance? ?It might be cool to see input >>>>> from listees and generally get some wheels turning... >>>>> >>>> raidz boot support in grub 2 is pretty high on my list to be honest.. >>>> >>>> Which brings up another question of where is the raidz stuff mostly? >>>> >>>> usr/src/uts/common/fs/zfs/vdev_raidz.c ? >>>> >>>> Any high level summary, docs or blog entries of what the process would >>>> look like for a raidz boot support is also appreciated. >>> >>> Given the threads that have appeared on this list lately, how about >>> codifying / standardizing the output of "zfs send" so that it can be backed >>> up to tape? :) >> >> It wouldn''t help. ?zfs send is a data stream which contains parts of files, >> not files (in the usual sense), so there is no real way to take a send >> stream and extract a file, other than by doing a receive. >> >> At the risk of repeating the Best Practices Guide (again): >> The zfs send and receive commands do not provide an enterprise-level backup >> solution. >> -- richard >> >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> >
On Feb 27, 2009, at 20:02, Richard Elling wrote:> It wouldn''t help. zfs send is a data stream which contains parts of > files, > not files (in the usual sense), so there is no real way to take a send > stream and extract a file, other than by doing a receive.If you create a non-incremental stream of a snapshot, doesn''t it contain all the information necessary to recreate the file system up to the point the snapshot was created? Now take that stream and drop it onto a tape (real or VTL) and you would potentially have a back up (a la ufsdump).> At the risk of repeating the Best Practices Guide (again): > The zfs send and receive commands do not provide an enterprise-level > backup solution.Yes, in its current state; hopefully that will change some point in the future (which is what we''re talking about with GSoC--the potential to change the status quo).
David Magda wrote:> On Feb 27, 2009, at 20:02, Richard Elling wrote: > >> It wouldn''t help. zfs send is a data stream which contains parts of >> files, >> not files (in the usual sense), so there is no real way to take a send >> stream and extract a file, other than by doing a receive. > > If you create a non-incremental stream of a snapshot, doesn''t it > contain all the information necessary to recreate the file system up > to the point the snapshot was created? Now take that stream and drop > it onto a tape (real or VTL) and you would potentially have a back up > (a la ufsdump).Who uses only one snapshot? Not many people.>> At the risk of repeating the Best Practices Guide (again): >> The zfs send and receive commands do not provide an enterprise-level >> backup solution. > > Yes, in its current state; hopefully that will change some point in > the future (which is what we''re talking about with GSoC--the potential > to change the status quo).I suppose, but considering that enterprise backup solutions exist, and some are open source, why reinvent the wheel? -- richard
Blake wrote:> Gnome GUI for desktop ZFS administration > > > > On Fri, Feb 27, 2009 at 9:13 PM, Blake <blake.irvin at gmail.com> wrote: > >> zfs send is great for moving a filesystem with lots of tiny files, >> since it just handles the blocks :) >> >> >> >> I''d like to see: >> >> pool-shrinking (and an option to shrink disk A when i want disk B to >> become a mirror, but A is a few blocks bigger) >>This may be interesting... I''m not sure how often you need to shrink a pool though? Could this be classified more as a Home or SME level feature?>> install to mirror from the liveCD gui >>I''m not working on OpenSolaris at all, but for when my projects installer is more ready /we/ can certainly do this..>> zfs recovery tools (sometimes bad things happen) >>Agreed.. part of what I think keeps zfs so stable though is the complete lack of dependence on any recovery tools.. It forces customers to bring up the issue instead of dirty hack and nobody knows.>> automated installgrub when mirroring an rpool >>This goes back to an installer option? ./C
Blake wrote:> Gnome GUI for desktop ZFS administration >With the libzfs java bindings I am plotting a web based interface.. I''m not sure if that would meet this gnome requirement though.. Knowing specifically what you''d want to do in that interface would be good.. I planned to compare it to fishworks and the nexenta appliance as a base..
I''m using opensolaris and zfs at my house for my photography storage as well as for an offsite backup location for my employer and several side web projects. I have an 80g drive as my root drive. I recently took posesion of 2 74g 10k drives which I''d love to add as a mirror to replace the 80 g drive.>From what I gather it is only possible if I zfs export my storagearray and reinstall solaris on the new disks. So I guess I''m hoping zfs shrink and grow commands show up sooner or later. Just a data point. Joe Esposito www.j-espo.com On 2/28/09, "C. Bergstr?m" <cbergstrom at netsyncro.com> wrote:> Blake wrote: >> Gnome GUI for desktop ZFS administration >> >> >> >> On Fri, Feb 27, 2009 at 9:13 PM, Blake <blake.irvin at gmail.com> wrote: >> >>> zfs send is great for moving a filesystem with lots of tiny files, >>> since it just handles the blocks :) >>> >>> >>> >>> I''d like to see: >>> >>> pool-shrinking (and an option to shrink disk A when i want disk B to >>> become a mirror, but A is a few blocks bigger) >>> > This may be interesting... I''m not sure how often you need to shrink a > pool though? Could this be classified more as a Home or SME level feature? >>> install to mirror from the liveCD gui >>> > I''m not working on OpenSolaris at all, but for when my projects > installer is more ready /we/ can certainly do this.. >>> zfs recovery tools (sometimes bad things happen) >>> > Agreed.. part of what I think keeps zfs so stable though is the complete > lack of dependence on any recovery tools.. It forces customers to bring > up the issue instead of dirty hack and nobody knows. >>> automated installgrub when mirroring an rpool >>> > This goes back to an installer option? > > ./C > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
On 28 Feb 2009, at 07:26, C. Bergstr?m wrote:> Blake wrote: >> Gnome GUI for desktop ZFS administration >> > With the libzfs java bindings I am plotting a web based interface.. > I''m not sure if that would meet this gnome requirement though.. > Knowing specifically what you''d want to do in that interface would > be good.. I planned to compare it to fishworks and the nexenta > appliance as a base..Recent builds of OpenSolaris come with SWT from the Eclipse project, which makes it possible for Java apps to use real GNOME/GTK native UIs. So your libzfs bindings may well be useful with that. Cheers, Chris
I for one would like an "interactive" attribute for zpools and filesystems, specifically for destroy. The existing behavior (no prompt) could be the default, but all filesystems would inherit from the zpool''s attrib. so I''d only need to set interactive=on for the pool itself, not for each filesystem. I have yet (in almost two years of using ZFS) to bone myself by accidentally destroying tank/worthmorethanyourjob, but it''s only a matter of time, regardless of how careful I am. The argument rm vs zfs destroy doesn''t hold much water to me. I don''t use rm -i, but destroying a single file or a hierarchy of directories is somewhat different than destroying a filesytem or entire pool. At least to my mind. As such, consider it a piece of mind feature. -- bda Cyberpunk is dead. Long live cyberpunk. http://mirrorshades.org
On 28 February, 2009 - Bryan Allen sent me these 1,0K bytes:> I for one would like an "interactive" attribute for zpools and > filesystems, specifically for destroy. > > The existing behavior (no prompt) could be the default, but all > filesystems would inherit from the zpool''s attrib. so I''d only > need to set interactive=on for the pool itself, not for each > filesystem. > > I have yet (in almost two years of using ZFS) to bone myself by > accidentally destroying tank/worthmorethanyourjob, but it''s only > a matter of time, regardless of how careful I am. > > The argument rm vs zfs destroy doesn''t hold much water to me. I > don''t use rm -i, but destroying a single file or a hierarchy of > directories is somewhat different than destroying a filesytem or > entire pool. At least to my mind. > > As such, consider it a piece of mind feature.Or a spinoff; -t argument to destroy, so you can pretty safely script snapshot destruction without risking the entire pool/fs being zapped.. That is: zfs destroy -t snapshot blah at foo And possibly: zfs destroy -r -t snapshot mypool/myfs to kill all snapshots below myfs. /Tomas -- Tomas ?gren, stric at acc.umu.se, http://www.acc.umu.se/~stric/ |- Student at Computing Science, University of Ume? `- Sysadmin at {cs,acc}.umu.se
Shrinking pools would also solve the right-sizing dilemma. Sent from my iPhone On Feb 28, 2009, at 3:37 AM, Joe Esposito <joe at j-espo.com> wrote:> I''m using opensolaris and zfs at my house for my photography storage > as well as for an offsite backup location for my employer and several > side web projects. > > I have an 80g drive as my root drive. I recently took posesion of 2 > 74g 10k drives which I''d love to add as a mirror to replace the 80 g > drive. > > From what I gather it is only possible if I zfs export my storage > array and reinstall solaris on the new disks. > > So I guess I''m hoping zfs shrink and grow commands show up sooner or > later. > > Just a data point. > > Joe Esposito > www.j-espo.com > > On 2/28/09, "C. Bergstr?m" <cbergstrom at netsyncro.com> wrote: >> Blake wrote: >>> Gnome GUI for desktop ZFS administration >>> >>> >>> >>> On Fri, Feb 27, 2009 at 9:13 PM, Blake <blake.irvin at gmail.com> >>> wrote: >>> >>>> zfs send is great for moving a filesystem with lots of tiny files, >>>> since it just handles the blocks :) >>>> >>>> >>>> >>>> I''d like to see: >>>> >>>> pool-shrinking (and an option to shrink disk A when i want disk B >>>> to >>>> become a mirror, but A is a few blocks bigger) >>>> >> This may be interesting... I''m not sure how often you need to >> shrink a >> pool though? Could this be classified more as a Home or SME level >> feature? >>>> install to mirror from the liveCD gui >>>> >> I''m not working on OpenSolaris at all, but for when my projects >> installer is more ready /we/ can certainly do this.. >>>> zfs recovery tools (sometimes bad things happen) >>>> >> Agreed.. part of what I think keeps zfs so stable though is the >> complete >> lack of dependence on any recovery tools.. It forces customers to >> bring >> up the issue instead of dirty hack and nobody knows. >>>> automated installgrub when mirroring an rpool >>>> >> This goes back to an installer option? >> >> ./C >> >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >>
On Sat, Feb 28, 2009 at 1:20 AM, Richard Elling <richard.elling at gmail.com> wrote:> David Magda wrote: >> On Feb 27, 2009, at 20:02, Richard Elling wrote: >>> At the risk of repeating the Best Practices Guide (again): >>> The zfs send and receive commands do not provide an enterprise-level >>> backup solution. >> >> Yes, in its current state; hopefully that will change some point in the >> future (which is what we''re talking about with GSoC--the potential to change >> the status quo). > > I suppose, but considering that enterprise backup solutions exist, > and some are open source, why reinvent the wheel? > -- richardThe default mode of operation for every enterprise backup tool that I have used is file level backups. The determination of which files need to be backed up seems to be to crawl the file system looking for files that have an mtime after the previous backup. Areas of strength for such tools include: - Works with any file system that provides a POSIX interface - Restore of a full backup is an accurate representation of the data backed up - Restore can happen to a different file system type - Restoring an individual file is possible Areas of weakness include: - Extremely inefficient for file systems with lots of files and little change. - Restore of full + incremental tends to have extra files because of spotty support or performance overhead of tool that would prevent it. - Large files that have blocks rewritten get backed up in full each time - Restores of file systems with lots of small files (especially in one directory) are extremely slow There exist features (sometimes expensive add-ons) that deal with some of these shortcomings via: - Keeping track of deleted files so that a restore is more representative of what is on disk during the incremental backup. Administration manuals typically warn that this has a big performance and/or size overhead on the database used by the backup software. - Including add-ons that hook into other components (e.g. VxFS storage checkpoints, Oracle RMAN) that provide something similar to block-level incremental backups Why re-invent the wheel? - People are more likely to have snapshots available for file-level restores, and as such a "zfs send" data stream would only be used in the event of a complete pool loss. - It is possible to provide a general block-level backup solution so that every product doesn''t have to invent it. This gives ZFS another feature benefit to put it higher in the procurement priority. - File creation slowness can likely be avoided allowing restore to happen at tape speed - To be competitive with NetApp "snapmirror to tape" - Even having a zfs(1M) option that could list the files that change between snapshots could be very helpful to prevent file system crawls and to avoid being fooled by bogus mtimes. -- Mike Gerdts http://mgerdts.blogspot.com/
>I''m using opensolaris and zfs at my house for my photography storage >as well as for an offsite backup location for my employer and several >side web projects. > >I have an 80g drive as my root drive. I recently took posesion of 2 >74g 10k drives which I''d love to add as a mirror to replace the 80 g >drive.Why do you want to use a small 10K rpm disk? A modern 1TB disk at 5400/7200 rpm (at $100) will put it to shame. Casper
On Sat, Feb 28, 2009 at 8:31 AM, <Casper.Dik at sun.com> wrote:> > >I''m using opensolaris and zfs at my house for my photography storage > >as well as for an offsite backup location for my employer and several > >side web projects. > > > >I have an 80g drive as my root drive. I recently took posesion of 2 > >74g 10k drives which I''d love to add as a mirror to replace the 80 g > >drive. > > Why do you want to use a small 10K rpm disk? > > A modern 1TB disk at 5400/7200 rpm (at $100) will put it to shame. > > Casper >fair enough. I just have a pair of these sitting here from a pull at work. The data array is currently 4x1TB with another hot swap bay ready for 4x??? when the need arises. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090228/88d4f3cd/attachment.html>
On Sat, Feb 28, 2009 at 8:28 AM, Joe Esposito <joe at j-espo.com> wrote:> > > On Sat, Feb 28, 2009 at 8:31 AM, <Casper.Dik at sun.com> wrote: > >> >> >I''m using opensolaris and zfs at my house for my photography storage >> >as well as for an offsite backup location for my employer and several >> >side web projects. >> > >> >I have an 80g drive as my root drive. I recently took posesion of 2 >> >74g 10k drives which I''d love to add as a mirror to replace the 80 g >> >drive. >> >> Why do you want to use a small 10K rpm disk? >> >> A modern 1TB disk at 5400/7200 rpm (at $100) will put it to shame. >> >> Casper >> > > fair enough. I just have a pair of these sitting here from a pull at work. > The data array is currently 4x1TB with another hot swap bay ready for 4x??? > when the need arises. >That''s not entirely true. Maybe it will put it to shame at streaming sequential I/O. The 10k drives will still wipe the floor with any modern 7200rpm drive for random IO and seek times. --Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090228/18b2edd8/attachment.html>
On Sat, 28 Feb 2009, Tim wrote:> > That''s not entirely true. Maybe it will put it to shame at streaming > sequential I/O. The 10k drives will still wipe the floor with any modern > 7200rpm drive for random IO and seek times.Or perhaps streaming sequential I/O will have similar performance, with much better performance for random IO and seek times. It is always best to consult the vendor spec sheet. Regardless, it is much easier to update with the same size, or a larger size drive. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
> >> pool-shrinking (and an option to shrink disk A when i want disk B to > >> become a mirror, but A is a few blocks bigger) > This may be interesting... I''m not sure how often you need to shrink a pool > though? Could this be classified more as a Home or SME level feature?Enterprise level especially in SAN environments need this. Projects own theyr own pools and constantly grow and *shrink* space. And they have no downtime available for that. give a +1 if you agree Thomas
I would really add : make insane zfs destroy <-r|> poolname as harmless
as zpool destroy poolname (recoverable)
zfs destroy <-r|> poolname<|/filesystem>
this should behave like that:
o snapshot the filesystem to be deleted (each, name it
@deletedby_operatorname_date)
o hide the snapshot as long as the pool has enough space and
property snapshotbeforedelete=on (default off) is set
''on''
o free space by removing those snapshots no earlier then configured
in a inheritable pool/filesystem property snapshotbeforedeleteremoval=3days
(=0 preserve forever, 30min preserve for 30 minutes, ...)
o prevent deletion of a pool or filesystem if at least one
snapshot from the above save actions exists down the tree
o purging of snapshots would be done by
To be honest, I don''t want a discussion like the "rm -rf" is
one.
In front of the keyboard or inside scripts we are all humans with
all theyr mistakes. In opposite to the rm -rf, the ZFS Design
should take this extension without major changes. It should be
a generic rule of dump to implement safety if it is possible
at resonable low cost.
I think the full range of users, Enterprise to Home will appreciate
that theyr multi-million-$$-business/home_data does not go down
accidentially with the interactive=on (Bryan) or the the idea
written here. This in case someone makes an error and all the
data could still be there (!)...ZFS should protect the user as well
and not only look at the hardware redundancy.
Thomas
PS: think of the day where simple operator $NAME makes a typo
zfs destroy -r poolname and all the data still sits on the
disk. But no one is able to bring that valueable data back,
except restoration from tape with hours of downtime.
Sorry for repeating that, it hurts so much to not having
this feature.
On Sat, Feb 28, 2009 at 04:35:05AM -0500, Bryan Allen
wrote:> I for one would like an "interactive" attribute for zpools and
> filesystems, specifically for destroy.
>
> The existing behavior (no prompt) could be the default, but all
> filesystems would inherit from the zpool''s attrib. so I''d
only
> need to set interactive=on for the pool itself, not for each
> filesystem.
>
> I have yet (in almost two years of using ZFS) to bone myself by
> accidentally destroying tank/worthmorethanyourjob, but it''s only
> a matter of time, regardless of how careful I am.
>
> The argument rm vs zfs destroy doesn''t hold much water to me. I
> don''t use rm -i, but destroying a single file or a hierarchy of
> directories is somewhat different than destroying a filesytem or
> entire pool. At least to my mind.
>
> As such, consider it a piece of mind feature.
> --
> bda
> Cyberpunk is dead. Long live cyberpunk.
> http://mirrorshades.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
--
Thomas Wagner
+49-171-6135989 http://www.wagner-net.net
On Sat, Feb 28, 2009 at 10:44:59PM +0100, Thomas Wagner wrote:> > >> pool-shrinking (and an option to shrink disk A when i want disk B to > > >> become a mirror, but A is a few blocks bigger) > > This may be interesting... I''m not sure how often you need to shrink a pool > > though? Could this be classified more as a Home or SME level feature? > > Enterprise level especially in SAN environments need this. > > Projects own theyr own pools and constantly grow and *shrink* space. > And they have no downtime available for that.Multiple pools on one server only makes sense if you are going to have different RAS for each pool for business reasons. It''s a lot easier to have a single pool though. I recommend it. Nico --
On Sat, Feb 28, 2009 at 4:33 PM, Nicolas Williams <Nicolas.Williams at sun.com> wrote:> On Sat, Feb 28, 2009 at 10:44:59PM +0100, Thomas Wagner wrote: >> > >> pool-shrinking (and an option to shrink disk A when i want disk B to >> > >> become a mirror, but A is a few blocks bigger) >> > ?This may be interesting... I''m not sure how often you need to shrink a pool >> > ?though? ?Could this be classified more as a Home or SME level feature? >> >> Enterprise level especially in SAN environments need this. >> >> Projects own theyr own pools and constantly grow and *shrink* space. >> And they have no downtime available for that. > > Multiple pools on one server only makes sense if you are going to have > different RAS for each pool for business reasons. ?It''s a lot easier to > have a single pool though. ?I recommend it.Other scenarios for multiple pools include: - Need independent portability of data between servers. For example, in a HA cluster environment, various workloads will be mapped to various pools. Since ZFS does not do active-active clustering, a single pool for anything other than a simple active-standby cluster is not useful. - Array based copies are needed. There are times when copies of data are performed at a storage array level to allow testing and support operations to happen "on different spindles". For example, in a consolidated database environment, each database may be constrained to a set of spindles so that each database can be replicated or copied independent of the various others. -- Mike Gerdts http://mgerdts.blogspot.com/
Absolutely agree. I''l love to be able to free up some LUNs that I don''t need in the pool any more. Also, concatenation of devices in a zpool would be great for devices that have LUN limits. It also seems like it may be an easy thing to implement. -Aaron On 2/28/09, Thomas Wagner <thomas.wagner at gmx.net> wrote:>> >> pool-shrinking (and an option to shrink disk A when i want disk B to >> >> become a mirror, but A is a few blocks bigger) >> This may be interesting... I''m not sure how often you need to shrink a >> pool >> though? Could this be classified more as a Home or SME level feature? > > Enterprise level especially in SAN environments need this. > > Projects own theyr own pools and constantly grow and *shrink* space. > And they have no downtime available for that. > > give a +1 if you agree > > Thomas > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-- Sent from my mobile device
> Multiple pools on one server only makes sense if you are going to have > different RAS for each pool for business reasons. It''s a lot easier to > have a single pool though. I recommend it.A couple of other things to consider to go with that recommendation. - never build a pool larger than you are willing to restore. Bad things can still happen that would require you to restore the entire pool. Convenience and SLAs aren''t always in agreement :-) The advances in ZFS availability might make me look at my worst case restore scenario a little different though - but there will still be a restore case that worries me. - as I look at the recent lifecycle improvements with zones (in the Solaris 10 context of zones), I really like upgrade on attach. That means I will be slinging zones more freely. So I need to design my pools to match that philosophy. - if you are using clustering technologies, pools will go hand in hand with failover boundaries. So if I have multiple failover zones, I will have multiple pools. Bob
On Sat, Feb 28, 2009 at 05:19:26PM -0600, Mike Gerdts wrote:> On Sat, Feb 28, 2009 at 4:33 PM, Nicolas Williams > <Nicolas.Williams at sun.com> wrote: > > On Sat, Feb 28, 2009 at 10:44:59PM +0100, Thomas Wagner wrote: > >> > >> pool-shrinking (and an option to shrink disk A when i want disk B to > >> > >> become a mirror, but A is a few blocks bigger) > >> > ?This may be interesting... I''m not sure how often you need to shrink a pool > >> > ?though? ?Could this be classified more as a Home or SME level feature? > >> > >> Enterprise level especially in SAN environments need this. > >> > >> Projects own theyr own pools and constantly grow and *shrink* space. > >> And they have no downtime available for that. > > > > Multiple pools on one server only makes sense if you are going to have > > different RAS for each pool for business reasons. ?It''s a lot easier to > > have a single pool though. ?I recommend it. > > Other scenarios for multiple pools include: > > - Need independent portability of data between servers. For example, > in a HA cluster environment, various workloads will be mapped to > various pools. Since ZFS does not do active-active clustering, a > single pool for anything other than a simple active-standby cluster is > not useful.Right, but normally each head in a cluster will have only one pool imported. The Sun Storage 7xxx do this. One pool per-head, two pools altogether in a cluster.> - Array based copies are needed. There are times when copies of data > are performed at a storage array level to allow testing and support > operations to happen "on different spindles". For example, in a > consolidated database environment, each database may be constrained to > a set of spindles so that each database can be replicated or copied > independent of the various others.This gets you back into managing physical space allocation. Do you really want that? If you''re using zvols you can do "array based copies" of you zvols. If you''re using filesystems then you should just use normal backup tools. Nico --
On Sat, Feb 28, 2009 at 8:34 PM, Nicolas Williams <Nicolas.Williams at sun.com> wrote:> On Sat, Feb 28, 2009 at 05:19:26PM -0600, Mike Gerdts wrote: >> On Sat, Feb 28, 2009 at 4:33 PM, Nicolas Williams >> <Nicolas.Williams at sun.com> wrote: >> > On Sat, Feb 28, 2009 at 10:44:59PM +0100, Thomas Wagner wrote: >> >> > >> pool-shrinking (and an option to shrink disk A when i want disk B to >> >> > >> become a mirror, but A is a few blocks bigger) >> >> > ?This may be interesting... I''m not sure how often you need to shrink a pool >> >> > ?though? ?Could this be classified more as a Home or SME level feature? >> >> >> >> Enterprise level especially in SAN environments need this. >> >> >> >> Projects own theyr own pools and constantly grow and *shrink* space. >> >> And they have no downtime available for that. >> > >> > Multiple pools on one server only makes sense if you are going to have >> > different RAS for each pool for business reasons. ?It''s a lot easier to >> > have a single pool though. ?I recommend it. >> >> Other scenarios for multiple pools include: >> >> - Need independent portability of data between servers. ?For example, >> in a HA cluster environment, various workloads will be mapped to >> various pools. ?Since ZFS does not do active-active clustering, a >> single pool for anything other than a simple active-standby cluster is >> not useful. > > Right, but normally each head in a cluster will have only one pool > imported.Not necessarily. Suppose I have a group of servers with a bunch of zones. Each zone represents a service group that needs to independently fail over between servers. In that case, I may have a zpool per zone. It seems this is how it is done in the real world.[1] 1. Upton, Tom. "A Conversation with Jason Hoffman." ACM Queue. January/February 2008. 9.> The Sun Storage 7xxx do this. ?One pool per-head, two pools altogether > in a cluster.Makes sense for your use case. If you are looking at a zpool per zone, it is likely a zpool created on a LUN provided by a Sun Storage 7xxx that is presented to multiple hosts. That is, ZFS on top of ZFS.>> - Array based copies are needed. ?There are times when copies of data >> are performed at a storage array level to allow testing and support >> operations to happen "on different spindles". ?For example, in a >> consolidated database environment, each database may be constrained to >> a set of spindles so that each database can be replicated or copied >> independent of the various others. > > This gets you back into managing physical space allocation. ?Do you > really want that? ?If you''re using zvols you can do "array based copies" > of you zvols. ?If you''re using filesystems then you should just use > normal backup tools.There are times when you have no real choice. If a regulation or a lawyer''s interpretation of a regulation says that you need to have physically separate components, you need to have physically separate components. If your disaster recovery requirements mean that you need to have a copy of data at a different site and array based copies have historically been used - it is unlikely that "while true ; do zfs send | ssh | zfs receive" will be adapted in the first round of implementation. Given this, zvols don''t do it today. When you have a smoking hole, the gap in transactions left by normal backup tools is not always good enough - especially if some of that smoke is coming from the tape library. Array based replication tends to allow you to keep much tighter tolerances on just how many committed transactions you are willing to lose. -- Mike Gerdts http://mgerdts.blogspot.com/
On Sat, 28 Feb 2009 21:45:12 -0600, Mike Gerdts <mgerdts at gmail.com> wrote:>On Sat, Feb 28, 2009 at 8:34 PM, Nicolas Williams ><Nicolas.Williams at sun.com> wrote:[snip]>> Right, but normally each head in a cluster will have only one pool >> imported. > >Not necessarily. Suppose I have a group of servers with a bunch of >zones. Each zone represents a service group that needs to >independently fail over between servers. In that case, I may have a >zpool per zone. It seems this is how it is done in the real world.[1] >1. Upton, Tom. "A Conversation with Jason Hoffman." ACM Queue. >January/February 2008. 9.Exactly. Or even a zpool per application, if there is more than one application in a zone. In that sense, I would call a zpool a "unit of maintenance". Or a "unit of failure". -- ( Kees Nuyt ) c[_]
David Magda wrote:> Given the threads that have appeared on this list lately, how about > codifying / standardizing the output of "zfs send" so that it can be > backed up to tape? :)We will soon be changing the manpage to indicate that the zfs send stream will be receivable on all future versions of ZFS. However, as has been mentioned, that does not necessarily make it an enterprise backup solution :-) --matt
Blake wrote:> zfs send is great for moving a filesystem with lots of tiny files, > since it just handles the blocks :) > > > > I''d like to see: > > pool-shrinking (and an option to shrink disk A when i want disk B to > become a mirror, but A is a few blocks bigger)I''m working on it.> install to mirror from the liveCD gui > > zfs recovery tools (sometimes bad things happen) > > automated installgrub when mirroring an rpool--matt
excellent! i wasn''t sure if that was the case, though i had heard rumors. On Mon, Mar 2, 2009 at 12:36 PM, Matthew Ahrens <Matthew.Ahrens at sun.com> wrote:> Blake wrote: >> >> zfs send is great for moving a filesystem with lots of tiny files, >> since it just handles the blocks :) >> >> >> >> I''d like to see: >> >> pool-shrinking (and an option to shrink disk A when i want disk B to >> become a mirror, but A is a few blocks bigger) > > I''m working on it. > >> install to mirror from the liveCD gui >> >> zfs recovery tools (sometimes bad things happen) >> >> automated installgrub when mirroring an rpool > > --matt >
On Sat, Feb 28, 2009 at 09:45:12PM -0600, Mike Gerdts wrote:> On Sat, Feb 28, 2009 at 8:34 PM, Nicolas Williams > > Right, but normally each head in a cluster will have only one pool > > imported. > > Not necessarily. Suppose I have a group of servers with a bunch of > zones. Each zone represents a service group that needs to > independently fail over between servers. In that case, I may have a > zpool per zone. It seems this is how it is done in the real world.[1]That model forces you to allocate storage. If you''re willing to take the pain of managine storage at a higher level of granularity then you''re welcome to it. As has just been posted (and as anyone who read Matt Ahrens'' blog entry on "block pointer rewrite" could have read between the lines), pool shrinking is coming. But I still don''t recommend it.> > The Sun Storage 7xxx do this. ?One pool per-head, two pools altogether > > in a cluster. > > Makes sense for your use case. If you are looking at a zpool per > zone, it is likely a zpool created on a LUN provided by a Sun Storage > 7xxx that is presented to multiple hosts. That is, ZFS on top of ZFS.Running ZFS on an LUN backed by a ZFS zvol is fine. That does not force you to manage storage at the physical partition/cylinder/chunk-of- sectors level.> > This gets you back into managing physical space allocation. ?Do you > > really want that? ?If you''re using zvols you can do "array based copies" > > of you zvols. ?If you''re using filesystems then you should just use > > normal backup tools. > > There are times when you have no real choice. If a regulation or a > lawyer''s interpretation of a regulation says that you need to have > physically separate components, you need to have physically separate > components. If your disaster recovery requirements mean that you need > to have a copy of data at a different site and array based copies have > historically been used - it is unlikely that "while true ; do zfs send > | ssh | zfs receive" will be adapted in the first round of > implementation. Given this, zvols don''t do it today.Physically separate components accessed by a single node are not likely to meet such a regulation -- either that or the lawyers need technology training. Chinese wall requirements might need to be met by physically separate storage heads, with physically separate networks. Except that no one builds networks like that any more, at least not in the corporat world -- it''s all switches and VLANs, so there will be some degree of logical separation, and the question really is: to what degree? Nico --
>>>>> "dm" == David Magda <dmagda at ee.ryerson.ca> writes:dm> Yes, in its current state; hopefully that will change some dm> point in the future I don''t think it will or should. A replication tool and a backup tool seem similar, but they''re not similar enough. With replication, you want an exact copy, and if for some reason the copy is not exact then you need something more: you want atomicity of operations so the overwatcher scheduler: * can safely retry the send|recv until it works, * can always tell its minder with certainty how much has safely been replicated so far, * can attempt further replication without risking existing data. These things are a bit hard to achieve. And zfs send|recv does them: * If a single bit is flipped the whole stream should be discarded * If, on a ''send'' timeline of <full>, <incremental>, <incremental>, <incremental>, one of the preceeding blobs did not make it, or became bad on the replication target (because somebody wrote to it, for example---a FAQ), it should become impossible to restore further incremental backups. The error should not be best-effort worked around, or simply silently concealed, the way it is and should be with restoring incremental backups. * reslicing the unit of replication after writing the stream is irrelevant, because you can just reslice on the replication-source if you need to do this. The possibility of reslicing interferes with the atomicity I spoke of which makes the replication scheduler so much easier to get correct. * there''s no need to test a stream''s validity without restoring it. The replication target will always be available and have enough free space to test-by-receiving. * there''s no need to restore the stream on an unimagined future filesystem. It''s more important that all fancyfeatures, ACL''s, gizmos, properties, compression modes, record sizes, whatever, make it to the replication target exactly to avoid surprises. No one is worried about data being locked in to a certain filesystem because it''s all there for you to get with rsync on both replication source and target. Try to use such a tool for backup, and you court disaster. Your pile of backups becomes an increasingly large time-lapse gamma ray detector, which signals a ray''s presence by destroying ALL the data not just the bit, not even just the file, that the ray hit. The impossibility of reslicing (extracting a single file from the backup) means that you can find yourself needing 48TB of empty disk on a development system somewhere to get out a 100kB file locked inside the atomic blob, which is an unacceptable burden in time and expense. The other points have obvious problems for backups, too---I''ll leave inventing imaginary restore scenarios as an exercise for the reader. All these harmful points are things that replication wants/needs. The goals are incompatible. If there''s going to be a snapshot-aware incremental backup tool for ZFS, I do not think zfs send|recv is a good starting point. And I''m getting frustrated pointing out these issues for the 10th time---it seems like, I mention five relatively unsolveable problems, and people seize onto the easiest one, misinterpret it, and then forget the other four. the versioning issue (NOT mentioned above) is a problem for replication among different Solaris releases, not just backup. It means you could potentially have to upgrade a whole mess of machines at once. At the very least you ought to be able to upgrade the target before you upgrade the source, so you don''t have to break replication while doing the upgrade---coincidentally that''s the right direction for upgrade-test-downgrade, too, since it''s on the target that you''d possibly have to destroy the zpools if you decide you need to downgrade. We should want this and don''t have it yet. It makes having a single backup pool for a lab full of different-aged systems impossible (even with backward-only compatibility, how do you restore?), so it is worth solving for that use case too. I think the ''zfs send'' format of a given filesystem should be bit-for-bit identical given a certain ZFS version, irrespective of zpool version or kernel release on the sending system. That''s enough to solve the single-lab-backup-pool problem, and it''s also regression-testable---keep some old streams around, recv them into the pool under test, send them back out, and make sure they come out identical. And the ''zfs recv'' panics need fixing. those would both be great things, but they would *NOT* make zfs send|recv into a backup system! They would make it into a better replication system. zfs send|recv will not become backup tools if you manage to check off a few other list-of-things-to-fix, either. They can''t be both a good replication system and a good backup system at the same time. no, I don''t think the si wiki explains the full size of the issue adequately. It makes it sound like the tools are just a little too small, and maybe someday they will get bigger, maybe some people should just ignore the advice and use them anyway. It''s CYA bullshit. i think in the mean time you should make backups with cpio (or some tool that uses something similar underneath like Amanda) or by archiving file-vdev zpools. not perfect but better. And you should store it on the medium in some way that the whole thing won''t be wiped by one flipped bit (not gzip). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090302/de235fec/attachment.bin>
>>>>> "ma" == Matthew Ahrens <Matthew.Ahrens at sun.com> writes:ma> We will soon be changing the manpage to indicate that the zfs ma> send stream will be receivable on all future versions of ZFS. still not strong enough statement for this case: old system new system 1. zfs send --backup---> zfs recv 2. zfs recv <--restore--- zfs send ***FAIL*** ...as I have said a handful of times already. Is this message short enough for everyone to read it? I''m almost certain it isn''t. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090302/c8ce7908/attachment.bin>
>>>>> "cb" == C Bergstr?m <cbergstrom at netsyncro.com> writes:cb> ideas for good zfs GSoC projects, but wanted to stir some cb> interest. Read-only vdev support. 1. possibility to import a zpool on DVD. All filesystems within would be read-only. DVD should be scrubbable: result would be a list of files with defects. 2. possible to import a zpool of writeable devices as ``read-mostly''''. All filesystems within it would be read-only, but you could still scrub it, and future ``vdev shrink'''' features would work on it, and the existing silent-fsck features like label rewriting or ZIL rolling or whatever would occur. This would be used for creating the DVD master images above. 3. possible to add a read-write vdev to the read-only zpool, and make the zpool become read-write. 4. (maybe) possible to import zpool as non-persistent. All writes would be stored in ARC/L2ARC, and the max size of writes could be capped. use cases: 0. mounting other machines'' pools without rewriting labels 1. live CD''s (obvious), 2. SOX compliant backups, when backups must be done to WORM media. you import several read-only vdev''s and keep attaching one read-write vdev to the ``end'''' of the pool when you want to add a blob of data. the first and each successive vdev becomes full, incremental, incremental, incremental---thus replication and backup converge. Through (2) ``read-mostly'''' import with bp-rewrite it''s possible to condense vdev''s, for example to coalesce fifteen WORM daily incrementals whose write fuse has already blown, into a single new big vdev, and then detach the fifteen tiny ones. A variety of apparent pools are importable depending on what combination of vdev''s you would like to use. ''zpool import'' would have to become a bit more complicated, to list the workable combinations and mask unwanted devices as you order it. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090302/a56a383e/attachment.bin>
So nobody is interested in Raidz grow support? i.e. you have 4 disks is a raidz and you only have room for a 5th disk(physically), so you add the 5th disk to the raidz. It would be a great feature for a home server and its the only thing stopping solaris going on my home file server. On Tue, Mar 3, 2009 at 12:06 AM, Miles Nordin <carton at ivy.net> wrote:> >>>>> "cb" == C Bergstr?m <cbergstrom at netsyncro.com> writes: > > cb> ideas for good zfs GSoC projects, but wanted to stir some > cb> interest. > > Read-only vdev support. > > 1. possibility to import a zpool on DVD. All filesystems within would > be read-only. DVD should be scrubbable: result would be a list of > files with defects. > > 2. possible to import a zpool of writeable devices as ``read-mostly''''. > All filesystems within it would be read-only, but you could still > scrub it, and future ``vdev shrink'''' features would work on it, and > the existing silent-fsck features like label rewriting or ZIL > rolling or whatever would occur. This would be used for creating > the DVD master images above. > > 3. possible to add a read-write vdev to the read-only zpool, and make > the zpool become read-write. > > 4. (maybe) possible to import zpool as non-persistent. All writes > would be stored in ARC/L2ARC, and the max size of writes could be > capped. > > > use cases: > > 0. mounting other machines'' pools without rewriting labels > > 1. live CD''s (obvious), > > 2. SOX compliant backups, when backups must be done to WORM media. > > you import several read-only vdev''s and keep attaching one > read-write vdev to the ``end'''' of the pool when you want to add a > blob of data. the first and each successive vdev becomes full, > incremental, incremental, incremental---thus replication and > backup converge. > > Through (2) ``read-mostly'''' import with bp-rewrite it''s possible > to condense vdev''s, for example to coalesce fifteen WORM daily > incrementals whose write fuse has already blown, into a single new > big vdev, and then detach the fifteen tiny ones. > > A variety of apparent pools are importable depending on what > combination of vdev''s you would like to use. ''zpool import'' would > have to become a bit more complicated, to list the workable > combinations and mask unwanted devices as you order it. > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090303/251563fa/attachment.html>
On Mar 2, 2009, at 19:31, David wrote:> So nobody is interested in Raidz grow support? i.e. you have 4 disks > is a > raidz and you only have room for a 5th disk(physically), so you add > the 5th > disk to the raidz. It would be a great feature for a home server and > its the > only thing stopping solaris going on my home file server.The block pointer rewrite / relocate code has already been written, so once that code is pushed back it:> will allow us to traverse all the blocks in the pool and move them > around. This will be used to move all the used space off a disk so > that it can be removed. But we realized that since bp relocate has > to visit all the blocks in the pool, it can also be used to scrub or > resilver the pool.http://blogs.sun.com/ahrens/entry/new_scrub_code
On Mar 2, 2009, at 18:37, Miles Nordin wrote:> And I''m getting frustrated pointing out these issues for the 10th > time [...]http://www.xkcd.com/386/
Matthew Ahrens wrote:> Blake wrote: >> zfs send is great for moving a filesystem with lots of tiny files, >> since it just handles the blocks :) >> >> >> >> I''d like to see: >> >> pool-shrinking (and an option to shrink disk A when i want disk B to >> become a mirror, but A is a few blocks bigger) > > I''m working on it. > >> install to mirror from the liveCD gui >> >> zfs recovery tools (sometimes bad things happen)We''ve actually discussed this at length and there will be some work started soon.>> >> automated installgrub when mirroring an rpoolI''m working on it. - George> > --matt > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Richard Elling wrote:> David Magda wrote: >> >> On Feb 27, 2009, at 18:23, C. Bergstr?m wrote: >> >>> Blake wrote: >>>> Care to share any of those in advance? It might be cool to see input >>>> from listees and generally get some wheels turning... >>>> >>> raidz boot support in grub 2 is pretty high on my list to be honest.. >>> >>> Which brings up another question of where is the raidz stuff mostly? >>> >>> usr/src/uts/common/fs/zfs/vdev_raidz.c ? >>> >>> Any high level summary, docs or blog entries of what the process >>> would look like for a raidz boot support is also appreciated. >> >> Given the threads that have appeared on this list lately, how about >> codifying / standardizing the output of "zfs send" so that it can be >> backed up to tape? :) > > It wouldn''t help. zfs send is a data stream which contains parts of > files, > not files (in the usual sense), so there is no real way to take a send > stream and extract a file, other than by doing a receive. > > At the risk of repeating the Best Practices Guide (again): > The zfs send and receive commands do not provide an enterprise-level > backup solution. > -- richardAlong these lines you can envision a restore tool that is capable of reading multiple ''zfs send'' streams to construct the various versions of files which are available. In addition, it would be nice if the tool could read in the streams and then make it easy to traverse and construct a single file from all available streams. For example, if I have 5 send streams then the tool would be able to ingest all the data and provide a directory structure similar to .zfs which would allow you to restore any file which is completely intact. - George
George Wilson wrote:> Along these lines you can envision a restore tool that is capable of > reading multiple ''zfs send'' streams to construct the various versions > of files which are available. In addition, it would be nice if the > tool could read in the streams and then make it easy to traverse and > construct a single file from all available streams. For example, if I > have 5 send streams then the tool would be able to ingest all the data > and provide a directory structure similar to .zfs which would allow > you to restore any file which is completely intact.In essence, this is how HSM works (qv ADM project). You get a view of the file system for which the data in the files may reside on media elsewhere. Good stuff. -- richard
Just my $0.02, but would pool shrinking be the same as vdev evacuation? I''m quite interested in vdev evacuation as an upgrade path for multi-disk pools. This would be yet another reason to for folks to use ZFS at home (you only have to buy cheap disks), but it would also be a good to have that ability from an enterprise perspective, as I''m sure we''ve all engineered ourselves into a corner one time or another... It''s a much cleaner, safer, and possibly much faster alternative to systematically pulling drives and letting zfs resilver onto a larger disk, in order to upgrade a pool in-place, and in production. basically, what I''m thinking is: zpool remove mypool <list of devices/vdevs> Allow time for ZFS to vacate the vdev(s), and then light up the "OK to remove" light on each evacuated disk. -Greg Blake Irvin wrote:> Shrinking pools would also solve the right-sizing dilemma. > > Sent from my iPhone > > On Feb 28, 2009, at 3:37 AM, Joe Esposito <joe at j-espo.com> wrote: > >> I''m using opensolaris and zfs at my house for my photography storage >> as well as for an offsite backup location for my employer and several >> side web projects. >> >> I have an 80g drive as my root drive. I recently took posesion of 2 >> 74g 10k drives which I''d love to add as a mirror to replace the 80 g >> drive. >> >> From what I gather it is only possible if I zfs export my storage >> array and reinstall solaris on the new disks. >> >> So I guess I''m hoping zfs shrink and grow commands show up sooner or >> later. >> >> Just a data point. >> >> Joe Esposito >> www.j-espo.com >> >> On 2/28/09, "C. Bergstr?m" <cbergstrom at netsyncro.com> wrote: >>> Blake wrote: >>>> Gnome GUI for desktop ZFS administration >>>> >>>> >>>> >>>> On Fri, Feb 27, 2009 at 9:13 PM, Blake <blake.irvin at gmail.com> wrote: >>>> >>>>> zfs send is great for moving a filesystem with lots of tiny files, >>>>> since it just handles the blocks :) >>>>> >>>>> >>>>> >>>>> I''d like to see: >>>>> >>>>> pool-shrinking (and an option to shrink disk A when i want disk B to >>>>> become a mirror, but A is a few blocks bigger) >>>>> >>> This may be interesting... I''m not sure how often you need to shrink a >>> pool though? Could this be classified more as a Home or SME level >>> feature? >>>>> install to mirror from the liveCD gui >>>>> >>> I''m not working on OpenSolaris at all, but for when my projects >>> installer is more ready /we/ can certainly do this.. >>>>> zfs recovery tools (sometimes bad things happen) >>>>> >>> Agreed.. part of what I think keeps zfs so stable though is the complete >>> lack of dependence on any recovery tools.. It forces customers to bring >>> up the issue instead of dirty hack and nobody knows. >>>>> automated installgrub when mirroring an rpool >>>>> >>> This goes back to an installer option? >>> >>> ./C >>> >>> _______________________________________________ >>> zfs-discuss mailing list >>> zfs-discuss at opensolaris.org >>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >>> > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Greg Mason wrote:> Just my $0.02, but would pool shrinking be the same as vdev evacuation?Yes.> basically, what I''m thinking is: > > zpool remove mypool <list of devices/vdevs> > > Allow time for ZFS to vacate the vdev(s), and then light up the "OK to > remove" light on each evacuated disk.That''s the goal. --matt
Hello George, Tuesday, March 3, 2009, 3:01:43 PM, you wrote: GW> Matthew Ahrens wrote:>>> >>> pool-shrinking (and an option to shrink disk A when i want disk B to >>> become a mirror, but A is a few blocks bigger) >> >> I''m working on it. >>>>> >>> automated installgrub when mirroring an rpoolGW> I''m working on it. Wouldn''t it be useful for everyone if the ZFS team could set-up and maintain a publicly available web page with a list of features currently being worked on or scheduled for a near future with a one or two sentence description if needed? That would probably reduce the amount of questions being asked hundreds of times - like - can I shrink a pool? I know that you probably can''t share all of it but still some of it would be useful for the community. -- Best regards, Robert Milkowski http://milek.blogspot.com
Hello Thomas,
Saturday, February 28, 2009, 10:14:20 PM, you wrote:
TW> I would really add : make insane zfs destroy <-r|> poolname as
TW> harmless as zpool destroy poolname (recoverable)
TW> zfs destroy <-r|> poolname<|/filesystem>
TW> this should behave like that:
TW> o snapshot the filesystem to be deleted (each, name it
@deletedby_operatorname_date)
TW> o hide the snapshot as long as the pool has enough space and
TW> property snapshotbeforedelete=on (default off) is set
''on''
[...]
I can''t disagree more. You can always write a small script/wrapper
doing something like this for you.
--
Best regards,
Robert Milkowski
http://milek.blogspot.com
How I do recursive, selective snapshot destroys: http://blog.clockworm.com/2008/03/remove-old-zfs-snapshots.html> > Saturday, February 28, 2009, 10:14:20 PM, you wrote: > > TW> I would really add : make insane zfs destroy <-r|> poolname as > TW> harmless as zpool destroy poolname (recoverable) >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090305/7d8aac16/attachment.html>