Besides copying data from /dev/md/dsk/x volume manager filesystems to new zfs filesystems does anyone know of any zfs conversion tools to make the conversion/migration from svm to zfs easier? Thanks Morris Hooten Unix SME Integrated Technology Delivery mhooten at us.ibm.com Office: 720-342-5614 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20130226/e7001202/attachment.html>
On 2013-02-26 21:30, Morris Hooten wrote:> Besides copying data from /dev/md/dsk/x volume manager filesystems to > new zfs filesystems > does anyone know of any zfs conversion tools to make the > conversion/migration from svm to zfs > easier?Do you mean something like a tool that would change metadata around your userdata in-place and turn an SVM volume into a ZFS pool, like Windows'' built-in FAT -> NTFS conversion? No, there''s nothing like it. However, depending on your old system''s configuration, you might have to be careful about choice of "copy" programs. Namely, if your setup used some ACLs (beyond standard POSIX access bits), then you''d need ACL-aware copying tools. Sun tar and cpio are some (see manpages about usage examples), rsync 3.0.10 was recently reported to support Solaris ACLs as well, but I didn''t test that myself. GNU tar and cpio are known to do a poor job with intimate Solaris features, though they might be superior for some other tasks. Basic (Sun, not GNU) cp and mv should work correctly too. I most often use "rsync -avPHK /src/ /dst/", especially if there are no ACLs to think about, or the target''s inheritable ACLs are acceptable (and overriding them with original''s access rights might even be wrong). Also, before you do the migration, think ahead of the storage and IO requirements for the datasets. For example, log files are often huge, compress into orders of magnitude less, and the IOPS loss might be negligible (or even boost, due to smaller hardware IOs and less seeks). Randomly accessed (written) data might not like heavier compressions. Databases or VM images might benefit from smaller maximum block sizes, although often these are not made 1:1 with DB block size, but rather balance about 4 DB entries in an FS block of 32Kb or 64Kb (from what I saw suggested on the list). Singly-written data, like OS images, might benefit from compression as well. If you have local zones, you might benefit from carrying over (or installing from scratch) one as a typical example "DUMMY" into a dedicated dataset, then cloning it into many actual zone roots as you''d need, and "rsync -cavPHK --delete-after" from originals into this dataset - this way only differing files (or parts thereof) would be transferred, giving you the benefits of cloning (space saving) without the downsides of deduplication. Also, for data in the zones (such as database files, tomcat/glassfish application server roots, etc.) you might like to use separate dataset hierarchies mounted via delegation of a "root" ZFS dataset into zones. This way your zoneroots would live a separate life from application data and non-packaged applications, which might simplify backups, etc. and you might be able to store these pieces in different pools (i.e. SSDs for some data and HDDs for other - though most list members would rightfully argue in favor of L2ARC on the SSDs). HTH, //Jim Klimov
On Feb 26, 2013, at 6:05 PM, Jim Klimov <jimklimov at cos.ru> wrote:> On 2013-02-26 21:30, Morris Hooten wrote: >> Besides copying data from /dev/md/dsk/x volume manager filesystems to >> new zfs filesystems >> does anyone know of any zfs conversion tools to make the >> conversion/migration from svm to zfs >> easier?> However, depending on your old system''s configuration, you might have > to be careful about choice of "copy" programs. Namely, if your setup > used some ACLs (beyond standard POSIX access bits), then you''d need > ACL-aware copying tools. Sun tar and cpio are some (see manpages about > usage examples), rsync 3.0.10 was recently reported to support Solaris > ACLs as well, but I didn''t test that myself. GNU tar and cpio are known > to do a poor job with intimate Solaris features, though they might be > superior for some other tasks. Basic (Sun, not GNU) cp and mv should > work correctly too.Under Solaris 10 I found ''cp -pr'' to be the both most reliable and fastest way to move data into, out of, and between ZFS datasets. -- Paul Kraus Deputy Technical Director, LoneStarCon 3 Sound Coordinator, Schenectady Light Opera Company
Ah, I forgot to mention - ufsdump|ufsrestore was at some time also a recommended way of such transition ;) I think it should be aware of all intimacies of the FS, including sparse files which reportedly may puzzle some other archivers. Although with any sort of ZFS compression (including lightweight zle) zero-filled blocks should translate into zero IOs. (Maybe some metadata would appear, to address the "holes", however). With proper handling of sparse files you don''t write any of that voidness into the FS and you don''t process anything on reads. Have fun, //Jim
On Feb 26, 2013, at 6:19 PM, Jim Klimov <jimklimov at cos.ru> wrote:> Ah, I forgot to mention - ufsdump|ufsrestore was at some time also > a recommended way of such transition ;)The last time I looked at using ufsdump/ufsrestore for this ufsrestore was NOT aware of ZFS ACL semantics. That was under Solaris 10, but I would be surprised if the ufsrestore code has changed since then. -- Paul Kraus Deputy Technical Director, LoneStarCon 3 Sound Coordinator, Schenectady Light Opera Company
what about Solaris live upgrade? On Wed, Feb 27, 2013 at 10:36 AM, Paul Kraus <paul at kraus-haus.org> wrote:> On Feb 26, 2013, at 6:19 PM, Jim Klimov <jimklimov at cos.ru> wrote: > > > Ah, I forgot to mention - ufsdump|ufsrestore was at some time also > > a recommended way of such transition ;) > > The last time I looked at using ufsdump/ufsrestore for this > ufsrestore was NOT aware of ZFS ACL semantics. That was under Solaris 10, > but I would be surprised if the ufsrestore code has changed since then. > > -- > Paul Kraus > Deputy Technical Director, LoneStarCon 3 > Sound Coordinator, Schenectady Light Opera Company > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-- *Alfredo* -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20130227/25027535/attachment.html>
Alfredo De Luca wrote:> On Wed, Feb 27, 2013 at 10:36 AM, Paul Kraus <paul at kraus-haus.org > <mailto:paul at kraus-haus.org>> wrote: > > On Feb 26, 2013, at 6:19 PM, Jim Klimov <jimklimov at cos.ru > <mailto:jimklimov at cos.ru>> wrote: > > > Ah, I forgot to mention - ufsdump|ufsrestore was at some time also > > a recommended way of such transition ;) > > The last time I looked at using ufsdump/ufsrestore for > this ufsrestore was NOT aware of ZFS ACL semantics. That was under > Solaris 10, but I would be surprised if the ufsrestore code has > changed since then. >> what about Solaris live upgrade? >It''s been a long time, but I''m sure LU only supports UFS->ZFS for the root pool. -- Ian.
Not sure http://docs.oracle.com/cd/E19253-01/821-0438/ggzdo/index.html On Wed, Feb 27, 2013 at 2:30 PM, Ian Collins <ian at ianshome.com> wrote:> Alfredo De Luca wrote: > >> On Wed, Feb 27, 2013 at 10:36 AM, Paul Kraus <paul at kraus-haus.org<mailto: >> paul at kraus-haus.org>> wrote: >> >> On Feb 26, 2013, at 6:19 PM, Jim Klimov <jimklimov at cos.ru >> <mailto:jimklimov at cos.ru>> wrote: >> >> > Ah, I forgot to mention - ufsdump|ufsrestore was at some time also >> > a recommended way of such transition ;) >> >> The last time I looked at using ufsdump/ufsrestore for >> this ufsrestore was NOT aware of ZFS ACL semantics. That was under >> Solaris 10, but I would be surprised if the ufsrestore code has >> changed since then. >> >> > what about Solaris live upgrade? >> >> > It''s been a long time, but I''m sure LU only supports UFS->ZFS for the root > pool. > > -- > Ian. > > > ______________________________**_________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/**mailman/listinfo/zfs-discuss<http://mail.opensolaris.org/mailman/listinfo/zfs-discuss> >-- *Alfredo* -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20130227/8685d7fc/attachment.html>
On 02/26/13 20:30, Morris Hooten wrote:> Besides copying data from /dev/md/dsk/x volume manager filesystems to > new zfs filesystems > does anyone know of any zfs conversion tools to make the > conversion/migration from svm to zfs > easier?With Solaris 11 you can use shadow migration, it is really a VFS layer feature but it is integrated into the ZFS CLI tools for easy of use # zfs create -o shadow=file:///path/to/old mypool/new The new filesystem will appear to instantly have all the data, and it will be copied over as it is access as well as shadowd pulling it over in advance. You can use shadowstat(1M) to show progress. -- Darren J Moffat
Hi Darren. you''re right! With solaris 11 and shadow migration feature it''s fantastic. Not sure which Solaria vers we are talking about here. Alfredo On Wed, Feb 27, 2013 at 10:22 PM, Darren J Moffat <darrenm at opensolaris.org>wrote:> > > On 02/26/13 20:30, Morris Hooten wrote: > >> Besides copying data from /dev/md/dsk/x volume manager filesystems to >> new zfs filesystems >> does anyone know of any zfs conversion tools to make the >> conversion/migration from svm to zfs >> easier? >> > > With Solaris 11 you can use shadow migration, it is really a VFS layer > feature but it is integrated into the ZFS CLI tools for easy of use > > # zfs create -o shadow=file:///path/to/old mypool/new > > The new filesystem will appear to instantly have all the data, and it will > be copied over as it is access as well as shadowd pulling it over in > advance. > > You can use shadowstat(1M) to show progress. > > -- > Darren J Moffat > > ______________________________**_________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/**mailman/listinfo/zfs-discuss<http://mail.opensolaris.org/mailman/listinfo/zfs-discuss> >-- *Alfredo* -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20130228/b7da220b/attachment.html>