Hi everyone, I have a couple of questions regarding FreeBSD''s ZFS support. Firstly, I believe it currently stands at zpool v28. Is this correct? Will this be updated any time soon? Also, looking at the Wikipedia page, the updates beyond this are: 29 Solaris Nevada b148 RAID-Z/mirror hybrid allocator. 30 Solaris Nevada b149 ZFS encryption. 31 Solaris Nevada b150 improved ''zfs list'' performance. 32 Solaris Nevada b151 One MB block support 33 Solaris Nevada b163 Improved share support I am not currently interested in encryption, but what are the advantages of the other improvements? If I were to use Solaris 11 11/11 on a small file server (running 16GB RAM and 3TB storage in 2 mirrored pairs) would I see any improvement in upgrading from v28 created under FreeBSD 9? Thanks Karl
2012-08-09 13:57, Karl Wagner wrote:> Hi everyone, > > I have a couple of questions regarding FreeBSD''s ZFS support. > > Firstly, I believe it currently stands at zpool v28. Is this correct? > Will this be updated any time soon? > > Also, looking at the Wikipedia page, the updates beyond this are: > 29 Solaris Nevada b148 RAID-Z/mirror hybrid allocator. > 30 Solaris Nevada b149 ZFS encryption. > 31 Solaris Nevada b150 improved ''zfs list'' performance. > 32 Solaris Nevada b151 One MB block support > 33 Solaris Nevada b163 Improved share support > > I am not currently interested in encryption, but what are the advantages > of the other improvements? If I were to use Solaris 11 11/11 on a small > file server (running 16GB RAM and 3TB storage in 2 mirrored pairs) would > I see any improvement in upgrading from v28 created under FreeBSD 9?From what I gather, ZFS features v29 and beyond are proprietary to Oracle, so unless their licensing changes and/or the code is officially legally published as grabbable open-source, it is not likely that these features will ever appear in non-Oracle ZFS implementations. There is even some FUD regarding the use of same zpool version numbers for open-sourced reimplementations of identical features (so open and proprietary zpools are compatible), and whether that won''t be sued. In the end, the open-sourced ZFS community got no public replies from Oracle regarding collaboration or lack thereof, and decided to part ways and implement things independently from Oracle. AFAIK main ZFS development converges in illumos-gate, contributed to by some OpenSolaris-derived distros and being the upstream for FreeBSD port of ZFS (probably others too). Lacking an authority to assign zpool version numbers to particular features, they instead went for enumeratable feature flags which report whether a particular zfs/zpool format feature is in use on the pool and supported by the software trying to import it. New features in the works include modernized compression and checksum algorithms, among others. Nominal zpool version is 5000 for pools which enabled feature flags, and that is currently supported by oi_151a5 prebuilt distro (I don''t know of other builds with that - feature integrated into code this summer). Regarding your other question, what the v29+ features provide, here''s my understanding: 29 Solaris Nevada b148 RAID-Z/mirror hybrid allocator. The miniature metadata blocks are allocated by mirroring sectors instead of raidz-encumbering them, which makes tasks with metadata faster and probably reduces associated storage and processing overheads. 30 Solaris Nevada b149 ZFS encryption. Encryption of datasets, pools and/or objects? 31 Solaris Nevada b150 improved ''zfs list'' performance. Probably a performance bump 32 Solaris Nevada b151 One MB block support Should improve efficiency of large file storage, especially on modern 4Kb sectored disks, by reducing the needed portion of metadata overhead and fragmentation (more data is written sequentially, low-level prefetches win more). Writes on *very full* pools might suffer, because it is less likely to quickly find an available block big enough. 33 Solaris Nevada b163 Improved share support Probably a performance and/or interoperability bump HTH, //Jim Klimov BTW, are you sure your intended use of Solaris 11 fits into the free usage license restrictions (dev/POC for Solaris, basically)? This is not a rhethorical question, because I know some home-users who were uncertain if they can use Sol11 as their home-NAS OS or a home desktop or their small office server, and just to be surely on the safe side, switched to some of the other distros. I would welcome enlightened comments to this part ;)
Jim Klimov <jimklimov at cos.ru> wrote:> In the end, the open-sourced ZFS community got no public replies > from Oracle regarding collaboration or lack thereof, and decided > to part ways and implement things independently from Oracle. > AFAIK main ZFS development converges in illumos-gate, contributed > to by some OpenSolaris-derived distros and being the upstream for > FreeBSD port of ZFS (probably others too).To me it seems that the "open-sourced ZFS community" is not open, or could you point me to their mailing list archives? J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
On 2012-08-09 11:35, Jim Klimov wrote:> 2012-08-09 13:57, Karl Wagner wrote: >> Hi everyone, >> >> I have a couple of questions regarding FreeBSD''s ZFS support. >> >> Firstly, I believe it currently stands at zpool v28. Is this >> correct? >> Will this be updated any time soon? >> >> Also, looking at the Wikipedia page, the updates beyond this are: >> 29 Solaris Nevada b148 RAID-Z/mirror hybrid allocator. >> 30 Solaris Nevada b149 ZFS encryption. >> 31 Solaris Nevada b150 improved ''zfs list'' performance. >> 32 Solaris Nevada b151 One MB block support >> 33 Solaris Nevada b163 Improved share support >> >> I am not currently interested in encryption, but what are the >> advantages >> of the other improvements? If I were to use Solaris 11 11/11 on a >> small >> file server (running 16GB RAM and 3TB storage in 2 mirrored pairs) >> would >> I see any improvement in upgrading from v28 created under FreeBSD 9? > > From what I gather, ZFS features v29 and beyond are proprietary > to Oracle, so unless their licensing changes and/or the code is > officially legally published as grabbable open-source, it is not > likely that these features will ever appear in non-Oracle ZFS > implementations. > > There is even some FUD regarding the use of same zpool version > numbers for open-sourced reimplementations of identical features > (so open and proprietary zpools are compatible), and whether that > won''t be sued. > > In the end, the open-sourced ZFS community got no public replies > from Oracle regarding collaboration or lack thereof, and decided > to part ways and implement things independently from Oracle. > AFAIK main ZFS development converges in illumos-gate, contributed > to by some OpenSolaris-derived distros and being the upstream forThank you for the info. Looking at your responses, I believe I may gain some advantage from an upgrade to v28, particularly from the hybrid alloc and 1MB blocks. However, I don''t think it is likely to be worth sacrificing compatibility with other solutions. Regarding licensing, I am not 100% certain of this. I, personally, count my entire home network as a dev platform (much to the dismay of my other half), and use it to learn stuff for work and/or personal projects. I doubt, however, that this fits Oracle''s definition of development. This is another good reason for me to maintain "backwards" compatibility, so even if I decide to try out Sol11 I doubt I will be upgrading the pool. Thanks Karl> FreeBSD port of ZFS (probably others too). > > Lacking an authority to assign zpool version numbers to particular > features, they instead went for enumeratable feature flags which > report whether a particular zfs/zpool format feature is in use > on the pool and supported by the software trying to import it. > New features in the works include modernized compression and > checksum algorithms, among others. Nominal zpool version is 5000 > for pools which enabled feature flags, and that is currently > supported by oi_151a5 prebuilt distro (I don''t know of other > builds with that - feature integrated into code this summer). > > Regarding your other question, what the v29+ features provide, > here''s my understanding: > 29 Solaris Nevada b148 RAID-Z/mirror hybrid allocator. > The miniature metadata blocks are allocated by mirroring > sectors instead of raidz-encumbering them, which makes > tasks with metadata faster and probably reduces associated > storage and processing overheads. > > 30 Solaris Nevada b149 ZFS encryption. > Encryption of datasets, pools and/or objects? > > 31 Solaris Nevada b150 improved ''zfs list'' performance. > Probably a performance bump > > 32 Solaris Nevada b151 One MB block support > Should improve efficiency of large file storage, especially > on modern 4Kb sectored disks, by reducing the needed portion > of metadata overhead and fragmentation (more data is written > sequentially, low-level prefetches win more). Writes on *very > full* pools might suffer, because it is less likely to quickly > find an available block big enough. > > 33 Solaris Nevada b163 Improved share support > Probably a performance and/or interoperability bump > > HTH, > //Jim Klimov > > BTW, are you sure your intended use of Solaris 11 fits into the > free usage license restrictions (dev/POC for Solaris, basically)? > This is not a rhethorical question, because I know some home-users > who were uncertain if they can use Sol11 as their home-NAS OS or > a home desktop or their small office server, and just to be surely > on the safe side, switched to some of the other distros. I would > welcome enlightened comments to this part ;)
On 08/09/2012 12:52 PM, Joerg Schilling wrote:> Jim Klimov <jimklimov at cos.ru> wrote: > >> In the end, the open-sourced ZFS community got no public replies >> from Oracle regarding collaboration or lack thereof, and decided >> to part ways and implement things independently from Oracle. >> AFAIK main ZFS development converges in illumos-gate, contributed >> to by some OpenSolaris-derived distros and being the upstream for >> FreeBSD port of ZFS (probably others too). > > To me it seems that the "open-sourced ZFS community" is not open, or could you > point me to their mailing list archives? > > J?rg >zfs at lists.illumos.org Welcome. -- Saso
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Joerg Schilling > > Jim Klimov <jimklimov at cos.ru> wrote: > > > In the end, the open-sourced ZFS community got no public replies > > from Oracle regarding collaboration or lack thereof, and decided > > to part ways and implement things independently from Oracle. > > AFAIK main ZFS development converges in illumos-gate, contributed > > to by some OpenSolaris-derived distros and being the upstream for > > FreeBSD port of ZFS (probably others too). > > To me it seems that the "open-sourced ZFS community" is not open, or could > you > point me to their mailing list archives?The last publicly released under opensolaris zpool version, v28, is incorporated into illumos / openindiana / nexenta / etc. Jim is talking about how Oracle closed-source after v28, and never issued any public statement confirming or denying that they were going closed-source. They just never released any source anymore. So the community spun off with the latest open source code, and there it is... Since that time, open source ZFS development has continued, but not at the same rate and not in the same direction as the oracle closed-source.
Sa?o Kiselkov <skiselkov.ml at gmail.com> wrote:> > To me it seems that the "open-sourced ZFS community" is not open, or could you > > point me to their mailing list archives? > > > > J?rg > > > > zfs at lists.illumos.orgWell, why then has there been a discussion about a "closed zfs mailing list"? Is this no longer true? J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
On 08/09/2012 01:05 PM, Joerg Schilling wrote:> Sa?o Kiselkov <skiselkov.ml at gmail.com> wrote: > >>> To me it seems that the "open-sourced ZFS community" is not open, or could you >>> point me to their mailing list archives? >>> >>> J?rg >>> >> >> zfs at lists.illumos.org > > Well, why then has there been a discussion about a "closed zfs mailing list"? > Is this no longer true?Not that I know of. The above one is where I post my changes and Matt, George, Garrett and all the others are lurking there. Cheers, -- Saso
Sa?o Kiselkov <skiselkov.ml at gmail.com> wrote:> On 08/09/2012 01:05 PM, Joerg Schilling wrote: > > Sa?o Kiselkov <skiselkov.ml at gmail.com> wrote: > > > >>> To me it seems that the "open-sourced ZFS community" is not open, or could you > >>> point me to their mailing list archives? > >>> > >>> J?rg > >>> > >> > >> zfs at lists.illumos.org > > > > Well, why then has there been a discussion about a "closed zfs mailing list"? > > Is this no longer true? > > Not that I know of. The above one is where I post my changes and Matt, > George, Garrett and all the others are lurking there.So if you frequently read this list, can you tell me whether they discuss the on-disk format in this list? J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
On 08/09/2012 01:11 PM, Joerg Schilling wrote:> Sa?o Kiselkov <skiselkov.ml at gmail.com> wrote: > >> On 08/09/2012 01:05 PM, Joerg Schilling wrote: >>> Sa?o Kiselkov <skiselkov.ml at gmail.com> wrote: >>> >>>>> To me it seems that the "open-sourced ZFS community" is not open, or could you >>>>> point me to their mailing list archives? >>>>> >>>>> J?rg >>>>> >>>> >>>> zfs at lists.illumos.org >>> >>> Well, why then has there been a discussion about a "closed zfs mailing list"? >>> Is this no longer true? >> >> Not that I know of. The above one is where I post my changes and Matt, >> George, Garrett and all the others are lurking there. > > So if you frequently read this list, can you tell me whether they discuss the > on-disk format in this list?It''s more of a list for development discussion and integration of changes, not a list for general ZFS discussion like zfs-discuss is. -- Saso
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Joerg Schilling > > Well, why then has there been a discussion about a "closed zfs mailing list"? > Is this no longer true?Oracle can do anything internally they want. I would presume they have an internal mailing list for zfs developers, but that''s in relation to the closed-source zfs that they develop. Not in relation to the open-source zfs that''s used in illumos, etc.
opensolarisisdeadlongliveopensolaris <opensolarisisdeadlongliveopensolaris at nedharvey.com> wrote:> > From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > > bounces at opensolaris.org] On Behalf Of Joerg Schilling > > > > Well, why then has there been a discussion about a "closed zfs mailing list"? > > Is this no longer true? > > Oracle can do anything internally they want. I would presume they have an internal mailing list for zfs developers, but that''s in relation to the closed-source zfs that they develop. Not in relation to the open-source zfs that''s used in illumos, etc.I was talking about illumos... J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
> From: Joerg Schilling [mailto:Joerg.Schilling at fokus.fraunhofer.de] > Sent: Thursday, August 09, 2012 11:35 AM > > > > From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > > > bounces at opensolaris.org] On Behalf Of Joerg Schilling > > > > > > Well, why then has there been a discussion about a "closed zfs mailing > list"? > > > Is this no longer true? > > > > Oracle can do anything internally they want. I would presume they have an > internal mailing list for zfs developers, but that''s in relation to the closed- > source zfs that they develop. Not in relation to the open-source zfs that''s > used in illumos, etc. > > I was talking about illumos...Then why are you talking about a closed zfs mailing list? Have you heard of an illumos closed-zfs mailing list that I haven''t heard of?
On 2012-Aug-09 16:05:00 +0530, Jim Klimov <jimklimov at cos.ru> wrote:>2012-08-09 13:57, Karl Wagner wrote: >> Firstly, I believe it currently stands at zpool v28. Is this correct?For FreeBSD 8.x and 9.x, yes. FreeBSD-head includes "feature flags" and "com.delphix:async_destroy".>> Will this be updated any time soon?I expect 8-stable and 9-stable will be update to match -head once FreeBSD 9.1 is released (ie 9.1 won''t support feature flags but 9.2 and a potential 8.4 will). In general, FreeBSD imports ZFS fixes and enhancements, generally from Illumos, as they become available. The Oracle v29 and later updates won''t be available in FreeBSD unless they are open-sourced by Oracle.>New features in the works include modernized compression and >checksum algorithms, among others. Nominal zpool version is 5000 >for pools which enabled feature flags, and that is currently >supported by oi_151a5 prebuilt distro (I don''t know of other >builds with that - feature integrated into code this summer).FreeBSD-head does. -- Peter Jeremy -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20120810/d283c226/attachment.bin>
On Aug 9, 2012, at 4:11 AM, Joerg.Schilling at fokus.fraunhofer.de (Joerg Schilling) wrote:> Sa?o Kiselkov <skiselkov.ml at gmail.com> wrote: > >> On 08/09/2012 01:05 PM, Joerg Schilling wrote: >>> Sa?o Kiselkov <skiselkov.ml at gmail.com> wrote: >>> >>>>> To me it seems that the "open-sourced ZFS community" is not open, or could you >>>>> point me to their mailing list archives? >>>>> >>>>> J?rg >>>>> >>>> >>>> zfs at lists.illumos.org >>> >>> Well, why then has there been a discussion about a "closed zfs mailing list"? >>> Is this no longer true? >> >> Not that I know of. The above one is where I post my changes and Matt, >> George, Garrett and all the others are lurking there. > > So if you frequently read this list, can you tell me whether they discuss the > on-disk format in this list?Yes, but nobody has posted proposals for new on-disk format changes since feature flags was first announced. NB, the zfs at lists.illumos.org is but one of the many discuss groups where ZFS users can get questions answered. There is also active Mac OSX, ZFS on Linux, and OTN lists. IMHO, zfs-discuss at opensolaris is shrinking, not growing. -- richard