I notice from the release notes that ZFS has been changed in an incompatible way. This concerns me deeply and I hope that the ARM porting team can be convinced to reverse this change to ensure that the ZFS in the ARM port is fully on disk compatible with the SPARC, Intel, etc ports of ZFS on OpenSolaris and other operating systems. I can fully understand the need to reduce the in RAM impact of ZFS I can even understand why being able to compile out certain features is desirable but what I don''t understand is why the on disk format was changed. What is actually being gained by this and why is it so important for ARM ? I see from looking at the code changes a completely new filesystem ''czfs'' type was created does this mean that the ARM port supports CZFS and ZFS ? My concern here is that the changes could make it difficult for adding encryption to the ARM port if it can''t use the normal ZFS - in particular the encryption support needs to take the 3rd DVA to store the IV and the upper two 64bit words of the checksum for the crypto MAC. However aside from encryption also I''m very concerned about being able to remove drives (encrypted or not) from a system running the ARM port and read them on SPARC or Intel running default OpenSolaris. Is there a document somewhere I can read about the design choices for CFS, particularly the on disk changes ? -- Darren J Moffat
I think this reveals a more interesting challenge. If the on-disk format changes when we add features, and feature additions are represented by incrementing the version, then we might create a reason to fork. Perhaps we should look at having an implemented features list instead? For example, if CZFS does not implement user quotas, then it could just ignore them if they do not exist on the medium. -- richard Darren J Moffat wrote:> I notice from the release notes that ZFS has been changed in an > incompatible way. This concerns me deeply and I hope that the ARM > porting team can be convinced to reverse this change to ensure that > the ZFS in the ARM port is fully on disk compatible with the SPARC, > Intel, etc ports of ZFS on OpenSolaris and other operating systems. > > I can fully understand the need to reduce the in RAM impact of ZFS I > can even understand why being able to compile out certain features is > desirable but what I don''t understand is why the on disk format was > changed. What is actually being gained by this and why is it so > important for ARM ? > > I see from looking at the code changes a completely new filesystem > ''czfs'' type was created does this mean that the ARM port supports CZFS > and ZFS ? > > My concern here is that the changes could make it difficult for adding > encryption to the ARM port if it can''t use the normal ZFS - in > particular the encryption support needs to take the 3rd DVA to store > the IV and the upper two 64bit words of the checksum for the crypto > MAC. However aside from encryption also I''m very concerned about being > able to remove drives (encrypted or not) from a system running the ARM > port and read them on SPARC or Intel running default OpenSolaris. > > Is there a document somewhere I can read about the design choices for > CFS, particularly the on disk changes ? >
Richard Elling wrote:> I think this reveals a more interesting challenge. If the on-disk format > changes when we add features, and feature additions are represented > by incrementing the version, then we might create a reason to fork. > Perhaps we should look at having an implemented features list instead? > For example, if CZFS does not implement user quotas, then it could > just ignore them if they do not exist on the medium.CZFS effectively does that already by returning ENOTSUP from some functions like ACL or quotas. The big issue though is that CZFS compacted some critical on disk structure like the blkptr_t, compare the CZFS blkptr at with the normal ZFS one at: http://src.opensolaris.org/source/xref/osarm/onarm-gate/usr/src/uts/common/fs/czfs/sys/spa.h#47 http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/sys/spa.h#120 That is the incompatible on disk part that is problematic, in particular it means CZFS can''t support encryption because the space needed to store the IV has been taken away. It might also not be able to support bp_write since there may be a need to store additional info in the blkptr_t for "moved" blocks. The feature additions as a list could be done still and has been discussed before. -- Darren J Moffat
Koji Uno wrote:> Hi, > > I''d just like you to know that UFS and normal ZFS are also > available in OpenSolaris for ARM.That is good to know.> The CZFS is one of challenges for the embedded systems. > > There are many requests to remove unused functions > in the embedded system. When it is the closed embedded > system, it become more strong request. The reason is > to reduce the costs of hardware or testing.I fully understand the need for a low memory footprint version of ZFS and in particular one that doesn''t support all the features: eg no RAIDZ2 no Quotas etc, no compression even, only supports limited checksums. However that doesn''t, require changing the size of fundamental on disk structures like the block pointer such that they are no longer compatible with a full featured ZFS. Is disk space for the target systems really that tight that we have to squash the size of the block pointer ? The bit I''m missing about CZFS isn''t the feature removal part but the part that changes on disk structures such that a "normal" ZFS can''t read it anymore. In other words why is this effectively a fork of the on disk ZFS structure rather than a partially functioning smaller memory footprint ZFS with the same on disk structure. For example the cut down readonly ZFS that is in GRUB is a little like and embedded version it has tight memory constraints and doesn''t need to understand all ZFS features. It does however need to be able to read a "normal" ZFS pool. The GRUB version has restrictions though, eg no gzip support. -- Darren J Moffat
I dont like an incompatibility. In my POW, compatibility is essential. I would prefer to loose some functions in ARM version of ZFS, instead of loosing entire compatibility -- This message posted from opensolaris.org
On Thu, Jun 25, 2009 at 9:19 AM, Koji Uno<Koji.Uno at sun.com> wrote:> Hi, > > NEC who are the contributor of ARM code, provided their outcomes > of study to us.[details snipped]> > It is the screen shot when OpenSolaris is booted in 16MB memory (ARC is > 1MB).May be when operating devices with such a small memory ZFS is not the right choice at all ? The on-disk image would not be compatible with the rest of the world, so you very well may just have other file system, more suitable for limited memory configuration. How about squashfs for example ? -- Regards, Cyril
Koji Uno wrote:> Hi, > > NEC who are the contributor of ARM code, provided their outcomes > of study to us. > > The size of ZFS runtime memory is measured by > - Dump content of runtime memory into file after booting > - Work it out by NEC internal tool > - Accumulate values of kmem > > Without changes of disk structure, they confirmed that the > consumption of ZFS runtime memory could be reduced to 8MB. > The ARC was limited to 4MB and it was minimum value in this > case. > > When the disk structure was changed as follows, further reduction > was realized. It resulted in 1MB ARC and 2MB ZFS runtime memory. > > - The size of dnode_phys_t structure was changed 512B into 256B. > > * See dnode_phys_t in uts/common/fs/zfs/sys/dnode.h > with uts/common/fs/czfs/sys/dnode.h > * See blkptr_t in uts/common/fs/zfs/sys/spa.h > with uts/common/fs/czfs/sys/spa.hThis change means that CZFS can''t support crypto in the same way that ZFS does and that IMO is really bad. An encrypted ZFS is highly desirable for embedded systems (think an ARM powered smartphone!). I can work around this but again it makes it different to ZFS and that isn''t good.> * See znode_phys_t in uts/common/fs/zfs/sys/zfs_znode.h > with uts/common/fs/czfs/sys/zfs_znode.hThis is going to be changing soon anyway because of another project, I''ll let the person working on it speak up if they wish to discuss that in public just now. -- Darren J Moffat