zfs discuss - Oct 2008 - "zfs set sharenfs" takes a long time to return.

I have an X4500 fileserver (NFS, Samba) running OpenSolaris 2008.05 pkg upgraded
to snv_91  with ~3200 filesystems (and ~27429 datasets, including snapshots).

I''ve been encountering some pretty big slow-downs on this system when
running certain zfs commands.  The one causing me the most pain at the moment is
setting the "sharenfs" property on a filesystem takes a little under 7
minutes per operation.

Tracing it, I see that it''s making a huge number of ioctl calls:

$ truss -f -c  zfs set sharenfs=on zpool/test

syscall               seconds   calls  errors
_exit                    .000       1
read                     .150    8618
open                     .166    6826       2
close                    .049    6839
brk                      .022    1246
lseek                    .000       3
getpid                   .039    7357
sysi86                   .000       1
ioctl                  32.834 6008913   10239
execve                   .000       1
fcntl                    .000      18
openat                   .000       1
getppriv                 .000       1
getprivimplinfo          .000       1
issetugid                .000       4
sigaction                .000       1
sigfillset               .000       1
getcontext               .000       1
setustack                .000       1
mmap                     .000      74
munmap                   .000      29
xstat                    .003      43      19
getrlimit                .000       1
memcntl                  .000      15
sysconfig                .000       5
lwp_sigmask              .000       2
lwp_private              .000       1
llseek                   .089   13688
door_info                .000      14
door_call                .086    7130
schedctl                 .000       1
resolvepath              .000      18
getdents64               .000       2
stat64                   .000       3
fstat64                  .112   20377
zone_getattr             .000       1
zone_lookup              .000       3
                     --------  ------   ----
sys totals:            33.556 6081241  10260
usr time:              41.354
elapsed:              386.130

Looking at exactly which ioctls get called:

5991763 MNTIOC_GETMNTENT
6782 TCGETA
6892 ZFS_IOC_DATASET_LIST_NEXT
3466 ZFS_IOC_OBJSET_STATS
   2 ZFS_IOC_POOL_CONFIGS
   1 ZFS_IOC_SET_PROP
   1 ZFS_IOC_SHARE
   8 ZFS_IOC_SNAPSHOT_LIST_NEXT

Is anyone else experiencing this, or does anyone have any ideas on how to solve
it? Is this solved in more recently snv releases?  (And are there changelogs
posted somewhere?)

Thanks!
-Laen
--
This message posted from opensolaris.org