Eugene M. Zheganin
2017-May-22 06:18 UTC
system is unresponsive and the amount of wired memory is cycling - zfs/iscsi ?
Hi. I'm using a FreeBSD 11.0-R server as a SAN system (with a native iSCSI target). It has 12 disks attached via external enclosure and a Megaraid SAS 3003 mrsas(4) controller. Actually I'm using several FreeBSD in similar configurations as SAN systems, but this one is frequently unresponsive. This often looks like top/gstat/dmesg are locked on start and waiting for something, then, after several minutes, everything is back to normal. In the same time ls and zpool list are working fine, but zfs clone is not. In the dmesg I got tonns if messages like ctl_datamove: tag 0x1086bc on (16:34:0) aborted ctl_datamove: tag 0x1086a8 on (16:34:0) aborted ctl_datamove: tag 0x10ec50 on (7:34:0) aborted ctl_datamove: tag 0xf1cd6 on (13:34:0) aborted ctl_datamove: tag 0xfcc88 on (15:34:0) aborted ctl_datamove: tag 0xd3ed1 on (21:34:0) aborted ctl_datamove: tag 0x1056f8 on (1:34:0) aborted Not sure if they are related to the lock issue I'm having, so I thought I will just mention them. It seems like the system is starving on some resources, but I really have no idea about what could it be. When this lock is happening, a launched top stops to update it's output, so I don't know what is happening, I let the system run with default zfs settings, top show that the ARC has eaten out all of the memory. One more importang thing - just befor top stops refereshing the screen it shows that ARC has eaten all the memory, then, after lock is gone, top shows the server has 11G free memory back, so this happens in cycles and definitely has something to do with the amount of the wired memory. Should I tune the zfs subsystem in some way ? If yes - what exactly should I tune ? I'm also running this system with a CTL patch increasing the target limit to 1024, may be this is eating some additional kernel memory, don't know if this is the source of this locking issue. I'm running other systems with the same patch, but they are running smoothly. Thanks. Eugene.