TL;DR Summary: I can run FreeBSD out of memory quite consistently, and it?s
not a TOS/mbuf exhaustion issue. It?s quite possible that ZFS is the
culprit, but shouldn?t the pager be able to handle aggressive memory
requests in a low memory situation gracefully, without needing custom
tuning of ZFS / VM?
Hello,
I?ve been dealing with some instability in my 10.1-RELEASE and
STABLEr282701M machines for the last few months.
These machines are NFS/iSCSI storage machines, running on Dell M610x or
similar hardware, 96 Gig Memory, 10Gig Network Cards, dual Xeon Processors
? Fairly beefy stuff.
Initially I thought it was more issues with TOS / jumbo mbufs, as I had
this problem last year. I had thought that this was properly resolved, but
setting my MTU to 1500, and turning off TOS did give me a bit more
stability. Currently all my machines are set this way.
Crashes were usually represented by loss of network connectivity, and the
ctld daemon scrolling messages across the screen at full speed about lost
connections.
All of this did seem like more network stack problems, but with each crash
I?d be able to learn a bit more.
Usually there was nothing of any use in the logfile, but every now and then
I?d get this:
Jun 3 13:02:04 san0 kernel: WARNING: 172.16.0.97
(iqn.1998-01.com.vmware:esx5a-3387a188): failed to allocate memory
Jun 3 13:02:04 san0 kernel: WARNING: icl_pdu_new: failed to allocate 80
bytes
Jun 3 13:02:04 san0 kernel: WARNING: 172.16.0.97
(iqn.1998-01.com.vmware:esx5a-3387a188): failed to allocate memory
Jun 3 13:02:04 san0 kernel: WARNING: icl_pdu_new: failed to allocate 80
bytes
Jun 3 13:02:04 san0 kernel: WARNING: 172.16.0.97
(iqn.1998-01.com.vmware:esx5a-3387a188): failed to allocate memory
---------
Jun 4 03:03:09 san0 kernel: WARNING: icl_pdu_new: failed to allocate 80
bytes
Jun 4 03:03:09 san0 kernel: WARNING: icl_pdu_new: failed to allocate 80
bytes
Jun 4 03:03:09 san0 kernel: WARNING: 172.16.0.97
(iqn.1998-01.com.vmware:esx5a-3387a188): failed to allocate memory
Jun 4 03:03:09 san0 kernel: WARNING: 172.16.0.97
(iqn.1998-01.com.vmware:esx5a-3387a188): connection error; dropping
connection
Jun 4 03:03:09 san0 kernel: WARNING: 172.16.0.97
(iqn.1998-01.com.vmware:esx5a-3387a188): connection error; dropping
connection
Jun 4 03:03:10 san0 kernel: WARNING: 172.16.0.97
(iqn.1998-01.com.vmware:esx5a-3387a188): waiting for CTL to terminate
tasks, 1 remaining
Jun 4 06:04:27 san0 syslogd: kernel boot file is /boot/kernel/kernel
So knowing that it seemed to be running out of memory, I started leaving
leaving ?vmstat 5? running on a console, to see what it was displaying
during the crash.
It was always the same thing:
0 0 0 1520M 4408M 15 0 0 0 25 19 0 0 21962 1667 91390
0 33 67
0 0 0 1520M 4310M 9 0 0 0 2 15 3 0 21527 1385 95165
0 31 69
0 0 0 1520M 4254M 7 0 0 0 14 19 0 0 17664 1739 72873
0 18 82
0 0 0 1520M 4145M 2 0 0 0 0 19 0 0 23557 1447 96941
0 36 64
0 0 0 1520M 4013M 4 0 0 0 14 19 0 0 4288 490 34685
0 72 28
0 0 0 1520M 3885M 2 0 0 0 0 19 0 0 11141 1038 69242
0 52 48
0 0 0 1520M 3803M 10 0 0 0 14 19 0 0 24102 1834 91050
0 33 67
0 0 0 1520M 8192B 2 0 0 0 2 15 1 0 19037 1131 77470
0 45 55
0 0 0 1520M 8192B 0 22 0 0 2 0 6 0 146 82 578 0
0 100
0 0 0 1520M 8192B 1 0 0 0 0 0 0 0 130 40 510 0
0 100
0 0 0 1520M 8192B 0 0 0 0 0 0 0 0 143 40 501 0
0 100
0 0 0 1520M 8192B 0 0 0 0 0 0 0 0 201 62 660 0
0 100
0 0 0 1520M 8192B 0 0 0 0 0 0 0 0 101 28 404 0
0 100
0 0 0 1520M 8192B 0 0 0 0 0 0 0 0 97 27 398 0
0 100
0 0 0 1520M 8192B 0 0 0 0 0 0 0 0 93 28 377 0
0 100
0 0 0 1520M 8192B 0 0 0 0 0 0 0 0 92 27 373 0
0 100
I?d go from a decent amount of free memory to suddenly having none. Vmstat
would stop outputting, console commands would hang, etc. The whole system
would be useless.
Looking into this, I came across a similar issue;
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199189
I started increasing v.v_free_min, and it helped ? My crashes went from
being ~every 6 hours to every few days.
Currently I?m running with vm.v_free_min=1254507 ? That?s (1254507 * 4KiB)
, or 4.78GiB of Reserve. The vmstat above is of a machine with that
setting still running to 8B of memory.
I have two issues here:
1) I don?t think I should ever be able to run the system into the ground on
memory. Deny me new memory until the pager can free more.
2) Setting ?min? doesn?t really mean ?min? as it can obviously go below
that threshold.
I have plenty of local UFS swap (non-ZFS drives)
Adrian requested that I output a few more diagnostic items, and this is
what I?m running on a console now, in a loop:
vmstat
netstat -m
vmstat -z
sleep 1
The output of four crashes are attached here, as they can be a bit long.
Let me know if that?s not a good way to report them. They will each start
mid-way through a vmstat ?z output, as that?s as far back as my terminal
buffer allows.
Now, I have a good idea of the conditions that are causing this: ZFS
Snapshots, run by cron, during times of high ZFS writes.
The crashes are all nearly on the hour, as that?s when crontab triggers my
python scripts to make new snapshots, and delete old ones.
My average FreeBSD machine has ~ 30 zfs datasets, with each pool having ~20
TiB used. These all need to snapshot on the hour.
By staggering the snapshots by a few minutes, I have been able to reduce
crashing from every other day to perhaps once a week if I?m lucky ? But if
I start moving a lot of data around, I can cause daily crashes again.
It?s looking to be the memory demand of snapshotting lots of ZFS datasets
at the same time while accepting a lot of write traffic.
Now perhaps the answer is ?don?t do that? but I feel that FreeBSD should be
robust enough to handle this. I don?t mind tuning for now to
reduce/eliminate this, but others shouldn?t run into this pain just because
they heavily load their machines ? There must be a way of avoiding this
condition.
Here are the contents of my /boot/loader.conf and sysctl.conf, so show my
minimal tuning to make this problem a little more bearable:
/boot/loader.conf
vfs.zfs.arc_meta_limit=49656727553
vfs.zfs.arc_max = 91489280512
/etc/sysctl.conf
vm.v_free_min=1254507
Any suggestions/help is appreciated.
Thank you.
-------------- next part --------------
SAN2 before Crash
zio_data_buf_57344: 57344, 0, 1, 27, 2879517, 0, 0
zio_buf_61440: 61440, 0, 0, 53, 8211932, 0, 0
zio_data_buf_61440: 61440, 0, 1, 21, 2089, 0, 0
zio_buf_65536: 65536, 0, 451, 614,332616682, 0, 0
zio_data_buf_65536: 65536, 0, 270683, 1219,483915352, 0, 0
zio_buf_69632: 69632, 0, 0, 22, 6244181, 0, 0
zio_data_buf_69632: 69632, 0, 1, 22, 2442, 0, 0
zio_buf_73728: 73728, 0, 0, 22, 7105463, 0, 0
zio_data_buf_73728: 73728, 0, 0, 25, 2078, 0, 0
zio_buf_77824: 77824, 0, 0, 22, 5360376, 0, 0
zio_data_buf_77824: 77824, 0, 0, 22, 2076, 0, 0
zio_buf_81920: 81920, 0, 0, 25, 6976233, 0, 0
zio_data_buf_81920: 81920, 0, 0, 20, 2077, 0, 0
zio_buf_86016: 86016, 0, 0, 27, 5387262, 0, 0
zio_data_buf_86016: 86016, 0, 0, 21, 2070, 0, 0
zio_buf_90112: 90112, 0, 2, 21, 6362572, 0, 0
zio_data_buf_90112: 90112, 0, 1, 23, 2551, 0, 0
zio_buf_94208: 94208, 0, 1, 22, 6067105, 0, 0
zio_data_buf_94208: 94208, 0, 0, 24, 2114, 0, 0
zio_buf_98304: 98304, 0, 1, 21, 8587915, 0, 0
zio_data_buf_98304: 98304, 0, 0, 21, 2104, 0, 0
zio_buf_102400: 102400, 0, 0, 26, 6589062, 0, 0
zio_data_buf_102400: 102400, 0, 0, 21, 2061, 0, 0
zio_buf_106496: 106496, 0, 1, 23, 7280570, 0, 0
zio_data_buf_106496: 106496, 0, 0, 24, 2009, 0, 0
zio_buf_110592: 110592, 0, 6, 19, 7767692, 0, 0
zio_data_buf_110592: 110592, 0, 0, 20, 1903, 0, 0
zio_buf_114688: 114688, 0, 1, 24, 9118617, 0, 0
zio_data_buf_114688: 114688, 0, 1, 21, 1901, 0, 0
zio_buf_118784: 118784, 0, 1, 23, 8268813, 0, 0
zio_data_buf_118784: 118784, 0, 0, 23, 1740, 0, 0
zio_buf_122880: 122880, 0, 0, 31,11224918, 0, 0
zio_data_buf_122880: 122880, 0, 0, 20, 1668, 0, 0
zio_buf_126976: 126976, 0, 0, 27,10713491, 0, 0
zio_data_buf_126976: 126976, 0, 0, 19, 1648, 0, 0
zio_buf_131072: 131072, 0, 13511, 1208,254193013, 0, 0
zio_data_buf_131072: 131072, 0, 0, 14, 56696, 0, 0
lz4_ctx: 16384, 0, 0, 52,1658805651, 0, 0
sa_cache: 80, 0, 234301, 83949, 5033876, 0, 0
dnode_t: 752, 0, 325173, 23822, 7846128, 0, 0
dmu_buf_impl_t: 232, 0, 2420613, 1092794,1018835522, 0, 0
arc_buf_hdr_t: 216, 0,31470560, 6580072,704775126, 0, 0
arc_buf_t: 72, 0, 2303174, 1444251,842979860, 0, 0
zil_lwb_cache: 192, 0, 32, 1928,118978505, 0, 0
zfs_znode_cache: 368, 0, 234301, 38139, 5033876, 0, 0
Mountpoints: 816, 0, 33, 137, 37, 0, 0
ksiginfo: 112, 0, 267, 3128, 51422, 0, 0
itimer: 352, 0, 1, 10, 1, 0, 0
KNOTE: 128, 0, 0, 2139, 386041, 0, 0
socket: 696, 1571620, 67, 283, 551957, 0, 0
ipq: 56, 95708, 0, 0, 0, 0, 0
udp_inpcb: 392, 1571620, 18, 612, 243215, 0, 0
udpcb: 16, 1571762, 18, 5002, 243215, 0, 0
tcp_inpcb: 392, 1571620, 33, 597, 295619, 0, 0
tcpcb: 1024, 1571620, 33, 283, 295619, 0, 0
tcptw: 88, 27810, 0, 1665, 5059, 0, 0
syncache: 160, 15375, 0, 500, 279331, 0, 0
hostcache: 136, 15370, 2, 375, 85, 0, 0
tcpreass: 40, 191400, 0, 2100, 9504, 0, 0
sackhole: 32, 0, 0, 2125, 107528, 0, 0
sctp_ep: 1408, 1571620, 0, 0, 0, 0, 0
sctp_asoc: 2416, 40000, 0, 0, 0, 0, 0
sctp_laddr: 48, 80012, 0, 0, 4, 0, 0
sctp_raddr: 728, 80000, 0, 0, 0, 0, 0
sctp_chunk: 136, 400026, 0, 0, 0, 0, 0
sctp_readq: 104, 400026, 0, 0, 0, 0, 0
sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0
sctp_asconf: 40, 400000, 0, 0, 0, 0, 0
sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0
udplite_inpcb: 392, 1571620, 0, 0, 0, 0, 0
ripcb: 392, 1571620, 0, 150, 17, 0, 0
unpcb: 240, 1571632, 15, 929, 13101, 0, 0
rtentry: 200, 0, 14, 146, 14, 0, 0
selfd: 56, 0, 519, 4309,25644232, 0, 0
SWAPMETA: 288, 6123806, 884, 2288, 494387, 0, 0
icl_conn: 152, 0, 8, 330, 82, 0, 0
icl_pdu: 80, 0, 1, 3099,619432242, 0, 0
iscsi_outstanding: 40, 0, 0, 0, 0, 0, 0
cfi_lun: 344, 0, 9, 46, 11, 0, 0
cfi_metatask: 320, 0, 0, 0, 0, 0, 0
cfiscsi_data_wait: 56, 0, 0, 5325,239655272, 0, 0
beio: 360, 0, 1, 538,284409034, 0, 0
cblk2: 131072, 0, 0, 27, 3344935, 0, 0
cblk3: 131072, 0, 0, 23, 2757, 0, 0
cblk4: 131072, 0, 0, 55,46151917, 0, 0
cblk5: 131072, 0, 0, 33,90961907, 0, 0
cblk6: 131072, 0, 0, 17,35264386, 0, 0
cblk7: 131072, 0, 0, 28,56989359, 0, 0
cblk8: 131072, 0, 0, 33,37747123, 0, 0
cblk9: 131072, 0, 1, 26, 7098146, 0, 0
cblk10: 131072, 0, 0, 26, 9556935, 0, 0
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr mf0 mf1 in sy cs us sy
id
8 0 7 1328M 2054M 606 0 0 0 2285 10 0 0 836 615 2438 0 4
96
19237/14213/33450 mbufs in use (current/cache/total)
18714/8170/26884/3061900 mbuf clusters in use (current/cache/total/max)
18648/6374 mbuf+clusters out of packet secondary zone in use (current/cache)
0/119/119/1530950 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/453614 9k jumbo clusters in use (current/cache/total/max)
0/0/0/255158 16k jumbo clusters in use (current/cache/total/max)
42237K/20369K/62606K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
UMA Kegs: 384, 0, 217, 3, 219, 0, 0
UMA Zones: 2688, 0, 217, 0, 219, 0, 0
UMA Slabs: 80, 0, 2649602, 1395898,431882332, 0, 0
UMA RCntSlabs: 88, 0, 13561, 5159, 1882437, 0, 0
UMA Hash: 256, 0, 20, 55, 92, 0, 0
4 Bucket: 32, 0, 3111, 13389,256363538, 0, 0
6 Bucket: 48, 0, 1132, 8247,63180373, 0, 0
8 Bucket: 64, 0, 350, 8082,69327066, 11, 0
12 Bucket: 96, 0, 744, 3479,81208647,6458, 0
16 Bucket: 128, 0, 394, 5062,53286276, 0, 0
32 Bucket: 256, 0, 354, 2166,68923809, 63, 0
64 Bucket: 512, 0, 638, 794,57591693,3563055, 0
128 Bucket: 1024, 0, 642, 254,47574372,65273, 0
256 Bucket: 2048, 0, 8439, 345,503759168,70395, 0
vmem btag: 56, 0, 627092, 156890,17555071,5535, 0
VM OBJECT: 256, 0, 54690, 22230,19586749, 0, 0
RADIX NODE: 144, 0, 452301, 219999,118434678, 0, 0
MAP: 240, 0, 3, 61, 3, 0, 0
KMAP ENTRY: 128, 0, 8, 116, 8, 0, 0
MAP ENTRY: 128, 0, 1439, 2715,54526204, 0, 0
VMSPACE: 448, 0, 41, 616, 1337735, 0, 0
fakepg: 104, 0, 0, 152, 25, 0, 0
mt_zone: 4112, 0, 369, 0, 369, 0, 0
16: 16, 0, 110513, 14234,2006288065, 0, 0
32: 32, 0,58122843,11652657,5037573408, 0, 0
64: 64, 0, 1357036, 622500,12845488547, 0, 0
128: 128, 0, 2012481, 834621,11032028232, 0, 0
256: 256, 0, 54950, 119020,8130142155, 0, 0
512: 512, 0, 418314, 358,2018523319, 0, 0
1024: 1024, 0, 15983, 289,134301395, 0, 0
2048: 2048, 0, 42445, 685,1931137837, 0, 0
4096: 4096, 0, 13874, 62,102907936, 0, 0
8192: 8192, 0, 26, 27, 3066882, 0, 0
16384: 16384, 0, 42, 16, 7611, 0, 0
32768: 32768, 0, 39, 17, 67805, 0, 0
65536: 65536, 0, 81, 21, 1042606, 0, 0
64 pcpu: 8, 0, 1654, 522, 1814, 0, 0
SLEEPQUEUE: 80, 0, 1297, 811, 1579, 0, 0
Files: 80, 0, 135, 2665,11736505, 0, 0
TURNSTILE: 136, 0, 1297, 563, 1579, 0, 0
rl_entry: 40, 0, 330, 3270, 341, 0, 0
umtx pi: 96, 0, 0, 0, 0, 0, 0
MAC labels: 40, 0, 0, 0, 0, 0, 0
PROC: 1216, 0, 61, 215, 1337752, 0, 0
THREAD: 1168, 0, 1169, 127, 5681, 0, 0
cpuset: 72, 0, 870, 1165, 1637, 0, 0
audit_record: 1248, 0, 0, 0, 0, 0, 0
mbuf_packet: 256, 19596165, 18655, 6367,10745933894, 0, 0
mbuf: 256, 19596165, 527, 7901,12874604748, 0, 0
mbuf_cluster: 2048, 3061900, 25022, 1862,1737637980, 0, 0
mbuf_jumbo_page: 4096, 1530950, 0, 119,1421717140, 0, 0
mbuf_jumbo_9k: 9216, 453614, 0, 0, 0, 0, 0
mbuf_jumbo_16k: 16384, 255158, 0, 0, 0, 0, 0
mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0
g_bio: 248, 0, 52, 2748,7325214508, 0, 0
ttyinq: 160, 0, 240, 210, 1125, 0, 0
ttyoutq: 256, 0, 126, 144, 593, 0, 0
DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0
ata_request: 336, 0, 0, 0, 0, 0, 0
vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0
FPU_save_area: 512, 0, 0, 0, 0, 0, 0
taskq_zone: 48, 0, 0, 6723,170080476, 0, 0
VNODE: 472, 0, 234386, 27478, 5034664, 0, 0
VNODEPOLL: 112, 0, 0, 0, 1, 0, 0
BUF TRIE: 144, 0, 0, 105948, 0, 0, 0
S VFS Cache: 108, 0, 146238, 83152, 5732698, 0, 0
STS VFS Cache: 148, 0, 0, 0, 12, 0, 0
L VFS Cache: 328, 0, 56810, 1846, 810988, 0, 0
LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0
NAMEI: 1024, 0, 0, 260,44885768, 0, 0
NCLNODE: 528, 0, 0, 0, 16, 0, 0
DIRHASH: 1024, 0, 0, 0, 0, 0, 0
pipe: 744, 0, 6, 239, 108265, 0, 0
procdesc: 128, 0, 0, 0, 0, 0, 0
range_seg_cache: 64, 0, 2248955, 1634849,68602909067, 0, 0
zio_cache: 920, 0, 5199, 100293,31417037045, 0, 0
zio_link_cache: 48, 0, 5470, 100355,34091891329, 0, 0
zio_buf_512: 512, 0, 275195, 68013,71262466, 0, 0
zio_data_buf_512: 512, 0, 30, 570, 9512527, 0, 0
zio_buf_1024: 1024, 0, 5, 287, 8475039, 0, 0
zio_data_buf_1024: 1024, 0, 22, 774, 4260371, 0, 0
zio_buf_1536: 1536, 0, 3, 183, 5277075, 0, 0
zio_data_buf_1536: 1536, 0, 7, 179, 6604805, 0, 0
zio_buf_2048: 2048, 0, 361, 121,13320298, 0, 0
zio_data_buf_2048: 2048, 0, 16, 96, 6060249, 0, 0
zio_buf_2560: 2560, 0, 7, 145, 4841768, 0, 0
zio_data_buf_2560: 2560, 0, 2, 66, 4327820, 0, 0
zio_buf_3072: 3072, 0, 108, 766, 8844569, 0, 0
zio_data_buf_3072: 3072, 0, 2, 61, 3435615, 0, 0
zio_buf_3584: 3584, 0, 3, 44, 4756617, 0, 0
zio_data_buf_3584: 3584, 0, 6, 26, 3238962, 0, 0
zio_buf_4096: 4096, 0, 1599414, 28976,3385517814, 0, 0
zio_data_buf_4096: 4096, 0, 305346, 1452,234760561, 0, 0
zio_buf_5120: 5120, 0, 300, 24,19194968, 0, 0
zio_data_buf_5120: 5120, 0, 4, 26, 1815808, 0, 0
zio_buf_6144: 6144, 0, 0, 24,13487047, 0, 0
zio_data_buf_6144: 6144, 0, 6, 25, 3353554, 0, 0
zio_buf_7168: 7168, 0, 0, 22, 9530478, 0, 0
zio_data_buf_7168: 7168, 0, 0, 27, 2315875, 0, 0
zio_buf_8192: 8192, 0, 141, 527,251242133, 0, 0
zio_data_buf_8192: 8192, 0, 5, 26, 1764072, 0, 0
zio_buf_10240: 10240, 0, 0, 39,15850440, 0, 0
zio_data_buf_10240: 10240, 0, 2, 27, 3405015, 0, 0
zio_buf_12288: 12288, 0, 1, 90,77829004, 0, 0
zio_data_buf_12288: 12288, 0, 3, 30, 3223381, 0, 0
zio_buf_14336: 14336, 0, 1, 69,13180271, 0, 0
zio_data_buf_14336: 14336, 0, 4, 25, 2869463, 0, 0
zio_buf_16384: 16384, 0, 273193, 3666,359348991, 0, 0
zio_data_buf_16384: 16384, 0, 5826, 62, 4478607, 0, 0
zio_buf_20480: 20480, 0, 0, 92,58858000, 0, 0
zio_data_buf_20480: 20480, 0, 8, 26, 3694179, 0, 0
zio_buf_24576: 24576, 0, 0, 65,57632331, 0, 0
zio_data_buf_24576: 24576, 0, 2, 26, 3241802, 0, 0
zio_buf_28672: 28672, 0, 0, 55,57194299, 0, 0
zio_data_buf_28672: 28672, 0, 0, 29, 3055270, 0, 0
zio_buf_32768: 32768, 0, 0, 104,55422934, 0, 0
zio_data_buf_32768: 32768, 0, 33, 80,95994809, 0, 0
zio_buf_36864: 36864, 0, 2, 45,30944956, 0, 0
zio_data_buf_36864: 36864, 0, 1, 26, 2905780, 0, 0
zio_buf_40960: 40960, 0, 0, 57,30084320, 0, 0
zio_data_buf_40960: 40960, 0, 6, 26, 3463744, 0, 0
zio_buf_45056: 45056, 0, 0, 69,28275199, 0, 0
zio_data_buf_45056: 45056, 0, 1, 29, 3648526, 0, 0
zio_buf_49152: 49152, 0, 0, 81,31247414, 0, 0
zio_data_buf_49152: 49152, 0, 1, 29, 3107296, 0, 0
zio_buf_53248: 53248, 0, 0, 135,25156727, 0, 0
zio_data_buf_53248: 53248, 0, 0, 26, 2534717, 0, 0
zio_buf_57344: 57344, 0, 0, 94,26579174, 0, 0
zio_data_buf_57344: 57344, 0, 1, 27, 2879517, 0, 0
zio_buf_61440: 61440, 0, 0, 53, 8211951, 0, 0
zio_data_buf_61440: 61440, 0, 1, 21, 2089, 0, 0
zio_buf_65536: 65536, 0, 0, 1065,332616711, 0, 0
zio_data_buf_65536: 65536, 0, 270981, 921,483916113, 0, 0
zio_buf_69632: 69632, 0, 0, 22, 6244183, 0, 0
zio_data_buf_69632: 69632, 0, 1, 22, 2442, 0, 0
zio_buf_73728: 73728, 0, 0, 22, 7105466, 0, 0
zio_data_buf_73728: 73728, 0, 0, 25, 2078, 0, 0
zio_buf_77824: 77824, 0, 0, 22, 5360380, 0, 0
zio_data_buf_77824: 77824, 0, 0, 22, 2076, 0, 0
zio_buf_81920: 81920, 0, 0, 25, 6976237, 0, 0
zio_data_buf_81920: 81920, 0, 0, 20, 2077, 0, 0
zio_buf_86016: 86016, 0, 0, 27, 5387269, 0, 0
zio_data_buf_86016: 86016, 0, 0, 21, 2070, 0, 0
zio_buf_90112: 90112, 0, 0, 23, 6362576, 0, 0
zio_data_buf_90112: 90112, 0, 1, 23, 2551, 0, 0
zio_buf_94208: 94208, 0, 0, 23, 6067112, 0, 0
zio_data_buf_94208: 94208, 0, 0, 24, 2114, 0, 0
zio_buf_98304: 98304, 0, 0, 22, 8587922, 0, 0
zio_data_buf_98304: 98304, 0, 0, 21, 2104, 0, 0
zio_buf_102400: 102400, 0, 0, 26, 6589071, 0, 0
zio_data_buf_102400: 102400, 0, 0, 21, 2061, 0, 0
zio_buf_106496: 106496, 0, 0, 24, 7280583, 0, 0
zio_data_buf_106496: 106496, 0, 0, 24, 2009, 0, 0
zio_buf_110592: 110592, 0, 0, 25, 7767698, 0, 0
zio_data_buf_110592: 110592, 0, 0, 20, 1903, 0, 0
zio_buf_114688: 114688, 0, 0, 25, 9118622, 0, 0
zio_data_buf_114688: 114688, 0, 1, 21, 1901, 0, 0
zio_buf_118784: 118784, 0, 0, 24, 8268816, 0, 0
zio_data_buf_118784: 118784, 0, 0, 23, 1740, 0, 0
zio_buf_122880: 122880, 0, 0, 31,11224925, 0, 0
zio_data_buf_122880: 122880, 0, 0, 20, 1668, 0, 0
zio_buf_126976: 126976, 0, 1, 26,10713515, 0, 0
zio_data_buf_126976: 126976, 0, 0, 19, 1648, 0, 0
zio_buf_131072: 131072, 0, 14002, 717,254193922, 0, 0
zio_data_buf_131072: 131072, 0, 0, 14, 56696, 0, 0
lz4_ctx: 16384, 0, 0, 52,1658814743, 0, 0
sa_cache: 80, 0, 234294, 83956, 5033876, 0, 0
dnode_t: 752, 0, 325194, 23801, 7846149, 0, 0
dmu_buf_impl_t: 232, 0, 2420742, 1092665,1018837476, 0, 0
arc_buf_hdr_t: 216, 0,31469718, 6580914,704776192, 0, 0
arc_buf_t: 72, 0, 2303607, 1443818,842981455, 0, 0
zil_lwb_cache: 192, 0, 162, 1798,118978648, 0, 0
zfs_znode_cache: 368, 0, 234294, 38146, 5033876, 0, 0
Mountpoints: 816, 0, 33, 137, 37, 0, 0
ksiginfo: 112, 0, 267, 3128, 51422, 0, 0
itimer: 352, 0, 1, 10, 1, 0, 0
KNOTE: 128, 0, 0, 2139, 386041, 0, 0
socket: 696, 1571620, 67, 283, 551957, 0, 0
ipq: 56, 95708, 0, 0, 0, 0, 0
udp_inpcb: 392, 1571620, 18, 612, 243215, 0, 0
udpcb: 16, 1571762, 18, 5002, 243215, 0, 0
tcp_inpcb: 392, 1571620, 33, 597, 295619, 0, 0
tcpcb: 1024, 1571620, 33, 283, 295619, 0, 0
tcptw: 88, 27810, 0, 1665, 5059, 0, 0
syncache: 160, 15375, 0, 500, 279331, 0, 0
hostcache: 136, 15370, 2, 375, 85, 0, 0
tcpreass: 40, 191400, 0, 2100, 9504, 0, 0
sackhole: 32, 0, 0, 2125, 107528, 0, 0
sctp_ep: 1408, 1571620, 0, 0, 0, 0, 0
sctp_asoc: 2416, 40000, 0, 0, 0, 0, 0
sctp_laddr: 48, 80012, 0, 0, 4, 0, 0
sctp_raddr: 728, 80000, 0, 0, 0, 0, 0
sctp_chunk: 136, 400026, 0, 0, 0, 0, 0
sctp_readq: 104, 400026, 0, 0, 0, 0, 0
sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0
sctp_asconf: 40, 400000, 0, 0, 0, 0, 0
sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0
udplite_inpcb: 392, 1571620, 0, 0, 0, 0, 0
ripcb: 392, 1571620, 0, 150, 17, 0, 0
unpcb: 240, 1571632, 15, 929, 13101, 0, 0
rtentry: 200, 0, 14, 146, 14, 0, 0
selfd: 56, 0, 519, 4309,25644346, 0, 0
SWAPMETA: 288, 6123806, 884, 2288, 494389, 0, 0
icl_conn: 152, 0, 8, 330, 82, 0, 0
icl_pdu: 80, 0, 6, 3094,619433674, 0, 0
iscsi_outstanding: 40, 0, 0, 0, 0, 0, 0
cfi_lun: 344, 0, 9, 46, 11, 0, 0
cfi_metatask: 320, 0, 0, 0, 0, 0, 0
cfiscsi_data_wait: 56, 0, 0, 5325,239655955, 0, 0
beio: 360, 0, 2, 537,284409733, 0, 0
cblk2: 131072, 0, 0, 27, 3344957, 0, 0
cblk3: 131072, 0, 0, 23, 2757, 0, 0
cblk4: 131072, 0, 0, 55,46151942, 0, 0
cblk5: 131072, 0, 1, 32,90961935, 0, 0
cblk6: 131072, 0, 0, 17,35264386, 0, 0
cblk7: 131072, 0, 0, 28,56989562, 0, 0
cblk8: 131072, 0, 1, 32,37747322, 0, 0
cblk9: 131072, 0, 0, 27, 7098349, 0, 0
cblk10: 131072, 0, 0, 26, 9556963, 0, 0
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr mf0 mf1 in sy cs us sy
id
0 0 7 1328M 1920M 606 0 0 0 2285 10 0 0 836 615 2438 0 4
96
19012/14438/33450 mbufs in use (current/cache/total)
18494/8390/26884/3061900 mbuf clusters in use (current/cache/total/max)
18457/6565 mbuf+clusters out of packet secondary zone in use (current/cache)
0/137/137/1530950 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/453614 9k jumbo clusters in use (current/cache/total/max)
0/0/0/255158 16k jumbo clusters in use (current/cache/total/max)
41741K/20937K/62678K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
UMA Kegs: 384, 0, 217, 3, 219, 0, 0
UMA Zones: 2688, 0, 217, 0, 219, 0, 0
UMA Slabs: 80, 0, 2652758, 1392742,431885488, 0, 0
UMA RCntSlabs: 88, 0, 13579, 5141, 1882455, 0, 0
UMA Hash: 256, 0, 20, 55, 92, 0, 0
4 Bucket: 32, 0, 3274, 13226,256364791, 0, 0
6 Bucket: 48, 0, 1309, 8070,63180681, 0, 0
8 Bucket: 64, 0, 367, 8065,69327210, 11, 0
12 Bucket: 96, 0, 762, 3461,81208911,6458, 0
16 Bucket: 128, 0, 405, 5051,53286290, 0, 0
32 Bucket: 256, 0, 354, 2166,68924050, 63, 0
64 Bucket: 512, 0, 7266, 366,57598810,3563055, 0
128 Bucket: 1024, 0, 449, 447,47578183,65273, 0
256 Bucket: 2048, 0, 9564, 100,503760707,70395, 0
vmem btag: 56, 0, 627092, 156890,17555071,5535, 0
VM OBJECT: 256, 0, 54690, 22230,19586816, 0, 0
RADIX NODE: 144, 0, 452304, 219996,118434964, 0, 0
MAP: 240, 0, 3, 61, 3, 0, 0
KMAP ENTRY: 128, 0, 8, 116, 8, 0, 0
MAP ENTRY: 128, 0, 1439, 2715,54526402, 0, 0
VMSPACE: 448, 0, 41, 616, 1337740, 0, 0
fakepg: 104, 0, 0, 152, 25, 0, 0
mt_zone: 4112, 0, 369, 0, 369, 0, 0
16: 16, 0, 110516, 14231,2006293901, 0, 0
32: 32, 0,57869263,11906237,5037586029, 0, 0
64: 64, 0, 1357790, 621746,12845953207, 0, 0
128: 128, 0, 1983142, 863960,11032064477, 0, 0
256: 256, 0, 550279, 1451,8130644436, 0, 0
512: 512, 0, 28473, 405767,2019303683, 0, 0
1024: 1024, 0, 15991, 281,134301412, 0, 0
2048: 2048, 0, 42464, 666,1931139731, 0, 0
4096: 4096, 0, 13872, 64,102908632, 0, 0
8192: 8192, 0, 26, 27, 3066882, 0, 0
16384: 16384, 0, 42, 16, 7611, 0, 0
32768: 32768, 0, 39, 17, 67805, 0, 0
65536: 65536, 0, 81, 21, 1042606, 0, 0
64 pcpu: 8, 0, 1654, 522, 1814, 0, 0
SLEEPQUEUE: 80, 0, 1297, 811, 1579, 0, 0
Files: 80, 0, 135, 2665,11736530, 0, 0
TURNSTILE: 136, 0, 1297, 563, 1579, 0, 0
rl_entry: 40, 0, 330, 3270, 341, 0, 0
umtx pi: 96, 0, 0, 0, 0, 0, 0
MAC labels: 40, 0, 0, 0, 0, 0, 0
PROC: 1216, 0, 61, 215, 1337757, 0, 0
THREAD: 1168, 0, 1169, 127, 5681, 0, 0
cpuset: 72, 0, 870, 1165, 1637, 0, 0
audit_record: 1248, 0, 0, 0, 0, 0, 0
mbuf_packet: 256, 19596165, 18425, 6597,10745970214, 0, 0
mbuf: 256, 19596165, 545, 7883,12874643548, 0, 0
mbuf_cluster: 2048, 3061900, 25055, 1829,1737649087, 0, 0
mbuf_jumbo_page: 4096, 1530950, 0, 137,1421717220, 0, 0
mbuf_jumbo_9k: 9216, 453614, 0, 0, 0, 0, 0
mbuf_jumbo_16k: 16384, 255158, 0, 0, 0, 0, 0
mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0
g_bio: 248, 0, 132, 2668,7325223092, 0, 0
ttyinq: 160, 0, 240, 210, 1125, 0, 0
ttyoutq: 256, 0, 126, 144, 593, 0, 0
DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0
ata_request: 336, 0, 0, 0, 0, 0, 0
vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0
FPU_save_area: 512, 0, 0, 0, 0, 0, 0
taskq_zone: 48, 0, 0, 6723,170080532, 0, 0
VNODE: 472, 0, 234386, 27478, 5034664, 0, 0
VNODEPOLL: 112, 0, 0, 0, 1, 0, 0
BUF TRIE: 144, 0, 0, 105948, 0, 0, 0
S VFS Cache: 108, 0, 146238, 83152, 5732698, 0, 0
STS VFS Cache: 148, 0, 0, 0, 12, 0, 0
L VFS Cache: 328, 0, 56810, 1846, 810988, 0, 0
LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0
NAMEI: 1024, 0, 0, 260,44885850, 0, 0
NCLNODE: 528, 0, 0, 0, 16, 0, 0
DIRHASH: 1024, 0, 0, 0, 0, 0, 0
pipe: 744, 0, 6, 239, 108265, 0, 0
procdesc: 128, 0, 0, 0, 0, 0, 0
range_seg_cache: 64, 0, 2258640, 1625164,68602919111, 0, 0
zio_cache: 920, 0, 234, 105258,31417113383, 0, 0
zio_link_cache: 48, 0, 230, 105595,34091959338, 0, 0
zio_buf_512: 512, 0, 275198, 68010,71262483, 0, 0
zio_data_buf_512: 512, 0, 30, 570, 9512561, 0, 0
zio_buf_1024: 1024, 0, 4, 288, 8475050, 0, 0
zio_data_buf_1024: 1024, 0, 22, 774, 4260377, 0, 0
zio_buf_1536: 1536, 0, 3, 183, 5277098, 0, 0
zio_data_buf_1536: 1536, 0, 7, 179, 6604827, 0, 0
zio_buf_2048: 2048, 0, 361, 121,13320325, 0, 0
zio_data_buf_2048: 2048, 0, 16, 96, 6060314, 0, 0
zio_buf_2560: 2560, 0, 7, 145, 4841781, 0, 0
zio_data_buf_2560: 2560, 0, 2, 66, 4327840, 0, 0
zio_buf_3072: 3072, 0, 108, 766, 8844573, 0, 0
zio_data_buf_3072: 3072, 0, 2, 61, 3435617, 0, 0
zio_buf_3584: 3584, 0, 3, 44, 4756623, 0, 0
zio_data_buf_3584: 3584, 0, 6, 26, 3238962, 0, 0
zio_buf_4096: 4096, 0, 1600211, 28179,3385518732, 0, 0
zio_data_buf_4096: 4096, 0, 304769, 2029,234760767, 0, 0
zio_buf_5120: 5120, 0, 300, 24,19194980, 0, 0
zio_data_buf_5120: 5120, 0, 4, 26, 1815813, 0, 0
zio_buf_6144: 6144, 0, 0, 24,13487079, 0, 0
zio_data_buf_6144: 6144, 0, 6, 25, 3353561, 0, 0
zio_buf_7168: 7168, 0, 0, 22, 9530491, 0, 0
zio_data_buf_7168: 7168, 0, 0, 27, 2315880, 0, 0
zio_buf_8192: 8192, 0, 1, 667,251242195, 0, 0
zio_data_buf_8192: 8192, 0, 5, 26, 1764074, 0, 0
zio_buf_10240: 10240, 0, 0, 39,15850464, 0, 0
zio_data_buf_10240: 10240, 0, 2, 27, 3405031, 0, 0
zio_buf_12288: 12288, 0, 1, 90,77829185, 0, 0
zio_data_buf_12288: 12288, 0, 3, 30, 3223400, 0, 0
zio_buf_14336: 14336, 0, 1, 69,13180306, 0, 0
zio_data_buf_14336: 14336, 0, 4, 25, 2869467, 0, 0
zio_buf_16384: 16384, 0, 272638, 4221,359349071, 0, 0
zio_data_buf_16384: 16384, 0, 5636, 252, 4478622, 0, 0
zio_buf_20480: 20480, 0, 0, 92,58858093, 0, 0
zio_data_buf_20480: 20480, 0, 8, 26, 3694182, 0, 0
zio_buf_24576: 24576, 0, 0, 65,57632464, 0, 0
zio_data_buf_24576: 24576, 0, 2, 26, 3241808, 0, 0
zio_buf_28672: 28672, 0, 1, 54,57194576, 0, 0
zio_data_buf_28672: 28672, 0, 0, 29, 3055273, 0, 0
zio_buf_32768: 32768, 0, 0, 104,55422964, 0, 0
zio_data_buf_32768: 32768, 0, 42, 71,95995510, 0, 0
zio_buf_36864: 36864, 0, 0, 47,30944970, 0, 0
zio_data_buf_36864: 36864, 0, 1, 26, 2905782, 0, 0
zio_buf_40960: 40960, 0, 0, 57,30084339, 0, 0
zio_data_buf_40960: 40960, 0, 6, 26, 3463747, 0, 0
zio_buf_45056: 45056, 0, 0, 69,28275218, 0, 0
zio_data_buf_45056: 45056, 0, 1, 29, 3648533, 0, 0
zio_buf_49152: 49152, 0, 0, 81,31247437, 0, 0
zio_data_buf_49152: 49152, 0, 1, 29, 3107299, 0, 0
zio_buf_53248: 53248, 0, 0, 135,25156769, 0, 0
zio_data_buf_53248: 53248, 0, 0, 26, 2534719, 0, 0
zio_buf_57344: 57344, 0, 0, 94,26579226, 0, 0
zio_data_buf_57344: 57344, 0, 1, 27, 2879519, 0, 0
zio_buf_61440: 61440, 0, 0, 53, 8211952, 0, 0
zio_data_buf_61440: 61440, 0, 1, 21, 2089, 0, 0
zio_buf_65536: 65536, 0, 0, 1065,332616712, 0, 0
zio_data_buf_65536: 65536, 0, 271017, 885,483917463, 0, 0
zio_buf_69632: 69632, 0, 0, 22, 6244186, 0, 0
zio_data_buf_69632: 69632, 0, 1, 22, 2442, 0, 0
zio_buf_73728: 73728, 0, 0, 22, 7105467, 0, 0
zio_data_buf_73728: 73728, 0, 0, 25, 2078, 0, 0
zio_buf_77824: 77824, 0, 0, 22, 5360384, 0, 0
zio_data_buf_77824: 77824, 0, 0, 22, 2076, 0, 0
zio_buf_81920: 81920, 0, 0, 25, 6976239, 0, 0
zio_data_buf_81920: 81920, 0, 0, 20, 2077, 0, 0
zio_buf_86016: 86016, 0, 0, 27, 5387273, 0, 0
zio_data_buf_86016: 86016, 0, 0, 21, 2070, 0, 0
zio_buf_90112: 90112, 0, 0, 23, 6362580, 0, 0
zio_data_buf_90112: 90112, 0, 1, 23, 2551, 0, 0
zio_buf_94208: 94208, 0, 0, 23, 6067115, 0, 0
zio_data_buf_94208: 94208, 0, 0, 24, 2114, 0, 0
zio_buf_98304: 98304, 0, 0, 22, 8587929, 0, 0
zio_data_buf_98304: 98304, 0, 0, 21, 2104, 0, 0
zio_buf_102400: 102400, 0, 0, 26, 6589077, 0, 0
zio_data_buf_102400: 102400, 0, 0, 21, 2061, 0, 0
zio_buf_106496: 106496, 0, 0, 24, 7280588, 0, 0
zio_data_buf_106496: 106496, 0, 0, 24, 2009, 0, 0
zio_buf_110592: 110592, 0, 0, 25, 7767706, 0, 0
zio_data_buf_110592: 110592, 0, 0, 20, 1903, 0, 0
zio_buf_114688: 114688, 0, 0, 25, 9118633, 0, 0
zio_data_buf_114688: 114688, 0, 1, 21, 1901, 0, 0
zio_buf_118784: 118784, 0, 0, 24, 8268830, 0, 0
zio_data_buf_118784: 118784, 0, 0, 23, 1740, 0, 0
zio_buf_122880: 122880, 0, 0, 31,11224940, 0, 0
zio_data_buf_122880: 122880, 0, 0, 20, 1668, 0, 0
zio_buf_126976: 126976, 0, 0, 27,10713525, 0, 0
zio_data_buf_126976: 126976, 0, 0, 19, 1648, 0, 0
zio_buf_131072: 131072, 0, 14056, 663,254194720, 0, 0
zio_data_buf_131072: 131072, 0, 0, 14, 56696, 0, 0
lz4_ctx: 16384, 0, 0, 52,1658815406, 0, 0
sa_cache: 80, 0, 234294, 83956, 5033876, 0, 0
dnode_t: 752, 0, 325197, 23798, 7846152, 0, 0
dmu_buf_impl_t: 232, 0, 2421139, 1092268,1018840854, 0, 0
arc_buf_hdr_t: 216, 0,31470141, 6580491,704778880, 0, 0
arc_buf_t: 72, 0, 2303529, 1443896,842984540, 0, 0
zil_lwb_cache: 192, 0, 845, 1115,118979331, 0, 0
zfs_znode_cache: 368, 0, 234294, 38146, 5033876, 0, 0
Mountpoints: 816, 0, 33, 137, 37, 0, 0
ksiginfo: 112, 0, 267, 3128, 51422, 0, 0
itimer: 352, 0, 1, 10, 1, 0, 0
KNOTE: 128, 0, 0, 2139, 386041, 0, 0
socket: 696, 1571620, 67, 283, 551957, 0, 0
ipq: 56, 95708, 0, 0, 0, 0, 0
udp_inpcb: 392, 1571620, 18, 612, 243215, 0, 0
udpcb: 16, 1571762, 18, 5002, 243215, 0, 0
tcp_inpcb: 392, 1571620, 33, 597, 295619, 0, 0
tcpcb: 1024, 1571620, 33, 283, 295619, 0, 0
tcptw: 88, 27810, 0, 1665, 5059, 0, 0
syncache: 160, 15375, 0, 500, 279331, 0, 0
hostcache: 136, 15370, 2, 375, 85, 0, 0
tcpreass: 40, 191400, 0, 2100, 9504, 0, 0
sackhole: 32, 0, 0, 2125, 107528, 0, 0
sctp_ep: 1408, 1571620, 0, 0, 0, 0, 0
sctp_asoc: 2416, 40000, 0, 0, 0, 0, 0
sctp_laddr: 48, 80012, 0, 0, 4, 0, 0
sctp_raddr: 728, 80000, 0, 0, 0, 0, 0
sctp_chunk: 136, 400026, 0, 0, 0, 0, 0
sctp_readq: 104, 400026, 0, 0, 0, 0, 0
sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0
sctp_asconf: 40, 400000, 0, 0, 0, 0, 0
sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0
udplite_inpcb: 392, 1571620, 0, 0, 0, 0, 0
ripcb: 392, 1571620, 0, 150, 17, 0, 0
unpcb: 240, 1571632, 15, 929, 13101, 0, 0
rtentry: 200, 0, 14, 146, 14, 0, 0
selfd: 56, 0, 519, 4309,25644450, 0, 0
SWAPMETA: 288, 6123806, 884, 2288, 494391, 0, 0
icl_conn: 152, 0, 8, 330, 82, 0, 0
icl_pdu: 80, 0, 0, 3100,619436489, 0, 0
iscsi_outstanding: 40, 0, 0, 0, 0, 0, 0
cfi_lun: 344, 0, 9, 46, 11, 0, 0
cfi_metatask: 320, 0, 0, 0, 0, 0, 0
cfiscsi_data_wait: 56, 0, 0, 5325,239657305, 0, 0
beio: 360, 0, 0, 539,284411124, 0, 0
cblk2: 131072, 0, 0, 27, 3344972, 0, 0
cblk3: 131072, 0, 0, 23, 2757, 0, 0
cblk4: 131072, 0, 0, 55,46152106, 0, 0
cblk5: 131072, 0, 0, 33,90962317, 0, 0
cblk6: 131072, 0, 0, 17,35264386, 0, 0
cblk7: 131072, 0, 0, 28,56989675, 0, 0
cblk8: 131072, 0, 0, 35,37747916, 0, 0
cblk9: 131072, 0, 0, 27, 7098449, 0, 0
cblk10: 131072, 0, 0, 26, 9556976, 0, 0
-------------- next part --------------
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy
id
0 0 4 759M 9969M 0 0 0 0 0 0 14 10 5326 33 24785 0 2
98
0 0 4 759M 9960M 0 0 0 0 0 0 6 13 3865 36 18945 0 1
99
1 0 4 759M 9885M 0 0 0 0 0 0 37 35 4621 32 22473 0 22
78
0 0 4 768M 9695M 11 0 0 0 0 0 8 10 2858 42 12492 0 52
48
0 0 4 768M 9492M 0 0 0 0 0 0 18 34 2495 31 16006 0 86
14
0 0 4 768M 9414M 0 0 0 0 0 0 6 7 2271 32 11384 0 32
68
1 0 4 768M 4667M 0 0 0 0 0 0 6 4 1772 30 7294 0 63
37
0 0 4 768M 8192B 0 0 0 0 0 0 1 6 577 32 2057 0 40
60
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 82 33 372 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 114 37 516 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 61 34 327 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 82 34 394 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 59 36 358 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 76 34 369 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 63 35 318 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 76 35 367 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 64 34 337 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 76 35 380 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 77 34 379 0 0
100
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 74 35 621 0 1
99
0 0 4 768M 8192B 0 0 0 0 10 0 0 0 57 35 1232 0 2
98
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 66 33 1402 0 2
98
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 50 33 1375 0 2
98
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 49 33 1487 0 2
98
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 47 33 1977 0 3
97
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 43 33 2251 0 4
96
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 46 33 2297 0 4
96
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 41 33 2387 0 4
96
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 38 33 2391 0 5
95
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 36 33 2427 0 6
94
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 29 33 2416 0 5
95
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 35 33 2439 0 6
94
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 33 33 2482 0 6
94
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 27 33 2432 0 6
94
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 32 33 2477 0 5
95
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 25 33 2412 0 5
95
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 30 33 2413 0 6
94
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 30 33 2481 0 7
93
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 44 33 2507 0 8
92
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 22 33 2457 0 8
92
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 26 32 2408 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 24 33 2404 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 22 33 2396 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 25 33 2402 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 25 33 2402 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 26 33 2410 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 19 33 2396 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 20 33 2392 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 24 33 2401 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 20 33 2456 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 22 33 2411 0 12
88
0 0 4 768M 8192B 0 0 0 0 0 0 0 0 19 33 2387 0 12
88
-------------- next part --------------
lz4_ctx: 16384, 0, 0, 94,50237120, 0, 0
sa_cache: 80, 0, 3843, 1307, 5471, 0, 0
dnode_t: 752, 0, 38683, 522, 5927648, 0, 0
dmu_buf_impl_t: 232, 0, 4689089, 953330,26438186, 0, 0
arc_buf_hdr_t: 216, 0, 9692075, 835,16552792, 0, 0
arc_buf_t: 72, 0, 4846982, 2024223,20064972, 0, 0
zil_lwb_cache: 192, 0, 20528, 652, 853333, 0, 0
zfs_znode_cache: 368, 0, 3843, 337, 5471, 0, 0
Mountpoints: 816, 0, 23, 147, 23, 0, 0
ksiginfo: 112, 0, 374, 3406, 1886, 0, 0
itimer: 352, 0, 0, 0, 0, 0, 0
KNOTE: 128, 0, 0, 3007, 8746, 0, 0
socket: 696, 3144485, 69, 336, 23791, 0, 0
ipq: 56, 191487, 0, 0, 0, 0, 0
udp_inpcb: 392, 3144490, 11, 749, 9050, 0, 0
udpcb: 16, 3144528, 11, 6264, 9050, 0, 0
tcp_inpcb: 392, 3144490, 45, 655, 3016, 0, 0
tcpcb: 1024, 3144488, 45, 267, 3016, 0, 0
tcptw: 88, 27810, 0, 0, 0, 0, 0
syncache: 160, 15375, 0, 500, 87, 0, 0
hostcache: 136, 15370, 0, 0, 0, 0, 0
tcpreass: 40, 383000, 0, 800, 57, 0, 0
sackhole: 32, 0, 0, 1625, 208, 0, 0
sctp_ep: 1408, 3144486, 0, 0, 0, 0, 0
sctp_asoc: 2416, 40000, 0, 0, 0, 0, 0
sctp_laddr: 48, 80012, 0, 1079, 4, 0, 0
sctp_raddr: 728, 80000, 0, 0, 0, 0, 0
sctp_chunk: 136, 400026, 0, 0, 0, 0, 0
sctp_readq: 104, 400026, 0, 0, 0, 0, 0
sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0
sctp_asconf: 40, 400000, 0, 0, 0, 0, 0
sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0
udplite_inpcb: 392, 3144490, 0, 0, 0, 0, 0
ripcb: 392, 3144490, 0, 0, 0, 0, 0
unpcb: 240, 3144496, 13, 1459, 11720, 0, 0
rtentry: 200, 0, 15, 485, 15, 0, 0
selfd: 56, 0, 468, 5638, 2586844, 0, 0
SWAPMETA: 288, 12254307, 0, 0, 0, 0, 0
icl_conn: 152, 0, 20, 656, 20, 0, 0
icl_pdu: 80, 0, 79, 7321,18907523, 0, 0
iscsi_outstanding: 40, 0, 0, 0, 0, 0, 0
cfi_lun: 344, 0, 24, 295, 24, 0, 0
cfi_metatask: 320, 0, 0, 0, 0, 0, 0
cfiscsi_data_wait: 56, 0, 0, 7100, 6656404, 0, 0
beio: 360, 0, 40, 840, 8245154, 0, 0
cblk0: 131072, 0, 0, 10, 29, 0, 0
cblk1: 131072, 0, 0, 34, 1028, 0, 0
cblk2: 131072, 0, 0, 14, 29, 0, 0
cblk3: 131072, 0, 0, 10, 29, 0, 0
cblk4: 131072, 0, 0, 7, 29, 0, 0
cblk5: 131072, 0, 0, 20, 94, 0, 0
cblk6: 131072, 0, 1, 79, 2218167, 0, 0
cblk7: 131072, 0, 0, 10, 23, 0, 0
cblk8: 131072, 0, 0, 52, 29547, 0, 0
cblk9: 131072, 0, 0, 0, 0, 0, 0
cblk10: 131072, 0, 0, 0, 0, 0, 0
cblk11: 131072, 0, 0, 79, 20324, 0, 0
cblk12: 131072, 0, 0, 9, 94, 0, 0
cblk13: 131072, 0, 0, 0, 0, 0, 0
cblk14: 131072, 0, 0, 0, 0, 0, 0
cblk15: 131072, 0, 11, 24, 306167, 0, 0
cblk16: 131072, 0, 10, 25, 635505, 0, 0
cblk17: 131072, 0, 11, 25, 2140785, 0, 0
cblk18: 131072, 0, 0, 13, 94, 0, 0
cblk19: 131072, 0, 0, 13, 94, 0, 0
cblk20: 131072, 0, 2, 76, 1173706, 0, 0
cblk21: 131072, 0, 1, 58, 1275285, 0, 0
cblk22: 131072, 0, 2, 35, 35182, 0, 0
cblk23: 131072, 0, 2, 19, 418905, 0, 0
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy
id
2 0 0 2010M 2249M 2205 0 0 0 2901 2 0 0 1982 46326 10468 0 3
97
40602/23148/63750 mbufs in use (current/cache/total)
39043/11353/50396/6127148 mbuf clusters in use (current/cache/total/max)
39030/7663 mbuf+clusters out of packet secondary zone in use (current/cache)
0/485/485/3063574 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/907725 9k jumbo clusters in use (current/cache/total/max)
0/0/0/510595 16k jumbo clusters in use (current/cache/total/max)
88236K/30433K/118669K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
UMA Kegs: 384, 0, 232, 8, 232, 0, 0
UMA Zones: 3712, 0, 232, 0, 232, 0, 0
UMA Slabs: 80, 0, 5499474, 2549626,17407298, 0, 0
UMA RCntSlabs: 88, 0, 25683, 4872, 35793, 0, 0
UMA Hash: 256, 0, 31, 44, 105, 0, 0
4 Bucket: 32, 0, 3483, 14142, 3247114, 0, 0
6 Bucket: 48, 0, 471, 12477, 1181551, 0, 0
8 Bucket: 64, 0, 227, 10685, 550157, 11, 0
12 Bucket: 96, 0, 264, 6747, 779000, 0, 0
16 Bucket: 128, 0, 647, 7258, 737573, 0, 0
32 Bucket: 256, 0, 605, 5650, 1438779, 51, 0
64 Bucket: 512, 0, 454, 5154, 1338457, 103, 0
128 Bucket: 1024, 0, 4099, 177, 1199430, 0, 0
256 Bucket: 2048, 0, 2778, 3088,41077412, 64, 0
vmem btag: 56, 0, 1283986, 21846, 2033663,9217, 0
VM OBJECT: 256, 0, 7113, 1962, 2962913, 0, 0
RADIX NODE: 144, 0, 1205194, 31271,14392452, 0, 0
MAP: 240, 0, 3, 61, 3, 0, 0
KMAP ENTRY: 128, 0, 9, 394, 9, 0, 0
MAP ENTRY: 128, 0, 2784, 6175, 8723723, 0, 0
VMSPACE: 448, 0, 70, 830, 195609, 0, 0
fakepg: 104, 0, 0, 0, 0, 0, 0
mt_zone: 4112, 0, 369, 0, 369, 0, 0
16: 16, 0, 30236, 5908,54245492, 0, 0
32: 32, 0, 9867998, 1052752,128558801, 0, 0
64: 64, 0, 3272549, 2265415,626206792, 0, 0
128: 128, 0, 953557, 3452690,347436440, 0, 0
256: 256, 0, 1335250, 3470,223771204, 0, 0
512: 512, 0, 1476, 665700,72979498, 0, 0
1024: 1024, 0, 26865, 663,60897413, 0, 0
2048: 2048, 0, 105437, 1851,54908239, 0, 0
4096: 4096, 0, 4244, 119, 2623674, 0, 0
8192: 8192, 0, 38, 79, 1909565, 0, 0
16384: 16384, 0, 46, 42, 5118, 0, 0
32768: 32768, 0, 47, 27, 6411, 0, 0
65536: 65536, 0, 106, 32, 175832, 0, 0
64 pcpu: 8, 0, 2110, 1986, 2110, 0, 0
SLEEPQUEUE: 80, 0, 2008, 1867, 2008, 0, 0
Files: 80, 0, 244, 3256, 1335758, 0, 0
TURNSTILE: 136, 0, 2008, 812, 2008, 0, 0
rl_entry: 40, 0, 163, 5937, 163, 0, 0
umtx pi: 96, 0, 0, 0, 0, 0, 0
MAC labels: 40, 0, 0, 0, 0, 0, 0
PROC: 1216, 0, 91, 314, 195630, 0, 0
THREAD: 1168, 0, 1823, 184, 4040, 0, 0
cpuset: 72, 0, 1461, 2169, 3694, 0, 0
audit_record: 1248, 0, 0, 0, 0, 0, 0
mbuf_packet: 256, 39213750, 39030, 7663,147171860, 0, 0
mbuf: 256, 39213750, 1572, 15485,193576128, 0, 0
mbuf_cluster: 2048, 6127148, 46706, 3690, 2856062, 0, 0
mbuf_jumbo_page: 4096, 3063574, 0, 485,12201317, 0, 0
mbuf_jumbo_9k: 9216, 907725, 0, 0, 0, 0, 0
mbuf_jumbo_16k: 16384, 510595, 0, 0, 0, 0, 0
mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0
g_bio: 248, 0, 13, 7843,201376813, 0, 0
ttyinq: 160, 0, 240, 1060, 825, 0, 0
ttyoutq: 256, 0, 126, 1164, 438, 0, 0
DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0
ata_request: 336, 0, 0, 0, 0, 0, 0
vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0
FPU_save_area: 512, 0, 0, 0, 0, 0, 0
taskq_zone: 48, 0, 0, 7636, 527203, 0, 0
VNODE: 472, 0, 3945, 351, 5618, 0, 0
VNODEPOLL: 112, 0, 0, 0, 0, 0, 0
BUF TRIE: 144, 0, 0, 105948, 0, 0, 0
S VFS Cache: 108, 0, 861, 2919, 43860, 0, 0
STS VFS Cache: 148, 0, 0, 0, 0, 0, 0
L VFS Cache: 328, 0, 8, 172, 9, 0, 0
LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0
NAMEI: 1024, 0, 12, 392, 4473407, 0, 0
NCLNODE: 528, 0, 0, 0, 0, 0, 0
DIRHASH: 1024, 0, 0, 0, 0, 0, 0
pipe: 744, 0, 50, 395, 17667, 0, 0
procdesc: 128, 0, 0, 0, 0, 0, 0
range_seg_cache: 64, 0,12378290, 4539464,9289819353, 0, 0
zio_cache: 920, 0, 1514976, 5105,1040425795, 0, 0
zio_link_cache: 48, 0, 1518753, 4961,1208862238, 0, 0
zio_buf_512: 512, 0, 12839, 969, 1068392, 0, 0
zio_data_buf_512: 512, 0, 23, 729, 334489, 0, 0
zio_buf_1024: 1024, 0, 28, 464, 361812, 0, 0
zio_data_buf_1024: 1024, 0, 499, 413, 6239, 0, 0
zio_buf_1536: 1536, 0, 26, 440, 300139, 0, 0
zio_data_buf_1536: 1536, 0, 202, 370, 119693, 0, 0
zio_buf_2048: 2048, 0, 714, 342, 2473795, 0, 0
zio_data_buf_2048: 2048, 0, 31, 581, 462343, 0, 0
zio_buf_2560: 2560, 0, 5, 128, 2488704, 0, 0
zio_data_buf_2560: 2560, 0, 50, 331, 468494, 0, 0
zio_buf_3072: 3072, 0, 58, 217, 574596, 0, 0
zio_data_buf_3072: 3072, 0, 4, 138, 141761, 0, 0
zio_buf_3584: 3584, 0, 405, 85, 183874, 0, 0
zio_data_buf_3584: 3584, 0, 7, 35, 6161, 0, 0
zio_buf_4096: 4096, 0, 3114395, 4549,157598225, 0, 0
zio_data_buf_4096: 4096, 0, 988702, 1285,10589829, 0, 0
zio_buf_5120: 5120, 0, 194, 24, 147156, 0, 0
zio_data_buf_5120: 5120, 0, 8, 67, 10992, 0, 0
zio_buf_6144: 6144, 0, 0, 28, 201018, 0, 0
zio_data_buf_6144: 6144, 0, 5, 36, 6969, 0, 0
zio_buf_7168: 7168, 0, 1, 22, 122127, 0, 0
zio_data_buf_7168: 7168, 0, 0, 29, 5943, 0, 0
zio_buf_8192: 8192, 0, 33, 425,14729804, 0, 0
zio_data_buf_8192: 8192, 0, 1204, 21, 16130, 0, 0
zio_buf_10240: 10240, 0, 2, 25, 148283, 0, 0
zio_data_buf_10240: 10240, 0, 3, 38, 8082, 0, 0
zio_buf_12288: 12288, 0, 5, 139, 4453217, 0, 0
zio_data_buf_12288: 12288, 0, 3, 39, 7069, 0, 0
zio_buf_14336: 14336, 0, 1, 23, 94394, 0, 0
zio_data_buf_14336: 14336, 0, 5, 37, 6987, 0, 0
zio_buf_16384: 16384, 0, 212167, 899,19249190, 0, 0
zio_data_buf_16384: 16384, 0, 744, 27, 93601, 0, 0
zio_buf_20480: 20480, 0, 5, 33, 1153983, 0, 0
zio_data_buf_20480: 20480, 0, 7, 37, 18193, 0, 0
zio_buf_24576: 24576, 0, 7, 39, 1938507, 0, 0
zio_data_buf_24576: 24576, 0, 2, 42, 25428, 0, 0
zio_buf_28672: 28672, 0, 6, 36, 1088596, 0, 0
zio_data_buf_28672: 28672, 0, 0, 41, 30325, 0, 0
zio_buf_32768: 32768, 0, 6, 34, 1682944, 0, 0
zio_data_buf_32768: 32768, 0, 1555, 31, 41421, 0, 0
zio_buf_36864: 36864, 0, 3, 69, 709999, 0, 0
zio_data_buf_36864: 36864, 0, 2, 43, 21459, 0, 0
zio_buf_40960: 40960, 0, 7, 27, 838198, 0, 0
zio_data_buf_40960: 40960, 0, 5, 42, 13220, 0, 0
zio_buf_45056: 45056, 0, 1, 33, 497669, 0, 0
zio_data_buf_45056: 45056, 0, 2, 36, 9262, 0, 0
zio_buf_49152: 49152, 0, 2, 36, 900646, 0, 0
zio_data_buf_49152: 49152, 0, 0, 41, 7357, 0, 0
zio_buf_53248: 53248, 0, 0, 33, 404803, 0, 0
zio_data_buf_53248: 53248, 0, 0, 37, 5590, 0, 0
zio_buf_57344: 57344, 0, 2, 29, 621778, 0, 0
zio_data_buf_57344: 57344, 0, 2, 37, 4943, 0, 0
zio_buf_61440: 61440, 0, 1, 30, 154851, 0, 0
zio_data_buf_61440: 61440, 0, 0, 23, 62, 0, 0
zio_buf_65536: 65536, 0, 3, 223, 6078554, 0, 0
zio_data_buf_65536: 65536, 0, 1005043, 639, 7121724, 0, 0
zio_buf_69632: 69632, 0, 1, 30, 141531, 0, 0
zio_data_buf_69632: 69632, 0, 1, 25, 68, 0, 0
zio_buf_73728: 73728, 0, 2, 30, 273374, 0, 0
zio_data_buf_73728: 73728, 0, 0, 26, 62, 0, 0
zio_buf_77824: 77824, 0, 1, 31, 125265, 0, 0
zio_data_buf_77824: 77824, 0, 0, 27, 62, 0, 0
zio_buf_81920: 81920, 0, 1, 25, 304491, 0, 0
zio_data_buf_81920: 81920, 0, 0, 22, 64, 0, 0
zio_buf_86016: 86016, 0, 2, 33, 119172, 0, 0
zio_data_buf_86016: 86016, 0, 0, 20, 62, 0, 0
zio_buf_90112: 90112, 0, 3, 33, 218576, 0, 0
zio_data_buf_90112: 90112, 0, 1, 26, 70, 0, 0
zio_buf_94208: 94208, 0, 0, 31, 117052, 0, 0
zio_data_buf_94208: 94208, 0, 0, 23, 54, 0, 0
zio_buf_98304: 98304, 0, 2, 30, 276076, 0, 0
zio_data_buf_98304: 98304, 0, 1, 16, 42, 0, 0
zio_buf_102400: 102400, 0, 2, 31, 110060, 0, 0
zio_data_buf_102400: 102400, 0, 0, 23, 43, 0, 0
zio_buf_106496: 106496, 0, 1, 37, 190369, 0, 0
zio_data_buf_106496: 106496, 0, 0, 18, 45, 0, 0
zio_buf_110592: 110592, 0, 0, 39, 112439, 0, 0
zio_data_buf_110592: 110592, 0, 0, 18, 33, 0, 0
zio_buf_114688: 114688, 0, 0, 43, 232331, 0, 0
zio_data_buf_114688: 114688, 0, 0, 18, 33, 0, 0
zio_buf_118784: 118784, 0, 0, 36, 106538, 0, 0
zio_data_buf_118784: 118784, 0, 0, 15, 19, 0, 0
zio_buf_122880: 122880, 0, 2, 37, 275512, 0, 0
zio_data_buf_122880: 122880, 0, 0, 5, 8, 0, 0
zio_buf_126976: 126976, 0, 0, 32, 311151, 0, 0
zio_data_buf_126976: 126976, 0, 0, 8, 10, 0, 0
zio_buf_131072: 131072, 0, 4192, 146, 6540964, 0, 0
zio_data_buf_131072: 131072, 0, 1839, 47, 230507, 0, 0
lz4_ctx: 16384, 0, 1, 93,50363911, 0, 0
sa_cache: 80, 0, 3843, 1307, 5471, 0, 0
dnode_t: 752, 0, 38683, 522, 5927648, 0, 0
dmu_buf_impl_t: 232, 0, 4688886, 953533,26438459, 0, 0
arc_buf_hdr_t: 216, 0, 9692328, 834,16553065, 0, 0
arc_buf_t: 72, 0, 4847203, 2024002,20065244, 0, 0
zil_lwb_cache: 192, 0, 20528, 652, 853333, 0, 0
zfs_znode_cache: 368, 0, 3843, 337, 5471, 0, 0
Mountpoints: 816, 0, 23, 147, 23, 0, 0
ksiginfo: 112, 0, 374, 3406, 1886, 0, 0
itimer: 352, 0, 0, 0, 0, 0, 0
KNOTE: 128, 0, 0, 3007, 8746, 0, 0
socket: 696, 3144485, 69, 336, 23791, 0, 0
ipq: 56, 191487, 0, 0, 0, 0, 0
udp_inpcb: 392, 3144490, 11, 749, 9050, 0, 0
udpcb: 16, 3144528, 11, 6264, 9050, 0, 0
tcp_inpcb: 392, 3144490, 45, 655, 3016, 0, 0
tcpcb: 1024, 3144488, 45, 267, 3016, 0, 0
tcptw: 88, 27810, 0, 0, 0, 0, 0
syncache: 160, 15375, 0, 500, 87, 0, 0
hostcache: 136, 15370, 0, 0, 0, 0, 0
tcpreass: 40, 383000, 0, 800, 57, 0, 0
sackhole: 32, 0, 0, 1625, 208, 0, 0
sctp_ep: 1408, 3144486, 0, 0, 0, 0, 0
sctp_asoc: 2416, 40000, 0, 0, 0, 0, 0
sctp_laddr: 48, 80012, 0, 1079, 4, 0, 0
sctp_raddr: 728, 80000, 0, 0, 0, 0, 0
sctp_chunk: 136, 400026, 0, 0, 0, 0, 0
sctp_readq: 104, 400026, 0, 0, 0, 0, 0
sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0
sctp_asconf: 40, 400000, 0, 0, 0, 0, 0
sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0
udplite_inpcb: 392, 3144490, 0, 0, 0, 0, 0
ripcb: 392, 3144490, 0, 0, 0, 0, 0
unpcb: 240, 3144496, 13, 1459, 11720, 0, 0
rtentry: 200, 0, 15, 485, 15, 0, 0
selfd: 56, 0, 468, 5638, 2586984, 0, 0
SWAPMETA: 288, 12254307, 0, 0, 0, 0, 0
icl_conn: 152, 0, 20, 656, 20, 0, 0
icl_pdu: 80, 0, 78, 7322,18907562, 0, 0
iscsi_outstanding: 40, 0, 0, 0, 0, 0, 0
cfi_lun: 344, 0, 24, 295, 24, 0, 0
cfi_metatask: 320, 0, 0, 0, 0, 0, 0
cfiscsi_data_wait: 56, 0, 0, 7100, 6656404, 0, 0
beio: 360, 0, 40, 840, 8245154, 0, 0
cblk0: 131072, 0, 0, 10, 29, 0, 0
cblk1: 131072, 0, 0, 34, 1028, 0, 0
cblk2: 131072, 0, 0, 14, 29, 0, 0
cblk3: 131072, 0, 0, 10, 29, 0, 0
cblk4: 131072, 0, 0, 7, 29, 0, 0
cblk5: 131072, 0, 0, 20, 94, 0, 0
cblk6: 131072, 0, 1, 79, 2218167, 0, 0
cblk7: 131072, 0, 0, 10, 23, 0, 0
cblk8: 131072, 0, 0, 52, 29547, 0, 0
cblk9: 131072, 0, 0, 0, 0, 0, 0
cblk10: 131072, 0, 0, 0, 0, 0, 0
cblk11: 131072, 0, 0, 79, 20324, 0, 0
cblk12: 131072, 0, 0, 9, 94, 0, 0
cblk13: 131072, 0, 0, 0, 0, 0, 0
cblk14: 131072, 0, 0, 0, 0, 0, 0
cblk15: 131072, 0, 11, 24, 306167, 0, 0
cblk16: 131072, 0, 10, 25, 635505, 0, 0
cblk17: 131072, 0, 11, 25, 2140785, 0, 0
cblk18: 131072, 0, 0, 13, 94, 0, 0
cblk19: 131072, 0, 0, 13, 94, 0, 0
cblk20: 131072, 0, 2, 76, 1173706, 0, 0
cblk21: 131072, 0, 1, 58, 1275285, 0, 0
cblk22: 131072, 0, 2, 35, 35182, 0, 0
cblk23: 131072, 0, 2, 19, 418905, 0, 0
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy
id
0 0 0 2010M 695M 2205 0 0 0 2911 2 0 0 1982 46325 10468 0 3
97
40606/23009/63615 mbufs in use (current/cache/total)
39044/11232/50276/6127148 mbuf clusters in use (current/cache/total/max)
39031/5385 mbuf+clusters out of packet secondary zone in use (current/cache)
0/485/485/3063574 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/907725 9k jumbo clusters in use (current/cache/total/max)
0/0/0/510595 16k jumbo clusters in use (current/cache/total/max)
88239K/30156K/118395K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
UMA Kegs: 384, 0, 232, 8, 232, 0, 0
UMA Zones: 3712, 0, 232, 0, 232, 0, 0
UMA Slabs: 80, 0, 5959753, 2088947,18087329, 0, 0
UMA RCntSlabs: 88, 0, 24769, 5786, 35793, 0, 0
UMA Hash: 256, 0, 31, 44, 105, 0, 0
4 Bucket: 32, 0, 0, 375, 3247539, 0, 0
6 Bucket: 48, 0, 0, 0, 1181564, 0, 0
8 Bucket: 64, 0, 0, 310, 550175, 11, 0
12 Bucket: 96, 0, 0, 0, 779002, 0, 0
16 Bucket: 128, 0, 0, 31, 740829, 0, 0
32 Bucket: 256, 0, 0, 15, 1438785, 51, 0
64 Bucket: 512, 0, 0, 16, 1339041, 103, 0
128 Bucket: 1024, 0, 0, 16, 1200275, 0, 0
256 Bucket: 2048, 0, 0, 8,41096782, 64, 0
vmem btag: 56, 0, 1249155, 56677, 2033663,9217, 0
VM OBJECT: 256, 0, 7113, 1962, 2962980, 0, 0
RADIX NODE: 144, 0, 1161374, 75091,14393249, 0, 0
MAP: 240, 0, 3, 61, 3, 0, 0
KMAP ENTRY: 128, 0, 9, 115, 9, 0, 0
MAP ENTRY: 128, 0, 2784, 5958, 8723921, 0, 0
VMSPACE: 448, 0, 70, 830, 195614, 0, 0
fakepg: 104, 0, 0, 0, 0, 0, 0
mt_zone: 4112, 0, 369, 0, 369, 0, 0
16: 16, 0, 30236, 5657,54245526, 0, 0
32: 32, 0, 9868648, 1052102,128559528, 0, 0
64: 64, 0, 4389252, 1148711,627415425, 0, 0
128: 128, 0, 1453889, 2946933,348010543, 0, 0
256: 256, 0, 1515556, 14,223971924, 0, 0
512: 512, 0, 1476, 4956,72979498, 0, 0
1024: 1024, 0, 26862, 382,60897417, 0, 0
2048: 2048, 0, 105505, 691,54919693, 0, 0
4096: 4096, 0, 4244, 0, 2623680, 0, 0
8192: 8192, 0, 38, 0, 1909565, 0, 0
16384: 16384, 0, 46, 0, 5118, 0, 0
32768: 32768, 0, 47, 0, 6411, 0, 0
65536: 65536, 0, 106, 0, 175832, 0, 0
64 pcpu: 8, 0, 1790, 1026, 2110, 0, 0
SLEEPQUEUE: 80, 0, 1930, 643, 2008, 0, 0
Files: 80, 0, 244, 3256, 1335783, 0, 0
TURNSTILE: 136, 0, 1930, 890, 2008, 0, 0
rl_entry: 40, 0, 163, 2337, 163, 0, 0
umtx pi: 96, 0, 0, 0, 0, 0, 0
MAC labels: 40, 0, 0, 0, 0, 0, 0
PROC: 1216, 0, 91, 314, 195635, 0, 0
THREAD: 1168, 0, 1823, 106, 4040, 0, 0
cpuset: 72, 0, 1461, 1289, 3694, 0, 0
audit_record: 1248, 0, 0, 0, 0, 0, 0
mbuf_packet: 256, 39213750, 39033, 0,147171889, 0, 0
mbuf: 256, 39213750, 1572, 22095,193576870, 0, 0
mbuf_cluster: 2048, 6127148, 39046, 10492, 2856064, 0, 0
mbuf_jumbo_page: 4096, 3063574, 0, 0,12201317, 0, 0
mbuf_jumbo_9k: 9216, 907725, 0, 0, 0, 0, 0
mbuf_jumbo_16k: 16384, 510595, 0, 0, 0, 0, 0
mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0
g_bio: 248, 0, 285, 259,201422605, 0, 0
ttyinq: 160, 0, 240, 260, 825, 0, 0
ttyoutq: 256, 0, 126, 189, 438, 0, 0
DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0
ata_request: 336, 0, 0, 0, 0, 0, 0
vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0
FPU_save_area: 512, 0, 0, 0, 0, 0, 0
taskq_zone: 48, 0, 0, 0, 527206, 0, 0
VNODE: 472, 0, 3945, 151, 5618, 0, 0
VNODEPOLL: 112, 0, 0, 0, 0, 0, 0
BUF TRIE: 144, 0, 0, 105948, 0, 0, 0
S VFS Cache: 108, 0, 861, 1834, 43860, 0, 0
STS VFS Cache: 148, 0, 0, 0, 0, 0, 0
L VFS Cache: 328, 0, 8, 76, 9, 0, 0
LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0
NAMEI: 1024, 0, 12, 36, 4473489, 0, 0
NCLNODE: 528, 0, 0, 0, 0, 0, 0
DIRHASH: 1024, 0, 0, 0, 0, 0, 0
pipe: 744, 0, 50, 160, 17667, 0, 0
procdesc: 128, 0, 0, 0, 0, 0, 0
range_seg_cache: 64, 0,12378247, 4539507,9289886970, 0, 0
zio_cache: 920, 0, 3447907, 1,1042602026, 0, 0
zio_link_cache: 48, 0, 3450416, 60,1211176250, 0, 0
zio_buf_512: 512, 0, 12839, 681, 1068401, 0, 0
zio_data_buf_512: 512, 0, 23, 113, 334497, 0, 0
zio_buf_1024: 1024, 0, 26, 62, 361839, 0, 0
zio_data_buf_1024: 1024, 0, 450, 350, 6240, 0, 0
zio_buf_1536: 1536, 0, 26, 26, 300143, 0, 0
zio_data_buf_1536: 1536, 0, 183, 125, 119693, 0, 0
zio_buf_2048: 2048, 0, 713, 143, 2473799, 0, 0
zio_data_buf_2048: 2048, 0, 29, 29, 462343, 0, 0
zio_buf_2560: 2560, 0, 5, 0, 2488704, 0, 0
zio_data_buf_2560: 2560, 0, 40, 0, 468494, 0, 0
zio_buf_3072: 3072, 0, 58, 0, 574596, 0, 0
zio_data_buf_3072: 3072, 0, 4, 0, 141761, 0, 0
zio_buf_3584: 3584, 0, 405, 0, 183874, 0, 0
zio_data_buf_3584: 3584, 0, 7, 0, 6161, 0, 0
zio_buf_4096: 4096, 0, 3797391, 0,158363763, 0, 0
zio_data_buf_4096: 4096, 0, 947505, 0,10589848, 0, 0
zio_buf_5120: 5120, 0, 194, 0, 147156, 0, 0
zio_data_buf_5120: 5120, 0, 7, 0, 10993, 0, 0
zio_buf_6144: 6144, 0, 0, 0, 201032, 0, 0
zio_data_buf_6144: 6144, 0, 5, 0, 6969, 0, 0
zio_buf_7168: 7168, 0, 1, 0, 122127, 0, 0
zio_data_buf_7168: 7168, 0, 0, 0, 5943, 0, 0
zio_buf_8192: 8192, 0, 33, 22,14731779, 0, 0
zio_data_buf_8192: 8192, 0, 1204, 0, 16130, 0, 0
zio_buf_10240: 10240, 0, 2, 0, 148291, 0, 0
zio_data_buf_10240: 10240, 0, 3, 0, 8082, 0, 0
zio_buf_12288: 12288, 0, 9, 8, 4453649, 0, 0
zio_data_buf_12288: 12288, 0, 3, 0, 7069, 0, 0
zio_buf_14336: 14336, 0, 1, 0, 94394, 0, 0
zio_data_buf_14336: 14336, 0, 5, 0, 6987, 0, 0
zio_buf_16384: 16384, 0, 212111, 4,19250025, 0, 0
zio_data_buf_16384: 16384, 0, 744, 0, 93601, 0, 0
zio_buf_20480: 20480, 0, 8, 4, 1154237, 0, 0
zio_data_buf_20480: 20480, 0, 7, 0, 18193, 0, 0
zio_buf_24576: 24576, 0, 9, 15, 1939001, 0, 0
zio_data_buf_24576: 24576, 0, 2, 0, 25428, 0, 0
zio_buf_28672: 28672, 0, 5, 3, 1088738, 0, 0
zio_data_buf_28672: 28672, 0, 0, 0, 30325, 0, 0
zio_buf_32768: 32768, 0, 13, 3, 1683317, 0, 0
zio_data_buf_32768: 32768, 0, 1478, 0, 41421, 0, 0
zio_buf_36864: 36864, 0, 6, 2, 710105, 0, 0
zio_data_buf_36864: 36864, 0, 2, 0, 21459, 0, 0
zio_buf_40960: 40960, 0, 4, 8, 838398, 0, 0
zio_data_buf_40960: 40960, 0, 5, 0, 13220, 0, 0
zio_buf_45056: 45056, 0, 2, 3, 497740, 0, 0
zio_data_buf_45056: 45056, 0, 2, 0, 9262, 0, 0
zio_buf_49152: 49152, 0, 9, 0, 900864, 0, 0
zio_data_buf_49152: 49152, 0, 0, 0, 7357, 0, 0
zio_buf_53248: 53248, 0, 1, 4, 404869, 0, 0
zio_data_buf_53248: 53248, 0, 0, 0, 5590, 0, 0
zio_buf_57344: 57344, 0, 5, 2, 621923, 0, 0
zio_data_buf_57344: 57344, 0, 2, 0, 4943, 0, 0
zio_buf_61440: 61440, 0, 1, 2, 154901, 0, 0
zio_data_buf_61440: 61440, 0, 0, 0, 62, 0, 0
zio_buf_65536: 65536, 0, 15, 3, 6078895, 0, 0
zio_data_buf_65536: 65536, 0, 924828, 86, 7121974, 0, 0
zio_buf_69632: 69632, 0, 0, 2, 141570, 0, 0
zio_data_buf_69632: 69632, 0, 1, 0, 68, 0, 0
zio_buf_73728: 73728, 0, 0, 2, 273469, 0, 0
zio_data_buf_73728: 73728, 0, 0, 0, 62, 0, 0
zio_buf_77824: 77824, 0, 1, 0, 125296, 0, 0
zio_data_buf_77824: 77824, 0, 0, 0, 62, 0, 0
zio_buf_81920: 81920, 0, 4, 3, 304592, 0, 0
zio_data_buf_81920: 81920, 0, 0, 0, 64, 0, 0
zio_buf_86016: 86016, 0, 1, 0, 119206, 0, 0
zio_data_buf_86016: 86016, 0, 0, 0, 62, 0, 0
zio_buf_90112: 90112, 0, 0, 1, 218641, 0, 0
zio_data_buf_90112: 90112, 0, 1, 0, 70, 0, 0
zio_buf_94208: 94208, 0, 1, 2, 117085, 0, 0
zio_data_buf_94208: 94208, 0, 0, 0, 54, 0, 0
zio_buf_98304: 98304, 0, 0, 5, 276178, 0, 0
zio_data_buf_98304: 98304, 0, 1, 0, 42, 0, 0
zio_buf_102400: 102400, 0, 0, 0, 110085, 0, 0
zio_data_buf_102400: 102400, 0, 0, 0, 43, 0, 0
zio_buf_106496: 106496, 0, 0, 0, 190424, 0, 0
zio_data_buf_106496: 106496, 0, 0, 0, 45, 0, 0
zio_buf_110592: 110592, 0, 0, 4, 112462, 0, 0
zio_data_buf_110592: 110592, 0, 0, 0, 33, 0, 0
zio_buf_114688: 114688, 0, 1, 2, 232388, 0, 0
zio_data_buf_114688: 114688, 0, 0, 0, 33, 0, 0
zio_buf_118784: 118784, 0, 1, 1, 106571, 0, 0
zio_data_buf_118784: 118784, 0, 0, 0, 19, 0, 0
zio_buf_122880: 122880, 0, 0, 2, 275561, 0, 0
zio_data_buf_122880: 122880, 0, 0, 0, 8, 0, 0
zio_buf_126976: 126976, 0, 0, 0, 311168, 0, 0
zio_data_buf_126976: 126976, 0, 0, 0, 10, 0, 0
zio_buf_131072: 131072, 0, 4143, 10, 6544064, 0, 0
zio_data_buf_131072: 131072, 0, 1832, 0, 230507, 0, 0
lz4_ctx: 16384, 0, 4, 8,50555610, 0, 0
sa_cache: 80, 0, 3843, 807, 5471, 0, 0
dnode_t: 752, 0, 38683, 447, 5927648, 0, 0
dmu_buf_impl_t: 232, 0, 4616146, 1021819,26438472, 0, 0
arc_buf_hdr_t: 216, 0, 9645576, 47262,16553270, 0, 0
arc_buf_t: 72, 0, 4765129, 2100466,20065448, 0, 0
zil_lwb_cache: 192, 0, 20528, 432, 853333, 0, 0
zfs_znode_cache: 368, 0, 3843, 197, 5471, 0, 0
Mountpoints: 816, 0, 23, 147, 23, 0, 0
ksiginfo: 112, 0, 374, 1236, 1887, 0, 0
itimer: 352, 0, 0, 0, 0, 0, 0
KNOTE: 128, 0, 0, 0, 8746, 0, 0
socket: 696, 3144485, 69, 336, 23791, 0, 0
ipq: 56, 191487, 0, 0, 0, 0, 0
udp_inpcb: 392, 3144490, 11, 749, 9050, 0, 0
udpcb: 16, 3144528, 11, 6264, 9050, 0, 0
tcp_inpcb: 392, 3144490, 45, 655, 3016, 0, 0
tcpcb: 1024, 3144488, 45, 267, 3016, 0, 0
tcptw: 88, 27810, 0, 0, 0, 0, 0
syncache: 160, 15375, 0, 0, 87, 0, 0
hostcache: 136, 15370, 0, 0, 0, 0, 0
tcpreass: 40, 383000, 0, 800, 57, 0, 0
sackhole: 32, 0, 0, 1625, 208, 0, 0
sctp_ep: 1408, 3144486, 0, 0, 0, 0, 0
sctp_asoc: 2416, 40000, 0, 0, 0, 0, 0
sctp_laddr: 48, 80012, 0, 0, 4, 0, 0
sctp_raddr: 728, 80000, 0, 0, 0, 0, 0
sctp_chunk: 136, 400026, 0, 0, 0, 0, 0
sctp_readq: 104, 400026, 0, 0, 0, 0, 0
sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0
sctp_asconf: 40, 400000, 0, 0, 0, 0, 0
sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0
udplite_inpcb: 392, 3144490, 0, 0, 0, 0, 0
ripcb: 392, 3144490, 0, 0, 0, 0, 0
unpcb: 240, 3144496, 13, 147, 11720, 0, 0
rtentry: 200, 0, 15, 165, 15, 0, 0
selfd: 56, 0, 468, 2798, 2587713, 0, 0
SWAPMETA: 288, 12254307, 4, 9, 4, 0, 0
icl_conn: 152, 0, 20, 266, 20, 0, 0
icl_pdu: 80, 0, 78, 3022,18907642, 0, 0
iscsi_outstanding: 40, 0, 0, 0, 0, 0, 0
cfi_lun: 344, 0, 24, 108, 24, 0, 0
cfi_metatask: 320, 0, 0, 0, 0, 0, 0
cfiscsi_data_wait: 56, 0, 0, 0, 6656404, 0, 0
beio: 360, 0, 40, 323, 8245154, 0, 0
cblk0: 131072, 0, 0, 0, 29, 0, 0
cblk1: 131072, 0, 0, 0, 1028, 0, 0
cblk2: 131072, 0, 0, 0, 29, 0, 0
cblk3: 131072, 0, 0, 0, 29, 0, 0
cblk4: 131072, 0, 0, 0, 29, 0, 0
cblk5: 131072, 0, 0, 0, 94, 0, 0
cblk6: 131072, 0, 1, 0, 2218167, 0, 0
cblk7: 131072, 0, 0, 0, 23, 0, 0
cblk8: 131072, 0, 0, 0, 29547, 0, 0
cblk9: 131072, 0, 0, 0, 0, 0, 0
cblk10: 131072, 0, 0, 0, 0, 0, 0
cblk11: 131072, 0, 0, 0, 20324, 0, 0
cblk12: 131072, 0, 0, 0, 94, 0, 0
cblk13: 131072, 0, 0, 0, 0, 0, 0
cblk14: 131072, 0, 0, 0, 0, 0, 0
cblk15: 131072, 0, 11, 0, 306167, 0, 0
cblk16: 131072, 0, 10, 0, 635505, 0, 0
cblk17: 131072, 0, 11, 0, 2140785, 0, 0
cblk18: 131072, 0, 0, 0, 94, 0, 0
cblk19: 131072, 0, 0, 0, 94, 0, 0
cblk20: 131072, 0, 2, 0, 1173706, 0, 0
cblk21: 131072, 0, 1, 0, 1275285, 0, 0
cblk22: 131072, 0, 2, 0, 35182, 0, 0
cblk23: 131072, 0, 2, 0, 418905, 0, 0
-------------- next part --------------
zio_data_buf_94208: 94208, 0, 0, 24, 623, 0, 0
zio_buf_98304: 98304, 0, 0, 21, 2455384, 0, 0
zio_data_buf_98304: 98304, 0, 0, 21, 619, 0, 0
zio_buf_102400: 102400, 0, 0, 24, 1828532, 0, 0
zio_data_buf_102400: 102400, 0, 0, 21, 609, 0, 0
zio_buf_106496: 106496, 0, 0, 22, 1954601, 0, 0
zio_data_buf_106496: 106496, 0, 0, 22, 606, 0, 0
zio_buf_110592: 110592, 0, 0, 23, 1985097, 0, 0
zio_data_buf_110592: 110592, 0, 0, 19, 575, 0, 0
zio_buf_114688: 114688, 0, 0, 43, 2372902, 0, 0
zio_data_buf_114688: 114688, 0, 0, 25, 581, 0, 0
zio_buf_118784: 118784, 0, 0, 18, 2051745, 0, 0
zio_data_buf_118784: 118784, 0, 0, 24, 584, 0, 0
zio_buf_122880: 122880, 0, 0, 17, 2492560, 0, 0
zio_data_buf_122880: 122880, 0, 0, 20, 582, 0, 0
zio_buf_126976: 126976, 0, 0, 22, 3582892, 0, 0
zio_data_buf_126976: 126976, 0, 0, 21, 581, 0, 0
zio_buf_131072: 131072, 0, 10165, 220,66089692, 0, 0
zio_data_buf_131072: 131072, 0, 3, 22, 16674, 0, 0
lz4_ctx: 16384, 0, 0, 39,466946404, 0, 0
sa_cache: 80, 0, 234131, 29619, 1545572, 0, 0
dnode_t: 752, 0, 301784, 10351, 1557965, 0, 0
dmu_buf_impl_t: 232, 0, 2555190, 555130,249496860, 0, 0
arc_buf_hdr_t: 216, 0,17591090, 28768,172228503, 0, 0
arc_buf_t: 72, 0, 2453676, 680609,186458062, 0, 0
zil_lwb_cache: 192, 0, 480, 2340,33626227, 0, 0
zfs_znode_cache: 368, 0, 234131, 18899, 1545572, 0, 0
Mountpoints: 816, 0, 37, 123, 37, 0, 0
ksiginfo: 112, 0, 229, 2641, 15054, 0, 0
itimer: 352, 0, 1, 10, 1, 0, 0
KNOTE: 128, 0, 0, 2139, 151176, 0, 0
socket: 696, 1571620, 70, 270, 185600, 0, 0
ipq: 56, 95708, 0, 0, 0, 0, 0
udp_inpcb: 392, 1571620, 18, 522, 87692, 0, 0
udpcb: 16, 1571762, 18, 4751, 87692, 0, 0
tcp_inpcb: 392, 1571620, 36, 614, 91538, 0, 0
tcpcb: 1024, 1571620, 36, 272, 91538, 0, 0
tcptw: 88, 27810, 0, 540, 8, 0, 0
syncache: 160, 15375, 0, 500, 88577, 0, 0
hostcache: 136, 15370, 1, 202, 10, 0, 0
tcpreass: 40, 191400, 0, 2100, 3370, 0, 0
sackhole: 32, 0, 0, 1875, 5046, 0, 0
sctp_ep: 1408, 1571620, 0, 0, 0, 0, 0
sctp_asoc: 2416, 40000, 0, 0, 0, 0, 0
sctp_laddr: 48, 80012, 0, 0, 4, 0, 0
sctp_raddr: 728, 80000, 0, 0, 0, 0, 0
sctp_chunk: 136, 400026, 0, 0, 0, 0, 0
sctp_readq: 104, 400026, 0, 0, 0, 0, 0
sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0
sctp_asconf: 40, 400000, 0, 0, 0, 0, 0
sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0
udplite_inpcb: 392, 1571620, 0, 0, 0, 0, 0
ripcb: 392, 1571620, 0, 90, 5, 0, 0
unpcb: 240, 1571632, 15, 865, 6360, 0, 0
rtentry: 200, 0, 14, 146, 14, 0, 0
selfd: 56, 0, 356, 5040,11798434, 0, 0
SWAPMETA: 288, 6123806, 816, 757, 58324, 0, 0
icl_conn: 152, 0, 13, 247, 28, 0, 0
icl_pdu: 80, 0, 3, 2647,243044586, 0, 0
iscsi_outstanding: 40, 0, 0, 0, 0, 0, 0
cfi_lun: 344, 0, 14, 85, 15, 0, 0
cfi_metatask: 320, 0, 0, 0, 0, 0, 0
cfiscsi_data_wait: 56, 0, 0, 6319,102765061, 0, 0
beio: 360, 0, 2, 735,114743805, 0, 0
cblk0: 131072, 0, 0, 11, 58648, 0, 0
cblk1: 131072, 0, 0, 20, 1849, 0, 0
cblk2: 131072, 0, 1, 23, 9398207, 0, 0
cblk3: 131072, 0, 0, 25,13447196, 0, 0
cblk4: 131072, 0, 0, 0, 0, 0, 0
cblk5: 131072, 0, 0, 29,21650203, 0, 0
cblk6: 131072, 0, 0, 52,43568091, 0, 0
cblk7: 131072, 0, 1, 21,10769749, 0, 0
cblk8: 131072, 0, 0, 23, 2138715, 0, 0
cblk9: 131072, 0, 0, 24,11411640, 0, 0
cblk10: 131072, 0, 0, 34, 2683020, 0, 0
cblk11: 131072, 0, 0, 27, 39210, 0, 0
cblk12: 131072, 0, 0, 22, 404743, 0, 0
cblk13: 131072, 0, 0, 22, 2592, 0, 0
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr mf0 mf1 in sy cs us sy
id
0 0 6 1307M 2511M 696 0 0 0 1712 3 0 0 3412 1741 7709 0 3
97
18995/12505/31500 mbufs in use (current/cache/total)
18424/9636/28060/3061900 mbuf clusters in use (current/cache/total/max)
18424/6231 mbuf+clusters out of packet secondary zone in use (current/cache)
34/193/227/1530950 4k (page size) jumbo clusters in use
(current/cache/total/max)
0/0/0/453614 9k jumbo clusters in use (current/cache/total/max)
0/0/0/255158 16k jumbo clusters in use (current/cache/total/max)
41732K/23170K/64903K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
UMA Kegs: 384, 0, 222, 8, 223, 0, 0
UMA Zones: 2688, 0, 222, 0, 223, 0, 0
UMA Slabs: 80, 0, 2638317, 540783,67248442, 0, 0
UMA RCntSlabs: 88, 0, 14257, 2933, 323489, 0, 0
UMA Hash: 256, 0, 23, 52, 96, 0, 0
4 Bucket: 32, 0, 2885, 8615,66481595,3376, 0
6 Bucket: 48, 0, 271, 6950,18810051, 0, 0
8 Bucket: 64, 0, 514, 6058, 9143820, 11, 0
12 Bucket: 96, 0, 704, 3888,20508529,121062, 0
16 Bucket: 128, 0, 346, 5761,14989247,66229, 0
32 Bucket: 256, 0, 343, 2252,16409932, 51, 0
64 Bucket: 512, 0, 4122, 286,10129138,851803, 0
128 Bucket: 1024, 0, 433, 803,13814281, 905, 0
256 Bucket: 2048, 0, 3277, 1581,88561921,10775, 0
vmem btag: 56, 0, 763673, 85416, 4162400,6053, 0
VM OBJECT: 256, 0, 61288, 1292, 6702671, 0, 0
RADIX NODE: 144, 0, 564514, 56729,34128828, 0, 0
MAP: 240, 0, 3, 61, 3, 0, 0
KMAP ENTRY: 128, 0, 8, 116, 8, 0, 0
MAP ENTRY: 128, 0, 1281, 2997,18796726, 0, 0
VMSPACE: 448, 0, 37, 584, 458828, 0, 0
fakepg: 104, 0, 0, 266, 50, 0, 0
mt_zone: 4112, 0, 369, 0, 369, 0, 0
16: 16, 0, 109368, 6092,408372116, 0, 0
32: 32, 0,30248937, 62438,885337186, 0, 0
64: 64, 0, 1640783, 347743,3429375675, 0, 0
128: 128, 0, 1056501, 1306908,3187205989, 0, 0
256: 256, 0, 513689, 2536,2294317506, 0, 0
512: 512, 0, 165411, 232053,639797597, 0, 0
1024: 1024, 0, 15989, 287,20235850, 0, 0
2048: 2048, 0, 63005, 1433,483730389, 0, 0
4096: 4096, 0, 13029, 54,41864969, 0, 0
8192: 8192, 0, 26, 21, 451764, 0, 0
16384: 16384, 0, 35, 17, 2766, 0, 0
32768: 32768, 0, 39, 16, 17810, 0, 0
65536: 65536, 0, 86, 20, 304734, 0, 0
64 pcpu: 8, 0, 1654, 650, 1874, 0, 0
SLEEPQUEUE: 80, 0, 1261, 1033, 1342, 0, 0
Files: 80, 0, 121, 2279, 4017074, 0, 0
TURNSTILE: 136, 0, 1261, 519, 1342, 0, 0
rl_entry: 40, 0, 199, 2501, 199, 0, 0
umtx pi: 96, 0, 0, 0, 0, 0, 0
MAC labels: 40, 0, 0, 0, 0, 0, 0
PROC: 1216, 0, 55, 185, 458858, 0, 0
THREAD: 1168, 0, 1199, 61, 1767, 0, 0
cpuset: 72, 0, 959, 1076, 1522, 0, 0
audit_record: 1248, 0, 0, 0, 0, 0, 0
mbuf_packet: 256, 19596165, 18423, 6232,2677137598, 0, 0
mbuf: 256, 19596165, 535, 6310,2780612729, 0, 0
mbuf_cluster: 2048, 3061900, 24655, 3405,548627688, 0, 0
mbuf_jumbo_page: 4096, 1530950, 8, 219,144227225, 0, 0
mbuf_jumbo_9k: 9216, 453614, 0, 0, 0, 0, 0
mbuf_jumbo_16k: 16384, 255158, 0, 0, 0, 0, 0
mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0
g_bio: 248, 0, 44, 3444,1931890439, 0, 0
ttyinq: 160, 0, 240, 235, 825, 0, 0
ttyoutq: 256, 0, 126, 114, 438, 0, 0
DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0
ata_request: 336, 0, 0, 0, 0, 0, 0
vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0
FPU_save_area: 512, 0, 0, 0, 0, 0, 0
taskq_zone: 48, 0, 0, 6225, 5981831, 0, 0
VNODE: 472, 0, 234231, 17921, 1546031, 0, 0
VNODEPOLL: 112, 0, 0, 0, 0, 0, 0
BUF TRIE: 144, 0, 0, 105948, 0, 0, 0
S VFS Cache: 108, 0, 138464, 63661, 1825006, 0, 0
STS VFS Cache: 148, 0, 0, 0, 0, 0, 0
L VFS Cache: 328, 0, 56797, 527, 312470, 0, 0
LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0
NAMEI: 1024, 0, 0, 276,14270986, 0, 0
NCLNODE: 528, 0, 0, 0, 0, 0, 0
DIRHASH: 1024, 0, 0, 0, 0, 0, 0
pipe: 744, 0, 4, 256, 21165, 0, 0
procdesc: 128, 0, 0, 0, 0, 0, 0
range_seg_cache: 64, 0, 1891912, 1155450,5241307054, 0, 0
zio_cache: 920, 0, 4623, 307345,9478353285, 0, 0
zio_link_cache: 48, 0, 4647, 307682,10893193647, 0, 0
zio_buf_512: 512, 0, 253441, 31767,17337254, 0, 0
zio_data_buf_512: 512, 0, 10, 654, 1271635, 0, 0
zio_buf_1024: 1024, 0, 35, 357, 2281623, 0, 0
zio_data_buf_1024: 1024, 0, 1, 851, 405872, 0, 0
zio_buf_1536: 1536, 0, 31, 199, 1317397, 0, 0
zio_data_buf_1536: 1536, 0, 0, 194, 539831, 0, 0
zio_buf_2048: 2048, 0, 72, 168, 5942777, 0, 0
zio_data_buf_2048: 2048, 0, 0, 120, 718243, 0, 0
zio_buf_2560: 2560, 0, 0, 58, 2680883, 0, 0
zio_data_buf_2560: 2560, 0, 1, 63, 562169, 0, 0
zio_buf_3072: 3072, 0, 87, 60, 2322177, 0, 0
zio_data_buf_3072: 3072, 0, 0, 63, 207706, 0, 0
zio_buf_3584: 3584, 0, 1, 19, 851979, 0, 0
zio_data_buf_3584: 3584, 0, 1, 20, 187078, 0, 0
zio_buf_4096: 4096, 0, 1814858, 4449,1036609626, 0, 0
zio_data_buf_4096: 4096, 0, 40698, 2155,23724089, 0, 0
zio_buf_5120: 5120, 0, 209, 24, 5051131, 0, 0
zio_data_buf_5120: 5120, 0, 0, 24, 281978, 0, 0
zio_buf_6144: 6144, 0, 0, 29, 2959422, 0, 0
zio_data_buf_6144: 6144, 0, 1, 28, 522283, 0, 0
zio_buf_7168: 7168, 0, 0, 27, 2783788, 0, 0
zio_data_buf_7168: 7168, 0, 0, 26, 441549, 0, 0
zio_buf_8192: 8192, 0, 0, 156,95867606, 0, 0
zio_data_buf_8192: 8192, 0, 0, 24, 316220, 0, 0
zio_buf_10240: 10240, 0, 0, 36, 4984923, 0, 0
zio_data_buf_10240: 10240, 0, 1, 26, 549926, 0, 0
zio_buf_12288: 12288, 0, 0, 66,27978869, 0, 0
zio_data_buf_12288: 12288, 0, 1, 26, 506823, 0, 0
zio_buf_14336: 14336, 0, 0, 29, 4274427, 0, 0
zio_data_buf_14336: 14336, 0, 0, 27, 417679, 0, 0
zio_buf_16384: 16384, 0, 243974, 683,142682652, 0, 0
zio_data_buf_16384: 16384, 0, 668, 32, 772462, 0, 0
zio_buf_20480: 20480, 0, 0, 57,21001730, 0, 0
zio_data_buf_20480: 20480, 0, 0, 28, 522774, 0, 0
zio_buf_24576: 24576, 0, 0, 53,21137990, 0, 0
zio_data_buf_24576: 24576, 0, 0, 27, 430090, 0, 0
zio_buf_28672: 28672, 0, 0, 55,19522849, 0, 0
zio_data_buf_28672: 28672, 0, 0, 26, 423479, 0, 0
zio_buf_32768: 32768, 0, 0, 267,20511123, 0, 0
zio_data_buf_32768: 32768, 0, 144, 82,29293434, 0, 0
zio_buf_36864: 36864, 0, 0, 44,11033800, 0, 0
zio_data_buf_36864: 36864, 0, 0, 28, 399093, 0, 0
zio_buf_40960: 40960, 0, 0, 42,10596065, 0, 0
zio_data_buf_40960: 40960, 0, 0, 28, 461019, 0, 0
zio_buf_45056: 45056, 0, 0, 32, 8978100, 0, 0
zio_data_buf_45056: 45056, 0, 0, 29, 472202, 0, 0
zio_buf_49152: 49152, 0, 0, 23,10466463, 0, 0
zio_data_buf_49152: 49152, 0, 0, 24, 394231, 0, 0
zio_buf_53248: 53248, 0, 0, 24, 7603930, 0, 0
zio_data_buf_53248: 53248, 0, 0, 27, 298722, 0, 0
zio_buf_57344: 57344, 0, 0, 46, 7539298, 0, 0
zio_data_buf_57344: 57344, 0, 0, 24, 372719, 0, 0
zio_buf_61440: 61440, 0, 0, 16, 3313241, 0, 0
zio_data_buf_61440: 61440, 0, 0, 23, 623, 0, 0
zio_buf_65536: 65536, 0, 0, 1893,99134107, 0, 0
zio_data_buf_65536: 65536, 0, 374970, 182,116394545, 0, 0
zio_buf_69632: 69632, 0, 0, 19, 2368803, 0, 0
zio_data_buf_69632: 69632, 0, 0, 21, 672, 0, 0
zio_buf_73728: 73728, 0, 0, 18, 2435354, 0, 0
zio_data_buf_73728: 73728, 0, 0, 21, 622, 0, 0
zio_buf_77824: 77824, 0, 0, 18, 1974579, 0, 0
zio_data_buf_77824: 77824, 0, 0, 22, 629, 0, 0
zio_buf_81920: 81920, 0, 0, 19, 2421622, 0, 0
zio_data_buf_81920: 81920, 0, 0, 24, 621, 0, 0
zio_buf_86016: 86016, 0, 0, 18, 1811521, 0, 0
zio_data_buf_86016: 86016, 0, 0, 21, 615, 0, 0
zio_buf_90112: 90112, 0, 0, 18, 1965656, 0, 0
zio_data_buf_90112: 90112, 0, 0, 21, 694, 0, 0
zio_buf_94208: 94208, 0, 0, 19, 1786991, 0, 0
zio_data_buf_94208: 94208, 0, 0, 24, 623, 0, 0
zio_buf_98304: 98304, 0, 0, 21, 2455386, 0, 0
zio_data_buf_98304: 98304, 0, 0, 21, 619, 0, 0
zio_buf_102400: 102400, 0, 0, 24, 1828532, 0, 0
zio_data_buf_102400: 102400, 0, 0, 21, 609, 0, 0
zio_buf_106496: 106496, 0, 0, 22, 1954606, 0, 0
zio_data_buf_106496: 106496, 0, 0, 22, 606, 0, 0
zio_buf_110592: 110592, 0, 0, 23, 1985104, 0, 0
zio_data_buf_110592: 110592, 0, 0, 19, 575, 0, 0
zio_buf_114688: 114688, 0, 0, 43, 2372904, 0, 0
zio_data_buf_114688: 114688, 0, 0, 25, 581, 0, 0
zio_buf_118784: 118784, 0, 0, 18, 2051746, 0, 0
zio_data_buf_118784: 118784, 0, 0, 24, 584, 0, 0
zio_buf_122880: 122880, 0, 0, 18, 2492568, 0, 0
zio_data_buf_122880: 122880, 0, 0, 20, 582, 0, 0
zio_buf_126976: 126976, 0, 0, 22, 3582896, 0, 0
zio_data_buf_126976: 126976, 0, 0, 21, 581, 0, 0
zio_buf_131072: 131072, 0, 11754, 343,66094061, 0, 0
zio_data_buf_131072: 131072, 0, 3, 22, 16674, 0, 0
lz4_ctx: 16384, 0, 0, 39,466971031, 0, 0
sa_cache: 80, 0, 234131, 29619, 1545572, 0, 0
dnode_t: 752, 0, 301784, 10351, 1557965, 0, 0
dmu_buf_impl_t: 232, 0, 2600375, 509945,249554905, 0, 0
arc_buf_hdr_t: 216, 0,17597187, 22671,172239331, 0, 0
arc_buf_t: 72, 0, 2458221, 676064,186469281, 0, 0
zil_lwb_cache: 192, 0, 3989, 611,33629736, 0, 0
zfs_znode_cache: 368, 0, 234131, 18899, 1545572, 0, 0
Mountpoints: 816, 0, 37, 123, 37, 0, 0
ksiginfo: 112, 0, 229, 2641, 15054, 0, 0
itimer: 352, 0, 1, 10, 1, 0, 0
KNOTE: 128, 0, 0, 2139, 151176, 0, 0
socket: 696, 1571620, 70, 270, 185600, 0, 0
ipq: 56, 95708, 0, 0, 0, 0, 0
udp_inpcb: 392, 1571620, 18, 522, 87692, 0, 0
udpcb: 16, 1571762, 18, 4751, 87692, 0, 0
tcp_inpcb: 392, 1571620, 36, 614, 91538, 0, 0
tcpcb: 1024, 1571620, 36, 272, 91538, 0, 0
tcptw: 88, 27810, 0, 540, 8, 0, 0
syncache: 160, 15375, 0, 500, 88577, 0, 0
hostcache: 136, 15370, 1, 202, 10, 0, 0
tcpreass: 40, 191400, 0, 2100, 3370, 0, 0
sackhole: 32, 0, 0, 1875, 5047, 0, 0
sctp_ep: 1408, 1571620, 0, 0, 0, 0, 0
sctp_asoc: 2416, 40000, 0, 0, 0, 0, 0
sctp_laddr: 48, 80012, 0, 0, 4, 0, 0
sctp_raddr: 728, 80000, 0, 0, 0, 0, 0
sctp_chunk: 136, 400026, 0, 0, 0, 0, 0
sctp_readq: 104, 400026, 0, 0, 0, 0, 0
sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0
sctp_asconf: 40, 400000, 0, 0, 0, 0, 0
sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0
udplite_inpcb: 392, 1571620, 0, 0, 0, 0, 0
ripcb: 392, 1571620, 0, 90, 5, 0, 0
unpcb: 240, 1571632, 15, 865, 6360, 0, 0
rtentry: 200, 0, 14, 146, 14, 0, 0
selfd: 56, 0, 356, 5040,11798535, 0, 0
SWAPMETA: 288, 6123806, 816, 757, 58326, 0, 0
icl_conn: 152, 0, 13, 247, 28, 0, 0
icl_pdu: 80, 0, 8, 2642,243055499, 0, 0
iscsi_outstanding: 40, 0, 0, 0, 0, 0, 0
cfi_lun: 344, 0, 14, 85, 15, 0, 0
cfi_metatask: 320, 0, 0, 0, 0, 0, 0
cfiscsi_data_wait: 56, 0, 0, 6319,102768869, 0, 0
beio: 360, 0, 3, 734,114748708, 0, 0
cblk0: 131072, 0, 0, 11, 58648, 0, 0
cblk1: 131072, 0, 0, 20, 1849, 0, 0
cblk2: 131072, 0, 0, 26, 9398547, 0, 0
cblk3: 131072, 0, 0, 25,13447442, 0, 0
cblk4: 131072, 0, 0, 0, 0, 0, 0
cblk5: 131072, 0, 0, 29,21650563, 0, 0
cblk6: 131072, 0, 0, 52,43568951, 0, 0
cblk7: 131072, 0, 0, 23,10769922, 0, 0
cblk8: 131072, 0, 0, 23, 2138746, 0, 0
cblk9: 131072, 0, 0, 26,11412682, 0, 0
cblk10: 131072, 0, 0, 35, 2684836, 0, 0
cblk11: 131072, 0, 0, 27, 39210, 0, 0
cblk12: 131072, 0, 0, 22, 404782, 0, 0
cblk13: 131072, 0, 0, 22, 2592, 0, 0
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr mf0 mf1 in sy cs us sy
id
0 0 6 1307M 2241M 696 0 0 0 1712 3 0 0 3412 1741 7710 0 3
97
19025/12475/31500 mbufs in use (current/cache/total)
18495/9565/28060/3061900 mbuf clusters in use (current/cache/total/max)
18462/6446 mbuf+clusters out of packet secondary zone in use (current/cache)
0/299/299/1530950 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/453614 9k jumbo clusters in use (current/cache/total/max)
0/0/0/255158 16k jumbo clusters in use (current/cache/total/max)
41746K/23444K/65191K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
UMA Kegs: 384, 0, 222, 8, 223, 0, 0
UMA Zones: 2688, 0, 222, 0, 223, 0, 0
UMA Slabs: 80, 0, 2722107, 456993,67332232, 0, 0
UMA RCntSlabs: 88, 0, 14329, 2861, 323561, 0, 0
UMA Hash: 256, 0, 23, 52, 96, 0, 0
4 Bucket: 32, 0, 3218, 8282,66485794,3376, 0
6 Bucket: 48, 0, 306, 6915,18810333, 0, 0
8 Bucket: 64, 0, 747, 5825, 9144404, 11, 0
12 Bucket: 96, 0, 916, 3676,20515343,121062, 0
16 Bucket: 128, 0, 339, 5768,14989263,66229, 0
32 Bucket: 256, 0, 381, 2214,16410193, 51, 0
64 Bucket: 512, 0, 6815, 177,10131995,851803, 0
128 Bucket: 1024, 0, 471, 765,13814595, 905, 0
256 Bucket: 2048, 0, 1357, 3501,88566899,10775, 0
vmem btag: 56, 0, 763683, 85406, 4162410,6053, 0
VM OBJECT: 256, 0, 61288, 1292, 6702738, 0, 0
RADIX NODE: 144, 0, 564770, 56473,34129369, 0, 0
MAP: 240, 0, 3, 61, 3, 0, 0
KMAP ENTRY: 128, 0, 8, 116, 8, 0, 0
MAP ENTRY: 128, 0, 1281, 2997,18796924, 0, 0
VMSPACE: 448, 0, 37, 584, 458833, 0, 0
fakepg: 104, 0, 0, 266, 50, 0, 0
mt_zone: 4112, 0, 369, 0, 369, 0, 0
16: 16, 0, 109372, 6088,408385033, 0, 0
32: 32, 0,30189005, 122370,885454216, 0, 0
64: 64, 0, 1755206, 233320,3429680449, 0, 0
128: 128, 0, 1137055, 1226354,3187334897, 0, 0
256: 256, 0, 824680, 1040,2294645206, 0, 0
512: 512, 0, 862, 396602,639985839, 0, 0
1024: 1024, 0, 15993, 283,20235873, 0, 0
2048: 2048, 0, 63026, 1412,483737195, 0, 0
4096: 4096, 0, 13029, 54,41865552, 0, 0
8192: 8192, 0, 26, 21, 451764, 0, 0
16384: 16384, 0, 35, 17, 2766, 0, 0
32768: 32768, 0, 39, 16, 17810, 0, 0
65536: 65536, 0, 86, 20, 304734, 0, 0
64 pcpu: 8, 0, 1654, 650, 1874, 0, 0
SLEEPQUEUE: 80, 0, 1261, 1033, 1342, 0, 0
Files: 80, 0, 124, 2276, 4017109, 0, 0
TURNSTILE: 136, 0, 1261, 519, 1342, 0, 0
rl_entry: 40, 0, 199, 2501, 199, 0, 0
umtx pi: 96, 0, 0, 0, 0, 0, 0
MAC labels: 40, 0, 0, 0, 0, 0, 0
PROC: 1216, 0, 55, 185, 458863, 0, 0
THREAD: 1168, 0, 1199, 61, 1767, 0, 0
cpuset: 72, 0, 959, 1076, 1522, 0, 0
audit_record: 1248, 0, 0, 0, 0, 0, 0
mbuf_packet: 256, 19596165, 18464, 6444,2677244083, 0, 0
mbuf: 256, 19596165, 561, 6031,2780744081, 0, 0
mbuf_cluster: 2048, 3061900, 24941, 3119,548632931, 0, 0
mbuf_jumbo_page: 4096, 1530950, 0, 299,144250719, 0, 0
mbuf_jumbo_9k: 9216, 453614, 0, 0, 0, 0, 0
mbuf_jumbo_16k: 16384, 255158, 0, 0, 0, 0, 0
mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0
g_bio: 248, 0, 106, 3382,1931929619, 0, 0
ttyinq: 160, 0, 240, 235, 825, 0, 0
ttyoutq: 256, 0, 126, 114, 438, 0, 0
DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0
ata_request: 336, 0, 0, 0, 0, 0, 0
vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0
FPU_save_area: 512, 0, 0, 0, 0, 0, 0
taskq_zone: 48, 0, 0, 6225, 5982888, 0, 0
VNODE: 472, 0, 234231, 17921, 1546031, 0, 0
VNODEPOLL: 112, 0, 0, 0, 0, 0, 0
BUF TRIE: 144, 0, 0, 105948, 0, 0, 0
S VFS Cache: 108, 0, 138464, 63661, 1825006, 0, 0
STS VFS Cache: 148, 0, 0, 0, 0, 0, 0
L VFS Cache: 328, 0, 56797, 527, 312470, 0, 0
LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0
NAMEI: 1024, 0, 0, 276,14271074, 0, 0
NCLNODE: 528, 0, 0, 0, 0, 0, 0
DIRHASH: 1024, 0, 0, 0, 0, 0, 0
pipe: 744, 0, 4, 256, 21165, 0, 0
procdesc: 128, 0, 0, 0, 0, 0, 0
range_seg_cache: 64, 0, 1888793, 1158569,5241312679, 0, 0
zio_cache: 920, 0, 278778, 33190,9478726297, 0, 0
zio_link_cache: 48, 0, 279004, 33325,10893596367, 0, 0
zio_buf_512: 512, 0, 253441, 31767,17337312, 0, 0
zio_data_buf_512: 512, 0, 10, 654, 1271658, 0, 0
zio_buf_1024: 1024, 0, 35, 357, 2281661, 0, 0
zio_data_buf_1024: 1024, 0, 1, 851, 405927, 0, 0
zio_buf_1536: 1536, 0, 31, 199, 1317424, 0, 0
zio_data_buf_1536: 1536, 0, 0, 194, 539897, 0, 0
zio_buf_2048: 2048, 0, 72, 168, 5942869, 0, 0
zio_data_buf_2048: 2048, 0, 0, 120, 718328, 0, 0
zio_buf_2560: 2560, 0, 0, 58, 2681043, 0, 0
zio_data_buf_2560: 2560, 0, 1, 63, 562221, 0, 0
zio_buf_3072: 3072, 0, 87, 69, 2322631, 0, 0
zio_data_buf_3072: 3072, 0, 0, 63, 207718, 0, 0
zio_buf_3584: 3584, 0, 1, 23, 852019, 0, 0
zio_data_buf_3584: 3584, 0, 1, 20, 187085, 0, 0
zio_buf_4096: 4096, 0, 1897651, 1815,1036695078, 0, 0
zio_data_buf_4096: 4096, 0, 42385, 2025,23760897, 0, 0
zio_buf_5120: 5120, 0, 209, 24, 5051172, 0, 0
zio_data_buf_5120: 5120, 0, 0, 24, 281982, 0, 0
zio_buf_6144: 6144, 0, 0, 29, 2959504, 0, 0
zio_data_buf_6144: 6144, 0, 1, 28, 522306, 0, 0
zio_buf_7168: 7168, 0, 0, 28, 2783965, 0, 0
zio_data_buf_7168: 7168, 0, 0, 26, 441566, 0, 0
zio_buf_8192: 8192, 0, 2, 226,95868308, 0, 0
zio_data_buf_8192: 8192, 0, 0, 24, 316226, 0, 0
zio_buf_10240: 10240, 0, 0, 36, 4985020, 0, 0
zio_data_buf_10240: 10240, 0, 1, 26, 549938, 0, 0
zio_buf_12288: 12288, 0, 1, 65,27979095, 0, 0
zio_data_buf_12288: 12288, 0, 1, 26, 506831, 0, 0
zio_buf_14336: 14336, 0, 2, 27, 4274509, 0, 0
zio_data_buf_14336: 14336, 0, 0, 27, 417695, 0, 0
zio_buf_16384: 16384, 0, 243864, 793,142682906, 0, 0
zio_data_buf_16384: 16384, 0, 1050, 20, 773256, 0, 0
zio_buf_20480: 20480, 0, 2, 74,21002080, 0, 0
zio_data_buf_20480: 20480, 0, 0, 28, 522849, 0, 0
zio_buf_24576: 24576, 0, 0, 56,21138422, 0, 0
zio_data_buf_24576: 24576, 0, 0, 28, 430178, 0, 0
zio_buf_28672: 28672, 0, 0, 84,19523237, 0, 0
zio_data_buf_28672: 28672, 0, 0, 26, 423593, 0, 0
zio_buf_32768: 32768, 0, 0, 267,20511381, 0, 0
zio_data_buf_32768: 32768, 0, 170, 56,29295067, 0, 0
zio_buf_36864: 36864, 0, 0, 57,11034118, 0, 0
zio_data_buf_36864: 36864, 0, 0, 29, 399149, 0, 0
zio_buf_40960: 40960, 0, 0, 57,10596348, 0, 0
zio_data_buf_40960: 40960, 0, 0, 28, 461044, 0, 0
zio_buf_45056: 45056, 0, 0, 91, 8978539, 0, 0
zio_data_buf_45056: 45056, 0, 0, 29, 472234, 0, 0
zio_buf_49152: 49152, 0, 0, 90,10466900, 0, 0
zio_data_buf_49152: 49152, 0, 0, 24, 394256, 0, 0
zio_buf_53248: 53248, 0, 2, 80, 7604219, 0, 0
zio_data_buf_53248: 53248, 0, 0, 27, 298737, 0, 0
zio_buf_57344: 57344, 0, 0, 50, 7539502, 0, 0
zio_data_buf_57344: 57344, 0, 0, 25, 372730, 0, 0
zio_buf_61440: 61440, 0, 0, 17, 3313252, 0, 0
zio_data_buf_61440: 61440, 0, 0, 23, 623, 0, 0
zio_buf_65536: 65536, 0, 0, 1893,99134123, 0, 0
zio_data_buf_65536: 65536, 0, 374845, 1315,116400469, 0, 0
zio_buf_69632: 69632, 0, 0, 19, 2368814, 0, 0
zio_data_buf_69632: 69632, 0, 0, 21, 672, 0, 0
zio_buf_73728: 73728, 0, 0, 19, 2435364, 0, 0
zio_data_buf_73728: 73728, 0, 0, 21, 622, 0, 0
zio_buf_77824: 77824, 0, 0, 18, 1974591, 0, 0
zio_data_buf_77824: 77824, 0, 0, 22, 629, 0, 0
zio_buf_81920: 81920, 0, 0, 21, 2421632, 0, 0
zio_data_buf_81920: 81920, 0, 0, 24, 621, 0, 0
zio_buf_86016: 86016, 0, 0, 18, 1811531, 0, 0
zio_data_buf_86016: 86016, 0, 0, 21, 615, 0, 0
zio_buf_90112: 90112, 0, 0, 18, 1965678, 0, 0
zio_data_buf_90112: 90112, 0, 0, 21, 694, 0, 0
zio_buf_94208: 94208, 0, 0, 20, 1787020, 0, 0
zio_data_buf_94208: 94208, 0, 0, 24, 623, 0, 0
zio_buf_98304: 98304, 0, 0, 21, 2455407, 0, 0
zio_data_buf_98304: 98304, 0, 0, 21, 619, 0, 0
zio_buf_102400: 102400, 0, 0, 28, 1828550, 0, 0
zio_data_buf_102400: 102400, 0, 0, 21, 609, 0, 0
zio_buf_106496: 106496, 0, 0, 22, 1954622, 0, 0
zio_data_buf_106496: 106496, 0, 0, 22, 606, 0, 0
zio_buf_110592: 110592, 0, 0, 23, 1985116, 0, 0
zio_data_buf_110592: 110592, 0, 0, 19, 575, 0, 0
zio_buf_114688: 114688, 0, 0, 43, 2372918, 0, 0
zio_data_buf_114688: 114688, 0, 0, 25, 581, 0, 0
zio_buf_118784: 118784, 0, 1, 17, 2051768, 0, 0
zio_data_buf_118784: 118784, 0, 0, 24, 584, 0, 0
zio_buf_122880: 122880, 0, 0, 18, 2492585, 0, 0
zio_data_buf_122880: 122880, 0, 0, 20, 582, 0, 0
zio_buf_126976: 126976, 0, 0, 22, 3582924, 0, 0
zio_data_buf_126976: 126976, 0, 0, 21, 581, 0, 0
zio_buf_131072: 131072, 0, 10474, 1623,66097273, 0, 0
zio_data_buf_131072: 131072, 0, 15, 22, 16697, 0, 0
lz4_ctx: 16384, 0, 1, 38,467035925, 0, 0
sa_cache: 80, 0, 234131, 29619, 1545572, 0, 0
dnode_t: 752, 0, 301784, 10351, 1557965, 0, 0
dmu_buf_impl_t: 232, 0, 2598633, 511687,249567134, 0, 0
arc_buf_hdr_t: 216, 0,17602378, 17480,172248394, 0, 0
arc_buf_t: 72, 0, 2456993, 677292,186480000, 0, 0
zil_lwb_cache: 192, 0, 6024, 596,33631771, 0, 0
zfs_znode_cache: 368, 0, 234131, 18899, 1545572, 0, 0
Mountpoints: 816, 0, 37, 123, 37, 0, 0
ksiginfo: 112, 0, 229, 2641, 15054, 0, 0
itimer: 352, 0, 1, 10, 1, 0, 0
KNOTE: 128, 0, 2, 2137, 151178, 0, 0
socket: 696, 1571620, 73, 267, 185605, 0, 0
ipq: 56, 95708, 0, 0, 0, 0, 0
udp_inpcb: 392, 1571620, 19, 521, 87693, 0, 0
udpcb: 16, 1571762, 19, 4750, 87693, 0, 0
tcp_inpcb: 392, 1571620, 38, 612, 91542, 0, 0
tcpcb: 1024, 1571620, 38, 270, 91542, 0, 0
tcptw: 88, 27810, 0, 540, 8, 0, 0
syncache: 160, 15375, 0, 500, 88581, 0, 0
hostcache: 136, 15370, 1, 202, 10, 0, 0
tcpreass: 40, 191400, 0, 2100, 3370, 0, 0
sackhole: 32, 0, 0, 1875, 5049, 0, 0
sctp_ep: 1408, 1571620, 0, 0, 0, 0, 0
sctp_asoc: 2416, 40000, 0, 0, 0, 0, 0
sctp_laddr: 48, 80012, 0, 0, 4, 0, 0
sctp_raddr: 728, 80000, 0, 0, 0, 0, 0
sctp_chunk: 136, 400026, 0, 0, 0, 0, 0
sctp_readq: 104, 400026, 0, 0, 0, 0, 0
sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0
sctp_asconf: 40, 400000, 0, 0, 0, 0, 0
sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0
udplite_inpcb: 392, 1571620, 0, 0, 0, 0, 0
ripcb: 392, 1571620, 0, 90, 5, 0, 0
unpcb: 240, 1571632, 15, 865, 6360, 0, 0
rtentry: 200, 0, 14, 146, 14, 0, 0
selfd: 56, 0, 353, 5043,11798700, 0, 0
SWAPMETA: 288, 6123806, 816, 757, 58328, 0, 0
icl_conn: 152, 0, 13, 247, 28, 0, 0
icl_pdu: 80, 0, 15, 2635,243064056, 0, 0
iscsi_outstanding: 40, 0, 0, 0, 0, 0, 0
cfi_lun: 344, 0, 14, 85, 15, 0, 0
cfi_metatask: 320, 0, 0, 0, 0, 0, 0
cfiscsi_data_wait: 56, 0, 0, 6319,102771167, 0, 0
beio: 360, 0, 4, 733,114752315, 0, 0
cblk0: 131072, 0, 0, 11, 58648, 0, 0
cblk1: 131072, 0, 0, 20, 1849, 0, 0
cblk2: 131072, 0, 0, 26, 9398679, 0, 0
cblk3: 131072, 0, 1, 25,13447843, 0, 0
cblk4: 131072, 0, 0, 0, 0, 0, 0
cblk5: 131072, 0, 0, 29,21650798, 0, 0
cblk6: 131072, 0, 0, 52,43569738, 0, 0
cblk7: 131072, 0, 0, 25,10770009, 0, 0
cblk8: 131072, 0, 0, 24, 2138810, 0, 0
cblk9: 131072, 0, 0, 32,11413559, 0, 0
cblk10: 131072, 0, 0, 35, 2685841, 0, 0
cblk11: 131072, 0, 0, 27, 39210, 0, 0
cblk12: 131072, 0, 0, 22, 404811, 0, 0
cblk13: 131072, 0, 0, 22, 2592, 0, 0