Hi,
I posted this before, but I didn?t really get an answer and I?m still looking
for ways to debug this.
I have to servers (HP DL 380 Gen8, 192 GB RAM, 6C CPU).
I?ve outfitted them with HP?s H22x cards (really OEMed 9207-8x, three
altogether) that I recently cross-flashed to LSI?s latest firmware.
They have:
- 14 600GB SAS disks in their internal cages (2x ZFs mirror to boot from and 2x
RAIDZ2)
- another 12 1200 GB SAS disks in an external HP 2700 enclosure (2x RAID-Z2)
- all the RAID Z2s together form a single zpool
These are NFS-servers (master + warm standby) and MySQL slaves.
The client uses the NFS-server and previously used both MySQL-Slaves for SELECT
queries only (via a middleware that seems to be able to sort this out).
When this system was built, we had twelve single RAID0 logical volumes on HP
P4x0 RAID-controllers (with 2 GB cache), merged into two RAID-Z2 vdevs for
storage (and a RAID1 boot-drive).
However, the customer outgrew that solution and because there are no native HP
utilities for managing the RAID-cards on FreeBSD, each time a drive dies on such
a setup, you have to reboot the server to make sure ZFS ?sees? the new drive (go
into adapter BIOS, destroy RAID, create RAID?).
So, I switched to LSI-type JBOD HBAs.
I had to re-install both servers from scratch, obviously, and also upgraded to
10.3 (from 10.1).
I have zfsnap (sysutils/zfsnap) create hourly, daily and weekly snapshots that I
transfer via zxfer from server1 to server2.
Previously, this only took a couple of seconds. But since I switched to LSI
HBAs, it?s taking longer - even though the deltas are really small in most
cases.
After 03:00 AM, when zfsnap deletes some snapshots on the source system, it even
takes several minutes to zxfer the snapshots.
At that point, it?s really noticeable that the 2nd system complete freezes. The
first system (that zfs sends) is absolutely unaffected and continues like
normal.
I?ve setup a cron-job that outputs iostat and vmstat into second-timestamped
files and a few seconds after the zxfer starts, it doesn?t generate any new
files until shortly before it finishes.
With these hangs, the customer can?t use the 2nd MySQL-server and is thus
unhappy.
I?ve upgraded to 11.0-RELEAE on the 2nd system - but it hasn?t helped (it might
even be slower actually).
How can I debug this? What?s the best way forward?
I don?t have any ZFS errors (checksum etc.). So, I doubt it?s a cable.
Here?s various system information
------------------------------------------------------------------------
ZFS Subsystem Report Sat Oct 1 17:50:48 2016
------------------------------------------------------------------------
System Information:
Kernel Version: 1100122 (osreldate)
Hardware Platform: amd64
Processor Architecture: amd64
ZFS Storage pool Version: 5000
ZFS Filesystem Version: 5
FreeBSD 11.0-RELEASE #0 r306211: Thu Sep 22 21:43:30 UTC 2016 root
5:50PM up 2 days, 19:50, 1 users, load averages: 0.11, 0.09, 0.08
------------------------------------------------------------------------
System Memory:
0.26% 506.77 MiB Active, 11.52% 21.54 GiB Inact
62.29% 116.49 GiB Wired, 0.00% 0 Cache
25.93% 48.49 GiB Free, 0.00% 0 Gap
Real Installed: 192.00 GiB
Real Available: 99.96% 191.93 GiB
Real Managed: 97.44% 187.01 GiB
Logical Total: 192.00 GiB
Logical Used: 63.53% 121.97 GiB
Logical Free: 36.47% 70.03 GiB
Kernel Memory: 2.15 GiB
Data: 98.41% 2.12 GiB
Text: 1.59% 35.02 MiB
Kernel Memory Map: 187.01 GiB
Size: 40.16% 75.10 GiB
Free: 59.84% 111.92 GiB
------------------------------------------------------------------------
ARC Summary: (HEALTHY)
Memory Throttle Count: 0
ARC Misc:
Deleted: 2.16m
Recycle Misses: 0
Mutex Misses: 1.92k
Evict Skips: 54.73k
ARC Size: 64.39% 41.21 GiB
Target Size: (Adaptive) 100.00% 64.00 GiB
Min Size (Hard Limit): 36.33% 23.25 GiB
Max Size (High Water): 2:1 64.00 GiB
ARC Size Breakdown:
Recently Used Cache Size: 89.29% 57.14 GiB
Frequently Used Cache Size: 10.71% 6.86 GiB
ARC Hash Breakdown:
Elements Max: 5.37m
Elements Current: 92.43% 4.97m
Collisions: 2.58m
Chain Max: 5
Chains: 333.76k
------------------------------------------------------------------------
ARC Efficiency: 265.65m
Cache Hit Ratio: 61.83% 164.25m
Cache Miss Ratio: 38.17% 101.40m
Actual Hit Ratio: 55.18% 146.57m
Data Demand Efficiency: 10.17% 7.66m
Data Prefetch Efficiency: 0.00% 32
CACHE HITS BY CACHE LIST:
Anonymously Used: 8.77% 14.41m
Most Recently Used: 29.22% 48.00m
Most Frequently Used: 60.02% 98.58m
Most Recently Used Ghost: 1.67% 2.74m
Most Frequently Used Ghost: 0.33% 536.90k
CACHE HITS BY DATA TYPE:
Demand Data: 0.47% 778.95k
Prefetch Data: 0.00% 0
Demand Metadata: 88.30% 145.04m
Prefetch Metadata: 11.22% 18.43m
CACHE MISSES BY DATA TYPE:
Demand Data: 6.79% 6.88m
Prefetch Data: 0.00% 32
Demand Metadata: 84.42% 85.60m
Prefetch Metadata: 8.79% 8.91m
------------------------------------------------------------------------
L2ARC is disabled
------------------------------------------------------------------------
File-Level Prefetch: (HEALTHY)
DMU Efficiency: 970.94m
Hit Ratio: 0.22% 2.09m
Miss Ratio: 99.78% 968.85m
Colinear: 0
Hit Ratio: 100.00% 0
Miss Ratio: 100.00% 0
Stride: 0
Hit Ratio: 100.00% 0
Miss Ratio: 100.00% 0
DMU Misc:
Reclaim: 0
Successes: 100.00% 0
Failures: 100.00% 0
Streams: 0
+Resets: 100.00% 0
-Resets: 100.00% 0
Bogus: 0
------------------------------------------------------------------------
VDEV cache is disabled
------------------------------------------------------------------------
ZFS Tunables (sysctl):
kern.maxusers 12619
vm.kmem_size 200803987456
vm.kmem_size_scale 1
vm.kmem_size_min 0
vm.kmem_size_max 1319413950874
vfs.zfs.trim.max_interval 1
vfs.zfs.trim.timeout 30
vfs.zfs.trim.txg_delay 32
vfs.zfs.trim.enabled 1
vfs.zfs.vol.unmap_enabled 1
vfs.zfs.vol.recursive 0
vfs.zfs.vol.mode 1
vfs.zfs.version.zpl 5
vfs.zfs.version.spa 5000
vfs.zfs.version.acl 1
vfs.zfs.version.ioctl 6
vfs.zfs.debug 1
vfs.zfs.super_owner 0
vfs.zfs.sync_pass_rewrite 2
vfs.zfs.sync_pass_dont_compress 5
vfs.zfs.sync_pass_deferred_free 2
vfs.zfs.zio.exclude_metadata 0
vfs.zfs.zio.use_uma 1
vfs.zfs.cache_flush_disable 0
vfs.zfs.zil_replay_disable 0
vfs.zfs.min_auto_ashift 12
vfs.zfs.max_auto_ashift 13
vfs.zfs.vdev.trim_max_pending 10000
vfs.zfs.vdev.bio_delete_disable 0
vfs.zfs.vdev.bio_flush_disable 0
vfs.zfs.vdev.write_gap_limit 4096
vfs.zfs.vdev.read_gap_limit 32768
vfs.zfs.vdev.aggregation_limit 131072
vfs.zfs.vdev.trim_max_active 64
vfs.zfs.vdev.trim_min_active 1
vfs.zfs.vdev.scrub_max_active 2
vfs.zfs.vdev.scrub_min_active 1
vfs.zfs.vdev.async_write_max_active 10
vfs.zfs.vdev.async_write_min_active 1
vfs.zfs.vdev.async_read_max_active 3
vfs.zfs.vdev.async_read_min_active 1
vfs.zfs.vdev.sync_write_max_active 10
vfs.zfs.vdev.sync_write_min_active 10
vfs.zfs.vdev.sync_read_max_active 10
vfs.zfs.vdev.sync_read_min_active 10
vfs.zfs.vdev.max_active 1000
vfs.zfs.vdev.async_write_active_max_dirty_percent60
vfs.zfs.vdev.async_write_active_min_dirty_percent30
vfs.zfs.vdev.mirror.non_rotating_seek_inc1
vfs.zfs.vdev.mirror.non_rotating_inc 0
vfs.zfs.vdev.mirror.rotating_seek_offset1048576
vfs.zfs.vdev.mirror.rotating_seek_inc 5
vfs.zfs.vdev.mirror.rotating_inc 0
vfs.zfs.vdev.trim_on_init 1
vfs.zfs.vdev.cache.bshift 16
vfs.zfs.vdev.cache.size 0
vfs.zfs.vdev.cache.max 16384
vfs.zfs.vdev.metaslabs_per_vdev 200
vfs.zfs.txg.timeout 5
vfs.zfs.space_map_blksz 4096
vfs.zfs.spa_slop_shift 5
vfs.zfs.spa_asize_inflation 24
vfs.zfs.deadman_enabled 1
vfs.zfs.deadman_checktime_ms 5000
vfs.zfs.deadman_synctime_ms 1000000
vfs.zfs.debug_flags 0
vfs.zfs.recover 0
vfs.zfs.spa_load_verify_data 1
vfs.zfs.spa_load_verify_metadata 1
vfs.zfs.spa_load_verify_maxinflight 10000
vfs.zfs.ccw_retry_interval 300
vfs.zfs.check_hostid 1
vfs.zfs.mg_fragmentation_threshold 85
vfs.zfs.mg_noalloc_threshold 0
vfs.zfs.condense_pct 200
vfs.zfs.metaslab.bias_enabled 1
vfs.zfs.metaslab.lba_weighting_enabled 1
vfs.zfs.metaslab.fragmentation_factor_enabled1
vfs.zfs.metaslab.preload_enabled 1
vfs.zfs.metaslab.preload_limit 3
vfs.zfs.metaslab.unload_delay 8
vfs.zfs.metaslab.load_pct 50
vfs.zfs.metaslab.min_alloc_size 33554432
vfs.zfs.metaslab.df_free_pct 4
vfs.zfs.metaslab.df_alloc_threshold 131072
vfs.zfs.metaslab.debug_unload 0
vfs.zfs.metaslab.debug_load 0
vfs.zfs.metaslab.fragmentation_threshold70
vfs.zfs.metaslab.gang_bang 16777217
vfs.zfs.free_bpobj_enabled 1
vfs.zfs.free_max_blocks -1
vfs.zfs.no_scrub_prefetch 0
vfs.zfs.no_scrub_io 0
vfs.zfs.resilver_min_time_ms 3000
vfs.zfs.free_min_time_ms 1000
vfs.zfs.scan_min_time_ms 1000
vfs.zfs.scan_idle 50
vfs.zfs.scrub_delay 4
vfs.zfs.resilver_delay 2
vfs.zfs.top_maxinflight 32
vfs.zfs.zfetch.array_rd_sz 1048576
vfs.zfs.zfetch.max_distance 8388608
vfs.zfs.zfetch.min_sec_reap 2
vfs.zfs.zfetch.max_streams 8
vfs.zfs.prefetch_disable 0
vfs.zfs.delay_scale 500000
vfs.zfs.delay_min_dirty_percent 60
vfs.zfs.dirty_data_sync 67108864
vfs.zfs.dirty_data_max_percent 10
vfs.zfs.dirty_data_max_max 4294967296
vfs.zfs.dirty_data_max 4294967296
vfs.zfs.max_recordsize 1048576
vfs.zfs.mdcomp_disable 0
vfs.zfs.nopwrite_enabled 1
vfs.zfs.dedup.prefetch 1
vfs.zfs.l2c_only_size 0
vfs.zfs.mfu_ghost_data_lsize 57031680
vfs.zfs.mfu_ghost_metadata_lsize 24999013888
vfs.zfs.mfu_ghost_size 25056045568
vfs.zfs.mfu_data_lsize 7619584
vfs.zfs.mfu_metadata_lsize 2427032064
vfs.zfs.mfu_size 7355872768
vfs.zfs.mru_ghost_data_lsize 0
vfs.zfs.mru_ghost_metadata_lsize 27116232704
vfs.zfs.mru_ghost_size 27116232704
vfs.zfs.mru_data_lsize 12935902208
vfs.zfs.mru_metadata_lsize 18539549696
vfs.zfs.mru_size 31494193152
vfs.zfs.anon_data_lsize 0
vfs.zfs.anon_metadata_lsize 0
vfs.zfs.anon_size 233472
vfs.zfs.l2arc_norw 1
vfs.zfs.l2arc_feed_again 1
vfs.zfs.l2arc_noprefetch 1
vfs.zfs.l2arc_feed_min_ms 200
vfs.zfs.l2arc_feed_secs 1
vfs.zfs.l2arc_headroom 2
vfs.zfs.l2arc_write_boost 8388608
vfs.zfs.l2arc_write_max 8388608
vfs.zfs.arc_meta_limit 17179869184
vfs.zfs.arc_free_target 339900
vfs.zfs.arc_shrink_shift 7
vfs.zfs.arc_average_blocksize 8192
vfs.zfs.arc_min 24966280704
vfs.zfs.arc_max 68719476736
????????????????????????????????????
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-RELEASE #0 r306211: Thu Sep 22 21:43:30 UTC 2016
root at releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM
3.8.0)
VT(vga): resolution 640x480
CPU: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz (2294.52-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d
Stepping=7
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x1fbee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x1<LAHF>
XSAVE Features=0x1<XSAVEOPT>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
real memory = 206158430208 (196608 MB)
avail memory = 200330240000 (191049 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <HP ProLiant>
FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs
FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads
random: unblocking device.
ACPI BIOS Warning (bug): Invalid length for FADT/Pm1aControlBlock: 32, using
default 16 (20160527/tbfadt-733)
ACPI BIOS Warning (bug): Invalid length for FADT/Pm2ControlBlock: 32, using
default 8 (20160527/tbfadt-733)
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic0 <Version 2.0> irqs 0-23 on motherboard
random: entropy device external interface
kbd1 at kbdmux0
netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff8101c950, 0) error 19
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
acpi0: <HP ProLiant> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> numa-domain 0 on acpi0
cpu1: <ACPI CPU> numa-domain 0 on acpi0
cpu2: <ACPI CPU> numa-domain 0 on acpi0
cpu3: <ACPI CPU> numa-domain 0 on acpi0
cpu4: <ACPI CPU> numa-domain 0 on acpi0
cpu5: <ACPI CPU> numa-domain 0 on acpi0
cpu6: <ACPI CPU> numa-domain 0 on acpi0
cpu7: <ACPI CPU> numa-domain 0 on acpi0
cpu8: <ACPI CPU> numa-domain 0 on acpi0
cpu9: <ACPI CPU> numa-domain 0 on acpi0
cpu10: <ACPI CPU> numa-domain 0 on acpi0
cpu11: <ACPI CPU> numa-domain 0 on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 350
Event timer "HPET1" frequency 14318180 Hz quality 340
Event timer "HPET2" frequency 14318180 Hz quality 340
Event timer "HPET3" frequency 14318180 Hz quality 340
Event timer "HPET4" frequency 14318180 Hz quality 340
Event timer "HPET5" frequency 14318180 Hz quality 340
Event timer "HPET6" frequency 14318180 Hz quality 340
Event timer "HPET7" frequency 14318180 Hz quality 340
atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x908-0x90b on acpi0
pcib0: <ACPI Host-PCI bridge> numa-domain 0 on acpi0
pcib0: _OSC returned error 0x10
pci0: <ACPI PCI bus> numa-domain 0 on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 numa-domain 0 on pci0
pci1: <ACPI PCI bus> numa-domain 0 on pcib1
mps0: <Avago Technologies (LSI) SAS2308> port 0x6000-0x60ff mem
0xfbef0000-0xfbefffff,0xfbe80000-0xfbebffff irq 26 at device 0.0 numa-domain 0
on pci1
mps0: Firmware: 20.00.07.00, Driver: 21.01.00.00-fbsd
mps0: IOCCapabilities:
5285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
pcib2: <ACPI PCI-PCI bridge> at device 1.1 numa-domain 0 on pci0
pci2: <ACPI PCI bus> numa-domain 0 on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 2.0 numa-domain 0 on pci0
pci3: <ACPI PCI bus> numa-domain 0 on pcib3
bge0: <HP Ethernet 1Gb 4-port 331FLR Adapter, ASIC rev. 0x5719001> mem
0xfabf0000-0xfabfffff,0xfabe0000-0xfabeffff,0xfabd0000-0xfabdffff irq 32 at
device 0.0 numa-domain 0 on pci3
bge0: APE FW version: NCSI v1.3.12.0
bge0: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus0: <MII bus> numa-domain 0 on bge0
brgphy0: <BCM5719C 1000BASE-T media interface> PHY 1 on miibus0
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Using defaults for TSO: 65518/35/2048
bge0: Ethernet address: 2c:44:fd:91:c2:bc
bge1: <HP Ethernet 1Gb 4-port 331FLR Adapter, ASIC rev. 0x5719001> mem
0xfabc0000-0xfabcffff,0xfabb0000-0xfabbffff,0xfaba0000-0xfabaffff irq 36 at
device 0.1 numa-domain 0 on pci3
bge1: APE FW version: NCSI v1.3.12.0
bge1: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus1: <MII bus> numa-domain 0 on bge1
brgphy1: <BCM5719C 1000BASE-T media interface> PHY 2 on miibus1
brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Using defaults for TSO: 65518/35/2048
bge1: Ethernet address: 2c:44:fd:91:c2:bd
bge2: <HP Ethernet 1Gb 4-port 331FLR Adapter, ASIC rev. 0x5719001> mem
0xfab90000-0xfab9ffff,0xfab80000-0xfab8ffff,0xfab70000-0xfab7ffff irq 32 at
device 0.2 numa-domain 0 on pci3
bge2: APE FW version: NCSI v1.3.12.0
bge2: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus2: <MII bus> numa-domain 0 on bge2
brgphy2: <BCM5719C 1000BASE-T media interface> PHY 3 on miibus2
brgphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge2: Using defaults for TSO: 65518/35/2048
bge2: Ethernet address: 2c:44:fd:91:c2:be
bge3: <HP Ethernet 1Gb 4-port 331FLR Adapter, ASIC rev. 0x5719001> mem
0xfab60000-0xfab6ffff,0xfab50000-0xfab5ffff,0xfab40000-0xfab4ffff irq 36 at
device 0.3 numa-domain 0 on pci3
bge3: APE FW version: NCSI v1.3.12.0
bge3: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus3: <MII bus> numa-domain 0 on bge3
brgphy3: <BCM5719C 1000BASE-T media interface> PHY 4 on miibus3
brgphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge3: Using defaults for TSO: 65518/35/2048
bge3: Ethernet address: 2c:44:fd:91:c2:bf
pcib4: <ACPI PCI-PCI bridge> at device 2.1 numa-domain 0 on pci0
pci4: <ACPI PCI bus> numa-domain 0 on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 2.2 numa-domain 0 on pci0
pci5: <ACPI PCI bus> numa-domain 0 on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 2.3 numa-domain 0 on pci0
pci6: <ACPI PCI bus> numa-domain 0 on pcib6
pcib7: <ACPI PCI-PCI bridge> at device 3.0 numa-domain 0 on pci0
pci7: <ACPI PCI bus> numa-domain 0 on pcib7
mps1: <Avago Technologies (LSI) SAS2308> port 0x5000-0x50ff mem
0xfbdf0000-0xfbdfffff,0xfbd80000-0xfbdbffff irq 40 at device 0.0 numa-domain 0
on pci7
mps1: Firmware: 20.00.07.00, Driver: 21.01.00.00-fbsd
mps1: IOCCapabilities:
5285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
pcib8: <ACPI PCI-PCI bridge> at device 3.1 numa-domain 0 on pci0
pci8: <ACPI PCI bus> numa-domain 0 on pcib8
pcib9: <ACPI PCI-PCI bridge> at device 3.2 numa-domain 0 on pci0
pci9: <ACPI PCI bus> numa-domain 0 on pcib9
pcib10: <ACPI PCI-PCI bridge> at device 3.3 numa-domain 0 on pci0
pci10: <ACPI PCI bus> numa-domain 0 on pcib10
pcib11: <ACPI PCI-PCI bridge> at device 17.0 numa-domain 0 on pci0
pci11: <ACPI PCI bus> numa-domain 0 on pcib11
ehci0: <Intel Patsburg USB 2.0 controller> mem 0xfac60000-0xfac603ff irq
21 at device 26.0 numa-domain 0 on pci0
usbus0: EHCI version 1.0
usbus0 numa-domain 0 on ehci0
pcib12: <ACPI PCI-PCI bridge> at device 28.0 numa-domain 0 on pci0
pci12: <ACPI PCI bus> numa-domain 0 on pcib12
mps2: <Avago Technologies (LSI) SAS2308> port 0x7000-0x70ff mem
0xfbff0000-0xfbffffff,0xfbf80000-0xfbfbffff irq 16 at device 0.0 numa-domain 0
on pci12
mps2: Firmware: 20.00.07.00, Driver: 21.01.00.00-fbsd
mps2: IOCCapabilities:
5285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
pcib13: <ACPI PCI-PCI bridge> at device 28.7 numa-domain 0 on pci0
pci13: <ACPI PCI bus> numa-domain 0 on pcib13
vgapci0: <VGA-compatible display> mem
0xf9000000-0xf9ffffff,0xfbce0000-0xfbce3fff,0xfb000000-0xfb7fffff irq 16 at
device 0.1 numa-domain 0 on pci13
vgapci0: Boot video device
uhci0: <HP iLO Standard Virtual USB controller> port 0x3c00-0x3c1f irq 16
at device 0.4 numa-domain 0 on pci13
usbus1 numa-domain 0 on uhci0
ehci1: <Intel Patsburg USB 2.0 controller> mem 0xfac50000-0xfac503ff irq
20 at device 29.0 numa-domain 0 on pci0
usbus2: EHCI version 1.0
usbus2 numa-domain 0 on ehci1
pcib14: <PCI-PCI bridge> at device 30.0 numa-domain 0 on pci0
pci14: <PCI bus> numa-domain 0 on pcib14
isab0: <PCI-ISA bridge> at device 31.0 numa-domain 0 on pci0
isa0: <ISA bus> numa-domain 0 on isab0
atapci0: <Intel Patsburg SATA600 controller> port
0x4000-0x4007,0x4008-0x400b,0x4010-0x4017,0x4018-0x401b,0x4020-0x402f,0x4030-0x403f
irq 17 at device 31.2 numa-domain 0 on pci0
ata2: <ATA channel> at channel 0 on atapci0
ata3: <ATA channel> at channel 1 on atapci0
acpi_tz0: <Thermal Zone> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
uart1: <Non-standard ns8250 class UART with FIFOs> port 0x2f8-0x2ff irq 3
on acpi0
orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0
ppc0: cannot reserve I/O port range
uart0: <Non-standard ns8250 class UART with FIFOs> at port 0x3f8 irq 4
flags 0x10 on isa0
uart0: console (115200,n,8,1)
est0: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est0 attach returned 6
est1: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est1 attach returned 6
est2: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu2
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est2 attach returned 6
est3: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu3
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est3 attach returned 6
est4: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu4
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est4 attach returned 6
est5: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu5
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est5 attach returned 6
est6: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est6 attach returned 6
est7: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu7
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est7 attach returned 6
est8: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu8
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est8 attach returned 6
est9: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu9
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est9 attach returned 6
est10: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu10
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est10 attach returned 6
est11: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu11
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 221b00001a00
device_attach: est11 attach returned 6
usbus0: 480Mbps High Speed USB v2.0
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
nvme cam probe device init
usbus1: 12Mbps Full Speed USB v1.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <HP> at usbus1
uhub1: <HP UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
usbus2: 480Mbps High Speed USB v2.0
ugen2.1: <Intel> at usbus2
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
uhub1: 2 ports with 2 removable, self powered
uhub0: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
ugen1.2: <BMC> at usbus1
ukbd0: <Virtual Keyboard > on usbus1
kbd2 at ukbd0
ugen0.2: <vendor 0x8087> at usbus0
uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on
usbus0
ugen2.2: <vendor 0x8087> at usbus2
uhub4: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on
usbus2
uhub3: 6 ports with 6 removable, self powered
uhub4: 8 ports with 8 removable, self powered
ugen2.3: <vendor 0x0424> at usbus2
uhub5: <vendor 0x0424 product 0x2660, class 9/0, rev 2.00/8.01, addr 3> on
usbus2
uhub5: 2 ports with 1 removable, self powered
ses0 at mps2 bus 0 scbus2 target 10 lun 0
ses0: <HP D2700 SAS AJ941A 0147> Fixed Enclosure Services SPC-3 SCSI
device
ses0: Serial Number 7CE539P12M
ses0: 600.000MB/s transfers
ses0: Command Queueing enabled
ses0: SCSI-3 ENC Device
(da0:mps0:0:8:0): UNMAPPED
(da1:da0 at mps0 bus 0 scbus0 target 8 lun 0
da0: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number S0M1F2B10000B418HVFX
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 572325MB (1172123568 512 byte sectors)
mps0:0:9:0): UNMAPPED
(da2:da1 at mps0 bus 0 scbus0 target 9 lun 0
da1: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number S0M1F7CG0000M419BS87
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 572325MB (1172123568 512 byte sectors)
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 1, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec197c9
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 2, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ebde785
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 3, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec1ae59
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 4, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008eb2e04d
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 5, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ebdfe25
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 6, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ebfea71
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 7, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec3f06d
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 8, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ebf2ab5
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 9, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec3c2f9
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 10, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ebedb35
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 11, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec012b9
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 12, Not All Phys
ses0: phy 0: SAS device type 1 id 0
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec58d09
mps0:0:10:0): UNMAPPED
(da3:da2 at mps0 bus 0 scbus0 target 10 lun 0
da2: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number S0M1EQSL0000B419CE37
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 572325MB (1172123568 512 byte sectors)
mps0:0:11:0): UNMAPPED
(da4:da3 at mps0 bus 0 scbus0 target 11 lun 0
da3: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number S0M1EQHL0000B419CG8Y
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 572325MB (1172123568 512 byte sectors)
mps0:0:12:0): UNMAPPED
(da5:da4 at mps0 bus 0 scbus0 target 12 lun 0
da4: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da4: Serial Number S0M1ERQR0000M41919NZ
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 572325MB (1172123568 512 byte sectors)
mps0:0:13:0): UNMAPPED
(da6:da5 at mps0 bus 0 scbus0 target 13 lun 0
da5: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da5: Serial Number S0M1EP8H0000M419667Y
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 572325MB (1172123568 512 byte sectors)
mps0:0:14:0): UNMAPPED
(da7:da6 at mps0 bus 0 scbus0 target 14 lun 0
da6: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da6: Serial Number S0M1F9B00000B419CQF1
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 572325MB (1172123568 512 byte sectors)
mps0:0:15:0): UNMAPPED
(da8:da7 at mps0 bus 0 scbus0 target 15 lun 0
da7: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da7: Serial Number S0M1ER7Y0000B418J2VA
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 572325MB (1172123568 512 byte sectors)
mps1:0:8:0): UNMAPPED
(da9:da8 at mps1 bus 0 scbus1 target 8 lun 0
da8: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da8: Serial Number S0M1E2VZ0000B419CQAF
da8: 600.000MB/s transfers
da8: Command Queueing enabled
da8: 572325MB (1172123568 512 byte sectors)
mps1:0:9:0): UNMAPPED
(da10:da9 at mps1 bus 0 scbus1 target 9 lun 0
da9: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da9: Serial Number S0M1ESN20000M418DKFL
da9: 600.000MB/s transfers
da9: Command Queueing enabled
da9: 572325MB (1172123568 512 byte sectors)
mps1:0:10:0): UNMAPPED
(da11:da10 at mps1 bus 0 scbus1 target 10 lun 0
da10: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da10: Serial Number S0M1F8V00000B419CPWR
da10: 600.000MB/s transfers
da10: Command Queueing enabled
da10: 572325MB (1172123568 512 byte sectors)
mps1:0:11:0): UNMAPPED
(da12:da11 at mps1 bus 0 scbus1 target 11 lun 0
da11: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da11: Serial Number S0M1ESLL0000B419DALG
da11: 600.000MB/s transfers
da11: Command Queueing enabled
da11: 572325MB (1172123568 512 byte sectors)
mps1:0:12:0): UNMAPPED
(da13:da12 at mps1 bus 0 scbus1 target 12 lun 0
da12: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da12: Serial Number S0M1DXJR0000B419CJKG
da12: 600.000MB/s transfers
da12: Command Queueing enabled
da12: 572325MB (1172123568 512 byte sectors)
mps1:0:13:0): UNMAPPED
(da14:da13 at mps1 bus 0 scbus1 target 13 lun 0
da13: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da13: Serial Number S0M1ES7R0000B419CJZX
da13: 600.000MB/s transfers
da13: Command Queueing enabled
da13: 572325MB (1172123568 512 byte sectors)
mps1:0:14:0): UNMAPPED
(da15:da14 at mps1 bus 0 scbus1 target 14 lun 0
da14: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da14: Serial Number S0M19J9D0000B419CEEC
da14: 600.000MB/s transfers
da14: Command Queueing enabled
da14: 572325MB (1172123568 512 byte sectors)
mps1:0:15:0): UNMAPPED
(da16:da15 at mps1 bus 0 scbus1 target 15 lun 0
da15: <HP EG0600FCVBK HPD5> Fixed Direct Access SPC-4 SCSI device
da15: Serial Number S0M1EQPR0000B419CGAC
da15: 600.000MB/s transfers
da15: Command Queueing enabled
da15: 572325MB (1172123568 512 byte sectors)
mps2:0:11:0): UNMAPPED
(da17:da16 at mps2 bus 0 scbus2 target 11 lun 0
da16: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da16: Serial Number S3L29R3L0000M608CP3Y
da16: 600.000MB/s transfers
da16: Command Queueing enabled
da16: 1144641MB (2344225968 512 byte sectors)
mps2:0:13:0): UNMAPPED
(da18:da17 at mps2 bus 0 scbus2 target 13 lun 0
da17: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da17: Serial Number S3L29XFQ0000M608CQD6
da17: 600.000MB/s transfers
da17: Command Queueing enabled
da17: 1144641MB (2344225968 512 byte sectors)
mps2:0:14:0): UNMAPPED
(da19:da18 at mps2 bus 0 scbus2 target 14 lun 0
da18: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da18: Serial Number S3L29QTK0000M608CN9K
da18: 600.000MB/s transfers
da18: Command Queueing enabled
da18: 1144641MB (2344225968 512 byte sectors)
mps2:0:15:0): UNMAPPED
(da20:da19 at mps2 bus 0 scbus2 target 15 lun 0
da19: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da19: Serial Number S3L29TAA0000M608CS65
da19: 600.000MB/s transfers
da19: Command Queueing enabled
da19: 1144641MB (2344225968 512 byte sectors)
mps2:0:16:0): UNMAPPED
(da21:da20 at mps2 bus 0 scbus2 target 16 lun 0
da20: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da20: Serial Number S3L28ZFA0000M607T1JN
da20: 600.000MB/s transfers
da20: Command Queueing enabled
da20: 1144641MB (2344225968 512 byte sectors)
mps2:0:17:0): UNMAPPED
(da22:da21 at mps2 bus 0 scbus2 target 17 lun 0
da21: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da21: Serial Number S3L29PG90000M6089MMG
da21: 600.000MB/s transfers
da21: Command Queueing enabled
da21: 1144641MB (2344225968 512 byte sectors)
mps2:0:18:0): UNMAPPED
(da23:da22 at mps2 bus 0 scbus2 target 18 lun 0
da22: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da22: Serial Number S3L29RHR0000M608STP7
da22: 600.000MB/s transfers
da22: Command Queueing enabled
da22: 1144641MB (2344225968 512 byte sectors)
mps2:0:19:0): UNMAPPED
(da24:da23 at mps2 bus 0 scbus2 target 19 lun 0
da23: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da23: Serial Number S3L29VQT0000M6069ZU8
da23: 600.000MB/s transfers
da23: Command Queueing enabled
da23: 1144641MB (2344225968 512 byte sectors)
mps2:0:20:0): UNMAPPED
(da25:da24 at mps2 bus 0 scbus2 target 20 lun 0
da24: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da24: Serial Number S3L2A7WM0000M608SSU7
da24: 600.000MB/s transfers
da24: Command Queueing enabled
da24: 1144641MB (2344225968 512 byte sectors)
mps2:0:21:0): UNMAPPED
(da26:da25 at mps2 bus 0 scbus2 target 21 lun 0
da25: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da25: Serial Number S3L29GXN0000M605A4C6
da25: 600.000MB/s transfers
da25: Command Queueing enabled
da25: 1144641MB (2344225968 512 byte sectors)
mps2:0:22:0): UNMAPPED
(da27:da26 at mps2 bus 0 scbus2 target 22 lun 0
da26: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da26: Serial Number S3L29TPT0000M608CSAA
da26: 600.000MB/s transfers
da26: Command Queueing enabled
da26: 1144641MB (2344225968 512 byte sectors)
mps2:0:23:0): UNMAPPED
da27 at mps2 bus 0 scbus2 target 23 lun 0
da27: <HP EG1200FDJYT HPD4> Fixed Direct Access SPC-4 SCSI device
da27: Serial Number S3L2A4EJ0000M607J095
da27: 600.000MB/s transfers
da27: Command Queueing enabled
da27: 1144641MB (2344225968 512 byte sectors)
SMP: AP CPU #1 Launched!
SMP: AP CPU #10 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #8 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #11 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #9 Launched!
SMP: AP CPU #5 Launched!
Timecounter "TSC-low" frequency 1147258217 Hz quality 1000
Trying to mount root from zfs:zroot/ROOT/default []...
GEOM_MIRROR: Device mirror/swap launched (2/2).
GEOM_ELI: Device mirror/swap.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI: Crypto: software
bge0: link state changed to DOWN
bge1: link state changed to DOWN
bge2: link state changed to DOWN
ums0: <Virtual Mouse> on usbus1
bge0: link state changed to UP
bge1: link state changed to UP
bge2: link state changed to UP
ugen1.2: <BMC> at usbus1 (disconnected)
ukbd0: at uhub1, port 1, addr 2 (disconnected)
ums0: at uhub1, port 1, addr 2 (disconnected)
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1ESLL_C1S03.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1ESLL_C1S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ESLL_C1S03.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F8V0_C1S04.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1F8V0_C1S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F8V0_C1S04.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1EQPR_C1S05.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1EQPR_C1S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQPR_C1S05.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M19J9D_C1S06.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M19J9D_C1S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M19J9D_C1S06.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1ES7R_C1S07.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1ES7R_C1S07.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ES7R_C1S07.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1DXJR_C1S08.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1DXJR_C1S08.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1DXJR_C1S08.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1EQHL_C2S01.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1EQHL_C2S01.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQHL_C2S01.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1EQSL_C2S02.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1EQSL_C2S02.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQSL_C2S02.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F7CG_C2S03.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1F7CG_C2S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F7CG_C2S03.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F2B1_C2S04.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1F2B1_C2S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F2B1_C2S04.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1ER7Y_C2S05.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1ER7Y_C2S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ER7Y_C2S05.
vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F9B0_C2S06.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1F9B0_C2S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F9B0_C2S06.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29R3L_EC1_S01.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29R3L_EC1_S01.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29R3L_EC1_S01.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29XFQ_EC1_S02.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29XFQ_EC1_S02.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29XFQ_EC1_S02.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29QTK_EC1_S03.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29QTK_EC1_S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29QTK_EC1_S03.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L28ZFA_EC1_S04.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L28ZFA_EC1_S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L28ZFA_EC1_S04.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29PG9_EC1_S05.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29PG9_EC1_S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29PG9_EC1_S05.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29TAA_EC1_S06.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29TAA_EC1_S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TAA_EC1_S06.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29RHR_EC1_S07.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29RHR_EC1_S07.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29RHR_EC1_S07.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29VQT_EC1_S08.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29VQT_EC1_S08.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29VQT_EC1_S08.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L2A7WM_EC1_S09.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L2A7WM_EC1_S09.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A7WM_EC1_S09.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29GXN_EC1_S10.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29GXN_EC1_S10.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29GXN_EC1_S10.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29TPT_EC1_S11.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29TPT_EC1_S11.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TPT_EC1_S11.
vdev_geom_close_locked:336[1]: Closing access to gpt/S3L2A4EJ_EC1_S12.
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L2A4EJ_EC1_S12.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A4EJ_EC1_S12.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1ESLL_C1S03.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ESLL_C1S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ESLL_C1S03.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1ESLL_C1S03...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1ESLL_C1S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ESLL_C1S03.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1ESLL_C1S03.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ESLL_C1S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ESLL_C1S03.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1F8V0_C1S04.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F8V0_C1S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F8V0_C1S04.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F8V0_C1S04...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1F8V0_C1S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F8V0_C1S04.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F8V0_C1S04.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F8V0_C1S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F8V0_C1S04.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1EQPR_C1S05.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQPR_C1S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQPR_C1S05.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1EQPR_C1S05...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1EQPR_C1S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQPR_C1S05.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1EQPR_C1S05.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQPR_C1S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQPR_C1S05.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M19J9D_C1S06.
vdev_geom_attach:192[1]: Attaching to gpt/S0M19J9D_C1S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M19J9D_C1S06.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M19J9D_C1S06...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M19J9D_C1S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M19J9D_C1S06.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M19J9D_C1S06.
vdev_geom_attach:192[1]: Attaching to gpt/S0M19J9D_C1S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M19J9D_C1S06.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1ES7R_C1S07.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ES7R_C1S07.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ES7R_C1S07.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1ES7R_C1S07...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1ES7R_C1S07.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ES7R_C1S07.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1ES7R_C1S07.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ES7R_C1S07.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ES7R_C1S07.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1DXJR_C1S08.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1DXJR_C1S08.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1DXJR_C1S08.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1DXJR_C1S08...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1DXJR_C1S08.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1DXJR_C1S08.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1DXJR_C1S08.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1DXJR_C1S08.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1DXJR_C1S08.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1EQHL_C2S01.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQHL_C2S01.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQHL_C2S01.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1EQHL_C2S01...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1EQHL_C2S01.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQHL_C2S01.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1EQHL_C2S01.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQHL_C2S01.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQHL_C2S01.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1EQSL_C2S02.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQSL_C2S02.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQSL_C2S02.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1EQSL_C2S02...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1EQSL_C2S02.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQSL_C2S02.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1EQSL_C2S02.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQSL_C2S02.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQSL_C2S02.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1F7CG_C2S03.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F7CG_C2S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F7CG_C2S03.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F7CG_C2S03...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1F7CG_C2S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F7CG_C2S03.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F7CG_C2S03.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F7CG_C2S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F7CG_C2S03.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1F2B1_C2S04.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F2B1_C2S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F2B1_C2S04.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F2B1_C2S04...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1F2B1_C2S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F2B1_C2S04.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F2B1_C2S04.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F2B1_C2S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F2B1_C2S04.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1ER7Y_C2S05.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ER7Y_C2S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ER7Y_C2S05.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1ER7Y_C2S05...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1ER7Y_C2S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ER7Y_C2S05.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1ER7Y_C2S05.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1ER7Y_C2S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ER7Y_C2S05.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S0M1F9B0_C2S06.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F9B0_C2S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F9B0_C2S06.
vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F9B0_C2S06...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S0M1F9B0_C2S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F9B0_C2S06.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F9B0_C2S06.
vdev_geom_attach:192[1]: Attaching to gpt/S0M1F9B0_C2S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F9B0_C2S06.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L29R3L_EC1_S01.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29R3L_EC1_S01.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29R3L_EC1_S01.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L29R3L_EC1_S01...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29R3L_EC1_S01.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29R3L_EC1_S01.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L29R3L_EC1_S01.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29R3L_EC1_S01.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29R3L_EC1_S01.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L29XFQ_EC1_S02.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29XFQ_EC1_S02.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29XFQ_EC1_S02.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L29XFQ_EC1_S02...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29XFQ_EC1_S02.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29XFQ_EC1_S02.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L29XFQ_EC1_S02.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29XFQ_EC1_S02.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29XFQ_EC1_S02.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L29QTK_EC1_S03.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29QTK_EC1_S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29QTK_EC1_S03.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L29QTK_EC1_S03...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29QTK_EC1_S03.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29QTK_EC1_S03.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L29QTK_EC1_S03.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29QTK_EC1_S03.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29QTK_EC1_S03.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L28ZFA_EC1_S04.
vdev_geom_attach:192[1]: Attaching to gpt/S3L28ZFA_EC1_S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L28ZFA_EC1_S04.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L28ZFA_EC1_S04...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L28ZFA_EC1_S04.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L28ZFA_EC1_S04.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L28ZFA_EC1_S04.
vdev_geom_attach:192[1]: Attaching to gpt/S3L28ZFA_EC1_S04.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L28ZFA_EC1_S04.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L29PG9_EC1_S05.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29PG9_EC1_S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29PG9_EC1_S05.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L29PG9_EC1_S05...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29PG9_EC1_S05.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29PG9_EC1_S05.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L29PG9_EC1_S05.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29PG9_EC1_S05.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29PG9_EC1_S05.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L29TAA_EC1_S06.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29TAA_EC1_S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TAA_EC1_S06.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L29TAA_EC1_S06...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29TAA_EC1_S06.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TAA_EC1_S06.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L29TAA_EC1_S06.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29TAA_EC1_S06.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TAA_EC1_S06.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L29RHR_EC1_S07.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29RHR_EC1_S07.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29RHR_EC1_S07.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L29RHR_EC1_S07...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29RHR_EC1_S07.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29RHR_EC1_S07.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L29RHR_EC1_S07.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29RHR_EC1_S07.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29RHR_EC1_S07.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L29VQT_EC1_S08.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29VQT_EC1_S08.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29VQT_EC1_S08.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L29VQT_EC1_S08...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29VQT_EC1_S08.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29VQT_EC1_S08.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L29VQT_EC1_S08.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29VQT_EC1_S08.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29VQT_EC1_S08.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L2A7WM_EC1_S09.
vdev_geom_attach:192[1]: Attaching to gpt/S3L2A7WM_EC1_S09.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A7WM_EC1_S09.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L2A7WM_EC1_S09...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L2A7WM_EC1_S09.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A7WM_EC1_S09.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L2A7WM_EC1_S09.
vdev_geom_attach:192[1]: Attaching to gpt/S3L2A7WM_EC1_S09.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A7WM_EC1_S09.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L29GXN_EC1_S10.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29GXN_EC1_S10.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29GXN_EC1_S10.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L29GXN_EC1_S10...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29GXN_EC1_S10.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29GXN_EC1_S10.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L29GXN_EC1_S10.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29GXN_EC1_S10.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29GXN_EC1_S10.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L29TPT_EC1_S11.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29TPT_EC1_S11.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TPT_EC1_S11.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L29TPT_EC1_S11...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L29TPT_EC1_S11.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TPT_EC1_S11.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L29TPT_EC1_S11.
vdev_geom_attach:192[1]: Attaching to gpt/S3L29TPT_EC1_S11.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TPT_EC1_S11.
vdev_geom_open_by_path:744[1]: Found provider by name /dev/gpt/S3L2A4EJ_EC1_S12.
vdev_geom_attach:192[1]: Attaching to gpt/S3L2A4EJ_EC1_S12.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A4EJ_EC1_S12.
vdev_geom_read_config:429[1]: Reading config from gpt/S3L2A4EJ_EC1_S12...
vdev_geom_detach:297[1]: Detaching consumer. Provider gpt/S3L2A4EJ_EC1_S12.
vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A4EJ_EC1_S12.
vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S3L2A4EJ_EC1_S12.
vdev_geom_attach:192[1]: Attaching to gpt/S3L2A4EJ_EC1_S12.
vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A4EJ_EC1_S12.
console="comconsole,vidconsole"
comconsole_speed="115200"
geom_mirror_load="YES"
kern.geom.label.gptid.enable="0"
kern.geom.label.disk_ident.enable="0"
zfs_load="YES"
kern.cam.scsi_delay=15000
# $FreeBSD: releng/11.0/etc/sysctl.conf 112200 2003-03-13 18:43:50Z mux $
#
# This file is read when going to multi-user and its contents piped thru
# ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf''
for details.
#
# Uncomment this to prevent users from seeing information about processes that
# are being run under another UID.
#security.bsd.see_other_uids=0
kern.ipc.shm_use_phys=1
kern.ipc.somaxconn=16384
# seems to be higher anyway
#kern.ipc.maxsockets=131072
#kern.maxfiles=131072
#kern.maxfilesperproc=104856
#kern.threads.max_threads_per_proc=4096
net.inet.tcp.fast_finwait2_recycle=1
net.inet.tcp.finwait2_timeout=15000
net.inet.tcp.msl=5000
machdep.panic_on_nmi=0
net.inet6.ip6.auto_flowlabel=0
security.bsd.see_other_gids=0
security.bsd.see_other_uids=0
# was 49152
net.inet.ip.portrange.hifirst=10000
security.bsd.unprivileged_proc_debug=0
net.inet.ip.redirect=0
net.inet6.ip6.redirect=0
net.inet.icmp.drop_redirect=1
net.inet6.icmp6.rediraccept=0
security.bsd.hardlink_check_uid=1
security.bsd.hardlink_check_gid=1
kern.coredump=0
kern.nodump_coredump=1
net.inet.ip.random_id=1
net.inet.ip.check_interface=1
net.inet.tcp.blackhole=1
net.inet.udp.blackhole=1
security.bsd.unprivileged_read_msgbuf=0
vfs.zfs.min_auto_ashift=12
vfs.zfs.arc_max=68719476736
I?ve set vfs.zfs.debug to 1.
Does anybody know what the attach/detach messages from GEOM mean?