Hi,
I'm running RELENG_9 and I'm trying to play with CTL.
Initially I've setup an isp(4) interface in TARGET mode and tried to export
a LUN to a directly connected
Linux RHEL host, but for some reason that failed with the block backend (ramdisk
was exported properly) :
This is how I export the volume :
zfs create -V1000G tank/oracle
ctladm create -b block -o file=/dev/zvol/tank/oracle -S ZFSSERIAL001 -d
ZFSLUN001
ctladm port -o on
ctladm realsync off
camcontrol and ctladm show the device correctly :
[10:25]root@goliath:/home/ndenev# ctladm port -l
Port Online Type Name pp vp WWNN WWPN
0 YES IOCTL CTL ioctl 0 0 0 0
1 YES INTERNAL ctl2cam 0 0 0x500000091a1f4700 0x500000091a1f4702
2 YES INTERNAL CTL internal 0 0 0 0
3 YES FC isp0 0 0 0x20000024ff376b98 0x21000024ff376b98
4 YES FC isp1 1 0 0x20000024ff376b99 0x21000024ff376b99
[10:26]root@goliath:/home/ndenev# ctladm devlist -v
LUN Backend Size (Blocks) BS Serial Number Device ID
0 block 2097152000 512 ZFSSERIAL001 ZFSLUN001
lun_type=0
num_threads=14
file=/dev/zvol/tank/oracle
[10:26]root@goliath:/home/ndenev# camcontrol devlist
<FREEBSD CTLDISK 0001> at scbus2 target 1 lun 0 (da0,pass0)
This is what I see on the Linux host for the exported zvol:
qla2xxx 0000:0a:00.1: LOOP UP detected (8 Gbps).
qla2xxx 0000:0a:00.1: qla2xxx_eh_host_reset: reset succeeded
qla2xxx 0000:0a:00.1: scsi(4:0:0): Abort command issued -- 1 c 2002.
sd 4:0:0:0: scsi: Device offlined - not ready after error recovery
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sdq : READ CAPACITY failed.
sdq : status=0, message=00, host=1, driver=00
sdq : sense not available.
sd 4:0:0:0: rejecting I/O to offline device
sdq: Write Protect is off
sdq: Mode Sense: 00 00 00 00
sd 4:0:0:0: rejecting I/O to offline device
sdq: asking for cache data failed
sdq: assuming drive cache: write through
sd 4:0:0:0: Attached scsi disk sdq
sd 4:0:0:0: Attached scsi generic sg18 type 0
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
I've noticed the READ CAPACITY failed message and tried to issue a several
"camcontrol readcap" commands,
but they got stuck and are now unkillable :
[10:21]root@goliath:/home/ndenev# ps axuw|grep cam
root 29739 0.0 0.0 16300 1628 0- DL+ 8:09AM 0:00.00 camcontrol
readcap da0
root 30033 0.0 0.0 16300 1628 1- DL+ 8:12AM 0:00.00 camcontrol
readcap 2:1:0
root 30135 0.0 0.0 16300 1632 2 D+ 8:21AM 0:00.00 camcontrol
start pass0
procstat shows the same kernel stack for them :
[10:23]root@goliath.:/home/ndenev# procstat -kk 30033
PID TID COMM TDNAME KSTACK
30033 101161 camcontrol - mi_switch+0x186 sleepq_wait+0x42
_sleep+0x390 cam_periph_runccb+0x5a passioctl+0x171 devfs_ioctl_f+0x7b
kern_ioctl+0x115 sys_ioctl+0xfd amd64_syscall+0x546 Xfast_syscall+0xf7
Also there is this suspicious message on the FreeBSD machine :
cfcs_action: unsupported CCB type 0x918
Then I've tried to remove the volume from CTL but the command also got stuck
:
[10:29]root@goliath:/home/ndenev# ps axuww|grep ctladm
root 54269 0.0 0.0 18484 1692 3 I 8:24AM 0:00.00 ctladm remove
-b block -l 0
root 20969 0.0 0.0 16280 1720 5 S+ 10:30AM 0:00.00 grep ctladm
[10:30]root@goliath:/home/ndenev# procstat -kk 54269
PID TID COMM TDNAME KSTACK
54269 101580 ctladm - mi_switch+0x186
sleepq_catch_signals+0x2cc sleepq_wait_sig+0xc _sleep+0x2b9
ctl_be_block_ioctl+0x9aa ctl_ioctl+0x9e6 devfs_ioctl_f+0x7b kern_ioctl+0x115
sys_ioctl+0xfd amd64_syscall+0x546 Xfast_syscall+0xf7
Some Linux mailing lists suggest that maybe a firmware upgrade on the Linux box
can help, but
this does not explain the stuck readcap and ctladm commands.
Any ideas on how to debug this further are welcome.