freebsd stable - Jul 2018 - ctld(8) 11.2-release lockup with w2k16 [Was: Re: ctld(8), multiple 'portal-group' on same socket (individual 'discovery-auth-group' restrictions)]

On 1020T1035, Harald Schmalzbauer wrote:
>  Hello,
> 
> I'm trying to move from istgt(1) to ctld(8), but it seems my setup
isn't
> possible with ctld.
> Besides missing support for virtual-DVDs ('UnitType DVD' in istgt)
and
> real ODD-devices ('UnitType pass' in istgt),
Yup, we don't implement virtual DVDs and passthrough.  Especially the
latter would be a nice feature to have.
> I guess it's impossible to
> define multiple "portal-group"s, listening on the same socket,
but with
> different "discovery-auth-group".
> The idea is, to present initiators only targets at discovery, which they
> are allowed to connect to.
> Am I missing something which could provide such selective discovery with
> ctld(8)?
I thought about it, but I don't like the way istgt does it.  By allowing
multiple portal groups to bind to a single address (portal), we would
introduce ambiguity as for which portal-group and associated
discovery-auth-group is being used.  On the other hand, a simplistic
approach of only showing targets with auth-group being the same
as discovery-auth-group for the portal probably wouldn't be very useful
in real-world cases.

Bez?glich Edward Tomasz Napiera?a's Nachricht vom 21.10.2014 12:43
(localtime):
> On 1020T1035, Harald Schmalzbauer wrote:
>>  Hello,
>>
>> I'm trying to move from istgt(1) to ctld(8), but it seems my setup
isn't
>> possible with ctld.
>> Besides missing support for virtual-DVDs ('UnitType DVD' in
istgt) and
>> real ODD-devices ('UnitType pass' in istgt),
> Yup, we don't implement virtual DVDs and passthrough.  Especially the
> latter would be a nice feature to have.
>
>> I guess it's impossible to
>> define multiple "portal-group"s, listening on the same
socket, but with
>> different "discovery-auth-group".
>> The idea is, to present initiators only targets at discovery, which
they
>> are allowed to connect to.
>> Am I missing something which could provide such selective discovery
with
>> ctld(8)?
> I thought about it, but I don't like the way istgt does it.  By
allowing
> multiple portal groups to bind to a single address (portal), we would
> introduce ambiguity as for which portal-group and associated
> discovery-auth-group is being used.  On the other hand, a simplistic
> approach of only showing targets with auth-group being the same
> as discovery-auth-group for the portal probably wouldn't be very useful
> in real-world cases.
Thanks a lot for your attention and the highly appreciated work on ctl[d]*!

In my "real world" case, I dispense backup disks to win7 clients via
iSCSI.
I liked the possibility to limit target-discovery-results, because
curious people, garnated such extra resources, weren't able to see who
else was granted this extra resource (by definition, there's no private
data on the clients which use that backup-strategy, so no need for
qualified identity checks or encryption needed). Even with any security
mechanisms active, two consumers (some dozen in my case) of the same
portal know about the other. That's one example where this feature was
useful for me.
Another one actually is (since not implemented in ctld, yet?) with
virtual DVDs. If you provide 50 DVDs at one portal, it's confusing if
any initiator needs to select from the complete list.

I'd like to point to two options I'm missing:
option RPM
option FormFactor
And another one I don't know/understand:
option scsiname

Missing resp. wrong reported option RPM leads to identification as SSD
with ESXi initiators. I don't remember what defines a SSD vs. HDD, I
think it was a threshold of something about 400? Or was it 0? I think I
read the former? Don't have docs handy.
The FormFactor was more cosmetic, but in bigger environments I guess it
can be useful if you have multiple targets available, choosing the
better fitting backend-pool e.g.

If someone could point me to the meaning of option iscsiname?

Thanks a lot,

-Harry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20141021/117a0589/attachment.sig>

Am 21.10.2014 um 12:43 schrieb Edward Tomasz Napiera?a:
> On 1020T1035, Harald Schmalzbauer wrote:
>>   Hello,
>>
>> I'm trying to move from istgt(1) to ctld(8), but it seems my setup
isn't
>> possible with ctld.
>> Besides missing support for virtual-DVDs ('UnitType DVD' in
istgt) and
>> real ODD-devices ('UnitType pass' in istgt),
> Yup, we don't implement virtual DVDs and passthrough.  Especially the
> latter would be a nice feature to have.

Hello Edward,

my current problem is unrelated.
But this old mail illustrates the timeframe I've been happily using 
ctld(8) without problems :-) Thanks!

Recently, I discovered that WindowsServerBackup fails with Win2k16 
(never used 2k12).
Old initiators running 2008R2 (or ESXi 5.5) are still able to use 
ctld(8) ZVOL targets for WindowsServerBackup on 11.2-release without 
problems.

I haven't had time to do much analysis and I'm lacking skills/equipment 
to do them down at debugger level, but I wanted to ask if you're aware 
about problems with Windows Server 2016 as ctld(8) initiator.

The Symptoms:

The system locks up for about 30-60 seconds with iSCSI load from w2k16.
When the lockup happens, systat(1) shows 25% intr usage (which is one 
core) and not even the login session is responsive anymore. Neither 
updating userland-output nor reacting to input.
But, the input is queued and gets processed after the lockup releases.
The lockup vanishes as soon as iSCSI session was reset:
Jun 28 06:14:09 bansta kernel: WARNING: 172.24.32.172 
(iqn.1991-05.com.microsoft:dafus.mgn.mo1.psw-online.de): no ping reply 
(NOP-Out) after 5 seconds; dropping
connection
Jun 28 06:14:09 bansta kernel: WARNING: 172.24.32.172 
(iqn.1991-05.com.microsoft:dafus.mgn.mo1.psw-online.de): waiting for CTL 
to terminate 94 tasks
Jun 28 06:14:09 bansta kernel: WARNING: 172.24.32.172 
(iqn.1991-05.com.microsoft:dafus.mgn.mo1.psw-online.de): tasks terminated

Sometimes it's possible to transfer 30GB before the lockup happens, 
sometimes even a NTFS-quick-format leads to the lockup.


Yesterday I used istgt(1) instead of ctld(8) to export the exactly same 
ZVOL using the exactly same network backend, with exactly the same 
initiator.
The lockup hasn't occured anymore, the complete WindowsServerBackup taks 
finishes successfully on the Windows Server 2016 initiator.? So I 
strongly suspect a ctld(8) locking problem.
Like mentioned, target backed is a ZFS volume.? I already used a HDD as 
target backed (and observed a much better performance, which drops even 
if I use a UFS vnode backend on the same HDD), but I'm not sure anymore 
whether the lockup also occured...

For now I can't tell anything helpfuly, just describe the symptoms and 
ask if you have any hints for me what to try next to narrow down the 
problem, or if this is a already known problem.

Thanks,

-harry