Libguestfs - Sep 2020 - Re: [libnbd PATCH 0/3] opt_list_meta_context

On 9/29/20 6:23 AM, Richard W.M. Jones wrote:
> This is not related to the current patch, but generally for libnbd:
> 
> (1) Should we be testing interop separately for qemu-storage-daemon
> (qsd), or is qsd basically qemu-nbd in a new wrapper so it's not worth
> doing it?
> 
> (2) I found a bug in the new nbdinfo behaviour:
> 
> $ nbdkit -fv file dir=/scratch
> 
> (where /scratch is a directory with a lot of files in it)
> 
> $ nbdinfo --version
> libnbd 1.5.3
> $ nbdinfo --list nbd://localhost
> [... lots of files shown ...]
> nbd_opt_info: recv: Connection reset by peer
> 
> nbdkit gives this error:
> 
> nbdkit: file[1]: error: client exceeded maximum number of options (32)
Yeah, we really ought to raise nbdkit's limits when .list_exports 
returns a large list.  We haven't yet released nbdkit 1.24 where the 
file driver actually has a large .list_exports; but it may also be a 
reason to figure out if libnbd should go to the effort of re-connecting 
for the later portion of a list on servers like older nbdkit that limit 
the length of the negotiation phase.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

On Tue, Sep 29, 2020 at 06:51:06AM -0500, Eric Blake
wrote:
> On 9/29/20 6:23 AM, Richard W.M. Jones wrote:
> >This is not related to the current patch, but generally for libnbd:
> >
> >(1) Should we be testing interop separately for qemu-storage-daemon
> >(qsd), or is qsd basically qemu-nbd in a new wrapper so it's not
worth
> >doing it?
Any thoughts on this one?
> >(2) I found a bug in the new nbdinfo behaviour:
> >
> >$ nbdkit -fv file dir=/scratch
> >
> >(where /scratch is a directory with a lot of files in it)
> >
> >$ nbdinfo --version
> >libnbd 1.5.3
> >$ nbdinfo --list nbd://localhost
> >[... lots of files shown ...]
> >nbd_opt_info: recv: Connection reset by peer
> >
> >nbdkit gives this error:
> >
> >nbdkit: file[1]: error: client exceeded maximum number of options (32)
> 
> Yeah, we really ought to raise nbdkit's limits when .list_exports
> returns a large list.  We haven't yet released nbdkit 1.24 where the
> file driver actually has a large .list_exports; but it may also be a
> reason to figure out if libnbd should go to the effort of
> re-connecting for the later portion of a list on servers like older
> nbdkit that limit the length of the negotiation phase.
It raises the question about what the limit needs to be.  If I point
nbdkit-file-plugin at a directory, the number of exports could be
pretty much unlimited.  But at the smae time the options limit is
there for a fairly good reason.  Should we change the client to
request info in groups of <N> (presumably requiring reconnection)?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

On 9/29/20 9:37 AM, Richard W.M. Jones wrote:
> On Tue, Sep 29, 2020 at 06:51:06AM -0500, Eric Blake wrote:
>> On 9/29/20 6:23 AM, Richard W.M. Jones wrote:
>>> This is not related to the current patch, but generally for libnbd:
>>>
>>> (1) Should we be testing interop separately for qemu-storage-daemon
>>> (qsd), or is qsd basically qemu-nbd in a new wrapper so it's
not worth
>>> doing it?
> 
> Any thoughts on this one?
It's a new wrapper, but using the same qemu NBD server code under the
hood.  Testing it might be good for demonstrating example command lines,
but won't tease out any new behavior interactions.
> 
>>> (2) I found a bug in the new nbdinfo behaviour:
>>>
>>> $ nbdkit -fv file dir=/scratch
>>>
>>> (where /scratch is a directory with a lot of files in it)
>>>
>>> $ nbdinfo --version
>>> libnbd 1.5.3
>>> $ nbdinfo --list nbd://localhost
>>> [... lots of files shown ...]
>>> nbd_opt_info: recv: Connection reset by peer
>>>
>>> nbdkit gives this error:
>>>
>>> nbdkit: file[1]: error: client exceeded maximum number of options
(32)
>>
>> Yeah, we really ought to raise nbdkit's limits when .list_exports
>> returns a large list.  We haven't yet released nbdkit 1.24 where
the
>> file driver actually has a large .list_exports; but it may also be a
>> reason to figure out if libnbd should go to the effort of
>> re-connecting for the later portion of a list on servers like older
>> nbdkit that limit the length of the negotiation phase.
> 
> It raises the question about what the limit needs to be.  If I point
> nbdkit-file-plugin at a directory, the number of exports could be
> pretty much unlimited.  But at the smae time the options limit is
> there for a fairly good reason.  Should we change the client to
> request info in groups of <N> (presumably requiring reconnection)?
I posted an nbdkit patch for upping the limit that nbdkit allows, as
'qemu-nbd --list' is also able to trigger the current low limit.
Changing nbdinfo to process in groups of N is lots of code for little
benefit, especially if we can't find any other server that has a limit
as low as the old nbdkit had (remember, it's not until nbdkit 1.24 that
you can come up  with an export list long enough to worry about the old
limit).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org