Libguestfs - Nov 2018 - Re: Further nbdkit patches for Haiku

Le 13/11/2018 à 14:40, Richard W.M. Jones a écrit :
> On Tue, Nov 06, 2018 at 09:02:17AM +0000, Richard W.M. Jones wrote:
>> Finally about the list of failing tests:
>>
>>   https://pastebin.com/1PFG85DS
>>
>> Mostly these are because of the old qemu-io binary.  The qemu-io -f
>> option was added 4 years ago :-/ so it could be time to upgrade.
> 
> FWIW this turns out to be a problem in RHEL 7 as well :-(  I worked
> around it (in RHEL 7 only) by patching the affected tests like this:
> 
>   # Ancient qemu-io in RHEL 7 doesn't support -f FORMAT option. 
However
>   # we can just omit it and the tests still work fine.
>   for f in tests/*.sh; do
>     sed -i -e 's/qemu-io -f raw/qemu-io/g' $f
>   done
> 

Possibly use something like this?

qemuioargs="-f raw"
qemu-io -h | grep -- -f > /dev/null 2>&1 || qemuioargs=""

qemu-io $qemuioargs ...

Seems to work here.
> I reproduced this on FreeBSD.  The path used in the tests is indeed
> relative:
> 
>   #!../nbdkit
> 
> However replacing this with an absolute path did *not* fix the issue.
> FreeBSD seems to not like running a shell script from a shebang,
> probably because this is (was?) insecure.  However because of the way
> our test harness works we really need to run the shell script.  As
> there seems to be no simple way to fix this for now, I left the bug in
> 1.8.0.
Possibly there are other restrictions, like having the interpreter
root-owned, dunno.

Looks like their ports attempt to fix those:
https://www.freebsd.org/doc/en/books/porters-handbook/uses-shebangfix.html

But the page doesn't list the restrictions.
> 
> Finally the way that iconv detection was implemented broke FreeBSD.  I
> added an interim replacement for this which at least fixes FreeBSD &
> Linux, however of couse I did not test Haiku:
> 
>  
https://github.com/libguestfs/nbdkit/commit/acbe7ad89e75efa8eea41d5891bca3972214200a
Well it doesn't work, because you don't link with the library, so the
floppy plugin will be skipped.
> So hopefully nbdkit 1.8.0 works for you, or at least is not completely
> broken.  If there are any problems then let us know.


François.

On Tue, Nov 13, 2018 at 03:09:16PM +0100, François Revol
wrote:
> Le 13/11/2018 à 14:40, Richard W.M. Jones a écrit :
> > On Tue, Nov 06, 2018 at 09:02:17AM +0000, Richard W.M. Jones wrote:
> >> Finally about the list of failing tests:
> >>
> >>   https://pastebin.com/1PFG85DS
> >>
> >> Mostly these are because of the old qemu-io binary.  The qemu-io
-f
> >> option was added 4 years ago :-/ so it could be time to upgrade.
> > 
> > FWIW this turns out to be a problem in RHEL 7 as well :-(  I worked
> > around it (in RHEL 7 only) by patching the affected tests like this:
> > 
> >   # Ancient qemu-io in RHEL 7 doesn't support -f FORMAT option. 
However
> >   # we can just omit it and the tests still work fine.
> >   for f in tests/*.sh; do
> >     sed -i -e 's/qemu-io -f raw/qemu-io/g' $f
> >   done
> > 
> 
> 
> Possibly use something like this?
> 
> qemuioargs="-f raw"
> qemu-io -h | grep -- -f > /dev/null 2>&1 ||
qemuioargs=""
> 
> qemu-io $qemuioargs ...
> 
> Seems to work here.
Yes we could do something like that, but perhaps with a better name
such as ‘$qemu_io_format_raw_args’.
> > I reproduced this on FreeBSD.  The path used in the tests is indeed
> > relative:
> > 
> >   #!../nbdkit
> > 
> > However replacing this with an absolute path did *not* fix the issue.
> > FreeBSD seems to not like running a shell script from a shebang,
> > probably because this is (was?) insecure.  However because of the way
> > our test harness works we really need to run the shell script.  As
> > there seems to be no simple way to fix this for now, I left the bug in
> > 1.8.0.
> 
> Possibly there are other restrictions, like having the interpreter
> root-owned, dunno.
> 
> Looks like their ports attempt to fix those:
> https://www.freebsd.org/doc/en/books/porters-handbook/uses-shebangfix.html
> 
> But the page doesn't list the restrictions.
> 
> > 
> > Finally the way that iconv detection was implemented broke FreeBSD.  I
> > added an interim replacement for this which at least fixes FreeBSD
&
> > Linux, however of couse I did not test Haiku:
> > 
> >  
https://github.com/libguestfs/nbdkit/commit/acbe7ad89e75efa8eea41d5891bca3972214200a
> 
> Well it doesn't work, because you don't link with the library, so
the
> floppy plugin will be skipped.
Right, that was expected.  It's a small regression for Haiku though so
if you can work out a better way then let me know.

However the problem is it's not easy to fix.  On FreeBSD by default
they have two <iconv.h> headers, and two iconv libraries, and if they
are mixed up then it all breaks.

Rich.
> > So hopefully nbdkit 1.8.0 works for you, or at least is not completely
> > broken.  If there are any problems then let us know.
> 
> 
> 
> François.
-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

On Tue, Nov 13, 2018 at 02:18:34PM +0000, Richard W.M. Jones
wrote:
> On Tue, Nov 13, 2018 at 03:09:16PM +0100, François Revol wrote:
> > Possibly use something like this?
> > 
> > qemuioargs="-f raw"
> > qemu-io -h | grep -- -f > /dev/null 2>&1 ||
qemuioargs=""
> > 
> > qemu-io $qemuioargs ...
> > 
> > Seems to work here.
> 
> Yes we could do something like that, but perhaps with a better name
> such as ‘$qemu_io_format_raw_args’.
I looked into this further.  Unfortunately although we are able to
work around the missing ‘-f’ flag using this technique, we are still
no closer to having the tests pass.  This old qemu does not support
various other features we need, at least:

 - newstyle protocol

 - --image-opts parameter

 - TLS client

It looks a bit hopeless trying to get the nbdkit tests to work with
this version of qemu, although of course this is only a test and does
not stop nbdkit itself from working (eg. if you use a non-qemu client
or qemu client is running on another machine).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top