Libguestfs - Mar 2020 - Re: Use libguestfs with SAS URI of a disk image

On Sat, Mar 14, 2020 at 04:48:24AM +0000, JP Zhang
wrote:
> HI Richard,
> 
> Thanks for the quick reply.
> 
> The Python program that works, that I mentioned earlier, uses subprocess to
issue/run "guestfish" commands below in that order.
> /libguestfs/run guestfish --listen -a <https storage-SAS-URL> -- ro
> then the Python code exported $PID to the environment. I believe so that
the same $PID could be used for the subsequent guestfish calls.
> /libguestfs/run guestfish --remote -- -version
> /libguestfs/run guestfish --remote launch
>
> Since it is running in the subprocess of "run.sh" shell, I
couldn't see the libguestfs debug output. Maybe there is a way, I just
don't know.
> 
> Then I manually typed similar guestfish commands in Linux bash shell, like
> guestfish --listen -a <https storage-SAS-URL> -- ro
> <this returned a $PID>
> guestfish --remote=<$PID> -- -version
> guestfish --remote=<$PID> launch
> this sequence of guestfish commands failed at "launch" cmd.
> 
> regarding params passed to add_drive call in libguestfs, the debug trace
from running the guestfish cmds above looks like this:
> libguestfs: trace: add_drive "/<container>/abcd"
"protocol:https"
"server:tcp:<account>.blob.core.windows.net"
> which is something I already tried, not any more insights.
You'll need the trace from the working call, not the
call that doesn't work.

Try ‘export LIBGUESTFS_TRACE=1’ before the working command.
> I really could not tell the difference between these two set of guestfish
commands.
> 
> (1) do you see any substance difference between doing things in these two
sets of commands? ones issued in Python program and ones issued in Bash shell
manually?
> (2) if I can add a local VM disk image to libguestfs and list its files,
can I safely say that I have libguestfs installed or set up correctly on my
machine? meaning I should be able to do "network" based disk image
calls/add "network" based disk image? correct, in theory? At this
point, I cannot 100% trust my local linguestfs setup since I don't know much
about libguestfs.
> (3) BTW, I see error message in manual run in Bash shell above. does this
tell us anything?
> Could not access KVM kernel module: No such file or directory
> qemu-system-x86_64: failed to initialize KVM: No such file or directory
> qemu-system-x86_64: Back to tcg accelerator
> I see the same KVM error when I ran "libguestfs-test-tool"
> Is KVM kernel absolutely needed? or is this just an option to try and does
libguestfs fall back to other Linux kernel to build supermin appliance
> 
> BTW, the working Python program was developed by others whom we have no
contact now. So far I don't see anything special about Python. It is a just
a shell program to issue libguestfs commands and wire up our own processing
logic on top of libguestfs.
> 
> Regarding using "nbdkit", it looks like it is another service I
need to run on my box. The goal of my project is to reduce the hops and improve
the performance. Also the code I develop will be a web service that takes many
real time requests. with nbdkit, does it mean that I need to create a instance
of nbdkit on the fly for every request that my service receives? will nbdkit
option work well when I have multiple concurrent libguestfs calls going through
nbdkit in multiple threads/processes?
Actually nbdkit doesn't need to be run as a separate service.  You can
run it "captive" from the main program.  It also has several
advantages over using libguestfs to connect to curl directly.

You probably want to look at what we do in virt-v2v, where we have
almost exactly the same scenario and we use nbdkit to proxy the
connection:

https://github.com/libguestfs/virt-v2v/blob/master/v2v/nbdkit.ml
https://github.com/libguestfs/virt-v2v/blob/f8e4e8643f105bc2abeb6b80b17a0ded3b636b7c/v2v/nbdkit_sources.ml#L276

Also at:

http://libguestfs.org/nbdkit.1.html
http://libguestfs.org/nbdkit-captive.1.html
http://libguestfs.org/nbdkit-curl-plugin.1.html
http://libguestfs.org/nbdkit-cache-filter.1.html

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org