Libguestfs - Mar 2020 - Re: Use libguestfs with SAS URI of a disk image

On Fri, Mar 13, 2020 at 09:41:37PM +0000, JP Zhang
wrote:
> Hi Richard,
> 
> Thanks for the great work you did on libguestfs.
>
> I try to call libguestfs lib API directly in C to "add drive" by
> using a Azure blob storage access SAS URI, which has shared access
> token/secret on the URL.
>
> The URL likes like this:
>
'https://<account>.blob.core.windows.net/<container>/<VMDiskImage>?sv=2017-04-17&sr=b&si=<guid>&sig=<secret>'
> I try to use this API:
> 
>  int
>  guestfs_add_drive_opts (guestfs_h *g,
>                          const char *filename,
>                          ...);
> 
> 
> My code looks like this:
> char const *servers[] = { (char const
*)"<account>.blob.core.windows.net", NULL };
> int addDriveRet =  guestfs_add_drive_opts (g,
"/<container>/<VMDiskImage>?sv=2017-04-17&sr=b&si=<guid>&sig=<secret>",
>                          GUESTFS_ADD_DRIVE_OPTS_FORMAT, "raw",
>                          GUESTFS_ADD_DRIVE_OPTS_PROTOCOL,
"https",
>                          GUESTFS_ADD_DRIVE_OPTS_SERVER, servers,
>                          GUESTFS_ADD_DRIVE_OPTS_READONLY, 1,
>                          -1);
Since you say you got it to work with guestfish, one trick is to run
‘guestfish -x’ (or the ‘trace true’ command, or set LIBGUESTFS_TRACE=1)
and it will actually print the full API call you should use.

I believe the problem may be to do with URI %-quoting of the filename
(2nd) parameter.
> I read the documentation and samples on libguestfs.org in depth. I
> could not find a good sample for "https" with SAS secret on URL.
I
> tried many different ways to break up the URL into "server",
> "protocol", and secret. It looks like "secret" option
is only for
> "rbd". From the debug log/printouts from libguestfs, it seemed to
> reassemble the parts of original URL back to a URL that matched the
> original one. But I got http 403 error( auth error). I am very sure
> the URL is correct because I can download the disk image using
> Curl. The libguestfs debug log printout displayed the URL with ?
> being turned into %3F and % into %25. I am not sure whether
> libguestfs URL-encoded printout was just for printout or whether it
> was the actual URL string being used to make the https server
> call. If it was, then the secret has been changed due to URL
> encoding again. This is just a guess. Maybe not related at all.
>
> BTW, I can run libguestfs-test-tool successfully. and I have another
> program in Python that call through /run.sh -> GuestFish ->
> libguestfs lib, it works with the same original SAS URL.
So you can contact the SAS URL through this?  If so I would try the
trace trick above to see the actual protocol call being made.

If that doesn't work, another way to do it would be to
forward the connection through nbdkit, ie:

  rm -f /tmp/socket
  nbdkit -f -U /tmp/socket curl 'https://...' &

and connect to the the NBD socket using the add_drive_opts API as
documented in the manual.

(in guestfish you'd use: guestfish -a 'nbd://?socket=/tmp/socket')

nbdkit can also be run captive from your program, see
http://libguestfs.org/nbdkit-captive.1.html
> also, on my dev machine, I can do "guestfs_add_drive" on the
local
> disk image successfully and mount the device, and list files after I
> downloaded the same disk image to my local machine.
>
> guestfs_add_drive(g, "/pathto/vmdiskimage");
>
> Could you share how I should fill in various params in
> "guest_add_drive_opts()? like "PROTOCOL",
"SERVER", "filename"( the
> 2nd param), or USERNAME ( not seem to apply here, I also tried with
> putting "secret" in SECRET, "sig" in USERNAME, the
rebuilt URL from
> libguestfs looks missing these parts in final URL, it probably put
> these in HTTP header.)
>
> I also try to call guestfs_add_drive() with the original URL
> below. it didn't work either.
>
> guestfs_add_drive(g,
"https://<account>.blob.core.windows.net/<container>/<VMDiskImage>?sv=2017-04-17&sr=b&si=<guid>&sig=<secret>");
Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW