thr3ads.net - Libguestfs - Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA [Nov 2017]

If this information is useful, please help other people find it:
Share via:

Gandalf Corvotempesta

2017-Nov-15 21:42 UTC

Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

I'm thinking on how to prove you a sample XVA
I have to create (and populate) a VM because an empty image will result in
an empty XVA
And a VM is 300-400Mb as minimum

Il 15 nov 2017 10:30 PM, "Max Reitz" <mreitz@redhat.com> ha
scritto:
> On 2017-11-15 21:41, Gandalf Corvotempesta wrote:
> > 2017-11-15 21:29 GMT+01:00 Richard W.M. Jones
<rjones@redhat.com>:
> >> Gandalf, is there an XVA file publically available (pref. not
> >> too big) that we can look at?
> >
> > I can try to provide one, but it's simple:
> >
> > # tar tvf 20160630_124823_aa72_.xva.gz | head -n 50
> > ---------- 0/0           42353 1970-01-01 01:00 ova.xml
> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000000
> > ---------- 0/0              40 1970-01-01 01:00
Ref:175/00000000.checksum
> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000001
> > ---------- 0/0              40 1970-01-01 01:00
Ref:175/00000001.checksum
> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000003
> > ---------- 0/0              40 1970-01-01 01:00
Ref:175/00000003.checksum
> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000004
> > ---------- 0/0              40 1970-01-01 01:00
Ref:175/00000004.checksum
> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000005
> > ---------- 0/0              40 1970-01-01 01:00
Ref:175/00000005.checksum
> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000006
> > ---------- 0/0              40 1970-01-01 01:00
Ref:175/00000006.checksum
> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000007
> > ---------- 0/0              40 1970-01-01 01:00
Ref:175/00000007.checksum
> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000009
> > ---------- 0/0              40 1970-01-01 01:00
Ref:175/00000009.checksum
> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000010
> > ---------- 0/0              40 1970-01-01 01:00
Ref:175/00000010.checksum
> >
> >
> > You can ignore the ova.xml and just use the "Ref:175"
directory.
> > Inside the XVA you'll fine one "Ref" directory for each
virtual disk
> > (ref number is different for each disk)
> > Inside each Ref directory, you'll get tons of 1MB file,
corrisponding
> > to the RAW image.
> > You have to merge these files in a single raw file with just an
> > exception: numbers are not sequential.
> > as you can see above, we have: 00000000, 00000001, 00000003
> >
> > The 00000002 is missing, because it's totally blank. XenServer
doesn't
> > export any empty block, thus it will skip the corrisponding 1MB file.
> > When building the raw image, you have to replace empty blocks with 1MB
> > full of zeros.
> >
> > This is (in addition to the tar extract) the most time-consuming part.
> > Currently I'm rebuilding a 250GB image, with tons of files to be
> > merge.
> > If qemu-img can be patched to automatically convert this kind of
> > format, I can save about 3 hours (30 minutes for extracting the
> > tarball, and about 2 hours to merge 250-300GB image)
>
> Hmmm...  OK, so as Rich said, you can use the raw driver with an offset
> and a size.  And you can use the null-co driver to emulate holes.  And
> qemu-img convert supports concatenation.
>
> Taking all of that together, it should be possible to create an input
> "file" per 1 MB block and concatenate all of those with qemu-img
> convert.  The only issue is that it'll get you a *really* long command
> line.
>
> (And you'll still have to gunzip the archive before, of course.)
>
> I'll try to write a script, but of course it's hart without an
example...
>
> Max
>
>

Richard W.M. Jones

2017-Nov-15 21:49 UTC

head link

Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

On Wed, Nov 15, 2017 at 10:42:42PM +0100, Gandalf Corvotempesta
wrote:> I'm thinking on how to prove you a sample XVA
> I have to create (and populate) a VM because an empty image will result in
> an empty XVA
> And a VM is 300-400Mb as minimum
That's fine to download.  Also you could compress it (for transfer)
which should make it smaller.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Gandalf Corvotempesta

2017-Nov-15 21:50 UTC

head link

Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

https://stacklet.com/downloads/XenServer-XVA-Template-Debian-7.8-Lightweight-x86

Some XVAs

Il 15 nov 2017 10:42 PM, "Gandalf Corvotempesta" <
gandalf.corvotempesta@gmail.com> ha scritto:
> I'm thinking on how to prove you a sample XVA
> I have to create (and populate) a VM because an empty image will result in
> an empty XVA
> And a VM is 300-400Mb as minimum
>
> Il 15 nov 2017 10:30 PM, "Max Reitz" <mreitz@redhat.com> ha
scritto:
>
>> On 2017-11-15 21:41, Gandalf Corvotempesta wrote:
>> > 2017-11-15 21:29 GMT+01:00 Richard W.M. Jones
<rjones@redhat.com>:
>> >> Gandalf, is there an XVA file publically available (pref. not
>> >> too big) that we can look at?
>> >
>> > I can try to provide one, but it's simple:
>> >
>> > # tar tvf 20160630_124823_aa72_.xva.gz | head -n 50
>> > ---------- 0/0           42353 1970-01-01 01:00 ova.xml
>> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000000
>> > ---------- 0/0              40 1970-01-01 01:00
>> Ref:175/00000000.checksum
>> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000001
>> > ---------- 0/0              40 1970-01-01 01:00
>> Ref:175/00000001.checksum
>> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000003
>> > ---------- 0/0              40 1970-01-01 01:00
>> Ref:175/00000003.checksum
>> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000004
>> > ---------- 0/0              40 1970-01-01 01:00
>> Ref:175/00000004.checksum
>> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000005
>> > ---------- 0/0              40 1970-01-01 01:00
>> Ref:175/00000005.checksum
>> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000006
>> > ---------- 0/0              40 1970-01-01 01:00
>> Ref:175/00000006.checksum
>> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000007
>> > ---------- 0/0              40 1970-01-01 01:00
>> Ref:175/00000007.checksum
>> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000009
>> > ---------- 0/0              40 1970-01-01 01:00
>> Ref:175/00000009.checksum
>> > ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000010
>> > ---------- 0/0              40 1970-01-01 01:00
>> Ref:175/00000010.checksum
>> >
>> >
>> > You can ignore the ova.xml and just use the "Ref:175"
directory.
>> > Inside the XVA you'll fine one "Ref" directory for
each virtual disk
>> > (ref number is different for each disk)
>> > Inside each Ref directory, you'll get tons of 1MB file,
corrisponding
>> > to the RAW image.
>> > You have to merge these files in a single raw file with just an
>> > exception: numbers are not sequential.
>> > as you can see above, we have: 00000000, 00000001, 00000003
>> >
>> > The 00000002 is missing, because it's totally blank. XenServer
doesn't
>> > export any empty block, thus it will skip the corrisponding 1MB
file.
>> > When building the raw image, you have to replace empty blocks with
1MB
>> > full of zeros.
>> >
>> > This is (in addition to the tar extract) the most time-consuming
part.
>> > Currently I'm rebuilding a 250GB image, with tons of files to
be
>> > merge.
>> > If qemu-img can be patched to automatically convert this kind of
>> > format, I can save about 3 hours (30 minutes for extracting the
>> > tarball, and about 2 hours to merge 250-300GB image)
>>
>> Hmmm...  OK, so as Rich said, you can use the raw driver with an offset
>> and a size.  And you can use the null-co driver to emulate holes.  And
>> qemu-img convert supports concatenation.
>>
>> Taking all of that together, it should be possible to create an input
>> "file" per 1 MB block and concatenate all of those with
qemu-img
>> convert.  The only issue is that it'll get you a *really* long
command
>> line.
>>
>> (And you'll still have to gunzip the archive before, of course.)
>>
>> I'll try to write a script, but of course it's hart without an
example...
>>
>> Max
>>
>>

Max Reitz

2017-Nov-15 21:50 UTC

head link

Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

On 2017-11-15 22:50, Gandalf Corvotempesta wrote:>
https://stacklet.com/downloads/XenServer-XVA-Template-Debian-7.8-Lightweight-x86
> 
> Some XVAs
Thanks!

Max Reitz

2017-Nov-15 22:28 UTC

head link

Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

On 2017-11-15 22:50, Gandalf Corvotempesta wrote:>
https://stacklet.com/downloads/XenServer-XVA-Template-Debian-7.8-Lightweight-x86
Hmmm, that gives me a timeout when downloading.  And another image I got
gives me an "argument list too long" -- which I had feared already...
It has a size of 3 MB and Linux "only" allows 2...

Max

Apparently Analagous Threads

Search for more seemingly similar threads

Libguestfs - Nov 2017 - Re: [Qemu-devel] [qemu-img] support for XVA

Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

Re: [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

Apparently Analagous Threads