libvirt users - Jan 2015 - Re: Using virsh blockcopy -- what's it supposed to accomplish?

On Thu, Jan 08, 2015 at 01:44:58PM -0600, Gary R Hook
wrote:
> On 12/24/14 4:42 AM, Kashyap Chamarthy wrote:
> >On Tue, Dec 23, 2014 at 12:38:57PM -0600, Gary R Hook wrote:
> >
> >[. . .]
> >
> >In my case, the block device is a QCOW2 disk image file. If I boot
> >without using the disk image file which has the operating system, the
> >domain will fail to boot, no?
> >
> >I see you're playing with NBD disks. I'll admit, I haven't
played much
> >with QEMU NBD, will have to experiment post holidays.
> 
> Back from the holidays, and back on this issue. I've learned a lot.
> 
> I've learned how to use the blockcopy command to create a local copy in
a
> simple disk file:
> 
> virsh dumpxml my_domain > my_domain.xml
> virsh undefine my_domain
> virsh blockcopy --domain my_domain vda $PWD/dsk.copy.qcow2 --wait --verbose
> --finish
> virsh define my_domain.xml
Yes, that's correct.
> and the resulting copy in dsk.copy.qcow2 is, indeed, bootable. It appears
to
> be a perfect copy, as I expect it to be.
Good, this time you find it bootable, compared to your previous test.
> But while I see (per Kashyap's article, etc) that it can be useful in
> certain scenarios, it's not interesting to me. I would like to my copy
to be
> off-system, and was hoping to use the NBD interface to accomplish that. So
I
> tried this (a variant of the above), working on the same system because
it's
> easier:
> 
> qemu-img create -f qcow2 /tmp/dsk.test.qcow2
A typo? You also need to provide a size here:

    $ qemu-img create -f qcow2 /tmp/dsk.test.qcow2 1G


For the rest, I'm afraid I still didn't manage time to test
the NBD scenario to give a meaningful response here. I'll let the others
who deal with NBD more often respond to it.
> qemu-nbd -f qcow2 -p11112 /tmp/dsk.test.qcow2
> nbd-client localhost 11112 /dev/nbd2
> virsh dumpxml my_domain > my_domain.xml
> virsh undefine my_domain
> virsh blockcopy --domain my_domain --wait --verbose --finish
> virsh define my_domain.xml
> nbd-client -d /dev/nbd2
> 
> and the qemu-nbd process exits, as I wish. I presume at this point that the
> new file has integrity.
> 
> I can take the qcow2 file that belongs to the domain and serve it up via
> NBD:
> 
> qemu-nbd --partition=1 -p11112 /path/to/my/qcow2/file.qcow2
> nbd-client localhost 11112 /dev/nbd2
> mount /dev/nbd2 -oloop /mnt/foo
> 
> and lo! in /mnt/foo I found my root filesytem. Seems perfectly reasonable.
> 
> If, however, I try to use my generated-via-NBD file, I get this:
> 
> # qemu-nbd --partition=1 -p11112 $PWD/dsk.test.qcow2 &
> [1] 7672
> # qemu-nbd: Could not find partition 1: Invalid argument
> 
> [1]+  Exit 1                  qemu-nbd --partition=1 -p11112
> $PWD/dsk.test.qcow2
> # qemu-nbd --partition=0 -p11112 $PWD/dsk.test.qcow2 &
> [1] 7686
> # qemu-nbd: Invalid partition 0
> ^C
> [1]+  Exit 1                  qemu-nbd --partition=0 -p11112
> $PWD/dsk.test.qcow2
> # qemu-nbd --partition=2 -p11112 $PWD/dsk.test.qcow2 &
> [1] 7699
> # qemu-nbd: Could not find partition 2: Invalid argument
> ^C
> [1]+  Exit 1                  qemu-nbd --partition=2 -p11112
> $PWD/dsk.test.qcow2
> # qemu-nbd --partition=3 -p11112 $PWD/dsk.test.qcow2 &
> [1] 7830
> # qemu-nbd: Could not find partition 3: Invalid argument
> 
> [1]+  Exit 1                  qemu-nbd --partition=3 -p11112
> $PWD/dsk.test.qcow2
> 
> I don't know what has been created, but it's not a copy of the
original
> guest's disk. There's no partition there, it seems.
> 
> So yes, blockcopy works fine under certain conditions. But the NBD layer
> seems to really muck things up.
> 
> Or, more likely, I'm doing things wrong. I'm hoping someone can
point out
> something obvious.
> 
> There's a recent thread about "Block Replication for Continuous
> Checkpointing" that is heading towards using NBD. I fail to understand
how
> this is ever going to work, based on my explorations.
-- 
/kashyap

On 1/8/15 2:21 PM, Kashyap Chamarthy wrote:
>> qemu-img create -f qcow2 /tmp/dsk.test.qcow2
>
> A typo? You also need to provide a size here:
>
>      $ qemu-img create -f qcow2 /tmp/dsk.test.qcow2 1G
Yes, my mistake. The size is set to the potential size of the source 
disk, which in this case is 20G.
> For the rest, I'm afraid I still didn't manage time to test
> the NBD scenario to give a meaningful response here. I'll let the
others
> who deal with NBD more often respond to it.
Well, that's the trick, right? No one that may have any experience has 
managed to pitch in to this conversation.

I can add this now: qemu-nbd seems to function like nbd-server, and 
concern itself with serving a filesystem. What's need, however, is a 
simple _file_. The challenge is getting that NBD-served thing to be 
viewed the same as a local disk file.

It would appear that the virsh option --raw is required. As far as I can 
tell, both virsh and qemu-nbd munge data with the intent of making it 
suitable for a qcow2 destination; the problem is that we don't need 
double-munging. So telling the blockcopy operation to "knock it off" 
seems to make it treat the NBD device the same as it does a simple disk 
file.

Yea!

So I find that the following:
>> qemu-nbd -f qcow2 -p11112 /tmp/dsk.test.qcow2
>> nbd-client localhost 11112 /dev/nbd2
>> virsh dumpxml my_domain > my_domain.xml
>> virsh undefine my_domain
>> virsh blockcopy --domain my_domain --wait --verbose --finish
only requires the addition of "--raw" to the above command. Or,
rather,
what I really need, which is

virsh blockcopy --domain my_domain --raw

which I can then control with subsequent commands.

I'm kinda surprised no one else has tried to do this and lived to write 
about it.

-- 
Gary R Hook
Senior Kernel Engineer
NIMBOXX, Inc

On Thu, Jan 08, 2015 at 06:04:20PM -0600, Gary R Hook
wrote:
> On 1/8/15 2:21 PM, Kashyap Chamarthy wrote:
> 
> >>qemu-img create -f qcow2 /tmp/dsk.test.qcow2
> >
> >A typo? You also need to provide a size here:
> >
> >     $ qemu-img create -f qcow2 /tmp/dsk.test.qcow2 1G
> 
> Yes, my mistake. The size is set to the potential size of the source disk,
> which in this case is 20G.
> 
> >For the rest, I'm afraid I still didn't manage time to test
> >the NBD scenario to give a meaningful response here. I'll let the
others
> >who deal with NBD more often respond to it.
> 
> Well, that's the trick, right? No one that may have any experience has
> managed to pitch in to this conversation.
>
> 
> I can add this now: qemu-nbd seems to function like nbd-server, and concern
> itself with serving a filesystem. What's need, however, is a simple
_file_.
> The challenge is getting that NBD-served thing to be viewed the same as a
> local disk file.
> 
> It would appear that the virsh option --raw is required. As far as I can
> tell, both virsh and qemu-nbd munge data with the intent of making it
> suitable for a qcow2 destination; the problem is that we don't need
> double-munging. So telling the blockcopy operation to "knock it
off" seems
> to make it treat the NBD device the same as it does a simple disk file.
> 
> Yea!
> 
> So I find that the following:
> 
> >>qemu-nbd -f qcow2 -p11112 /tmp/dsk.test.qcow2
> >>nbd-client localhost 11112 /dev/nbd2
> >>virsh dumpxml my_domain > my_domain.xml
> >>virsh undefine my_domain
> >>virsh blockcopy --domain my_domain --wait --verbose --finish
> 
> only requires the addition of "--raw" to the above command. Or,
rather, what
> I really need, which is
> 
> virsh blockcopy --domain my_domain --raw
> 
> which I can then control with subsequent commands.
Nice, I see that Eric and Paolo also concur with your observations.
> 
> I'm kinda surprised no one else has tried to do this and lived to write
> about it.
Well, the above is slightly an exaggerated statement, as you can guess
from other responses in this thread, not everyone will invest the time
to write everything they test explaining it to the world. Anyhow, the
issue seems to be resolved, and as you suggested in your other response
to write about this, that'd be useful. :-)

-- 
/kashyap