libvirt users - Mar 2012 - Suggestions on building VM disks from scratch

This may not be the best forum to ask, but it does
seem to be one where it is as likely as anywhere
that someone will have dealt with a similar problem.

I need to define a procedure for last resort disaster 
recovery from an incremental file level backup of the 
root partition (and any others that are critical.)

Now it is easy enough to create a raw virtual disk
with dd, then to losetup and do a cfdisk and mkfs, 
then mount it and rsync the backup onto it.

At this point I would like to be able to install
grub and make the virtual disk bootable. I have
'issues' with the way grub does things and the way
it seems to make certain assumptions for you. The
virtual disk is in actuality going to become an hd(0)
on the virtual machine; but my past experience with
grub is that it is going to assume you really
intend to install the MBR on your running system disk;
but if you tell it hd(1) then when you try to boot
it ain't gonna work.

Has anyone dealt with these issues? It's more than
just a VM one; I run up against it time and again. 
A few months ago I attempted to attach a brand
new SATA disk via a USB converter to my laptop so
I could build a system on it for a machine that
has not CD, no floppy, etc, etc. I damn near clobbered
my laptop but fortunately caught the problem in time
and fixed it before a reboot... in which case I'd
have *really* had issues.

Here's how far I've progressed this evening:

$ dd if=/dev/zero of=/recover2.my.net.raw count=20M
$ losetup /dev/loop0 recover2.my.net.raw
$ cfdisk /dev/loop0
	create a 9G bootable linux partition and a 1G swap
$ kpartx -a -v /dev/loop0
$ mkfs.ext4 /dev/mapper/loop0p1 -m 0.01 -L "RecoverTest"
$ mkdir tmp
$ mount /dev/mapper/loop0p1 tmp
>From the backup server:
$ rsync -av tree/ root at vmserver.my.com:/srv/pool/tmp/
$ umount tmp
$ kpartx -d /dev/loop0
$ losetup -d /dev/loop0
$ virsh attach-disk recovery.my.net --source /srv/pool/recover2.my.net.raw
--target sdb

But no joy from the running VM itself. It does not show sdb, 
only its own vda1. rescan-scsi-bus.sh fails to bring make /dev/sdb
appear. Is the hotplugging failing perhaps?

Suggestions?

On 03/31/2012 01:39 PM, Dale Amon wrote:
> I need to define a procedure for last resort disaster 
> recovery from an incremental file level backup of the 
> root partition (and any others that are critical.)
> 
> Now it is easy enough to create a raw virtual disk
> with dd, then to losetup and do a cfdisk and mkfs, 
> then mount it and rsync the backup onto it.
It sounds like you want to use the features of libguestfs, possibly via
the command line program 'virt-rescue'; see http://libguestfs.org for
more details about what this can let you do.

-- 
Eric Blake   eblake at redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 620 bytes
Desc: OpenPGP digital signature
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20120331/fea963ee/attachment.sig>

> Then I created my blank 'disk' file and tried to
> run virt-rescue on it. It crashed out with an 
> error from febootstrap.
First of all, debug this properly:

(1) What is the full error message?

(2) What is the complete, unedited output of 'libguestfs-test-tool'?

(3) What version of libguestfs & febootstrap are using and where did
you get them from?

You can post the details on our mailing list libguestfs at redhat.com
(no need to subscribe if you don't want to).
> After finding nothing terribly
> useful or current on this in searching, I tried
> guestfish instead. After some fiddling I got it
> to attach my blank disk. However I cannot find a
> reasonable way to partition it with the part-add which
> seems to want me to count sectors. All I want is
> a 9G linux and a 2G swap.
Assuming the filesystem was in /tmp/root.tar.gz, the following code
will do this:

  guestfish <<EOF

  sparse /tmp/test.img 11G
  run

  part-init /dev/sda mbr
  # 9GB sda1
  part-add /dev/sda p 64 $(( 9*1024*1024*2 ))
  # remainder in sda2
  part-add /dev/sda p $(( 9*1024*1024*2 + 1 )) -64

  mkfs ext4 /dev/sda1
  mkswap /dev/sda2

  mount /dev/sda1 /
  tgz-in /tmp/root.tar.gz /

  EOF

Example:

  $ sh test.sh 
  $ virt-df -a test.img -h
  Filesystem                                Size       Used  Available  Use%
  test.img:/dev/sda1                        9.0G       276M       8.3G    4%
  $ ll -h test.img 
  -rw-rw-r--. 1 rjones rjones 11G Apr  4 09:39 test.img

Whether this would actually boot is another question: you may also
need to add some grub commands to set up the bootloader, *or* (better
and easier IMHO) set up libvirt so that it boots from an external
kernel + initrd.
> It also lacks access to rsync.
It's not the first time that someone has asked for rsync, and it
wouldn't be too hard to add.  However note that rsync really gives you
no benefit when you're creating a filesystem from scratch, because
there's no original to rsync against.  If you are updating a
filesystem image then rsync makes sense.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

On 4/4/2012 4:45 AM, Richard W.M. Jones wrote:
>> Then I created my blank 'disk' file and tried to
>> run virt-rescue on it. It crashed out with an
>> error from febootstrap.
>
> First of all, debug this properly:
>
> (1) What is the full error message?
>
> (2) What is the complete, unedited output of
'libguestfs-test-tool'?
>
> (3) What version of libguestfs&  febootstrap are using and where did
> you get them from?
>
> You can post the details on our mailing list libguestfs at redhat.com
> (no need to subscribe if you don't want to).
>
>> After finding nothing terribly
>> useful or current on this in searching, I tried
>> guestfish instead. After some fiddling I got it
>> to attach my blank disk. However I cannot find a
>> reasonable way to partition it with the part-add which
>> seems to want me to count sectors. All I want is
>> a 9G linux and a 2G swap.
>
> Assuming the filesystem was in /tmp/root.tar.gz, the following code
> will do this:
>
>    guestfish<<EOF
>
>    sparse /tmp/test.img 11G
>    run
>
>    part-init /dev/sda mbr
>    # 9GB sda1
>    part-add /dev/sda p 64 $(( 9*1024*1024*2 ))
>    # remainder in sda2
>    part-add /dev/sda p $(( 9*1024*1024*2 + 1 )) -64
>
>    mkfs ext4 /dev/sda1
>    mkswap /dev/sda2
>
>    mount /dev/sda1 /
>    tgz-in /tmp/root.tar.gz /
>
>    EOF
>
> Example:
>
>    $ sh test.sh
>    $ virt-df -a test.img -h
>    Filesystem                                Size       Used  Available 
Use%
>    test.img:/dev/sda1                        9.0G       276M       8.3G   
4%
>    $ ll -h test.img
>    -rw-rw-r--. 1 rjones rjones 11G Apr  4 09:39 test.img
>
> Whether this would actually boot is another question: you may also
> need to add some grub commands to set up the bootloader, *or* (better
> and easier IMHO) set up libvirt so that it boots from an external
> kernel + initrd.
>
>> It also lacks access to rsync.
>
> It's not the first time that someone has asked for rsync, and it
> wouldn't be too hard to add.  However note that rsync really gives you
> no benefit when you're creating a filesystem from scratch, because
> there's no original to rsync against.  If you are updating a
> filesystem image then rsync makes sense.
>
> Rich.
>
Also rsync does not do anything magical/differential when dealing with 
local copies. Even an update is still a full normal copy because there 
is no benefit to the differential processing (the same disk(s) have to 
be read and written just as much and it all has to pass through the same 
cpu and busses. You can force it by doing a network transfer over 
localhost, but "ya canna change the laws o physics" And, not only is 
there no differential magic benefit, there is still a major speed loss. 
rsync is usually a lot slower than the same tar/cpio/cp doing full 
copies, even over networks, even using -W and --inplace.

Definitely the point of using rsync locally is not speed of the data 
transfer itself, but speed of the the user.

The main benefit is just familiarity and convenience with the rsync 
command line arguments and behavior. You spend a lot of time learning 
how to get various common tasks done with rsync and it is really quite a 
swiss army knife. And you develop trust and confidence with it. You know 
exactly what will happen when you run a given command because you do the 
same thing routinely. Those features are handy enough to be worth taking 
the speed hit in return for not having to develop complicated, entirely 
different, find/cp/cpio commands that you use less often and are less 
sure about what they'll actually do when you press enter.

You could argue the find/cpio is better for writing into scripts and 
apps since they are a more universal dependency as well as the 
performance reason, but you could also argue that it's an advantage to 
use a command that works equally between any to points, be they local or 
remote, making your script/app more useful in more unpredicted 
situations for free.

-- 
bkw