Richard W.M. Jones
2010-Apr-10 12:47 UTC
[Libguestfs] [PATCH 0/9] Enhance virt-resize so it can really expand Linux and Windows guests
This is a set of bugfixes and enhancements to allow virt-resize to really expand Linux and Windows guests. Previously there were lots of bugs. This version has been tested on a variety of guests successfully. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://et.redhat.com/~rjones/libguestfs/ See what it can do: http://et.redhat.com/~rjones/libguestfs/recipes.html
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://et.redhat.com/~rjones/libguestfs/ See what it can do: http://et.redhat.com/~rjones/libguestfs/recipes.html -------------- next part -------------->From f8210da95888f0060b25dcca2ae299d5c16ff814 Mon Sep 17 00:00:00 2001From: Richard Jones <rjones at redhat.com> Date: Sat, 10 Apr 2010 13:13:07 +0100 Subject: [PATCH 1/9] New API: ntfsresize. This implements the ntfsresize operation, using the external program from ntfsprogs. --- daemon/ntfs.c | 23 +++++++++++++++++++++++ src/generator.ml | 8 ++++++++ 2 files changed, 31 insertions(+), 0 deletions(-) diff --git a/daemon/ntfs.c b/daemon/ntfs.c index 73ccbbc..8938dbd 100644 --- a/daemon/ntfs.c +++ b/daemon/ntfs.c @@ -35,6 +35,13 @@ optgroup_ntfs3g_available (void) } int +optgroup_ntfsprogs_available (void) +{ + int r = access ("/usr/sbin/ntfsresize", X_OK); + return r == 0; +} + +int do_ntfs_3g_probe (int rw, const char *device) { char *err; @@ -52,3 +59,19 @@ do_ntfs_3g_probe (int rw, const char *device) return r; } + +int +do_ntfsresize (const char *device) +{ + char *err; + int r; + + r = command (NULL, &err, "ntfsresize", "-P", device, NULL); + if (r == -1) { + reply_with_error ("%s: %s", device, err); + free (err); + return -1; + } + + return 0; +} diff --git a/src/generator.ml b/src/generator.ml index 9d19097..24b9bc0 100755 --- a/src/generator.ml +++ b/src/generator.ml @@ -4373,6 +4373,14 @@ I<xz compressed> tar file) into C<directory>."); This command packs the contents of C<directory> and downloads it to local file C<tarball> (as an xz compressed tar archive)."); + ("ntfsresize", (RErr, [Device "device"]), 231, [Optional "ntfsprogs"], + [], + "resize an NTFS filesystem", + "\ +This command resizes an NTFS filesystem, expanding or +shrinking it to the size of the underlying device. +See also L<ntfsresize(8)>."); + ] let all_functions = non_daemon_functions @ daemon_functions -- 1.6.6.1
This isn't related to the patch, but since I wrote it and it's a useful function, I didn't want to drop the feature. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora -------------- next part -------------->From 9752039e52d190c7d62281346fd2170a98434c86 Mon Sep 17 00:00:00 2001From: Richard Jones <rjones at redhat.com> Date: Sat, 10 Apr 2010 13:14:12 +0100 Subject: [PATCH 2/9] New API: vgscan Implement vgscan to allow for a full rescan of all LVM PVs, VGs and LVs. --- daemon/lvm.c | 18 ++++++++++++++++++ src/generator.ml | 8 ++++++++ 2 files changed, 26 insertions(+), 0 deletions(-) diff --git a/daemon/lvm.c b/daemon/lvm.c index e4fa54e..20f2fb3 100644 --- a/daemon/lvm.c +++ b/daemon/lvm.c @@ -594,3 +594,21 @@ do_vglvuuids (const char *vgname) { return get_lvm_fields ("vgs", "lv_uuid", vgname); } + +int +do_vgscan (void) +{ + char *err; + int r; + + r = command (NULL, &err, + "lvm", "vgscan", NULL); + if (r == -1) { + reply_with_error ("%s", err); + free (err); + return -1; + } + + free (err); + return 0; +} diff --git a/src/generator.ml b/src/generator.ml index 24b9bc0..b423025 100755 --- a/src/generator.ml +++ b/src/generator.ml @@ -4381,6 +4381,14 @@ This command resizes an NTFS filesystem, expanding or shrinking it to the size of the underlying device. See also L<ntfsresize(8)>."); + ("vgscan", (RErr, []), 232, [], + [InitEmpty, Always, TestRun ( + [["vgscan"]])], + "rescan for LVM physical volumes, volume groups and logical volumes", + "\ +This rescans all block devices and rebuilds the list of LVM +physical volumes, volume groups and logical volumes."); + ] let all_functions = non_daemon_functions @ daemon_functions -- 1.6.6.1
Richard W.M. Jones
2010-Apr-10 12:51 UTC
[Libguestfs] [PATCH 3/9] Documentation: Clarify documentation on the bootable flag.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top -------------- next part -------------->From 30752ac7ca042f533b8d0c4c6144400a8532b154 Mon Sep 17 00:00:00 2001From: Richard Jones <rjones at redhat.com> Date: Sat, 10 Apr 2010 13:14:54 +0100 Subject: [PATCH 3/9] Documentation: Clarify documentation on the bootable flag. --- src/generator.ml | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/src/generator.ml b/src/generator.ml index b423025..2b833c1 100755 --- a/src/generator.ml +++ b/src/generator.ml @@ -4095,10 +4095,9 @@ but other possible values are described in C<guestfs_part_init>."); This sets the bootable flag on partition numbered C<partnum> on device C<device>. Note that partitions are numbered from 1. -The bootable flag is used by some PC BIOSes to determine which -partition to boot from. It is by no means universally recognized, -and in any case if your operating system installed a boot -sector on the device itself, then that takes precedence."); +The bootable flag is used by some operating systems (notably +Windows) to determine which partition to boot from. It is by +no means universally recognized."); ("part_set_name", (RErr, [Device "device"; Int "partnum"; String "name"]), 212, [], [InitEmpty, Always, TestRun ( -- 1.6.6.1
Richard W.M. Jones
2010-Apr-10 12:52 UTC
[Libguestfs] [PATCH 4/9] daemon: Make the RUN_PARTED macro take an error statement.
Turns out we don't use this either. However still a nice change. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top -------------- next part -------------->From 440ad646cdf31bdb5ad2bc92fc51fa6df3fb9c63 Mon Sep 17 00:00:00 2001From: Richard Jones <rjones at redhat.com> Date: Sat, 10 Apr 2010 13:16:19 +0100 Subject: [PATCH 4/9] daemon: Make the RUN_PARTED macro take an error statement. This allows us to make the RUN_PARTED macro do something else along the error path, other than just returning -1. --- daemon/parted.c | 16 +++++++++------- 1 files changed, 9 insertions(+), 7 deletions(-) diff --git a/daemon/parted.c b/daemon/parted.c index 99417c2..ff6cca1 100644 --- a/daemon/parted.c +++ b/daemon/parted.c @@ -55,7 +55,7 @@ recover_blkrrpart (const char *device, const char *err) return 0; } -#define RUN_PARTED(device,...) \ +#define RUN_PARTED(error,device,...) \ do { \ int r; \ char *err; \ @@ -66,7 +66,7 @@ recover_blkrrpart (const char *device, const char *err) if (recover_blkrrpart ((device), err) == -1) { \ reply_with_error ("%s: parted: %s: %s", __func__, (device), err); \ free (err); \ - return -1; \ + error; \ } \ } \ \ @@ -107,7 +107,7 @@ do_part_init (const char *device, const char *parttype) return -1; } - RUN_PARTED (device, "mklabel", parttype, NULL); + RUN_PARTED (return -1, device, "mklabel", parttype, NULL); udev_settle (); @@ -151,7 +151,7 @@ do_part_add (const char *device, const char *prlogex, * name_ to prlogex, eg. "primary". I would essentially describe * this as a bug in the parted mkpart command. */ - RUN_PARTED (device, "mkpart", prlogex, startstr, endstr, NULL); + RUN_PARTED (return -1, device, "mkpart", prlogex, startstr, endstr, NULL); udev_settle (); @@ -183,7 +183,8 @@ do_part_disk (const char *device, const char *parttype) endstr = "-1s"; } - RUN_PARTED (device, + RUN_PARTED (return -1, + device, "mklabel", parttype, /* See comment about about the parted mkpart command. */ "mkpart", STREQ (parttype, "gpt") ? "p1" : "primary", @@ -201,7 +202,8 @@ do_part_set_bootable (const char *device, int partnum, int bootable) snprintf (partstr, sizeof partstr, "%d", partnum); - RUN_PARTED (device, "set", partstr, "boot", bootable ? "on" : "off", NULL); + RUN_PARTED (return -1, + device, "set", partstr, "boot", bootable ? "on" : "off", NULL); udev_settle (); @@ -215,7 +217,7 @@ do_part_set_name (const char *device, int partnum, const char *name) snprintf (partstr, sizeof partstr, "%d", partnum); - RUN_PARTED (device, "name", partstr, name, NULL); + RUN_PARTED (return -1, device, "name", partstr, name, NULL); udev_settle (); -- 1.6.6.1
Richard W.M. Jones
2010-Apr-10 12:52 UTC
[Libguestfs] [PATCH 5/9] daemon: More reliable parsing of the output from 'parted print'.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw -------------- next part -------------->From cb9350019cc6382a35c98f522c9c4d221c92b605 Mon Sep 17 00:00:00 2001From: Richard Jones <rjones at redhat.com> Date: Sat, 10 Apr 2010 13:17:50 +0100 Subject: [PATCH 5/9] daemon: More reliable parsing of the output from 'parted print'. Previously we used strtok. However this has the problem that strtok considers multiple delimiter characters to be like a single delimiter, eg. "1:::2" would be parsed the same as "1:2". In other words, the previous code would skip over or fail if there are empty fields. --- daemon/parted.c | 52 +++++++++++++++++++++++++++++++++------------------- 1 files changed, 33 insertions(+), 19 deletions(-) diff --git a/daemon/parted.c b/daemon/parted.c index ff6cca1..b1750a5 100644 --- a/daemon/parted.c +++ b/daemon/parted.c @@ -224,6 +224,36 @@ do_part_set_name (const char *device, int partnum, const char *name) return 0; } +/* Return the nth field from a string of ':'/';'-delimited strings. + * Useful for parsing the return value from print_partition_table + * function below. + */ +static char * +get_table_field (const char *line, int n) +{ + const char *p = line; + + while (*p && n > 0) { + p += strcspn (p, ":;") + 1; + n--; + } + + if (n > 0) { + reply_with_error ("not enough fields in output of parted print command: %s", + line); + return NULL; + } + + size_t len = strcspn (p, ":;"); + char *q = strndup (p, len); + if (q == NULL) { + reply_with_perror ("strndup"); + return NULL; + } + + return q; +} + static char ** print_partition_table (const char *device) { @@ -269,31 +299,15 @@ print_partition_table (const char *device) char * do_part_get_parttype (const char *device) { - char **lines; - char *r; - - lines = print_partition_table (device); + char **lines = print_partition_table (device); if (!lines) return NULL; /* lines[1] is something like: * "/dev/sda:1953525168s:scsi:512:512:msdos:ATA Hitachi HDT72101;" */ - if (strtok (lines[1], ":") == NULL /* device */ - || strtok (NULL, ":") == NULL /* size */ - || strtok (NULL, ":") == NULL /* transport */ - || strtok (NULL, ":") == NULL /* sector size */ - || strtok (NULL, ":") == NULL /* physical sector size */ - || (r = strtok (NULL, ":")) == NULL /* return value */ - ) { - reply_with_error ("too few fields in output from parted print command: %s", lines[1]); - free_strings (lines); - return NULL; - } - - r = strdup (r); - if (!r) { - reply_with_perror ("strdup"); + char *r = get_table_field (lines[1], 5); + if (r == NULL) { free_strings (lines); return NULL; } -- 1.6.6.1
Richard W.M. Jones
2010-Apr-10 12:53 UTC
[Libguestfs] [PATCH 6/9] New partition APIs: part_del, part_get_bootable, part_get/set_mbr_id.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw -------------- next part -------------->From b68c030adfdbefe65bc9ecdd673844c01bddb32a Mon Sep 17 00:00:00 2001From: Richard Jones <rjones at redhat.com> Date: Sat, 10 Apr 2010 13:19:43 +0100 Subject: [PATCH 6/9] New partition APIs: part_del, part_get_bootable, part_get/set_mbr_id These APIs flesh out further the partitioning API. --- daemon/parted.c | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/MAX_PROC_NR | 2 +- src/generator.ml | 55 +++++++++++++++++++++++++++++ 3 files changed, 157 insertions(+), 1 deletions(-) diff --git a/daemon/parted.c b/daemon/parted.c index b1750a5..9c65570 100644 --- a/daemon/parted.c +++ b/daemon/parted.c @@ -159,6 +159,18 @@ do_part_add (const char *device, const char *prlogex, } int +do_part_del (const char *device, int partnum) +{ + char partnum_str[16]; + snprintf (partnum_str, sizeof partnum_str, "%d", partnum); + + RUN_PARTED (return -1, device, "rm", partnum_str, NULL); + + udev_settle (); + return 0; +} + +int do_part_disk (const char *device, const char *parttype) { const char *startstr; @@ -371,3 +383,92 @@ do_part_list (const char *device) free_strings (lines); return NULL; } + +int +do_part_get_bootable (const char *device, int partnum) +{ + char **lines = print_partition_table (device); + if (!lines) + return -1; + + /* We want lines[1+partnum]. */ + if (count_strings (lines) < 1+partnum) { + reply_with_error ("partition number out of range: %d", partnum); + free_strings (lines); + return -1; + } + + char *boot = get_table_field (lines[1+partnum], 6); + if (boot == NULL) { + free_strings (lines); + return -1; + } + + int r = STREQ (boot, "boot"); + + free (boot); + free_strings (lines); + + return r; +} + +/* Currently we use sfdisk for getting and setting the ID byte. In + * future, extend parted to provide this functionality. As a result + * of using sfdisk, this won't work for non-MBR-style partitions, but + * that limitation is noted in the documentation and we can extend it + * later without breaking the ABI. + */ +int +do_part_get_mbr_id (const char *device, int partnum) +{ + char partnum_str[16]; + snprintf (partnum_str, sizeof partnum_str, "%d", partnum); + + char *out, *err; + int r; + + r = command (&out, &err, "sfdisk", "--print-id", device, partnum_str, NULL); + if (r == -1) { + reply_with_error ("sfdisk --print-id: %s", err); + free (out); + free (err); + return -1; + } + + free (err); + + /* It's printed in hex ... */ + int id; + if (sscanf (out, "%x", &id) != 1) { + reply_with_error ("sfdisk --print-id: cannot parse output: %s", out); + free (out); + return -1; + } + + free (out); + return id; +} + +int +do_part_set_mbr_id (const char *device, int partnum, int idbyte) +{ + char partnum_str[16]; + snprintf (partnum_str, sizeof partnum_str, "%d", partnum); + + char idbyte_str[16]; + snprintf (idbyte_str, sizeof partnum_str, "%x", idbyte); /* NB: hex */ + + char *err; + int r; + + r = command (NULL, &err, "sfdisk", + "--change-id", device, partnum_str, idbyte_str, NULL); + if (r == -1) { + reply_with_error ("sfdisk --change-id: %s", err); + free (err); + return -1; + } + + free (err); + return 0; +} diff --git a/src/MAX_PROC_NR b/src/MAX_PROC_NR index dcb6b5b..f1f094b 100644 --- a/src/MAX_PROC_NR +++ b/src/MAX_PROC_NR @@ -1 +1 @@ -230 +236 diff --git a/src/generator.ml b/src/generator.ml index 2b833c1..50ae91d 100755 --- a/src/generator.ml +++ b/src/generator.ml @@ -4388,6 +4388,61 @@ See also L<ntfsresize(8)>."); This rescans all block devices and rebuilds the list of LVM physical volumes, volume groups and logical volumes."); + ("part_del", (RErr, [Device "device"; Int "partnum"]), 233, [], + [InitEmpty, Always, TestRun ( + [["part_init"; "/dev/sda"; "mbr"]; + ["part_add"; "/dev/sda"; "primary"; "1"; "-1"]; + ["part_del"; "/dev/sda"; "1"]])], + "delete a partition", + "\ +This command deletes the partition numbered C<partnum> on C<device>. + +Note that in the case of MBR partitioning, deleting an +extended partition also deletes any logical partitions +it contains."); + + ("part_get_bootable", (RBool "bootable", [Device "device"; Int "partnum"]), 234, [], + [InitEmpty, Always, TestOutputTrue ( + [["part_init"; "/dev/sda"; "mbr"]; + ["part_add"; "/dev/sda"; "primary"; "1"; "-1"]; + ["part_set_bootable"; "/dev/sda"; "1"; "true"]; + ["part_get_bootable"; "/dev/sda"; "1"]])], + "return true if a partition is bootable", + "\ +This command returns true if the partition C<partnum> on +C<device> has the bootable flag set. + +See also C<guestfs_part_set_bootable>."); + + ("part_get_mbr_id", (RInt "idbyte", [Device "device"; Int "partnum"]), 235, [], + [InitEmpty, Always, TestOutputInt ( + [["part_init"; "/dev/sda"; "mbr"]; + ["part_add"; "/dev/sda"; "primary"; "1"; "-1"]; + ["part_set_mbr_id"; "/dev/sda"; "1"; "0x7f"]; + ["part_get_mbr_id"; "/dev/sda"; "1"]], 0x7f)], + "get the MBR type byte (ID byte) from a partition", + "\ +Returns the MBR type byte (also known as the ID byte) from +the numbered partition C<partnum>. + +Note that only MBR (old DOS-style) partitions have type bytes. +You will get undefined results for other partition table +types (see C<guestfs_part_get_parttype>)."); + + ("part_set_mbr_id", (RErr, [Device "device"; Int "partnum"; Int "idbyte"]), 236, [], + [], (* tested by part_get_mbr_id *) + "set the MBR type byte (ID byte) of a partition", + "\ +Sets the MBR type byte (also known as the ID byte) of +the numbered partition C<partnum> to C<idbyte>. Note +that the type bytes quoted in most documentation are +in fact hexadecimal numbers, but usually documented +without any leading \"0x\" which might be confusing. + +Note that only MBR (old DOS-style) partitions have type bytes. +You will get undefined results for other partition table +types (see C<guestfs_part_get_parttype>)."); + ] let all_functions = non_daemon_functions @ daemon_functions -- 1.6.6.1
Richard W.M. Jones
2010-Apr-10 12:53 UTC
[Libguestfs] [PATCH 7/9] Bugfixes for virt-resize.
Misc. bugfixes for virt-resize which are separated out from the other enhancements so we can cherry pick them for the stable branch. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v -------------- next part -------------->From add5e22563972210f5c8baf9e8cf651fc1a7bbd8 Mon Sep 17 00:00:00 2001From: Richard Jones <rjones at redhat.com> Date: Sat, 10 Apr 2010 13:35:31 +0100 Subject: [PATCH 7/9] Bugfixes for virt-resize. - copy more than 64 boot loader sectors across, since real boot loaders (eg. for Windows) can be much larger than this - copy bootable flag and ID byte to new partitions - start the first partition on the new disk at the same sector offset as on the old disk - sync the disks before existing --- tools/virt-resize | 88 +++++++++++++++++++++++++++++++++++++++++----------- 1 files changed, 69 insertions(+), 19 deletions(-) diff --git a/tools/virt-resize b/tools/virt-resize index 1c4006a..74f13b1 100755 --- a/tools/virt-resize +++ b/tools/virt-resize @@ -411,12 +411,23 @@ if ($debug) { print "$outfile size $outsize bytes\n"; } +# In reality the number of sectors containing boot loader data will be +# less than this (although Windows 7 defaults to putting the first +# partition on sector 2048, and has quite a large boot loader). +# +# However make this large enough to be sure that we have copied over +# the boot loader. We could also do this by looking for the sector +# offset of the first partition. +# +# It doesn't matter if we copy too much. +my $boot_sectors = 4096; + die __x("virt-resize: {file}: file is too small to be a disk image ({sz} bytes)\n", file => $infile, sz => $insize) - if $insize < 64 * 512; + if $insize < $boot_sectors * 512; die __x("virt-resize: {file}: file is too small to be a disk image ({sz} bytes)\n", file => $outfile, sz => $outsize) - if $outsize < 64 * 512; + if $outsize < $boot_sectors * 512; # Copy the boot loader across. do_copy_boot_loader () if $copy_boot_loader; @@ -426,12 +437,12 @@ sub do_copy_boot_loader print "copying boot loader ...\n" if $debug; open IFILE, $infile or die "$infile: $!"; my $s; - my $r = sysread (IFILE, $s, 64 * 512) or die "$infile: $!"; - die "$infile: short read" if $r < 64 * 512; + my $r = sysread (IFILE, $s, $boot_sectors * 512) or die "$infile: $!"; + die "$infile: short read" if $r < $boot_sectors * 512; open OFILE, "+<$outfile" or die "$outfile: $!"; sysseek OFILE, 0, SEEK_SET or die "$outfile: seek: $!"; - $r = syswrite (OFILE, $s, 64 * 512) or die "$outfile: $!"; - die "$outfile: short write" if $r < 64 * 512; + $r = syswrite (OFILE, $s, $boot_sectors * 512) or die "$outfile: $!"; + die "$outfile: short write" if $r < $boot_sectors * 512; } # Add them to the handle and launch the appliance. @@ -466,6 +477,8 @@ sub check_source_disk my %h = %$_; $h{name} = $name; + $h{bootable} = $g->part_get_bootable ("/dev/sda", $h{part_num}); + eval { $h{mbr_id} = $g->part_get_mbr_id ("/dev/sda", $h{part_num}); }; $partitions{$name} = \%h; } } @@ -676,7 +689,11 @@ sub calculate_surplus # We need some overhead for partitioning. Worst case would be for # EFI partitioning + massive per-partition alignment. - my $overhead = $sectsize * (2 * 64 + (64 * (@partitions + 1)) + 128); + my $overhead = $sectsize * ( + 2 * 64 + # GPT start and end + (64 * (@partitions + 1)) + # Maximum alignment + ($boot_sectors - 64) # Boot loader + ); my $required = 0; foreach (@partitions) { @@ -777,17 +794,36 @@ sub repartition if ($copy_boot_loader) { $parttype = $g->part_get_parttype ("/dev/sdb"); - print "partition table type: $parttype\n" if $debug; } else { - # Didn't copy over the initial boot loader, so we need - # to make a new partition type here. $parttype = "efi"; } + print "partition table type: $parttype\n" if $debug; - # Delete any existing partitions on the destination disk. - $g->part_init ("/dev/sdb", $parttype); + # Delete any existing partitions on the destination disk, + # but leave the bootloader that we copied over intact. + if ($copy_boot_loader) { + # Delete in reverse as an easy way to deal with extended + # partitions. + foreach (sort { $b cmp $a } $g->list_partitions ()) { + if (m{^/dev/.db(\d+)$}) { + $g->part_del ("/dev/sdb", $1); + } + } + } else { + # Didn't copy over the initial boot loader, so we need + # to make a new partition table here. + $g->part_init ("/dev/sdb", $parttype); + } + + # Work out where to start the first partition. + die __"virt-resize: source disk does not have a first partition\n" + unless exists ($partitions{"/dev/sda1"}); + my $start = $partitions{"/dev/sda1"}->{part_start} / $sectsize; - my $start = 64; + # Align to 64. + $start = ($start + 63) & ~63; + + print "starting to partition from $start\n" if $debug; # Create the new partitions. foreach my $part (@partitions) { @@ -803,9 +839,18 @@ sub repartition } # Create it. - my ($target, $end) = add_partition ($start, $size); + my ($target, $end, $part_num) = add_partition ($start, $size); $partitions{$part}->{target} = $target; + if ($partitions{$part}->{bootable}) { + $g->part_set_bootable ("/dev/sdb", $part_num, 1); + } + + if ($partitions{$part}->{mbr_id}) { + $g->part_set_mbr_id ("/dev/sdb", $part_num, + $partitions{$part}->{mbr_id}); + } + # Start of next partition + alignment. $start = $end + 1; $start = ($start + 63) & ~63; @@ -825,26 +870,26 @@ sub add_partition my $start = shift; my $size = shift; - my ($target, $end); + my ($target, $end, $part_num); if ($nextpart <= 3 || $parttype ne "msdos") { $target = "/dev/sdb$nextpart"; $end = $start + $size - 1; $g->part_add ("/dev/sdb", "primary", $start, $end); - $nextpart++; + $part_num = $nextpart++; } else { if ($nextpart == 4) { $g->part_add ("/dev/sdb", "extended", $start, -1); - $nextpart++; + $part_num = $nextpart++; $start += 64; } $target = "/dev/sdb$nextpart"; $end = $start + $size - 1; $g->part_add ("/dev/sdb", "logical", $start, $end); - $nextpart++; + $part_num = $nextpart++; } - return ($target, $end); + return ($target, $end, $part_num); } # Copy over the data. @@ -881,6 +926,11 @@ sub copy_data } } +# Sync disk and exit. +$g->umount_all (); +$g->sync (); +undef $g; + exit 0; sub sizebytes -- 1.6.6.1
Richard W.M. Jones
2010-Apr-10 12:54 UTC
[Libguestfs] [PATCH 8/9] virt-list-partitions: Add -t / --total option.
Note the enhanced virt-resize documentation (in patch 9/9) will mention this -t option. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora -------------- next part -------------->From e6bdd66fa6bdfa4e762f3c67ec1bac83a1d6643b Mon Sep 17 00:00:00 2001From: Richard Jones <rjones at redhat.com> Date: Sat, 10 Apr 2010 13:21:33 +0100 Subject: [PATCH 8/9] virt-list-partitions: Add -t / --total option. This option lists out block devices separately: $ virt-list-partitions -lht /tmp/WindowsResized.img /dev/sda1 ntfs 100.0M /dev/sda2 ntfs 11.9G /dev/sda device 12.0G --- tools/virt-list-partitions | 76 ++++++++++++++++++++++++++++++++----------- 1 files changed, 56 insertions(+), 20 deletions(-) diff --git a/tools/virt-list-partitions b/tools/virt-list-partitions index b8bc0cc..0fec872 100755 --- a/tools/virt-list-partitions +++ b/tools/virt-list-partitions @@ -82,6 +82,14 @@ at all. =cut +my $human; + +=item B<-h> | B<--human-readable> + +Show sizes in human-readable form (eg. "1G"). + +=cut + my $long; =item B<-l> | B<--long> @@ -91,11 +99,12 @@ and size of each partition too (where "type" means C<ext3>, C<pv> etc.) =cut -my $human; +my $total; -=item B<-h> | B<--human-readable> +=item B<-t> | B<--total> -Show sizes in human-readable form (eg. "1G"). +Display the total size of each block device (as a separate row or +rows). =back @@ -107,8 +116,9 @@ Getopt::Long::Configure ("bundling"); GetOptions ("help|?" => \$help, "version" => \$version, "connect|c=s" => \$uri, - "long|l" => \$long, "human-readable|h" => \$human, + "long|l" => \$long, + "total|t" => \$total, ) or pod2usage (2); pod2usage (1) if $help; if ($version) { @@ -131,44 +141,70 @@ if ($uri) { $g->launch (); # List of partitions and sizes. -my @partitions; -my @devices = $g->list_devices (); -foreach my $dev (@devices) { - my @p = $g->part_list ($dev); - foreach (@p) { - $_->{name} = canonicalize ("$dev" . $_->{part_num}); - push @partitions, $_; - } -} +my @partitions = $g->list_partitions (); +foreach my $name (@partitions) { + $name = canonicalize ($name); -# Print them. -foreach my $part (@partitions) { - print $part->{name}; + print $name; if ($long) { my $type; eval { - $type = $g->vfs_type ($part->{name}); + $type = $g->vfs_type ($name); }; $type ||= "unknown"; $type = "pv" if $type eq "LVM2_member"; print " $type "; + + my $size; + eval { + $size = $g->blockdev_getsize64 ($name); + }; + $size ||= "unknown"; + if ($human) { - print (human_size($part->{part_size})); + print (human_size($size)); } else { - print $part->{part_size}; + print $size; } } print "\n"; } +if ($total) { + # List of devices and sizes. + my @devices = $g->list_devices (); + foreach my $name (@devices) { + $name = canonicalize ($name); + + print $name; + + if ($long) { + print " device "; + + my $size; + eval { + $size = $g->blockdev_getsize64 ($name); + }; + $size ||= "unknown"; + + if ($human) { + print (human_size($size)); + } else { + print $size; + } + } + print "\n"; + } +} + # The reverse of device name translation, see # BLOCK DEVICE NAMING in guestfs(3). sub canonicalize { local $_ = shift; - if (m{^/dev/[hv]d([a-z]\d)$}) { + if (m{^/dev/[hv]d([a-z]+\d*)$}) { return "/dev/sd$1"; } $_; -- 1.6.6.1
Richard W.M. Jones
2010-Apr-10 12:55 UTC
[Libguestfs] [PATCH 9/9] virt-resize: Enhance virt-resize so it can expand partition content.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v -------------- next part -------------->From 5466638279b51d46e6b24d4f7148d520cb4f3c34 Mon Sep 17 00:00:00 2001From: Richard Jones <rjones at redhat.com> Date: Sat, 10 Apr 2010 13:38:26 +0100 Subject: [PATCH 9/9] virt-resize: Enhance virt-resize so it can expand partition content. Enhance virt-resize so it can expand "first level" partition content, including ext/2/3/4/ntfs filesystems and PVs. Also extensively update the documentation. This has been tested on a variety of Linux and Windows guests. --- tools/virt-resize | 321 +++++++++++++++++++++++++++++++++++++++++++++-------- 1 files changed, 273 insertions(+), 48 deletions(-) diff --git a/tools/virt-resize b/tools/virt-resize index 74f13b1..fbbf7f6 100755 --- a/tools/virt-resize +++ b/tools/virt-resize @@ -20,6 +20,7 @@ use warnings; use strict; use Sys::Guestfs; +use Sys::Guestfs::Lib qw(feature_available); use Fcntl qw(S_ISREG SEEK_SET); use POSIX qw(floor); use Pod::Usage; @@ -40,9 +41,9 @@ virt-resize - Resize a virtual machine disk =head1 SYNOPSIS - virt-resize [--resize /dev/sdaN=[+/-]<size>[%]] [--expand /dev/sdaN] - [--shrink /dev/sdaN] [--ignore /dev/sdaN] [--delete /dev/sdaN] [...] - indisk outdisk + virt-resize [--resize /dev/sdaN=[+/-]<size>[%]] + [--expand /dev/sdaN] [--shrink /dev/sdaN] + [--ignore /dev/sdaN] [--delete /dev/sdaN] [...] indisk outdisk =head1 DESCRIPTION @@ -60,41 +61,42 @@ L<virt-list-filesystems(1)> and L<virt-df(1)>, we recommend you go and read those manual pages first. -=head2 BASIC USAGE +=head1 BASIC USAGE -This describes the common case where you want to expand an image to -give your guest more space. Shrinking images is considerably more -complicated (unfortunately). +=head2 EXPANDING A VIRTUAL MACHINE DISK =over 4 -=item 1. Locate disk image +=item 1. Shut down the virtual machine -Locate the disk image that you want to resize. It could be in a local -file or device. If the guest is managed by libvirt, you can use -C<virsh dumpxml> like this to find the disk image name: +=item 2. Locate input disk image + +Locate the input disk image (ie. the file or device on the host +containing the guest's disk). If the guest is managed by libvirt, you +can use C<virsh dumpxml> like this to find the disk image name: # virsh dumpxml guestname | xpath /domain/devices/disk/source Found 1 nodes: -- NODE -- <source dev="/dev/vg/lv_guest" /> -=item 2. Look at current sizing +=item 3. Look at current sizing Use L<virt-list-partitions(1)> to display the current partitions and sizes: - # virt-list-partitions -lh /dev/vg/lv_guest + # virt-list-partitions -lht /dev/vg/lv_guest /dev/sda1 ext3 101.9M /dev/sda2 pv 7.9G + /dev/sda device 8.0G (This example is a virtual machine with an 8 GB disk which we would like to expand up to 10 GB). -=item 3. Create destination disk +=item 4. Create output disk Virt-resize cannot do in-place disk modifications. You have to have -space to store the resized destination disk. +space to store the resized output disk. To store the resized disk image in a file, create a file of a suitable size: @@ -102,7 +104,7 @@ size: # rm -f outdisk # truncate -s 10G outdisk -Use L<lvcreate(1)> to create a logical volume: +Or use L<lvcreate(1)> to create a logical volume: # lvcreate -L 10G -n lv_name vg_name @@ -111,9 +113,13 @@ Or use L<virsh(1)> vol-create-as to create a libvirt storage volume: # virsh pool-list # virsh vol-create-as poolname newvol 10G -=item 4. Resize +=item 5. Resize + +virt-resize takes two mandatory parameters, the input disk (eg. device +or file) and the output disk. The output disk is the one created in +the previous step. - virt-resize indisk outdisk + # virt-resize indisk outdisk This command just copies disk image C<indisk> to disk image C<outdisk> I<without> resizing or changing any existing partitions. If @@ -121,32 +127,37 @@ C<outdisk> is larger, then an extra, empty partition is created at the end of the disk covering the extra space. If C<outdisk> is smaller, then it will give an error. -To resize, you need to pass extra options (for the full list see the +More realistically you'd want to expand existing partitions in the +disk image by passing extra options (for the full list see the L</OPTIONS> section below). L</--expand> is the most useful option. It expands the named partition within the disk to fill any extra space: - virt-resize --expand /dev/sda2 indisk outdisk + # virt-resize --expand /dev/sda2 indisk outdisk (In this case, an extra partition is I<not> created at the end of the disk, because there will be no unused space). -If /dev/sda2 in the image contains a filesystem or LVM PV, then -this content is B<not> automatically resized. You can resize it -afterwards either using L<guestfish(1)> (offline) or using commands -inside the guest (online resizing). - L</--resize> is the other commonly used option. The following would increase the size of /dev/sda1 by 200M, and expand /dev/sda2 to fill the rest of the available space: - virt-resize --resize /dev/sda1=+200M --expand /dev/sda2 \ - indisk outdisk + # virt-resize --resize /dev/sda1=+200M --expand /dev/sda2 \ + indisk outdisk + +If the expanded partition in the image contains a filesystem or LVM +PV, then if virt-resize knows how, it will resize the contents, the +equivalent of calling a command such as L<pvresize(8)>, +L<resize2fs(8)> or L<ntfsresize(8)>. However virt-resize does not +know how to resize some filesystems, so you would have to online +resize them after booting the guest. And virt-resize also does not +resize anything inside an LVM PV, it just resizes the PV itself and +leaves the user to resize any LVs inside that PV as desired. Other options are covered below. -=item 5. Test +=item 6. Test Thoroughly test the new disk image I<before> discarding the old one. @@ -161,17 +172,47 @@ Then start up the domain with the new, resized disk: # virsh start guestname -and check that it still works. +and check that it still works. See also the L</NOTES> section below +for additional information. + +=item 7. Resize LVs etc inside the guest + +(This can also be done offline using L<guestfish(1)>) -Note that to see the extra space in the guest, you may need to use -guest commands to resize PVs, LVs and/or filesystems to fit the extra -space available. Three common guest commands for doing this for Linux -guests are L<pvresize(8)>, L<lvresize(8)> and L<resize2fs(8)>. It is -also possible to do this offline (eg. for scripting changes) using -L<guestfish(1)>. +Once the guest has booted you should see the new space available, at +least for filesystems that virt-resize knows how to resize, and for +PVs. The user may need to resize LVs inside PVs, and also resize +filesystem types that virt-resize does not know how to expand. =back +=head2 SHRINKING A VIRTUAL MACHINE DISK + +Shrinking is somewhat more complex than expanding, and only an +overview is given here. + +Firstly virt-resize will not attempt to shrink any partition content +(PVs, filesystems). The user has to shrink content before passing the +disk image to virt-resize, and virt-resize will check that the content +has been shrunk properly. + +(Shrinking can also be done offline using L<guestfish(1)>) + +After shrinking PVs and filesystems, shut down the guest, and proceed +with steps 3 and 4 above to allocate a new disk image. + +Then run virt-resize with any of the C<--shrink> and/or C<--resize> +options. + +=head2 IGNORING OR DELETING PARTITIONS + +virt-resize also gives a convenient way to ignore or delete partitions +when copying from the input disk to the output disk. Ignoring a +partition speeds up the copy where you don't care about the existing +contents of a partition. Deleting a partition removes it completely, +but note that it also renumbers any partitions after the one which is +deleted, which can leave some guests unbootable. + =head1 OPTIONS =over 4 @@ -220,9 +261,11 @@ size; or as a relative number or percentage. For example: --resize /dev/sda1=-10% -You can increase the size of any partition. +You can increase the size of any partition. Virt-resize will expand +the direct content of the partition if it knows how (see C<--expand> +below). -You can I<only> B<decrease> the size of partitions that contain +You can only I<decrease> the size of partitions that contain filesystems or PVs which have already been shrunk. Virt-resize will check this has been done before proceeding, or else will print an error (see also C<--resize-force>). @@ -252,9 +295,37 @@ my $expand; Expand the named partition so it uses up all extra space (space left over after any other resize changes that you request have been done). -Any filesystem inside the partition is I<not> expanded. You will need -to expand the filesystem (or PV) to fit the extra space either using -L<guestfish(1)> (offline) or online guest tools. +If virt-resize knows how, it will expand the direct content of the +partition. For example, if the partition is an LVM PV, it will expand +the PV to fit (like calling L<pvresize(8)>). Virt-resize leaves any +other content it doesn't know about alone. + +Currently virt-resize can resize: + +=over 4 + +=item * + +ext2, ext3 and ext4 filesystems when they are contained +directly inside a partition. + +=item * + +NTFS filesystems contained directly in a partition, if libguestfs was +compiled with support for NTFS. + +The filesystem must have been shut down consistently last time it was +used. Additionally, L<ntfsresize(8)> marks the resized filesystem as +requiring a consistency check, so at the first boot after resizing +Windows will check the disk. + +=item * + +LVM PVs (physical volumes). However virt-resize does I<not> +resize anything inside the PV. The user will have to resize +LVs as desired. + +=back Note that you cannot use C<--expand> and C<--shrink> together. @@ -342,6 +413,18 @@ partition will be created. =cut +my $expand_content = 1; + +=item B<--no-expand-content> + +By default, virt-resize will try to expand the direct contents +of partitions, if it knows how (see C<--expand> option above). + +If you give the C<--no-expand-content> option then virt-resize +will not attempt this. + +=cut + my $debug; =item B<-d> | B<--debug> @@ -378,8 +461,9 @@ GetOptions ("help|?" => \$help, "delete=s" => \@delete, "copy-boot-loader!" => \$copy_boot_loader, "extra-partition!" => \$extra_partition, + "expand-content!" => \$expand_content, "d|debug" => \$debug, - "n|dryrun" => \$dryrun, + "n|dryrun|dry-run" => \$dryrun, "q|quiet" => \$quiet, ) or pod2usage (2); pod2usage (1) if $help; @@ -524,6 +608,21 @@ sub examine_partition # that case user won't be allowed to shrink this partition except # by forcing it. $partitions{$part}->{fssize} = $fssize; + + # Is it partition content that we know how to expand? + $partitions{$part}->{can_expand_content} = 0; + if ($expand_content) { + if ($type eq "LVM2_member") { + $partitions{$part}->{can_expand_content} = 1; + $partitions{$part}->{expand_content_method} = "pvresize"; + } elsif ($type =~ /^ext[234]/) { + $partitions{$part}->{can_expand_content} = 1; + $partitions{$part}->{expand_content_method} = "resize2fs"; + } elsif ($type eq "ntfs" && feature_available ($g, "ntfsprogs")) { + $partitions{$part}->{can_expand_content} = 1; + $partitions{$part}->{expand_content_method} = "ntfsresize"; + } + } } if ($debug) { @@ -586,6 +685,8 @@ sub do_delete } # Handle --resize and --resize-force. +my $to_be_expanded = 0; + do_resize ($_, 0, "--resize") foreach @resize; do_resize ($_, 1, "--resize-force") foreach @resize_force; @@ -659,6 +760,11 @@ sub mark_partition_for_resize } $partitions{$part}->{newsize} = $newsize; + + if ($partitions{$part}->{can_expand_content} && $bigger) { + $partitions{$part}->{will_expand_content} = 1; + $to_be_expanded++; + } } # Handle --expand and --shrink. @@ -747,18 +853,22 @@ sub print_summary foreach my $part (@partitions) { if ($partitions{$part}->{ignore}) { - print __x("{p}: partition will be ignored", p => $part); + print __x("{p}: partition will be ignored\n", p => $part); } elsif ($partitions{$part}->{delete}) { - print __x("{p}: partition will be deleted", p => $part); + print __x("{p}: partition will be deleted\n", p => $part); } elsif ($partitions{$part}->{newsize}) { - print __x("{p}: partition will be resized from {oldsize} to {newsize}", + print __x("{p}: partition will be resized from {oldsize} to {newsize}\n", p => $part, oldsize => human_size ($partitions{$part}->{part_size}), newsize => human_size ($partitions{$part}->{newsize})); + if ($partitions{$part}->{will_expand_content}) { + print __x("{p}: content will be expanded using the '{meth}' method\n", + p => $part, + meth => $partitions{$part}->{expand_content_method}); + } } else { - print __x("{p}: partition will be left alone", p => $part); + print __x("{p}: partition will be left alone\n", p => $part); } - print "\n" } if ($surplus > 0) { @@ -912,20 +1022,110 @@ sub copy_data if (!$quiet && !$debug) { local $| = 1; - print "Copying $part ..."; + print __x("Copying {p} ...", p => $part); } $g->copy_size ($part, $target, $newsize < $oldsize ? $newsize : $oldsize); if (!$quiet && !$debug) { - print " done\n" + print " ", __"done", "\n"; + } + } + } + } +} + +# After copying the data over we must shut down and restart the +# appliance in order to expand the content. The reason for this may +# not be obvious, but it's because otherwise we'll have duplicate VGs +# (the old VG(s) and the new VG(s)) which breaks LVM. +# +# The restart is only required if we're going to expand something. + +if ($to_be_expanded > 0) { + restart_appliance (); + expand_partitions (); +} + +sub restart_appliance +{ + # Sync disk and exit. + $g->umount_all (); + $g->sync (); + undef $g; + + $g = Sys::Guestfs->new (); + $g->set_trace (1) if $debug; + $g->add_drive ($outfile); + $g->launch (); + + # Target partitions have changed from /dev/sdb to /dev/sda, + # so change them. + foreach my $part (@partitions) + { + my $target = $partitions{$part}->{target}; + if ($target) { + if ($target =~ m{/dev/(.)db(.*)}) { + $partitions{$part}->{target} = "/dev/$1da$2"; + } else { + die "internal error: unexpected partition target: $target"; + } + } + } +} + +sub expand_partitions +{ + foreach my $part (@partitions) + { + unless ($partitions{$part}->{ignore}) { + my $target = $partitions{$part}->{target}; + if ($target) { + # Expand if requested. + if ($partitions{$part}->{will_expand_content}) { + if (!$quiet && !$debug) { + print __x("Expanding {p} using the '{meth}' method", + p => $part, + meth => $partitions{$part}->{expand_content_method}); + } + expand_target_partition ($part) } } } } } +sub expand_target_partition +{ + local $_; + my $part = shift; + + # Assertions. + die unless $part; + die unless $partitions{$part}->{can_expand_content}; + die unless $partitions{$part}->{will_expand_content}; + die unless $partitions{$part}->{expand_content_method}; + die unless $partitions{$part}->{target}; + die unless $expand_content; + + my $target = $partitions{$part}->{target}; + my $method = $partitions{$part}->{expand_content_method}; + if ($method eq "pvresize") { + $g->pvresize ($target); + } + elsif ($method eq "resize2fs") { + $g->e2fsck_f ($target); + $g->resize2fs ($target); + } + elsif ($method eq "ntfsresize") { + $g->ntfsresize ($target); + } + else { + die "internal error: unknown method: $method"; + } +} + # Sync disk and exit. $g->umount_all (); $g->sync (); @@ -995,6 +1195,30 @@ sub canonicalize $_; } +=head1 NOTES + +=head2 "Partition 1 does not end on cylinder boundary." + +Virt-resize aligns partitions to multiples of 64 sectors. Usually +this means the partitions will not be aligned to the ancient CHS +geometry. However CHS geometry is meaningless for disks manufactured +since the early 1990s, and doubly so for virtual hard drives. +Alignment of partitions to cylinders is not required by any modern +operating system. + +=head2 RESIZING WINDOWS VIRTUAL MACHINES + +In Windows Vista and later versions, Microsoft switched to using a +separate boot partition. In these VMs, typically C</dev/sda1> is the +boot partition and C</dev/sda2> is the main (C:) drive. We have not +had any luck resizing the boot partition. Doing so seems to break the +guest completely. However expanding the second partition (ie. C: +drive) should work. + +Windows may initiate a lengthy "chkdsk" on first boot after a resize, +if NTFS partitions have been expanded. This is just a safety check +and (unless it find errors) is nothing to worry about. + =head1 SEE ALSO L<virt-list-partitions(1)>, @@ -1006,6 +1230,7 @@ L<lvm(8)>, L<pvresize(8)>, L<lvresize(8)>, L<resize2fs(8)>, +L<ntfsresize(8)>, L<virsh(1)>, L<Sys::Guestfs(3)>, L<http://libguestfs.org/>. -- 1.6.6.1
Apparently Analagous Threads
- [PATCH febootstrap 0/8] Add support for building an ext2-based appliance
- [PATCH 0/8 v2 DISCUSSION ONLY] Connecting to live virtual machines
- [PATCH 0/5] 5 conservative changes to errno handling
- [PATCH 0/13 v2] Prepare for adding write support to hivex (Windows registry) library
- [PATCH 0/7] Add libvirt domain to core API