I got a little way into this. The two attached patches are preliminary work. My proposed split is: libguestfs.git common -> git submodule libguestfs-common.git generator/ lib/ all language bindings C based tools (eg. virt-df, virt-edit, guestfish) guestfs-tools.git common -> git submodule libguestfs-common.git virt-builder, virt-customize, virt-sparsify, virt-sparsify, etc virt-v2v.git common -> git submodule libguestfs-common.git virt-p2v.git [already done] The current common/ subdirectory would become a git submodule. While git submodules are awkward, they do solve this particular problem with having common code shared across the repositories, there's only one git submodule and it's under our control. It does mean that any time there's a change to common/, we would need to add a commit to the other 3 repos updating the submodule hash. There are only 6 generated files which are not confined to the new libguestfs.git. 4 files are needed by virt-customize and 2 by virt-v2v (and these last 2 will probably go away in future). My proposal (in patch 2/2) is that we simply copy these 6 files into the common submodule. It's not very elegant but it only affects a tiny fraction of all generated files, these ones are updated very infrequently, and it solves the problem for now. A possible future path would be to split the generator so there's a mini-generator for virt-customize. libguestfs-common.git can be created from libguestfs.git using 'git filter-branch --prune-empty --subdirectory-filter common master'. I tested this already and it seems to work. The two other new repos can be created using the rather more complex technique described here which I didn't try yet: https://stackoverflow.com/questions/2797191/how-to-split-a-git-repository-while-preserving-subdirectories Previous discussion was in this thread over several months: https://www.redhat.com/archives/libguestfs/2018-February/thread.html#00067 https://www.redhat.com/archives/libguestfs/2019-April/thread.html#00260 https://www.redhat.com/archives/libguestfs/2019-June/thread.html#00095 https://www.redhat.com/archives/libguestfs/2019-July/thread.html#00000 Rich.
Richard W.M. Jones
2019-Oct-15 08:01 UTC
[Libguestfs] [PATCH 1/2] generator, customize, v2v: Only place generated files in libguestfs or common.
After the proposed split of the libguestfs repo, we will end up with the following layout: libguestfs.git common -> git submodule libguestfs-common.git generator virt-v2v.git common -> git submodule libguestfs-common.git guestfs-tools.git common -> git submodule libguestfs-common.git The generator will only be able to write to libguestfs directories and the common directory/submodule. This is mostly the case already with only 6 exceptions: customize/customize-options.pod customize/customize-synopsis.pod customize/customize_cmdline.ml customize/customize_cmdline.mli v2v/uefi.ml v2v/uefi.mli This commit moves these files around so they appear under common/ml* It is somewhat unsatisfactory because it involves copying files around, but there are some mitigating factors: (1) Any changes now give us more freedom to develop faster and thus clean things up in future. (2) The v2v/uefi files ought to go away in future anyway. This is simple code motion and should have no effect on the built programs or tests. --- .gitignore | 8 ++++++-- Makefile.am | 1 + builder/Makefile.am | 6 +++--- builder/test-docs.sh | 4 ++-- common/mlcustomize/Makefile.am | 12 ++++++++++++ common/mlv2v/Makefile.am | 28 +++++++++++++++++++++++++++ configure.ac | 1 + customize/Makefile.am | 24 +++++++++++++---------- customize/test-virt-customize-docs.sh | 4 ++-- docs/guestfs-hacking.pod | 4 ++++ generator/main.ml | 12 ++++++------ po-docs/podfiles | 4 ++-- v2v/Makefile.am | 16 +++++++++------ 13 files changed, 91 insertions(+), 33 deletions(-) diff --git a/.gitignore b/.gitignore index 657ebdb8c..2ea4bfbe6 100644 --- a/.gitignore +++ b/.gitignore @@ -128,6 +128,10 @@ Makefile.in /common/errnostring/errnostring.h /common/mlaugeas/.depend /common/mlcustomize/.depend +/common/mlcustomize/customize_cmdline.ml +/common/mlcustomize/customize_cmdline.mli +/common/mlcustomize/customize-options.pod +/common/mlcustomize/customize-synopsis.pod /common/mlcustomize/test-firstboot-*.sh /common/mlgettext/.depend /common/mlgettext/common_gettext.ml @@ -153,6 +157,8 @@ Makefile.in /common/mlutils/.depend /common/mlutils/c_utils_unit_tests /common/mlutils/oUnit-* +/common/mlv2v/uefi.ml +/common/mlv2v/uefi.mli /common/mlvisit/.depend /common/mlvisit/visit_tests /common/mlxml/.depend @@ -177,8 +183,6 @@ Makefile.in /customize/.depend /customize/customize_cmdline.ml /customize/customize_cmdline.mli -/customize/customize-options.pod -/customize/customize-synopsis.pod /customize/stamp-virt-customize.pod /customize/test-password-*.sh /customize/test-settings-*.sh diff --git a/Makefile.am b/Makefile.am index b5d5dcd6b..9ce0a75a7 100644 --- a/Makefile.am +++ b/Makefile.am @@ -164,6 +164,7 @@ SUBDIRS += common/mlvisit SUBDIRS += common/mlxml SUBDIRS += common/mltools SUBDIRS += common/mlcustomize +SUBDIRS += common/mlv2v if HAVE_LIBVIRT SUBDIRS += common/mllibvirt endif diff --git a/builder/Makefile.am b/builder/Makefile.am index 7b4c3a59e..daf4778c7 100644 --- a/builder/Makefile.am +++ b/builder/Makefile.am @@ -263,12 +263,12 @@ noinst_DATA += $(top_builddir)/website/virt-builder.1.html virt-builder.1 $(top_builddir)/website/virt-builder.1.html: stamp-virt-builder.pod -stamp-virt-builder.pod: virt-builder.pod $(top_srcdir)/customize/customize-synopsis.pod $(top_srcdir)/customize/customize-options.pod +stamp-virt-builder.pod: virt-builder.pod $(top_srcdir)/common/mlcustomize/customize-synopsis.pod $(top_srcdir)/common/mlcustomize/customize-options.pod $(PODWRAPPER) \ --man virt-builder.1 \ --html $(top_builddir)/website/virt-builder.1.html \ - --insert $(top_srcdir)/customize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \ - --insert $(top_srcdir)/customize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \ + --insert $(top_srcdir)/common/mlcustomize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \ + --insert $(top_srcdir)/common/mlcustomize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \ --license GPLv2+ \ --warning safe \ $< diff --git a/builder/test-docs.sh b/builder/test-docs.sh index 6f39b906d..7cac4fe2d 100755 --- a/builder/test-docs.sh +++ b/builder/test-docs.sh @@ -22,8 +22,8 @@ $TEST_FUNCTIONS skip_if_skipped $top_srcdir/podcheck.pl virt-builder.pod virt-builder \ - --insert $top_srcdir/customize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \ - --insert $top_srcdir/customize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \ + --insert $top_srcdir/common/mlcustomize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \ + --insert $top_srcdir/common/mlcustomize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \ --ignore=--check-signatures,--no-check-signatures $srcdir/../podcheck.pl virt-builder-repository.pod virt-builder-repository diff --git a/common/mlcustomize/Makefile.am b/common/mlcustomize/Makefile.am index 293a9d394..149a76175 100644 --- a/common/mlcustomize/Makefile.am +++ b/common/mlcustomize/Makefile.am @@ -18,12 +18,24 @@ include $(top_srcdir)/subdir-rules.mk EXTRA_DIST = \ + $(generator_built) \ $(SOURCES_MLI) \ $(SOURCES_ML) \ $(SOURCES_C) \ test-firstboot.sh \ test-selinuxrelabel.sh +# Note: So that I don't have to move many modules from virt-customize +# to here, we don't compile customize_cmdline.ml into the mlcustomize +# library. Instead virt-customize links to these files. They are +# only located here because they have to be placed in common/ because +# they are generated (since the repository split). +generator_built = \ + customize_cmdline.mli \ + customize_cmdline.ml \ + customize-options.pod \ + customize-synopsis.pod + SOURCES_MLI = \ firstboot.mli \ SELinux_relabel.mli diff --git a/common/mlv2v/Makefile.am b/common/mlv2v/Makefile.am new file mode 100644 index 000000000..9e82148ac --- /dev/null +++ b/common/mlv2v/Makefile.am @@ -0,0 +1,28 @@ +# libguestfs OCaml virt-v2v generated code +# Copyright (C) 2011-2019 Red Hat Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +include $(top_srcdir)/subdir-rules.mk + +EXTRA_DIST = \ + $(generator_built) + +# Note: These are not compiled into a library. They are used directly +# by virt-v2v and are only located here because the files are +# generated (since the split). +generator_built = \ + uefi.mli \ + uefi.ml diff --git a/configure.ac b/configure.ac index ff3dc5e99..c15541c97 100644 --- a/configure.ac +++ b/configure.ac @@ -246,6 +246,7 @@ AC_CONFIG_FILES([Makefile common/mlstdutils/guestfs_config.ml common/mltools/Makefile common/mlutils/Makefile + common/mlv2v/Makefile common/mlvisit/Makefile common/mlxml/Makefile common/options/Makefile diff --git a/customize/Makefile.am b/customize/Makefile.am index 181cba2b4..7a76159ca 100644 --- a/customize/Makefile.am +++ b/customize/Makefile.am @@ -18,7 +18,6 @@ include $(top_srcdir)/subdir-rules.mk EXTRA_DIST = \ - $(generator_built) \ $(SOURCES_MLI) $(SOURCES_ML) $(SOURCES_C) \ customize_main.ml \ test-password.pl \ @@ -27,12 +26,6 @@ EXTRA_DIST = \ test-virt-customize-docs.sh \ virt-customize.pod -generator_built = \ - customize_cmdline.mli \ - customize_cmdline.ml \ - customize-options.pod \ - customize-synopsis.pod - SOURCES_MLI = \ append_line.mli \ crypt.mli \ @@ -67,6 +60,17 @@ SOURCES_C = \ crypt-c.c \ perl_edit-c.c +BUILT_SOURCES = \ + customize_cmdline.mli \ + customize_cmdline.ml + +# These are created by the generator in common/mlcustomize and +# need to be copied here. +customize_cmdline.mli: $(top_srcdir)/common/mlcustomize/customize_cmdline.mli + cp $< $@ +customize_cmdline.ml: $(top_srcdir)/common/mlcustomize/customize_cmdline.ml + cp $< $@ + if HAVE_OCAML # Build the virt-customize objects into a library, since these are @@ -193,12 +197,12 @@ noinst_DATA += $(top_builddir)/website/virt-customize.1.html virt-customize.1 $(top_builddir)/website/virt-customize.1.html: stamp-virt-customize.pod -stamp-virt-customize.pod: virt-customize.pod $(top_srcdir)/customize/customize-synopsis.pod $(top_srcdir)/customize/customize-options.pod +stamp-virt-customize.pod: virt-customize.pod $(top_srcdir)/common/mlcustomize/customize-synopsis.pod $(top_srcdir)/common/mlcustomize/customize-options.pod $(PODWRAPPER) \ --man virt-customize.1 \ --html $(top_builddir)/website/virt-customize.1.html \ - --insert $(top_srcdir)/customize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \ - --insert $(top_srcdir)/customize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \ + --insert $(top_srcdir)/common/mlcustomize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \ + --insert $(top_srcdir)/common/mlcustomize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \ --license GPLv2+ \ --warning general \ $< diff --git a/customize/test-virt-customize-docs.sh b/customize/test-virt-customize-docs.sh index ddaedea9f..99e67cb10 100755 --- a/customize/test-virt-customize-docs.sh +++ b/customize/test-virt-customize-docs.sh @@ -21,6 +21,6 @@ set -e $TEST_FUNCTIONS $top_srcdir/podcheck.pl virt-customize.pod virt-customize \ - --insert $top_srcdir/customize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \ - --insert $top_srcdir/customize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \ + --insert $top_srcdir/common/mlcustomize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \ + --insert $top_srcdir/common/mlcustomize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \ --ignore=--dryrun diff --git a/docs/guestfs-hacking.pod b/docs/guestfs-hacking.pod index 42035fdb6..3130f04f3 100644 --- a/docs/guestfs-hacking.pod +++ b/docs/guestfs-hacking.pod @@ -133,6 +133,10 @@ C<virt-sysprep>, C<virt-v2v> etc.) OCaml bindings for C functions in C<common/utils>, and some POSIX bindings which are missing from the OCaml stdlib. +=item F<common/mlv2v> + +A few generated files used by virt-v2v. + =item F<common/mlvisit> OCaml bindings for the visit functions (see F<common/visit>). diff --git a/generator/main.ml b/generator/main.ml index eff417536..741954e8e 100644 --- a/generator/main.ml +++ b/generator/main.ml @@ -347,18 +347,18 @@ Run it from the top source directory using the command output_to "gobject/src/session.c" GObject.generate_gobject_session_source; - output_to "v2v/uefi.ml" + output_to "common/mlv2v/uefi.ml" UEFI.generate_uefi_ml; - output_to "v2v/uefi.mli" + output_to "common/mlv2v/uefi.mli" UEFI.generate_uefi_mli; - output_to "customize/customize_cmdline.mli" + output_to "common/mlcustomize/customize_cmdline.mli" Customize.generate_customize_cmdline_mli; - output_to "customize/customize_cmdline.ml" + output_to "common/mlcustomize/customize_cmdline.ml" Customize.generate_customize_cmdline_ml; - output_to "customize/customize-synopsis.pod" + output_to "common/mlcustomize/customize-synopsis.pod" Customize.generate_customize_synopsis_pod; - output_to "customize/customize-options.pod" + output_to "common/mlcustomize/customize-options.pod" Customize.generate_customize_options_pod; output_to "rust/src/guestfs.rs" diff --git a/po-docs/podfiles b/po-docs/podfiles index b0deb00df..d366a282a 100644 --- a/po-docs/podfiles +++ b/po-docs/podfiles @@ -8,8 +8,8 @@ ../cat/virt-log.pod ../cat/virt-ls.pod ../cat/virt-tail.pod -../customize/customize-options.pod -../customize/customize-synopsis.pod +../common/mlcustomize/customize-options.pod +../common/mlcustomize/customize-synopsis.pod ../customize/virt-customize.pod ../daemon/guestfsd.pod ../df/virt-df.pod diff --git a/v2v/Makefile.am b/v2v/Makefile.am index 6e0829f81..4972b877e 100644 --- a/v2v/Makefile.am +++ b/v2v/Makefile.am @@ -17,18 +17,15 @@ include $(top_srcdir)/subdir-rules.mk -generator_built = \ - uefi.ml \ - uefi.mli - BUILT_SOURCES = \ - $(generator_built) \ config.ml \ output_rhv_upload_createvm_source.ml \ output_rhv_upload_deletedisks_source.ml \ output_rhv_upload_plugin_source.ml \ output_rhv_upload_precheck_source.ml \ - output_rhv_upload_vmcheck_source.ml + output_rhv_upload_vmcheck_source.ml \ + uefi.ml \ + uefi.mli EXTRA_DIST = \ $(SOURCES_MLI) $(SOURCES_ML) $(SOURCES_C) \ @@ -188,6 +185,13 @@ output_rhv_upload_precheck_source.ml: $(srcdir)/rhv-upload-precheck.py output_rhv_upload_vmcheck_source.ml: $(srcdir)/rhv-upload-vmcheck.py $(srcdir)/embed.sh code $^ $@ +# These are created by the generator in common/mlv2v and +# need to be copied here. +uefi.mli: $(top_srcdir)/common/mlv2v/uefi.mli + cp $< $@ +uefi.ml: $(top_srcdir)/common/mlv2v/uefi.ml + cp $< $@ + if HAVE_OCAML bin_PROGRAMS = virt-v2v virt-v2v-copy-to-local -- 2.23.0
Richard W.M. Jones
2019-Oct-15 08:01 UTC
[Libguestfs] [PATCH 2/2] common/{mlcustomize, mlv2v}: Add generated files to git.
With the proposed split we will only run the generator from the libguestfs repo. When compiling virt-v2v or the guestfs-tools we will need certain generated files to be present already in the libguestfs-common repo, and therefore these files must be added to git. Hopefully they won't change very often. --- .gitignore | 6 - common/mlcustomize/customize-options.pod | 406 +++++++++++++++++ common/mlcustomize/customize-synopsis.pod | 16 + common/mlcustomize/customize_cmdline.ml | 515 ++++++++++++++++++++++ common/mlcustomize/customize_cmdline.mli | 118 +++++ common/mlv2v/uefi.ml | 71 +++ common/mlv2v/uefi.mli | 35 ++ 7 files changed, 1161 insertions(+), 6 deletions(-) diff --git a/.gitignore b/.gitignore index 2ea4bfbe6..2228d0b1a 100644 --- a/.gitignore +++ b/.gitignore @@ -128,10 +128,6 @@ Makefile.in /common/errnostring/errnostring.h /common/mlaugeas/.depend /common/mlcustomize/.depend -/common/mlcustomize/customize_cmdline.ml -/common/mlcustomize/customize_cmdline.mli -/common/mlcustomize/customize-options.pod -/common/mlcustomize/customize-synopsis.pod /common/mlcustomize/test-firstboot-*.sh /common/mlgettext/.depend /common/mlgettext/common_gettext.ml @@ -157,8 +153,6 @@ Makefile.in /common/mlutils/.depend /common/mlutils/c_utils_unit_tests /common/mlutils/oUnit-* -/common/mlv2v/uefi.ml -/common/mlv2v/uefi.mli /common/mlvisit/.depend /common/mlvisit/visit_tests /common/mlxml/.depend diff --git a/common/mlcustomize/customize-options.pod b/common/mlcustomize/customize-options.pod new file mode 100644 index 000000000..1c4fcb213 --- /dev/null +++ b/common/mlcustomize/customize-options.pod @@ -0,0 +1,406 @@ +=begin comment + +libguestfs generated file + WARNING: THIS FILE IS GENERATED FROM THE FOLLOWING FILES: + generator/customize.ml + and from the code in the generator/ subdirectory. + ANY CHANGES YOU MAKE TO THIS FILE WILL BE LOST. + + Copyright (C) 2009-2019 Red Hat Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License along + with this program; if not, write to the Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +=end comment + +=over 4 + +=item B<--append-line> FILE:LINE + +Append a single line of text to the C<FILE>. If the file does not already +end with a newline, then one is added before the appended +line. Also a newline is added to the end of the C<LINE> string +automatically. + +For example (assuming ordinary shell quoting) this command: + + --append-line '/etc/hosts:10.0.0.1 foo' + +will add either C<10.0.0.1 foo⏎> or C<⏎10.0.0.1 foo⏎> to +the file, the latter only if the existing file does not +already end with a newline. + +C<⏎> represents a newline character, which is guessed by +looking at the existing content of the file, so this command +does the right thing for files using Unix or Windows line endings. +It also works for empty or non-existent files. + +To insert several lines, use the same option several times: + + --append-line '/etc/hosts:10.0.0.1 foo' + --append-line '/etc/hosts:10.0.0.2 bar' + +To insert a blank line before the appended line, do: + + --append-line '/etc/hosts:' + --append-line '/etc/hosts:10.0.0.1 foo' + +=item B<--chmod> PERMISSIONS:FILE + +Change the permissions of C<FILE> to C<PERMISSIONS>. + +I<Note>: C<PERMISSIONS> by default would be decimal, unless you prefix +it with C<0> to get octal, ie. use C<0700> not C<700>. + +=item B<--commands-from-file> FILENAME + +Read the customize commands from a file, one (and its arguments) +each line. + +Each line contains a single customization command and its arguments, +for example: + + delete /some/file + install some-package + password some-user:password:its-new-password + +Empty lines are ignored, and lines starting with C<#> are comments +and are ignored as well. Furthermore, arguments can be spread across +multiple lines, by adding a C<\> (continuation character) at the of +a line, for example + + edit /some/file:\ + s/^OPT=.*/OPT=ok/ + +The commands are handled in the same order as they are in the file, +as if they were specified as I<--delete /some/file> on the command +line. + +=item B<--copy> SOURCE:DEST + +Copy files or directories recursively inside the guest. + +Wildcards cannot be used. + +=item B<--copy-in> LOCALPATH:REMOTEDIR + +Copy local files or directories recursively into the disk image, +placing them in the directory C<REMOTEDIR> (which must exist). + +Wildcards cannot be used. + +=item B<--delete> PATH + +Delete a file from the guest. Or delete a directory (and all its +contents, recursively). + +You can use shell glob characters in the specified path. Be careful +to escape glob characters from the host shell, if that is required. +For example: + + virt-customize --delete '/var/log/*.log'. + +See also: I<--upload>, I<--scrub>. + +=item B<--edit> FILE:EXPR + +Edit C<FILE> using the Perl expression C<EXPR>. + +Be careful to properly quote the expression to prevent it from +being altered by the shell. + +Note that this option is only available when Perl 5 is installed. + +See L<virt-edit(1)/NON-INTERACTIVE EDITING>. + +=item B<--firstboot> SCRIPT + +Install C<SCRIPT> inside the guest, so that when the guest first boots +up, the script runs (as root, late in the boot process). + +The script is automatically chmod +x after installation in the guest. + +The alternative version I<--firstboot-command> is the same, but it +conveniently wraps the command up in a single line script for you. + +You can have multiple I<--firstboot> options. They run in the same +order that they appear on the command line. + +Please take a look at L<virt-builder(1)/FIRST BOOT SCRIPTS> for more +information and caveats about the first boot scripts. + +See also I<--run>. + +=item B<--firstboot-command> 'CMD+ARGS' + +Run command (and arguments) inside the guest when the guest first +boots up (as root, late in the boot process). + +You can have multiple I<--firstboot> options. They run in the same +order that they appear on the command line. + +Please take a look at L<virt-builder(1)/FIRST BOOT SCRIPTS> for more +information and caveats about the first boot scripts. + +See also I<--run>. + +=item B<--firstboot-install> PKG,PKG.. + +Install the named packages (a comma-separated list). These are +installed when the guest first boots using the guest’s package manager +(eg. apt, yum, etc.) and the guest’s network connection. + +For an overview on the different ways to install packages, see +L<virt-builder(1)/INSTALLING PACKAGES>. + +=item B<--hostname> HOSTNAME + +Set the hostname of the guest to C<HOSTNAME>. You can use a +dotted hostname.domainname (FQDN) if you want. + +=item B<--install> PKG,PKG.. + +Install the named packages (a comma-separated list). These are +installed during the image build using the guest’s package manager +(eg. apt, yum, etc.) and the host’s network connection. + +For an overview on the different ways to install packages, see +L<virt-builder(1)/INSTALLING PACKAGES>. + +See also I<--update>, I<--uninstall>. + +=item B<--link> TARGET:LINK[:LINK..] + +Create symbolic link(s) in the guest, starting at C<LINK> and +pointing at C<TARGET>. + +=item B<--mkdir> DIR + +Create a directory in the guest. + +This uses S<C<mkdir -p>> so any intermediate directories are created, +and it also works if the directory already exists. + +=item B<--move> SOURCE:DEST + +Move files or directories inside the guest. + +Wildcards cannot be used. + +=item B<--no-logfile> + +Scrub C<builder.log> (log file from build commands) from the image +after building is complete. If you don't want to reveal precisely how +the image was built, use this option. + +See also: L</LOG FILE>. + +=item B<--password> USER:SELECTOR + +Set the password for C<USER>. (Note this option does I<not> +create the user account). + +See L<virt-builder(1)/USERS AND PASSWORDS> for the format of +the C<SELECTOR> field, and also how to set up user accounts. + +=item B<--password-crypto> md5|sha256|sha512 + +When the virt tools change or set a password in the guest, this +option sets the password encryption of that password to +C<md5>, C<sha256> or C<sha512>. + +C<sha256> and C<sha512> require glibc E<ge> 2.7 (check crypt(3) inside +the guest). + +C<md5> will work with relatively old Linux guests (eg. RHEL 3), but +is not secure against modern attacks. + +The default is C<sha512> unless libguestfs detects an old guest that +didn't have support for SHA-512, in which case it will use C<md5>. +You can override libguestfs by specifying this option. + +Note this does not change the default password encryption used +by the guest when you create new user accounts inside the guest. +If you want to do that, then you should use the I<--edit> option +to modify C</etc/sysconfig/authconfig> (Fedora, RHEL) or +C</etc/pam.d/common-password> (Debian, Ubuntu). + +=item B<--root-password> SELECTOR + +Set the root password. + +See L<virt-builder(1)/USERS AND PASSWORDS> for the format of +the C<SELECTOR> field, and also how to set up user accounts. + +Note: In virt-builder, if you I<don't> set I<--root-password> +then the guest is given a I<random> root password. + +=item B<--run> SCRIPT + +Run the shell script (or any program) called C<SCRIPT> on the disk +image. The script runs virtualized inside a small appliance, chrooted +into the guest filesystem. + +The script is automatically chmod +x. + +If libguestfs supports it then a limited network connection is +available but it only allows outgoing network connections. You can +also attach data disks (eg. ISO files) as another way to provide data +(eg. software packages) to the script without needing a network +connection (I<--attach>). You can also upload data files (I<--upload>). + +You can have multiple I<--run> options. They run +in the same order that they appear on the command line. + +See also: I<--firstboot>, I<--attach>, I<--upload>. + +=item B<--run-command> 'CMD+ARGS' + +Run the command and arguments on the disk image. The command runs +virtualized inside a small appliance, chrooted into the guest filesystem. + +If libguestfs supports it then a limited network connection is +available but it only allows outgoing network connections. You can +also attach data disks (eg. ISO files) as another way to provide data +(eg. software packages) to the script without needing a network +connection (I<--attach>). You can also upload data files (I<--upload>). + +You can have multiple I<--run-command> options. They run +in the same order that they appear on the command line. + +See also: I<--firstboot>, I<--attach>, I<--upload>. + +=item B<--scrub> FILE + +Scrub a file from the guest. This is like I<--delete> except that: + +=over 4 + +=item * + +It scrubs the data so a guest could not recover it. + +=item * + +It cannot delete directories, only regular files. + +=back + +=item B<--selinux-relabel> + +Relabel files in the guest so that they have the correct SELinux label. + +This will attempt to relabel files immediately, but if the operation fails +this will instead touch F</.autorelabel> on the image to schedule a +relabel operation for the next time the image boots. + +You should only use this option for guests which support SELinux. + +=item B<--sm-attach> SELECTOR + +Attach to a pool using C<subscription-manager>. + +See L<virt-builder(1)/SUBSCRIPTION-MANAGER> for the format of +the C<SELECTOR> field. + +=item B<--sm-credentials> SELECTOR + +Set the credentials for C<subscription-manager>. + +See L<virt-builder(1)/SUBSCRIPTION-MANAGER> for the format of +the C<SELECTOR> field. + +=item B<--sm-register> + +Register the guest using C<subscription-manager>. + +This requires credentials being set using I<--sm-credentials>. + +=item B<--sm-remove> + +Remove all the subscriptions from the guest using +C<subscription-manager>. + +=item B<--sm-unregister> + +Unregister the guest using C<subscription-manager>. + +=item B<--ssh-inject> USER[:SELECTOR] + +Inject an ssh key so the given C<USER> will be able to log in over +ssh without supplying a password. The C<USER> must exist already +in the guest. + +See L<virt-builder(1)/SSH KEYS> for the format of +the C<SELECTOR> field. + +You can have multiple I<--ssh-inject> options, for different users +and also for more keys for each user. + +=item B<--timezone> TIMEZONE + +Set the default timezone of the guest to C<TIMEZONE>. Use a location +string like C<Europe/London> + +=item B<--touch> FILE + +This command performs a L<touch(1)>-like operation on C<FILE>. + +=item B<--truncate> FILE + +This command truncates C<FILE> to a zero-length file. The file must exist +already. + +=item B<--truncate-recursive> PATH + +This command recursively truncates all files under C<PATH> to zero-length. + +=item B<--uninstall> PKG,PKG.. + +Uninstall the named packages (a comma-separated list). These are +removed during the image build using the guest’s package manager +(eg. apt, yum, etc.). Dependent packages may also need to be +uninstalled to satisfy the request. + +See also I<--install>, I<--update>. + +=item B<--update> + +Do the equivalent of C<yum update>, C<apt-get upgrade>, or whatever +command is required to update the packages already installed in the +template to their latest versions. + +See also I<--install>, I<--uninstall>. + +=item B<--upload> FILE:DEST + +Upload local file C<FILE> to destination C<DEST> in the disk image. +File owner and permissions from the original are preserved, so you +should set them to what you want them to be in the disk image. + +C<DEST> could be the final filename. This can be used to rename +the file on upload. + +If C<DEST> is a directory name (which must already exist in the guest) +then the file is uploaded into that directory, and it keeps the same +name as on the local filesystem. + +See also: I<--mkdir>, I<--delete>, I<--scrub>. + +=item B<--write> FILE:CONTENT + +Write C<CONTENT> to C<FILE>. + +=back + diff --git a/common/mlcustomize/customize-synopsis.pod b/common/mlcustomize/customize-synopsis.pod new file mode 100644 index 000000000..5f1854089 --- /dev/null +++ b/common/mlcustomize/customize-synopsis.pod @@ -0,0 +1,16 @@ + [--append-line FILE:LINE] [--chmod PERMISSIONS:FILE] + [--commands-from-file FILENAME] [--copy SOURCE:DEST] + [--copy-in LOCALPATH:REMOTEDIR] [--delete PATH] [--edit FILE:EXPR] + [--firstboot SCRIPT] [--firstboot-command 'CMD+ARGS'] + [--firstboot-install PKG,PKG..] [--hostname HOSTNAME] + [--install PKG,PKG..] [--link TARGET:LINK[:LINK..]] [--mkdir DIR] + [--move SOURCE:DEST] [--password USER:SELECTOR] + [--root-password SELECTOR] [--run SCRIPT] + [--run-command 'CMD+ARGS'] [--scrub FILE] [--sm-attach SELECTOR] + [--sm-register] [--sm-remove] [--sm-unregister] + [--ssh-inject USER[:SELECTOR]] [--truncate FILE] + [--truncate-recursive PATH] [--timezone TIMEZONE] [--touch FILE] + [--uninstall PKG,PKG..] [--update] [--upload FILE:DEST] + [--write FILE:CONTENT] [--no-logfile] + [--password-crypto md5|sha256|sha512] [--selinux-relabel] + [--sm-credentials SELECTOR] diff --git a/common/mlcustomize/customize_cmdline.ml b/common/mlcustomize/customize_cmdline.ml new file mode 100644 index 000000000..c06237987 --- /dev/null +++ b/common/mlcustomize/customize_cmdline.ml @@ -0,0 +1,515 @@ +(* libguestfs generated file + * WARNING: THIS FILE IS GENERATED FROM THE FOLLOWING FILES: + * generator/customize.ml + * and from the code in the generator/ subdirectory. + * ANY CHANGES YOU MAKE TO THIS FILE WILL BE LOST. + * + * Copyright (C) 2009-2019 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) + +(* Command line argument parsing, both for the virt-customize binary + * and for the other tools that share the same code. + *) + +open Printf + +open Std_utils +open Tools_utils +open Common_gettext.Gettext +open Getopt.OptionName + +type ops = { + ops : op list; + flags : flags; +} +and op = [ + | `AppendLine of string * string + (* --append-line FILE:LINE *) + | `Chmod of string * string + (* --chmod PERMISSIONS:FILE *) + | `CommandsFromFile of string + (* --commands-from-file FILENAME *) + | `Copy of string * string + (* --copy SOURCE:DEST *) + | `CopyIn of string * string + (* --copy-in LOCALPATH:REMOTEDIR *) + | `Delete of string + (* --delete PATH *) + | `Edit of string * string + (* --edit FILE:EXPR *) + | `FirstbootScript of string + (* --firstboot SCRIPT *) + | `FirstbootCommand of string + (* --firstboot-command 'CMD+ARGS' *) + | `FirstbootPackages of string list + (* --firstboot-install PKG,PKG.. *) + | `Hostname of string + (* --hostname HOSTNAME *) + | `InstallPackages of string list + (* --install PKG,PKG.. *) + | `Link of string * string list + (* --link TARGET:LINK[:LINK..] *) + | `Mkdir of string + (* --mkdir DIR *) + | `Move of string * string + (* --move SOURCE:DEST *) + | `Password of string * Password.password_selector + (* --password USER:SELECTOR *) + | `RootPassword of Password.password_selector + (* --root-password SELECTOR *) + | `Script of string + (* --run SCRIPT *) + | `Command of string + (* --run-command 'CMD+ARGS' *) + | `Scrub of string + (* --scrub FILE *) + | `SMAttach of Subscription_manager.sm_pool + (* --sm-attach SELECTOR *) + | `SMRegister + (* --sm-register *) + | `SMRemove + (* --sm-remove *) + | `SMUnregister + (* --sm-unregister *) + | `SSHInject of string * Ssh_key.ssh_key_selector + (* --ssh-inject USER[:SELECTOR] *) + | `Truncate of string + (* --truncate FILE *) + | `TruncateRecursive of string + (* --truncate-recursive PATH *) + | `Timezone of string + (* --timezone TIMEZONE *) + | `Touch of string + (* --touch FILE *) + | `UninstallPackages of string list + (* --uninstall PKG,PKG.. *) + | `Update + (* --update *) + | `Upload of string * string + (* --upload FILE:DEST *) + | `Write of string * string + (* --write FILE:CONTENT *) +] +and flags = { + scrub_logfile : bool; + (* --no-logfile *) + password_crypto : Password.password_crypto option; + (* --password-crypto md5|sha256|sha512 *) + selinux_relabel : bool; + (* --selinux-relabel *) + sm_credentials : Subscription_manager.sm_credentials option; + (* --sm-credentials SELECTOR *) +} + +type argspec = Getopt.keys * Getopt.spec * Getopt.doc + +let rec argspec () + let ops = ref [] in + let scrub_logfile = ref false in + let password_crypto = ref None in + let selinux_relabel = ref false in + let sm_credentials = ref None in + + let rec get_ops () = { + ops = List.rev !ops; + flags = get_flags (); + } + and get_flags () = { + scrub_logfile = !scrub_logfile; + password_crypto = !password_crypto; + selinux_relabel = !selinux_relabel; + sm_credentials = !sm_credentials; + } + in + + let split_string_pair option_name arg + let i + try String.index arg ':' + with Not_found -> + error (f_"invalid format for '--%s' parameter, see the man page") + option_name in + let len = String.length arg in + String.sub arg 0 i, String.sub arg (i+1) (len-(i+1)) + in + let split_string_list arg + String.nsplit "," arg + in + let split_links_list option_name arg + match String.nsplit ":" arg with + | [] | [_] -> + error (f_"invalid format for '--%s' parameter, see the man page") + option_name + | target :: lns -> target, lns + in + + let rec argspec = [ + ( + [ L"append-line" ], + Getopt.String ( + s_"FILE:LINE", + fun s -> + let p = split_string_pair "append-line" s in + List.push_front (`AppendLine p) ops + ), + s_"Append line(s) to the file" + ), + Some "FILE:LINE", "Append a single line of text to the C<FILE>. If the file does not already\nend with a newline, then one is added before the appended\nline. Also a newline is added to the end of the C<LINE> string\nautomatically.\n\nFor example (assuming ordinary shell quoting) this command:\n\n --append-line '/etc/hosts:10.0.0.1 foo'\n\nwill add either C<10.0.0.1 foo\226\143\142> or C<\226\143\14210.0.0.1 foo\226\143\142> to\nthe file, the latter only if the existing file does not\nalready end with a newline.\n\nC<\226\143\142> represents a newline character, which is guessed by\nlooking at the existing content of the file, so this command\ndoes the right thing for files using Unix or Windows line endings.\nIt also works for empty or non-existent files.\n\nTo insert several lines, use the same option several times:\n\n --append-line '/etc/hosts:10.0.0.1 foo'\n --append-line '/etc/hosts:10.0.0.2 bar'\n\nTo insert a blank line before the appended line, do:\n\n --append-line '/etc/hosts:'\n --append-line '/etc/hosts:10.0.0.1 foo'"; + ( + [ L"chmod" ], + Getopt.String ( + s_"PERMISSIONS:FILE", + fun s -> + let p = split_string_pair "chmod" s in + List.push_front (`Chmod p) ops + ), + s_"Change the permissions of a file" + ), + Some "PERMISSIONS:FILE", "Change the permissions of C<FILE> to C<PERMISSIONS>.\n\nI<Note>: C<PERMISSIONS> by default would be decimal, unless you prefix\nit with C<0> to get octal, ie. use C<0700> not C<700>."; + ( + [ L"commands-from-file" ], + Getopt.String ( + s_"FILENAME", + fun s -> + customize_read_from_file s; + List.push_front (`CommandsFromFile s) ops + ), + s_"Read customize commands from file" + ), + Some "FILENAME", "Read the customize commands from a file, one (and its arguments)\neach line.\n\nEach line contains a single customization command and its arguments,\nfor example:\n\n delete /some/file\n install some-package\n password some-user:password:its-new-password\n\nEmpty lines are ignored, and lines starting with C<#> are comments\nand are ignored as well. Furthermore, arguments can be spread across\nmultiple lines, by adding a C<\\> (continuation character) at the of\na line, for example\n\n edit /some/file:\\\n s/^OPT=.*/OPT=ok/\n\nThe commands are handled in the same order as they are in the file,\nas if they were specified as I<--delete /some/file> on the command\nline."; + ( + [ L"copy" ], + Getopt.String ( + s_"SOURCE:DEST", + fun s -> + let p = split_string_pair "copy" s in + List.push_front (`Copy p) ops + ), + s_"Copy files in disk image" + ), + Some "SOURCE:DEST", "Copy files or directories recursively inside the guest.\n\nWildcards cannot be used."; + ( + [ L"copy-in" ], + Getopt.String ( + s_"LOCALPATH:REMOTEDIR", + fun s -> + let p = split_string_pair "copy-in" s in + List.push_front (`CopyIn p) ops + ), + s_"Copy local files or directories into image" + ), + Some "LOCALPATH:REMOTEDIR", "Copy local files or directories recursively into the disk image,\nplacing them in the directory C<REMOTEDIR> (which must exist).\n\nWildcards cannot be used."; + ( + [ L"delete" ], + Getopt.String (s_"PATH", fun s -> List.push_front (`Delete s) ops), + s_"Delete a file or directory" + ), + Some "PATH", "Delete a file from the guest. Or delete a directory (and all its\ncontents, recursively).\n\nYou can use shell glob characters in the specified path. Be careful\nto escape glob characters from the host shell, if that is required.\nFor example:\n\n virt-customize --delete '/var/log/*.log'.\n\nSee also: I<--upload>, I<--scrub>."; + ( + [ L"edit" ], + Getopt.String ( + s_"FILE:EXPR", + fun s -> + let p = split_string_pair "edit" s in + List.push_front (`Edit p) ops + ), + s_"Edit file using Perl expression" + ), + Some "FILE:EXPR", "Edit C<FILE> using the Perl expression C<EXPR>.\n\nBe careful to properly quote the expression to prevent it from\nbeing altered by the shell.\n\nNote that this option is only available when Perl 5 is installed.\n\nSee L<virt-edit(1)/NON-INTERACTIVE EDITING>."; + ( + [ L"firstboot" ], + Getopt.String (s_"SCRIPT", fun s -> List.push_front (`FirstbootScript s) ops), + s_"Run script at first guest boot" + ), + Some "SCRIPT", "Install C<SCRIPT> inside the guest, so that when the guest first boots\nup, the script runs (as root, late in the boot process).\n\nThe script is automatically chmod +x after installation in the guest.\n\nThe alternative version I<--firstboot-command> is the same, but it\nconveniently wraps the command up in a single line script for you.\n\nYou can have multiple I<--firstboot> options. They run in the same\norder that they appear on the command line.\n\nPlease take a look at L<virt-builder(1)/FIRST BOOT SCRIPTS> for more\ninformation and caveats about the first boot scripts.\n\nSee also I<--run>."; + ( + [ L"firstboot-command" ], + Getopt.String (s_"'CMD+ARGS'", fun s -> List.push_front (`FirstbootCommand s) ops), + s_"Run command at first guest boot" + ), + Some "'CMD+ARGS'", "Run command (and arguments) inside the guest when the guest first\nboots up (as root, late in the boot process).\n\nYou can have multiple I<--firstboot> options. They run in the same\norder that they appear on the command line.\n\nPlease take a look at L<virt-builder(1)/FIRST BOOT SCRIPTS> for more\ninformation and caveats about the first boot scripts.\n\nSee also I<--run>."; + ( + [ L"firstboot-install" ], + Getopt.String ( + s_"PKG,PKG..", + fun s -> + let ss = split_string_list s in + List.push_front (`FirstbootPackages ss) ops + ), + s_"Add package(s) to install at first boot" + ), + Some "PKG,PKG..", "Install the named packages (a comma-separated list). These are\ninstalled when the guest first boots using the guest\226\128\153s package manager\n(eg. apt, yum, etc.) and the guest\226\128\153s network connection.\n\nFor an overview on the different ways to install packages, see\nL<virt-builder(1)/INSTALLING PACKAGES>."; + ( + [ L"hostname" ], + Getopt.String (s_"HOSTNAME", fun s -> List.push_front (`Hostname s) ops), + s_"Set the hostname" + ), + Some "HOSTNAME", "Set the hostname of the guest to C<HOSTNAME>. You can use a\ndotted hostname.domainname (FQDN) if you want."; + ( + [ L"install" ], + Getopt.String ( + s_"PKG,PKG..", + fun s -> + let ss = split_string_list s in + List.push_front (`InstallPackages ss) ops + ), + s_"Add package(s) to install" + ), + Some "PKG,PKG..", "Install the named packages (a comma-separated list). These are\ninstalled during the image build using the guest\226\128\153s package manager\n(eg. apt, yum, etc.) and the host\226\128\153s network connection.\n\nFor an overview on the different ways to install packages, see\nL<virt-builder(1)/INSTALLING PACKAGES>.\n\nSee also I<--update>, I<--uninstall>."; + ( + [ L"link" ], + Getopt.String ( + s_"TARGET:LINK[:LINK..]", + fun s -> + let ss = split_links_list "link" s in + List.push_front (`Link ss) ops + ), + s_"Create symbolic links" + ), + Some "TARGET:LINK[:LINK..]", "Create symbolic link(s) in the guest, starting at C<LINK> and\npointing at C<TARGET>."; + ( + [ L"mkdir" ], + Getopt.String (s_"DIR", fun s -> List.push_front (`Mkdir s) ops), + s_"Create a directory" + ), + Some "DIR", "Create a directory in the guest.\n\nThis uses S<C<mkdir -p>> so any intermediate directories are created,\nand it also works if the directory already exists."; + ( + [ L"move" ], + Getopt.String ( + s_"SOURCE:DEST", + fun s -> + let p = split_string_pair "move" s in + List.push_front (`Move p) ops + ), + s_"Move files in disk image" + ), + Some "SOURCE:DEST", "Move files or directories inside the guest.\n\nWildcards cannot be used."; + ( + [ L"password" ], + Getopt.String ( + s_"USER:SELECTOR", + fun s -> + let user, sel = split_string_pair "password" s in + let sel = Password.parse_selector sel in + List.push_front (`Password (user, sel)) ops + ), + s_"Set user password" + ), + Some "USER:SELECTOR", "Set the password for C<USER>. (Note this option does I<not>\ncreate the user account).\n\nSee L<virt-builder(1)/USERS AND PASSWORDS> for the format of\nthe C<SELECTOR> field, and also how to set up user accounts."; + ( + [ L"root-password" ], + Getopt.String ( + s_"SELECTOR", + fun s -> + let sel = Password.parse_selector s in + List.push_front (`RootPassword sel) ops + ), + s_"Set root password" + ), + Some "SELECTOR", "Set the root password.\n\nSee L<virt-builder(1)/USERS AND PASSWORDS> for the format of\nthe C<SELECTOR> field, and also how to set up user accounts.\n\nNote: In virt-builder, if you I<don't> set I<--root-password>\nthen the guest is given a I<random> root password."; + ( + [ L"run" ], + Getopt.String (s_"SCRIPT", fun s -> List.push_front (`Script s) ops), + s_"Run script in disk image" + ), + Some "SCRIPT", "Run the shell script (or any program) called C<SCRIPT> on the disk\nimage. The script runs virtualized inside a small appliance, chrooted\ninto the guest filesystem.\n\nThe script is automatically chmod +x.\n\nIf libguestfs supports it then a limited network connection is\navailable but it only allows outgoing network connections. You can\nalso attach data disks (eg. ISO files) as another way to provide data\n(eg. software packages) to the script without needing a network\nconnection (I<--attach>). You can also upload data files (I<--upload>).\n\nYou can have multiple I<--run> options. They run\nin the same order that they appear on the command line.\n\nSee also: I<--firstboot>, I<--attach>, I<--upload>."; + ( + [ L"run-command" ], + Getopt.String (s_"'CMD+ARGS'", fun s -> List.push_front (`Command s) ops), + s_"Run command in disk image" + ), + Some "'CMD+ARGS'", "Run the command and arguments on the disk image. The command runs\nvirtualized inside a small appliance, chrooted into the guest filesystem.\n\nIf libguestfs supports it then a limited network connection is\navailable but it only allows outgoing network connections. You can\nalso attach data disks (eg. ISO files) as another way to provide data\n(eg. software packages) to the script without needing a network\nconnection (I<--attach>). You can also upload data files (I<--upload>).\n\nYou can have multiple I<--run-command> options. They run\nin the same order that they appear on the command line.\n\nSee also: I<--firstboot>, I<--attach>, I<--upload>."; + ( + [ L"scrub" ], + Getopt.String (s_"FILE", fun s -> List.push_front (`Scrub s) ops), + s_"Scrub a file" + ), + Some "FILE", "Scrub a file from the guest. This is like I<--delete> except that:\n\n=over 4\n\n=item *\n\nIt scrubs the data so a guest could not recover it.\n\n=item *\n\nIt cannot delete directories, only regular files.\n\n=back"; + ( + [ L"sm-attach" ], + Getopt.String ( + s_"SELECTOR", + fun s -> + let sel = Subscription_manager.parse_pool_selector s in + List.push_front (`SMAttach sel) ops + ), + s_"Attach to a subscription-manager pool" + ), + Some "SELECTOR", "Attach to a pool using C<subscription-manager>.\n\nSee L<virt-builder(1)/SUBSCRIPTION-MANAGER> for the format of\nthe C<SELECTOR> field."; + ( + [ L"sm-register" ], + Getopt.Unit (fun () -> List.push_front `SMRegister ops), + s_"Register using subscription-manager" + ), + None, "Register the guest using C<subscription-manager>.\n\nThis requires credentials being set using I<--sm-credentials>."; + ( + [ L"sm-remove" ], + Getopt.Unit (fun () -> List.push_front `SMRemove ops), + s_"Remove all the subscriptions" + ), + None, "Remove all the subscriptions from the guest using\nC<subscription-manager>."; + ( + [ L"sm-unregister" ], + Getopt.Unit (fun () -> List.push_front `SMUnregister ops), + s_"Unregister using subscription-manager" + ), + None, "Unregister the guest using C<subscription-manager>."; + ( + [ L"ssh-inject" ], + Getopt.String ( + s_"USER[:SELECTOR]", + fun s -> + let user, selstr = String.split ":" s in + let sel = Ssh_key.parse_selector selstr in + List.push_front (`SSHInject (user, sel)) ops + ), + s_"Inject a public key into the guest" + ), + Some "USER[:SELECTOR]", "Inject an ssh key so the given C<USER> will be able to log in over\nssh without supplying a password. The C<USER> must exist already\nin the guest.\n\nSee L<virt-builder(1)/SSH KEYS> for the format of\nthe C<SELECTOR> field.\n\nYou can have multiple I<--ssh-inject> options, for different users\nand also for more keys for each user."; + ( + [ L"truncate" ], + Getopt.String (s_"FILE", fun s -> List.push_front (`Truncate s) ops), + s_"Truncate a file to zero size" + ), + Some "FILE", "This command truncates C<FILE> to a zero-length file. The file must exist\nalready."; + ( + [ L"truncate-recursive" ], + Getopt.String (s_"PATH", fun s -> List.push_front (`TruncateRecursive s) ops), + s_"Recursively truncate all files in directory" + ), + Some "PATH", "This command recursively truncates all files under C<PATH> to zero-length."; + ( + [ L"timezone" ], + Getopt.String (s_"TIMEZONE", fun s -> List.push_front (`Timezone s) ops), + s_"Set the default timezone" + ), + Some "TIMEZONE", "Set the default timezone of the guest to C<TIMEZONE>. Use a location\nstring like C<Europe/London>"; + ( + [ L"touch" ], + Getopt.String (s_"FILE", fun s -> List.push_front (`Touch s) ops), + s_"Run touch on a file" + ), + Some "FILE", "This command performs a L<touch(1)>-like operation on C<FILE>."; + ( + [ L"uninstall" ], + Getopt.String ( + s_"PKG,PKG..", + fun s -> + let ss = split_string_list s in + List.push_front (`UninstallPackages ss) ops + ), + s_"Uninstall package(s)" + ), + Some "PKG,PKG..", "Uninstall the named packages (a comma-separated list). These are\nremoved during the image build using the guest\226\128\153s package manager\n(eg. apt, yum, etc.). Dependent packages may also need to be\nuninstalled to satisfy the request.\n\nSee also I<--install>, I<--update>."; + ( + [ L"update" ], + Getopt.Unit (fun () -> List.push_front `Update ops), + s_"Update packages" + ), + None, "Do the equivalent of C<yum update>, C<apt-get upgrade>, or whatever\ncommand is required to update the packages already installed in the\ntemplate to their latest versions.\n\nSee also I<--install>, I<--uninstall>."; + ( + [ L"upload" ], + Getopt.String ( + s_"FILE:DEST", + fun s -> + let p = split_string_pair "upload" s in + List.push_front (`Upload p) ops + ), + s_"Upload local file to destination" + ), + Some "FILE:DEST", "Upload local file C<FILE> to destination C<DEST> in the disk image.\nFile owner and permissions from the original are preserved, so you\nshould set them to what you want them to be in the disk image.\n\nC<DEST> could be the final filename. This can be used to rename\nthe file on upload.\n\nIf C<DEST> is a directory name (which must already exist in the guest)\nthen the file is uploaded into that directory, and it keeps the same\nname as on the local filesystem.\n\nSee also: I<--mkdir>, I<--delete>, I<--scrub>."; + ( + [ L"write" ], + Getopt.String ( + s_"FILE:CONTENT", + fun s -> + let p = split_string_pair "write" s in + List.push_front (`Write p) ops + ), + s_"Write file" + ), + Some "FILE:CONTENT", "Write C<CONTENT> to C<FILE>."; + ( + [ L"no-logfile" ], + Getopt.Set scrub_logfile, + s_"Scrub build log file" + ), + None, "Scrub C<builder.log> (log file from build commands) from the image\nafter building is complete. If you don't want to reveal precisely how\nthe image was built, use this option.\n\nSee also: L</LOG FILE>."; + ( + [ L"password-crypto" ], + Getopt.String ( + s_"md5|sha256|sha512", + fun s -> + password_crypto := Some (Password.password_crypto_of_string s) + ), + s_"Set password crypto" + ), + Some "md5|sha256|sha512", "When the virt tools change or set a password in the guest, this\noption sets the password encryption of that password to\nC<md5>, C<sha256> or C<sha512>.\n\nC<sha256> and C<sha512> require glibc E<ge> 2.7 (check crypt(3) inside\nthe guest).\n\nC<md5> will work with relatively old Linux guests (eg. RHEL 3), but\nis not secure against modern attacks.\n\nThe default is C<sha512> unless libguestfs detects an old guest that\ndidn't have support for SHA-512, in which case it will use C<md5>.\nYou can override libguestfs by specifying this option.\n\nNote this does not change the default password encryption used\nby the guest when you create new user accounts inside the guest.\nIf you want to do that, then you should use the I<--edit> option\nto modify C</etc/sysconfig/authconfig> (Fedora, RHEL) or\nC</etc/pam.d/common-password> (Debian, Ubuntu)."; + ( + [ L"selinux-relabel" ], + Getopt.Set selinux_relabel, + s_"Relabel files with correct SELinux labels" + ), + None, "Relabel files in the guest so that they have the correct SELinux label.\n\nThis will attempt to relabel files immediately, but if the operation fails\nthis will instead touch F</.autorelabel> on the image to schedule a\nrelabel operation for the next time the image boots.\n\nYou should only use this option for guests which support SELinux."; + ( + [ L"sm-credentials" ], + Getopt.String ( + s_"SELECTOR", + fun s -> + sm_credentials := Some (Subscription_manager.parse_credentials_selector s) + ), + s_"Credentials for subscription-manager" + ), + Some "SELECTOR", "Set the credentials for C<subscription-manager>.\n\nSee L<virt-builder(1)/SUBSCRIPTION-MANAGER> for the format of\nthe C<SELECTOR> field."; + ] + and customize_read_from_file filename + let forbidden_commands = [ + "commands-from-file"; + ] in + let lines = read_whole_file filename in + let lines = String.lines_split lines in + let lines = List.filter ( + fun line -> + String.length line > 0 && line.[0] <> '#' + ) lines in + let cmds = List.map (fun line -> String.split " " line) lines in + (* Check for commands not allowed in files containing commands. *) + List.iter ( + fun (cmd, _) -> + if List.mem cmd forbidden_commands then + error (f_"command '%s' cannot be used in command files, see the man page") + cmd + ) cmds; + List.iter ( + fun (cmd, arg) -> + try + let ((_, spec, _), _, _) = List.find ( + fun ((keys, _, _), _, _) -> + List.mem (L cmd) keys + ) argspec in + (match spec with + | Getopt.Unit fn -> fn () + | Getopt.String (_, fn) -> fn arg + | Getopt.Set varref -> varref := true + | _ -> error "INTERNAL error: spec not handled for %s" cmd + ) + with Not_found -> + error (f_"command '%s' not valid, see the man page") + cmd + ) cmds + in + + argspec, get_ops diff --git a/common/mlcustomize/customize_cmdline.mli b/common/mlcustomize/customize_cmdline.mli new file mode 100644 index 000000000..9ac3c2c55 --- /dev/null +++ b/common/mlcustomize/customize_cmdline.mli @@ -0,0 +1,118 @@ +(* libguestfs generated file + * WARNING: THIS FILE IS GENERATED FROM THE FOLLOWING FILES: + * generator/customize.ml + * and from the code in the generator/ subdirectory. + * ANY CHANGES YOU MAKE TO THIS FILE WILL BE LOST. + * + * Copyright (C) 2009-2019 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) + +(** Command line argument parsing, both for the virt-customize binary + and for the other tools that share the same code. *) + +type ops = { + ops : op list; + flags : flags; +} +and op = [ + | `AppendLine of string * string + (* --append-line FILE:LINE *) + | `Chmod of string * string + (* --chmod PERMISSIONS:FILE *) + | `CommandsFromFile of string + (* --commands-from-file FILENAME *) + | `Copy of string * string + (* --copy SOURCE:DEST *) + | `CopyIn of string * string + (* --copy-in LOCALPATH:REMOTEDIR *) + | `Delete of string + (* --delete PATH *) + | `Edit of string * string + (* --edit FILE:EXPR *) + | `FirstbootScript of string + (* --firstboot SCRIPT *) + | `FirstbootCommand of string + (* --firstboot-command 'CMD+ARGS' *) + | `FirstbootPackages of string list + (* --firstboot-install PKG,PKG.. *) + | `Hostname of string + (* --hostname HOSTNAME *) + | `InstallPackages of string list + (* --install PKG,PKG.. *) + | `Link of string * string list + (* --link TARGET:LINK[:LINK..] *) + | `Mkdir of string + (* --mkdir DIR *) + | `Move of string * string + (* --move SOURCE:DEST *) + | `Password of string * Password.password_selector + (* --password USER:SELECTOR *) + | `RootPassword of Password.password_selector + (* --root-password SELECTOR *) + | `Script of string + (* --run SCRIPT *) + | `Command of string + (* --run-command 'CMD+ARGS' *) + | `Scrub of string + (* --scrub FILE *) + | `SMAttach of Subscription_manager.sm_pool + (* --sm-attach SELECTOR *) + | `SMRegister + (* --sm-register *) + | `SMRemove + (* --sm-remove *) + | `SMUnregister + (* --sm-unregister *) + | `SSHInject of string * Ssh_key.ssh_key_selector + (* --ssh-inject USER[:SELECTOR] *) + | `Truncate of string + (* --truncate FILE *) + | `TruncateRecursive of string + (* --truncate-recursive PATH *) + | `Timezone of string + (* --timezone TIMEZONE *) + | `Touch of string + (* --touch FILE *) + | `UninstallPackages of string list + (* --uninstall PKG,PKG.. *) + | `Update + (* --update *) + | `Upload of string * string + (* --upload FILE:DEST *) + | `Write of string * string + (* --write FILE:CONTENT *) +] +and flags = { + scrub_logfile : bool; + (* --no-logfile *) + password_crypto : Password.password_crypto option; + (* --password-crypto md5|sha256|sha512 *) + selinux_relabel : bool; + (* --selinux-relabel *) + sm_credentials : Subscription_manager.sm_credentials option; + (* --sm-credentials SELECTOR *) +} + +type argspec = Getopt.keys * Getopt.spec * Getopt.doc +val argspec : unit -> (argspec * string option * string) list * (unit -> ops) +(** This returns a pair [(list, get_ops)]. + + [list] is a list of the command line arguments, plus some extra data. + + [get_ops] is a function you can call {i after} command line parsing + which will return the actual operations specified by the user on the + command line. *) \ No newline at end of file diff --git a/common/mlv2v/uefi.ml b/common/mlv2v/uefi.ml new file mode 100644 index 000000000..a1f1518dd --- /dev/null +++ b/common/mlv2v/uefi.ml @@ -0,0 +1,71 @@ +(* libguestfs generated file + * WARNING: THIS FILE IS GENERATED + * from the code in the generator/ subdirectory. + * ANY CHANGES YOU MAKE TO THIS FILE WILL BE LOST. + * + * Copyright (C) 2009-2019 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) + +type uefi_firmware = { + code : string; + code_debug : string option; + vars : string; + flags : uefi_flags; +} +and uefi_flags = uefi_flag list +and uefi_flag = UEFI_FLAG_SECURE_BOOT_REQUIRED + +let uefi_aarch64_firmware = [ + { code = "/usr/share/AAVMF/AAVMF_CODE.fd"; + code_debug = Some "/usr/share/AAVMF/AAVMF_CODE.verbose.fd"; + vars = "/usr/share/AAVMF/AAVMF_VARS.fd"; + flags = []; + }; + { code = "/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw"; + code_debug = None; + vars = "/usr/share/edk2/aarch64/vars-template-pflash.raw"; + flags = []; + }; +] + +let uefi_x86_64_firmware = [ + { code = "/usr/share/OVMF/OVMF_CODE.fd"; + code_debug = None; + vars = "/usr/share/OVMF/OVMF_VARS.fd"; + flags = []; + }; + { code = "/usr/share/OVMF/OVMF_CODE.secboot.fd"; + code_debug = None; + vars = "/usr/share/OVMF/OVMF_VARS.fd"; + flags = [UEFI_FLAG_SECURE_BOOT_REQUIRED]; + }; + { code = "/usr/share/edk2/ovmf/OVMF_CODE.fd"; + code_debug = None; + vars = "/usr/share/edk2/ovmf/OVMF_VARS.fd"; + flags = []; + }; + { code = "/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd"; + code_debug = None; + vars = "/usr/share/edk2/ovmf/OVMF_VARS.fd"; + flags = [UEFI_FLAG_SECURE_BOOT_REQUIRED]; + }; + { code = "/usr/share/qemu/ovmf-x86_64-code.bin"; + code_debug = None; + vars = "/usr/share/qemu/ovmf-x86_64-vars.bin"; + flags = []; + }; +] diff --git a/common/mlv2v/uefi.mli b/common/mlv2v/uefi.mli new file mode 100644 index 000000000..df45218f9 --- /dev/null +++ b/common/mlv2v/uefi.mli @@ -0,0 +1,35 @@ +(* libguestfs generated file + * WARNING: THIS FILE IS GENERATED + * from the code in the generator/ subdirectory. + * ANY CHANGES YOU MAKE TO THIS FILE WILL BE LOST. + * + * Copyright (C) 2009-2019 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) + +(** UEFI paths. *) + +type uefi_firmware = { + code : string; (** code file *) + code_debug : string option; (** code debug file *) + vars : string; (** vars template file *) + flags : uefi_flags; (** flags *) +} +and uefi_flags = uefi_flag list +and uefi_flag = UEFI_FLAG_SECURE_BOOT_REQUIRED + +val uefi_aarch64_firmware : uefi_firmware list +val uefi_x86_64_firmware : uefi_firmware list -- 2.23.0
On Tuesday, 15 October 2019 10:01:28 CEST Richard W.M. Jones wrote:> I got a little way into this. The two attached patches are > preliminary work.I see the work was done already, so I guess providing alternative ideas (or opinions, apparently) is of no use now...> My proposed split is: > > libguestfs.git > common -> git submodule libguestfs-common.git > generator/ > lib/ > all language bindings > C based tools (eg. virt-df, virt-edit, guestfish) > > guestfs-tools.git > common -> git submodule libguestfs-common.git > virt-builder, virt-customize, virt-sparsify, virt-sparsify, etcI do not think splitting these tools in an own repository makes much sense. What is the goal/advantage you get by splitting them in an own repository, compared to the ones left in libguestfs.git? Users do not care about virt-customize written in OCaml rather than C, and it makes harder to eventually rewrite a C tool in OCaml.> The current common/ subdirectory would become a git submodule. While > git submodules are awkward, they do solve this particular problem with > having common code shared across the repositories, there's only one > git submodule and it's under our control. It does mean that any time > there's a change to common/, we would need to add a commit to the > other 3 repos updating the submodule hash.The current common/ subdirectory is a giant mixup of different components needed by some or just one tool each; few examples: - common/mlaugeas -> only for the daemon - common/mllibvirt -> only for v2v - common/mlxml -> only for v2v - some of the ml modules are need by any OCaml stuff - some of the ml modules are used by 1/2 tools - etc -- Pino Toscano
Richard W.M. Jones
2019-Oct-15 16:39 UTC
Re: [Libguestfs] Splitting the large libguestfs repo
On Tue, Oct 15, 2019 at 03:59:04PM +0200, Pino Toscano wrote:> On Tuesday, 15 October 2019 10:01:28 CEST Richard W.M. Jones wrote: > > I got a little way into this. The two attached patches are > > preliminary work. > > I see the work was done already, so I guess providing alternative ideas > (or opinions, apparently) is of no use now...It's always valued.> > My proposed split is: > > > > libguestfs.git > > common -> git submodule libguestfs-common.git > > generator/ > > lib/ > > all language bindings > > C based tools (eg. virt-df, virt-edit, guestfish) > > > > guestfs-tools.git > > common -> git submodule libguestfs-common.git > > virt-builder, virt-customize, virt-sparsify, virt-sparsify, etc > > I do not think splitting these tools in an own repository makes much > sense. What is the goal/advantage you get by splitting them in an own > repository, compared to the ones left in libguestfs.git? Users do not > care about virt-customize written in OCaml rather than C, and it makes > harder to eventually rewrite a C tool in OCaml.This part of the split isn't absolutely necessary, I really wanted to concentrate on virt-v2v and get that done, partly just because dealing with virt-v2v inside the bigger repo is such a pain. You're right that people don't care about what a tool is written in, so another idea might be to put the C _and_ OCaml tools into the guestfs-tools.git repo (leaving lib + daemon + language bindings only in libguestfs.git). In fact I like this better now I think about it.> > The current common/ subdirectory would become a git submodule. While > > git submodules are awkward, they do solve this particular problem with > > having common code shared across the repositories, there's only one > > git submodule and it's under our control. It does mean that any time > > there's a change to common/, we would need to add a commit to the > > other 3 repos updating the submodule hash. > > The current common/ subdirectory is a giant mixup of different > components needed by some or just one tool each; few examples: > - common/mlaugeas -> only for the daemon > - common/mllibvirt -> only for v2v > - common/mlxml -> only for v2v > - some of the ml modules are need by any OCaml stuff > - some of the ml modules are used by 1/2 tools > - etcIt's a step on the path rather than the end point. We can definitely move mllibvirt & mlxml to virt-v2v in future. We can also try to organize it better or split it in future. The alternatives to the submodule are somehow packaging everything up into a library, which would then become a build dep of the tools. This isn't particularly nice because (as you say) it's a big mix of miscellaneous "stuff", essentially a core library of missing C and OCaml functionality that we happened to need. So therefore not useful as an independent library. I'm not overjoyed by git submodules, but in this particular instance I think it can make sense, and there's only one of them (at the moment, we could change that). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
Possibly Parallel Threads
- [PATCH] customize: Move virt-customize-related code to a separate
- [PATCH 0/4] Various small build fixes
- [PATCH v2 0/6] Various small build fixes
- [PATCH v4 0/2] mllib: Various fixes and changes to Getopt module.
- [PATCH v3 0/3] mllib: Various fixes and changes to Getopt module.