Pkg xen devel - Jan 2021 - [PATCH 09/16] d/shuffle-boot-files: Rework version string handling

Presently PyGRUB and PvGRUB are the DomU bootloaders available in
Debian.  Several alternative DomU bootloaders exist, but are not yet
available.  Add a recommendation of "xen-domu-bootloader" meant to be
provided by these options.

While the "traditional" Linux bridge has been favored for a long time,
presently Open vSwitch is gaining favor.  Add ovn-host as an alternative
recommendation to bridge-utils.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/control | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index 5e24f5fd3c..fc4523945c 100644
--- a/debian/control
+++ b/debian/control
@@ -102,7 +102,7 @@ Section: admin
 Architecture: amd64 arm64 armhf i386
 Provides: xen-utils
 Depends: ${shlibs:Depends}, ${misc:Depends}, python3, xen-utils-common (>=
${source:Version})
-Recommends: bridge-utils, libc6-xen [i386], xen-hypervisor-4.14,
qemu-system-x86, grub-xen-host [i386 amd64]
+Recommends: ovn-host|bridge-utils, libc6-xen [i386], xen-hypervisor-4.14,
qemu-system-x86, xen-domu-bootloader|grub-xen-host
 Suggests: qemu-utils [i386 amd64], seabios [i386 amd64], ovmf
 Description: XEN administrative tools
  The userspace tools to manage a system virtualized through the XEN virtual
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

>From the description of messages, I'm left suspecting it is the list
server corrupting patches, not the MTAs.  Perhaps there is extra magic in
`git send-email` which tickles list servers in magic ways, but again that
needs `git send-email` in a place which mismatches my setup (fine,
I've got an odd setup, that doesn't make it wrong).

Bottom line, I'm directly Cc'ing Hans on this series, let us see whether
that gets things through correctly...


My primary goal is having cross-builds working.  I've ended up picking up
some extra pieces related to this along the way.  I'm a bit worried this
is turning into a misnamed trilogy.  The earlier patches are more
crucial/valuable, the later patches are pretty well optional.

Without the first four, cross-builds simply don't work for me.  One quirk
of my builds is I've been building with ACPI support, so this may well be
a case of making things work for me while breaking for others.

I grimace at `xen-utils-wrapper` having a built-in copy of `xen-version`,
but the issue is how often `xen-utils-wrapper` is used.
`xen-utils-wrapper` is used *really* often.  Every invocation of `xl`
uses `xen-utils-wrapper`, which means overhead in `xen-utils-wrapper` is
massively magnified.  I haven't done proper benchmarking, but my feeling
is this is a Big win.


My feeling is the current packaging is suboptimal.  "PyGRUB" really
should be a *separate* package from xen-utils-X.YY.  The reason is better
alternatives to PyGRUB exist for most cases and PyGRUB isn't at all
paired to the version of Xen.  Notably I've confirmed a 4.11 build of
PyGRUB works fine with a 4.14 hypervisor build.

Presently PvGRUB/grub-xen-host is the only packaged alternative to PyGRUB
(and only for x86).  Multiple viable and sometimes better alternatives
exist though.  The two I'm aware of are Tianocore and U-Boot.  I've
filed
wishlist bug #978595 against src:edk2 asking for the Xen-compatible
builds of Tianocore to be built.  With this list of options, the
dependency should be on xen-domu-bootloader (or a better name) and this
should be Provided by grub-xen-host.  A Tianocore build could then also
offer this virtual package name, and a broken-off PyGRUB package would
follow.

With the number of recommendations of Open vSwitch over the traditional
bridge, having ovn-host as an alternative Recommends to bridge-utils
seems a no-brainer.


Then there is the script work I suggest.  These aren't actually required
for functioning cross-builds, but they seemed like good ideas.

A target has been painted on xen-init-list.  Thing is the real solution
means some major work in `xl list`.  This is an in-progress submission to
Xen, at which point /etc/init.d/xendomains will be able to simply use
`xl list` directly.


Elliott Mitchell (16):
  d/shuffle-binaries: Fix binary shuffling script for cross-building
  debian/rules: Setup use of noOCAML profile
  Rework "debian/rules: Do not try to move EFI binaries on armhf"
  debian/rules: Correct shim install step for current Xen
  debian/scripts: Optimize runtime scripts
  debian/control: Update utils Recommends
  debian/xen-utils-common.examples: Remove xm examples
  d/shuffle-boot-files: Merge arguments together
  d/shuffle-boot-files: Rework version string handling
  d/xendomains.init: Use internal `xen` compat wrapper for `xl`
  debian/xendomains.init: Pipe xen-init-list instead of tmp file
  d/shuffle-boot-files: Support filenames besides .gz
  d/shuffle-boot-files: Simplify loop structure
  d/shuffle-binaries: Switch loop from for to while
  d/shuffle-binaries: Switch to POSIX shell, instead of Bash
  d/shuffle-boot-files: Switch to POSIX shell, instead of Bash

 debian/control                          |  7 ++--
 debian/rules                            | 30 +++++++++++------
 debian/scripts/xen-utils-wrapper        | 43 ++++++++++++++++++++++---
 debian/scripts/xen-version              |  6 ++--
 debian/shuffle-binaries                 | 36 ++++++++++-----------
 debian/shuffle-boot-files               | 19 +++++------
 debian/xen-utils-common.examples        |  1 -
 debian/xen-utils-common.xendomains.init | 21 +++++++-----
 8 files changed, 108 insertions(+), 55 deletions(-)

-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

`ldd` doesn't work with cross-builds, so use `file` to detect scripts
and `strings | grep` for identifying linked libraries.  Even though
debhelper depends on file, make the requirement explicit.

Heavily simplify script inner loop.  While the core of that inner loop
was often executed only once, it is best to shrink inner loops.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/control          |  1 +
 debian/shuffle-binaries | 31 ++++++++++++++++---------------
 2 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/debian/control b/debian/control
index 942bf6711c..1f88f1a40f 100644
--- a/debian/control
+++ b/debian/control
@@ -8,6 +8,7 @@ Build-Depends:
    debhelper (>= 10),
    dh-exec,
    dpkg-dev (>= 1.16.0~),
+   file,
    rdfind,
    lsb-release,
    flex, bison,
diff --git a/debian/shuffle-binaries b/debian/shuffle-binaries
index 8a823da10b..d7a832bbe0 100755
--- a/debian/shuffle-binaries
+++ b/debian/shuffle-binaries
@@ -23,28 +23,29 @@ cd=/usr/lib/xen-common/bin
 mkdir -p "$t/$vd"
 
 for binary in `find $t/usr/{bin,sbin} -type f`; do
-	reason=''
-	{ ldd "$binary" ||: ; } | { fgrep '=>' ||: ; } \
-	| (
-		while read lib dummy; do
+	# filter for executables (ignore scripts)
+	file "$binary" | grep -q -eELF.\\+version.\\+interpreter || continue
+
+	# ldd doesn't work for cross-building
+	reason=$(
+		strings "$binary" | grep -e^lib.\\+\\.so\\.\[.0-9\]\\+\$ | \
+		while read lib; do
 			lib=${lib%.so.*}
-			if grep -F "usr/lib/*/$lib.so.*" $list >/dev/null; then
-				reason+=" $lib"
+			if grep -q -F "usr/lib/*/$lib.so.*" $list; then
+				printf " %s" "$lib"
 			fi
 		done
+	)
 
-		if [ "x$reason" = x ]; then
-			exit 0
-		fi
+	# if no reason, then skip
+	[ -n "$reason" ] || continue
 
-		echo "shuffling $binary $reason"
+	echo "shuffling $binary$reason"
 
-		leaf=${binary##*}
-		mv -v $binary $t/$vd/$leaf
-		ln -vs $cd/xen-utils-wrapper $binary
+	mv -v "$binary" "$t/$vd/"
+	ln -vs "$cd/xen-utils-wrapper" "$binary"
 
-		touch debian/shuffle-binaries.stamp
-	)
+	touch "$0.stamp"
 done
 
 if [ ! -e "$0.stamp" ]; then
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

This adds support for use of the "noocaml" profile
(https://wiki.debian.org/BuildProfileSpec).  This allows disabling build
of OCAML portions of Xen.  Compatibility between non-OCAML and OCAML
builds needs to be maintained since updates might make cross-building of
OCAML possible or an installation might replace one with the other.

Presently the primary expected use is to allow cross-building of the Xen
packages.  At this time cross-building for OCAML is very broken.
Hopefully it may work in future, but at this time it does not.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/control |  4 ++--
 debian/rules   | 11 +++++++++++
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/debian/control b/debian/control
index 1f88f1a40f..5e24f5fd3c 100644
--- a/debian/control
+++ b/debian/control
@@ -32,8 +32,8 @@ Build-Depends:
    zlib1g-dev,
    pandoc,
    markdown,
-   ocaml-native-compilers | ocaml-nox,
-   ocaml-findlib,
+   ocaml-native-compilers <!noocaml> | ocaml-nox <!noocaml>,
+   ocaml-findlib <!noocaml>,
 Homepage: https://xenproject.org/
 Vcs-Browser: https://salsa.debian.org/xen-team/debian-xen
 Vcs-Git: https://salsa.debian.org/xen-team/debian-xen.git
diff --git a/debian/rules b/debian/rules
index 78aad5eeda..959542c25a 100755
--- a/debian/rules
+++ b/debian/rules
@@ -164,6 +164,11 @@ export XEN_BUILD_TIME=$(shell LC_ALL=C date -u -d
"@$(SOURCE_DATE_EPOCH)" "+%T")
 export SMBIOS_REL_DATE=$(shell LC_ALL=C date -u -d
"@$(SOURCE_DATE_EPOCH)" "+%m/%d/%Y")
 export VGABIOS_REL_DATE=$(shell LC_ALL=C date -u -d
"@$(SOURCE_DATE_EPOCH)" "+%d %b %Y")
 
+# Disable Xen's OCAML build if specified
+ifneq ($(filter noocaml,$(DEB_BUILD_PROFILES)),)
+make_args_tools += OCAML_TOOLS=n
+endif
+
 export PYBUILD_NAME=xen
 export PYBUILD_DISABLE=test
 export PYBUILD_SYSTEM=distutils
@@ -238,6 +243,12 @@ override_dh_auto_install: $(TEMPLATED_FILES)
 	$(MAKE) $(make_args_tools) DESTDIR=$t \
 		install-{tools,docs} CONFIG_PV_SHIM=n
 	:
+	@# xen-utils-common needs these against another installation w/OCAML
+	if [ -n "$(filter noocaml,$(DEB_BUILD_PROFILES))" ]; then \
+		ln tools/ocaml/xenstored/oxenstored.conf $t/etc/xen/oxenstored.conf ; \
+		ln -s /usr/lib/xen-common/bin/xen-utils-wrapper $t/usr/sbin/oxenstored ; \
+	fi
+	:
 	@# shim install target needs to be run separately because we
 	@# need to pass it the make_args_xen settings, in particular
 	@# on i386 bwe need to pass x86_64 here to actually build it.
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

What is actually needed is a move command which fails if the move
fails (i386/amd64, the EFI files were created, but the move failed), but
succeeds if no files are moved (armhf, no EFI files are created).  A
combination of find/xargs matches this.

This reworks commit 8ff478af61fa8a87806a89bbd618cd9da2354302.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/rules | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/debian/rules b/debian/rules
index 959542c25a..60d99e6dc2 100755
--- a/debian/rules
+++ b/debian/rules
@@ -258,10 +258,7 @@ override_dh_auto_install: $(TEMPLATED_FILES)
 		-C tools/firmware install-shim
 	:
 	@# Inexplicably, upstream puts the efi binares in usr/lib64
-	case $(flavour) in \
-		armhf) ;; \
-		*) mv $t/usr/lib64/efi/* $t/boot/. ;; \
-	esac
+	find "$t"/usr/lib*/efi -mindepth 1 -maxdepth 1 -print0 | xargs -r -0
mv -t "$t/boot"
 	:
 	@# This file contains an arch-specific path and we put it
 	@# in xen-utils-common, an arch-all package.  But the
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

When originally implemented, the separated shim install step relied on
the shim install being a NOP on shimless architectures.  Either this is
no longer the case, or else cross-building confuses the architecture
detection.

Take out a typo while at it.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/rules | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/debian/rules b/debian/rules
index 60d99e6dc2..0a7fcc9553 100755
--- a/debian/rules
+++ b/debian/rules
@@ -251,11 +251,13 @@ override_dh_auto_install: $(TEMPLATED_FILES)
 	:
 	@# shim install target needs to be run separately because we
 	@# need to pass it the make_args_xen settings, in particular
-	@# on i386 bwe need to pass x86_64 here to actually build it.
-	@# Luckily this target, unlike the build, is a noop on
-	@# shimless arches, so it does not need to be conditional.
-	$(MAKE) $(make_args_xen) DESTDIR=$t $(make_args_xen) \
-		-C tools/firmware install-shim
+	@# on i386 we need to pass x86_64 here to actually build it.
+	case $(flavour) in \
+	amd64|i386) \
+		$(MAKE) $(make_args_xen) DESTDIR=$t $(make_args_xen) \
+			-C tools/firmware install-shim \
+	;; \
+	esac
 	:
 	@# Inexplicably, upstream puts the efi binares in usr/lib64
 	find "$t"/usr/lib*/efi -mindepth 1 -maxdepth 1 -print0 | xargs -r -0
mv -t "$t/boot"
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Fewer fork()s and execve()s quickly add up to significant savings.  I'm
concerned Debian is slowly headed towards recreating SunOS^WSolaris
5.7^W2.7^W7 and the layers and layers of scripts which killed
performance.

As these runtime scripts are heavily used, avoid all uses of external
programs by them.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/scripts/xen-utils-wrapper | 43 +++++++++++++++++++++++++++++---
 debian/scripts/xen-version       |  6 +++--
 2 files changed, 43 insertions(+), 6 deletions(-)

diff --git a/debian/scripts/xen-utils-wrapper b/debian/scripts/xen-utils-wrapper
index 4d27a62d33..6247fd0b29 100755
--- a/debian/scripts/xen-utils-wrapper
+++ b/debian/scripts/xen-utils-wrapper
@@ -1,6 +1,41 @@
-#!/bin/sh -e
+#!/bin/sh
 
-COMMAND="$(basename $0)"
-DIR=$(/usr/lib/xen-common/bin/xen-dir)
+# This portion is fast-path, avoid external programs, NOTE: NOT -e HERE!
+
+COMMAND="${0##*/}"
+
+DIR=/sys/hypervisor/version
+# these return non-zero if hypervisor is absent, this is detected below
+read major 2> /dev/null < $DIR/major
+read minor 2> /dev/null < $DIR/minor
+VERSION="$major.$minor"
+
+DIR="/usr/lib/xen-$VERSION/bin"
+
+[ -x "$DIR/$COMMAND" ] && exec "$DIR/$COMMAND"
"$@"
+
+
+# Certainly didn't work, now report failures, slower-path
+set -e
+
+error() {
+	err="$1"
+	shift
+	echo "ERROR: " "$@" >&2
+	exit $err
+}
+
+[ -e "/sys/hypervisor/type" ] || error 1 "Can't find
hypervisor information in sysfs!"
+
+read type < /sys/hypervisor/type
+if [ "$type" != xen ]; then
+	[ -n "$type" ] || error 1 "Can't read hypervisor type from
sysfs!"
+	error 1 "Hypervisor is not xen but '$type'!"
+fi
+
+if [ ! -d "/usr/lib/xen-$VERSION" ]; then
+	error 127 "Can't find version $VERSION of xen utils (maybe
xen-utils-$VERSION was already removed before rebooting out of Xen $VERSION),
bailing out!"
+fi
+
+error 127 "Unable to execute $DIR/$COMMAND"
 
-exec "$DIR/bin/$COMMAND" "$@"
diff --git a/debian/scripts/xen-version b/debian/scripts/xen-version
index 492070a43b..5c42ad7351 100755
--- a/debian/scripts/xen-version
+++ b/debian/scripts/xen-version
@@ -6,10 +6,12 @@ error() {
 }
 
 if [ -e "/sys/hypervisor/type" ]; then
-    type="$(cat /sys/hypervisor/type)"
+    read type < /sys/hypervisor/type
     if [ "$type" = xen ]; then
         DIR=/sys/hypervisor/version
-        VERSION="$(cat $DIR/major).$(cat $DIR/minor)"
+        read major < $DIR/major
+        read minor < $DIR/minor
+        VERSION="$major.$minor"
     elif [ -z "$type" ]; then
         error "Can't read hypervisor type from sysfs!"
     else
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

`xm` is officially a tool of the past, remove reference to the examples.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/xen-utils-common.examples | 1 -
 1 file changed, 1 deletion(-)

diff --git a/debian/xen-utils-common.examples b/debian/xen-utils-common.examples
index 50cee55177..f296b66852 100644
--- a/debian/xen-utils-common.examples
+++ b/debian/xen-utils-common.examples
@@ -1,2 +1 @@
 debian/tmp/etc/xen/cpupool*
-debian/tmp/etc/xen/xm*
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Rather than passing two strings which are then joined repeatedly and
used to replace filename versions, turn them into a single string.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/rules              | 2 +-
 debian/shuffle-boot-files | 7 +++----
 2 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/debian/rules b/debian/rules
index 0a7fcc9553..77ddce6b12 100755
--- a/debian/rules
+++ b/debian/rules
@@ -298,7 +298,7 @@ xenstore_rm = $(addprefix debian/xen-utils-common/,		\
 override_dh_install:
 	debian/shuffle-binaries $(upstream_version)
 	:
-	debian/shuffle-boot-files $(upstream_version) $(flavour)
+	debian/shuffle-boot-files $(upstream_version)-$(flavour)
 	:
 	dh_install $(dh_install_excludes)
 	if test -d debian/xen-utils-common; then rm -v $(xenstore_rm); fi
diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index 683788d40a..0fb6567ad1 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -2,11 +2,10 @@
 
 set -e
 
-version="$1"; shift
-flavour="$1"; shift
+newvers="$1"; shift
 t=debian/tmp
 
-hv="xen-hypervisor-$version-$flavour"
+hv="xen-hypervisor-$newvers"
 dest="debian/$hv/boot"
 mkdir -p "$dest"
 
@@ -27,5 +26,5 @@ verstring="${verstring##*/}"
 verstring="${verstring%.gz}"
 
 for f in `cd "$t/boot" && find * -type f -print`; do
-	cp -v "$t/boot/$f"
"$dest/${f/$verstring/xen-$version-$flavour}"
+	cp -v "$t/boot/$f" "$dest/${f/$verstring/xen-$newvers}"
 done
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Rework $verstring to end up *strictly* with the version string, not
"xen-<version>".  This allows capturing files which *don't*
include
"xen-" in their filenames.  Notably "xenpolicy" has been
added recently.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-boot-files | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index 0fb6567ad1..bcb7197b1a 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -22,9 +22,9 @@ mkdir -p "$dest"
 
 verstring="$(readlink debian/tmp/boot/xen.gz ||
             readlink debian/tmp/boot/xen)"
-verstring="${verstring##*/}"
+verstring="${verstring##*xen-}"
 verstring="${verstring%.gz}"
 
 for f in `cd "$t/boot" && find * -type f -print`; do
-	cp -v "$t/boot/$f" "$dest/${f/$verstring/xen-$newvers}"
+	cp -v "$t/boot/$f" "$dest/${f/$verstring/$newvers}"
 done
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

During the transition from `xm` to `xl` there was some benefit to using
an external `xen` wrapper to call the appropriate one.  Now `xl` is the
only one in use and has been the only one for some time.  As such adjust
the `xendomains` script to use an internal wrapper instead.

I am highly tempted to purge the `xen` wrapper, but could understand
some users desiring to keep the compatibility around.  Perhaps for a
future where something replaces `xl`.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/xen-utils-common.xendomains.init | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/debian/xen-utils-common.xendomains.init
b/debian/xen-utils-common.xendomains.init
index ca03ae4cfe..e01d41beb6 100644
--- a/debian/xen-utils-common.xendomains.init
+++ b/debian/xen-utils-common.xendomains.init
@@ -17,6 +17,11 @@
 . /lib/init/vars.sh
 . /lib/lsb/init-functions
 
+xen()
+{
+	xl "$@"
+}
+
 xen list &> /dev/null
 if test $? -ne 0
 then
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Shell syntax is has some funky behavior in some areas/ways.  For these
sections /usr/lib/xen-common/bin/xen-init-list should be piped to the
loop, rather than creating a temporary file.  This is also pure-Bourne
shell syntax, rather than requiring Bash.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/xen-utils-common.xendomains.init | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/debian/xen-utils-common.xendomains.init
b/debian/xen-utils-common.xendomains.init
index e01d41beb6..3704fde27c 100644
--- a/debian/xen-utils-common.xendomains.init
+++ b/debian/xen-utils-common.xendomains.init
@@ -158,10 +158,10 @@ do_stop_migrate()
 {
   [ -n "$XENDOMAINS_MIGRATE" ] || return
 
-  while read id name rest; do
+  /usr/lib/xen-common/bin/xen-init-list | while read id name discard; do
     log_action_begin_msg "Migrating Xen domain $name ($id)"
     (timeout_coproc "$XENDOMAINS_STOP_MAXWAIT" xen migrate $id
$XENDOMAINS_MIGRATE)
-  done < <(/usr/lib/xen-common/bin/xen-init-list)
+  done
 }
 
 do_stop_save()
@@ -169,24 +169,24 @@ do_stop_save()
   [ -n "$XENDOMAINS_SAVE" ] || return
   [ -d "$XENDOMAINS_SAVE" ] || mkdir -m 0700 -p
"$XENDOMAINS_SAVE"
 
-  while read id name rest; do
+  /usr/lib/xen-common/bin/xen-init-list | while read id name discard; do
     log_action_begin_msg "Saving Xen domain $name ($id)"
     (timeout_coproc "$XENDOMAINS_STOP_MAXWAIT" xen save $id
$XENDOMAINS_SAVE/$name)
-  done < <(/usr/lib/xen-common/bin/xen-init-list)
+  done
 }
 
 do_stop_shutdown()
 {
-  while read id name rest; do
+  /usr/lib/xen-common/bin/xen-init-list | while read id name discard; do
     log_action_begin_msg "Shutting down Xen domain $name ($id)"
     xen shutdown $id 2>&1 1>/dev/null
     log_action_end_msg $?
-  done < <(/usr/lib/xen-common/bin/xen-init-list)
-  while read id name rest; do
+  done
+  /usr/lib/xen-common/bin/xen-init-list | while read id name discard; do
     log_action_begin_msg "Waiting for Xen domain $name ($id) to shut
down"
     timeout_domain "$name" "$XENDOMAINS_STOP_MAXWAIT"
     log_action_end_msg $?
-  done < <(/usr/lib/xen-common/bin/xen-init-list)
+  done
 }
 
 do_stop()
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

While I'm unaware of initiatives to support formats besides GNU zip, a
number of other formats have appeared which may potentially gain support.
BZIP, LZMA and XZ are notable for substantially better compression ratio,
LZ4 is notable for greater speed.  Adjust the glob patterns to allow for
their use in the future.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-boot-files | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index bcb7197b1a..88bcdd8d68 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -20,10 +20,10 @@ mkdir -p "$dest"
 # more useful when the -i386 flavour existed and was coinstallable
 # with the -amd64 flavour.)
 
-verstring="$(readlink debian/tmp/boot/xen.gz ||
+verstring="$(readlink debian/tmp/boot/xen.?z* ||
             readlink debian/tmp/boot/xen)"
 verstring="${verstring##*xen-}"
-verstring="${verstring%.gz}"
+verstring="${verstring%.?z*}"
 
 for f in `cd "$t/boot" && find * -type f -print`; do
 	cp -v "$t/boot/$f" "$dest/${f/$verstring/$newvers}"
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Rather than changing into the work directory and then running find,
instead simply run find and strip off the directory name later.  Since
the directory name was being re-added to the source filename later, this
reduces complexity.  Instead the work directory name needs to be stripped
off the target filename.

Put another way, remove all processing of source filename and do more
processing of target filename.

While at it use structures which conform to POSIX, allowing the use of
/bin/sh instead of bash.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-boot-files | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index 88bcdd8d68..f4b11eddb6 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -25,6 +25,8 @@ verstring="$(readlink debian/tmp/boot/xen.?z* ||
 verstring="${verstring##*xen-}"
 verstring="${verstring%.?z*}"
 
-for f in `cd "$t/boot" && find * -type f -print`; do
-	cp -v "$t/boot/$f" "$dest/${f/$verstring/$newvers}"
+find "$t/boot" -type f -print | while read f; do
+	head="${f%$verstring*}"
+	tail="${f#*$verstring}"
+	cp -vl "$f" "$dest${head#$t/boot}$newvers${tail}"
 done
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

A for loop forces the shell to allocate memory for every filename the
script runs on.  While not yet a concern, in a theoretical future where
hundreds or thousands of files are being worked with, this would become
a concern.  Additionally using "$t"/usr/*bin is compatible with shells
besides Bash, and will work in a future where /usr/xbin exists.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-binaries | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/shuffle-binaries b/debian/shuffle-binaries
index d7a832bbe0..d41d4e11aa 100755
--- a/debian/shuffle-binaries
+++ b/debian/shuffle-binaries
@@ -22,7 +22,7 @@ cd=/usr/lib/xen-common/bin
 
 mkdir -p "$t/$vd"
 
-for binary in `find $t/usr/{bin,sbin} -type f`; do
+find "$t"/usr/*bin -type f | while read binary; do
 	# filter for executables (ignore scripts)
 	file "$binary" | grep -q -eELF.\\+version.\\+interpreter || continue
 
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

While bash is great for an interactive shell, it is poor for scripting
use.  Loading many libraries means large overhead.  Plus its command
parser has to be slow to be user-friendly, instead of fast for scripts.
While pipefail is useful for reliability, with the inner loop
dramatically simplified it now has minimal value.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-binaries | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/debian/shuffle-binaries b/debian/shuffle-binaries
index d41d4e11aa..d2c5f14c90 100755
--- a/debian/shuffle-binaries
+++ b/debian/shuffle-binaries
@@ -1,6 +1,5 @@
-#!/bin/bash
+#!/bin/sh
 set -e
-set -o pipefail
 
 version="$1"; shift
 
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

While bash is great for an interactive shell, it is poor for scripting
use.  Loading many libraries means large overhead.  Plus its command
parser has to be slow to be user-friendly, instead of fast for scripts.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-boot-files | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index f4b11eddb6..7c0f53deff 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/sh
 
 set -e
 
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

On 1/3/21 4:19 AM, Elliott Mitchell wrote:
> From the description of messages, I'm left suspecting it is the list
> server corrupting patches, not the MTAs.  Perhaps there is extra magic in
> `git send-email` which tickles list servers in magic ways, but again that
> needs `git send-email` in a place which mismatches my setup (fine,
> I've got an odd setup, that doesn't make it wrong).
It is wrong. To be explicit, from now on I will silently ignore
everything that is not sent using git send-email.
> My primary goal is having cross-builds working.
And for that reason I'm trying to get some things in that will help
achieving that goal.

I had to throw some things out again, because of FTBFS or lintian
errors. I have been spending too much time discovering this while it is
avoidable. Do your builds, also the normal builds, run lintian etc.
before sending all of it.
> I've ended up picking up
> some extra pieces related to this along the way.  I'm a bit worried
this
> is turning into a misnamed trilogy.  The earlier patches are more
> crucial/valuable, the later patches are pretty well optional.
Well, learning some more about converting from bash to sh was
interesting. And I do also like the pipe into loop more than overly use
of sub shells.

But, really, trying to figure out what you are doing and why is rather
labor intensive and sometimes mind boggling. Try to just tell what you
want and why and do realize that if you send something, someone else has
to actually spend time to read it. Writing essays and contemplating
about hypothetical theoretical futures and overly use of unquantified
language like substantially better, greater speed, reduces complexity,
hundreds or thousands of files, large overhead, slow, fast,
dramatically, etc. is not that helpful.
> [...]
Have fun,
Hans