thr3ads.net - Pkg xen devel - [Pkg-xen-devel] [PATCH 00/19] Fixing cross-compilation, some script work, revised [Dec 2020]

If this information is useful, please help other people find it:
Share via:

Elliott Mitchell

2020-Dec-02 17:34 UTC

[Pkg-xen-devel] [PATCH 00/19] Fixing cross-compilation, some script work, revised

Hopefully this takes care of the comments which have been made.  Quite a
few commit messages were adjusted.  Two scripting commits were broken
into tiny pieces.

There are a number of small differences, but the end result will be the
same.

Hopefully I've figured out the right set of flags for the way you prefer
patches being sent.  Right level/type of threading, etc.  The dates may
differ since Git uses the author date, NOT the commit date when sending
messages.  These were mostly authored in July, those are the dates they
get.  Since some were originally added in September, it gets that date
header.  Your mailer might choose to sort them this way, which differs
from sequence number.

Elliott Mitchell (19):
  debian/rules: Correct shim install step for current Xen
  d/shuffle-binaries: Remove useless extra argument being passed in
  d/shuffle-binaries: Make error detection/message overt
  d/shuffle-binaries: Add quoting for potentially changeable variables
  d/shuffle-binaries: Switch loop from for to while
  d/shuffle-binaries: Fix binary shuffling script for cross-building
  d/shuffle-binaries: Switch to POSIX shell, instead of Bash
  d/shuffle-boot-files: The Great Quotification
  d/shuffle-boot-files: Merge arguments together
  d/shuffle-boot-files: Support filenames besides .gz
  d/shuffle-boot-files: Rework version string handling
  d/shuffle-boot-files: Simplify loop structure
  d/shuffle-boot-files: Switch to POSIX shell, instead of Bash
  debian/rules: Set CC/LD to enable cross-building
  debian/rules: Setup use of noOCAML profile
  Rework "debian/rules: Do not try to move EFI binaries on armhf"
  debian/xen.init: Load xen_acpi_processor on boot
  debian/scripts: Optimize runtime scripts
  d/xendomains.init: Remove use of `xen` compat wrapper for `xl`

 debian/control                          |  5 +--
 debian/rules                            | 34 ++++++++++++------
 debian/scripts/xen-utils-wrapper        | 43 +++++++++++++++++++---
 debian/scripts/xen-version              |  6 ++--
 debian/shuffle-binaries                 | 48 +++++++++++++------------
 debian/shuffle-boot-files               | 25 ++++++-------
 debian/xen-utils-common.xen.init        |  1 +
 debian/xen-utils-common.xendomains.init |  5 +++
 8 files changed, 114 insertions(+), 53 deletions(-)

-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:49 UTC

head link

[Pkg-xen-devel] [PATCH 01/19] debian/rules: Correct shim install step for current Xen

When originally implemented, the separated shim install step relied on
the shim install being a NOP on shimless architectures.  Either this is
no longer the case, or else cross-building confuses the architecture
detection.

Take out a typo while at it.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/rules | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/debian/rules b/debian/rules
index b21c9e6948..741676e617 100755
--- a/debian/rules
+++ b/debian/rules
@@ -223,11 +223,13 @@ override_dh_auto_install: $(TEMPLATED_FILES)
 	:
 	@# shim install target needs to be run separately because we
 	@# need to pass it the make_args_xen settings, in particular
-	@# on i386 bwe need to pass x86_64 here to actually build it.
-	@# Luckily this target, unlike the build, is a noop on
-	@# shimless arches, so it does not need to be conditional.
-	$(MAKE) $(make_args_xen) DESTDIR=$t $(make_args_xen) \
-		-C tools/firmware install-shim
+	@# on i386 we need to pass x86_64 here to actually build it.
+	case $(flavour) in \
+	amd64|i386) \
+		$(MAKE) $(make_args_xen) DESTDIR=$t $(make_args_xen) \
+			-C tools/firmware install-shim \
+	;; \
+	esac
 	:
 	@# Inexplicably, upstream puts the efi binares in usr/lib64
 	case $(flavour) in \
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:49 UTC

head link

[Pkg-xen-devel] [PATCH 02/19] d/shuffle-binaries: Remove useless extra argument being passed in

This doesn't do anything.  Really it is almost a bug the script
/doesn't/
complain.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/rules | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/rules b/debian/rules
index 741676e617..c7dc55ad03 100755
--- a/debian/rules
+++ b/debian/rules
@@ -271,7 +271,7 @@ xenstore_rm = $(addprefix debian/xen-utils-common/,		\
 		))
 
 override_dh_install:
-	debian/shuffle-binaries $(upstream_version) $(flavour)
+	debian/shuffle-binaries $(upstream_version)
 	:
 	debian/shuffle-boot-files $(upstream_version) $(flavour)
 	:
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:49 UTC

head link

[Pkg-xen-devel] [PATCH 03/19] d/shuffle-binaries: Make error detection/message overt

The reason for the `ls` at the end is pretty straightforward if you think
about it, mainly confirming the shuffle step did something.  Yet that
is a bit non-obvious, and the error message produced on failure is poor
("No such file or directory" simply means the script failed
somewhere?).
Add an overt error message.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-binaries | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/debian/shuffle-binaries b/debian/shuffle-binaries
index cff6de5428..31aa01e2bd 100755
--- a/debian/shuffle-binaries
+++ b/debian/shuffle-binaries
@@ -47,4 +47,7 @@ for binary in `find $t/usr/{bin,sbin} -type f`; do
 	)
 done
 
-ls debian/shuffle-binaries.stamp
+if [ ! -e "$0.stamp" ]; then
+	echo "Failed to shuffle binaries!" 1>&2
+	exit 1
+fi
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:49 UTC

head link

[Pkg-xen-devel] [PATCH 04/19] d/shuffle-binaries: Add quoting for potentially changeable variables

These should originate with the owner of a build system and are unlikely
to get hazardous values.  This script though *should* work on a system
with such a bizzare setup.  On general principle, add double-quotes.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-binaries | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/debian/shuffle-binaries b/debian/shuffle-binaries
index 31aa01e2bd..8a823da10b 100755
--- a/debian/shuffle-binaries
+++ b/debian/shuffle-binaries
@@ -2,7 +2,7 @@
 set -e
 set -o pipefail
 
-version=$1; shift
+version="$1"; shift
 
 # We wrap all of the programs in /usr/bin with a script
 # (xen-utils-wrapper) which chooses the version corresponding to the
@@ -17,10 +17,10 @@ version=$1; shift
 list=debian/libxenmiscV.install.vsn-in
 
 t=debian/tmp
-vd=/usr/lib/xen-$version/bin
+vd="/usr/lib/xen-$version/bin"
 cd=/usr/lib/xen-common/bin
 
-mkdir -p $t/$vd
+mkdir -p "$t/$vd"
 
 for binary in `find $t/usr/{bin,sbin} -type f`; do
 	reason=''
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:49 UTC

head link

[Pkg-xen-devel] [PATCH 05/19] d/shuffle-binaries: Switch loop from for to while

A for loop forces the shell to allocate memory for every filename the
script runs on.  While not yet a concern, in a theoretical future where
hundreds or thousands of files are being worked with, this would become
a concern.  Additionally using "$t"/usr/*bin is compatible with shells
besides Bash, and will work in a future where /usr/xbin exists.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-binaries | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/debian/shuffle-binaries b/debian/shuffle-binaries
index 8a823da10b..37176d26e7 100755
--- a/debian/shuffle-binaries
+++ b/debian/shuffle-binaries
@@ -22,7 +22,8 @@ cd=/usr/lib/xen-common/bin
 
 mkdir -p "$t/$vd"
 
-for binary in `find $t/usr/{bin,sbin} -type f`; do
+find "$t"/usr/*bin -type f | while read binary; do
+
 	reason=''
 	{ ldd "$binary" ||: ; } | { fgrep '=>' ||: ; } \
 	| (
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:49 UTC

head link

[Pkg-xen-devel] [PATCH 06/19] d/shuffle-binaries: Fix binary shuffling script for cross-building

`ldd` doesn't work with cross-builds, so use `file` to detect scripts
and `strings | grep` for identifying linked libraries.  Even though
debhelper depends on file, make the requirement explicit.

Heavily simplify script inner loop.  While the core of that inner loop
was often executed only once, it is best to shrink inner loops.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/control          |  1 +
 debian/shuffle-binaries | 31 ++++++++++++++++---------------
 2 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/debian/control b/debian/control
index 942bf6711c..90007f6a02 100644
--- a/debian/control
+++ b/debian/control
@@ -6,6 +6,7 @@ Section: admin
 Standards-Version: 4.5.0
 Build-Depends:
    debhelper (>= 10),
+   file,
    dh-exec,
    dpkg-dev (>= 1.16.0~),
    rdfind,
diff --git a/debian/shuffle-binaries b/debian/shuffle-binaries
index 37176d26e7..03b4bc4794 100755
--- a/debian/shuffle-binaries
+++ b/debian/shuffle-binaries
@@ -24,28 +24,29 @@ mkdir -p "$t/$vd"
 
 find "$t"/usr/*bin -type f | while read binary; do
 
-	reason=''
-	{ ldd "$binary" ||: ; } | { fgrep '=>' ||: ; } \
-	| (
-		while read lib dummy; do
+	# filter for executables (ignore scripts)
+	file "$binary" | grep -q -eELF.\\+version.\\+interpreter || continue
+
+	# ldd doesn't work for cross-building
+	reason=$(
+		strings "$binary" | grep -e^lib.\\+\\.so\\.\[.0-9\]\\+\$ | \
+		while read lib; do
 			lib=${lib%.so.*}
-			if grep -F "usr/lib/*/$lib.so.*" $list >/dev/null; then
-				reason+=" $lib"
+			if grep -q -F "usr/lib/*/$lib.so.*" $list; then
+				printf " %s" "$lib"
 			fi
 		done
+	)
 
-		if [ "x$reason" = x ]; then
-			exit 0
-		fi
+	# if no reason, then skip
+	[ -n "$reason" ] || continue
 
-		echo "shuffling $binary $reason"
+	echo "shuffling $binary$reason"
 
-		leaf=${binary##*}
-		mv -v $binary $t/$vd/$leaf
-		ln -vs $cd/xen-utils-wrapper $binary
+	mv -v "$binary" "$t/$vd/"
+	ln -vs "$cd/xen-utils-wrapper" "$binary"
 
-		touch debian/shuffle-binaries.stamp
-	)
+	touch "$0.stamp"
 done
 
 if [ ! -e "$0.stamp" ]; then
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:50 UTC

head link

[Pkg-xen-devel] [PATCH 07/19] d/shuffle-binaries: Switch to POSIX shell, instead of Bash

While bash is great for an interactive shell, it is poor for scripting
use.  Loading many libraries means large overhead.  Plus its command
parser has to be slow to be user-friendly, instead of fast for scripts.
While pipefail is useful for reliability, with the inner loop
dramatically simplified it now has minimal value.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-binaries | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/debian/shuffle-binaries b/debian/shuffle-binaries
index 03b4bc4794..19467d6496 100755
--- a/debian/shuffle-binaries
+++ b/debian/shuffle-binaries
@@ -1,6 +1,5 @@
-#!/bin/bash
+#!/bin/sh
 set -e
-set -o pipefail
 
 version="$1"; shift
 
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:50 UTC

head link

[Pkg-xen-devel] [PATCH 08/19] d/shuffle-boot-files: The Great Quotification

These should originate with the owner of a build system and are unlikely
to get hazardous values.  This script though *should* work on a system
with such a bizzare setup.  On general principle, add lots of double-quotes.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-boot-files | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index f7492c0c72..683788d40a 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -2,13 +2,13 @@
 
 set -e
 
-version=$1; shift
-flavour=$1; shift
+version="$1"; shift
+flavour="$1"; shift
 t=debian/tmp
 
-hv=xen-hypervisor-$version-$flavour
-dest=debian/$hv/boot
-mkdir -p $dest
+hv="xen-hypervisor-$version-$flavour"
+dest="debian/$hv/boot"
+mkdir -p "$dest"
 
 # The upstream build system puts a pile of needless symlinks in /boot.
 #
@@ -21,11 +21,11 @@ mkdir -p $dest
 # more useful when the -i386 flavour existed and was coinstallable
 # with the -amd64 flavour.)
 
-verstring=$(readlink debian/tmp/boot/xen.gz ||
-            readlink debian/tmp/boot/xen)
-verstring=${verstring##*/}
-verstring=${verstring%.gz}
+verstring="$(readlink debian/tmp/boot/xen.gz ||
+            readlink debian/tmp/boot/xen)"
+verstring="${verstring##*/}"
+verstring="${verstring%.gz}"
 
-for f in `cd $t/boot && find * -type f -print`; do
-	cp -v $t/boot/$f $dest/${f/$verstring/xen-$version-$flavour}
+for f in `cd "$t/boot" && find * -type f -print`; do
+	cp -v "$t/boot/$f"
"$dest/${f/$verstring/xen-$version-$flavour}"
 done
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:50 UTC

head link

[Pkg-xen-devel] [PATCH 09/19] d/shuffle-boot-files: Merge arguments together

Rather than passing two strings which are then joined repeatedly and
used to replace filename versions, turn them into a single string.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/rules              | 2 +-
 debian/shuffle-boot-files | 7 +++----
 2 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/debian/rules b/debian/rules
index c7dc55ad03..95f3cc4311 100755
--- a/debian/rules
+++ b/debian/rules
@@ -273,7 +273,7 @@ xenstore_rm = $(addprefix debian/xen-utils-common/,		\
 override_dh_install:
 	debian/shuffle-binaries $(upstream_version)
 	:
-	debian/shuffle-boot-files $(upstream_version) $(flavour)
+	debian/shuffle-boot-files $(upstream_version)-$(flavour)
 	:
 	dh_install $(dh_install_excludes)
 	if test -d debian/xen-utils-common; then rm -v $(xenstore_rm); fi
diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index 683788d40a..0fb6567ad1 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -2,11 +2,10 @@
 
 set -e
 
-version="$1"; shift
-flavour="$1"; shift
+newvers="$1"; shift
 t=debian/tmp
 
-hv="xen-hypervisor-$version-$flavour"
+hv="xen-hypervisor-$newvers"
 dest="debian/$hv/boot"
 mkdir -p "$dest"
 
@@ -27,5 +26,5 @@ verstring="${verstring##*/}"
 verstring="${verstring%.gz}"
 
 for f in `cd "$t/boot" && find * -type f -print`; do
-	cp -v "$t/boot/$f"
"$dest/${f/$verstring/xen-$version-$flavour}"
+	cp -v "$t/boot/$f" "$dest/${f/$verstring/xen-$newvers}"
 done
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:50 UTC

head link

[Pkg-xen-devel] [PATCH 10/19] d/shuffle-boot-files: Support filenames besides .gz

While I'm unaware of initiatives to support formats besides GNU zip, a
number of other formats have appeared which may potentially gain support.
BZIP, LZMA and XZ are notable for substantially better compression ratio,
LZ4 is notable for greater speed.  Adjust the glob patterns to allow for
their use in the future.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-boot-files | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index 0fb6567ad1..e9058d0e3a 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -20,10 +20,10 @@ mkdir -p "$dest"
 # more useful when the -i386 flavour existed and was coinstallable
 # with the -amd64 flavour.)
 
-verstring="$(readlink debian/tmp/boot/xen.gz ||
+verstring="$(readlink debian/tmp/boot/xen.?z* ||
             readlink debian/tmp/boot/xen)"
 verstring="${verstring##*/}"
-verstring="${verstring%.gz}"
+verstring="${verstring%.?z*}"
 
 for f in `cd "$t/boot" && find * -type f -print`; do
 	cp -v "$t/boot/$f" "$dest/${f/$verstring/xen-$newvers}"
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:50 UTC

head link

[Pkg-xen-devel] [PATCH 11/19] d/shuffle-boot-files: Rework version string handling

Rework $verstring to end up *strictly* with the version string, not
"xen-<version>".  This allows capturing files which *don't*
include
"xen-" in their filenames.  Notably "xenpolicy" has been
added recently.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-boot-files | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index e9058d0e3a..88bcdd8d68 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -22,9 +22,9 @@ mkdir -p "$dest"
 
 verstring="$(readlink debian/tmp/boot/xen.?z* ||
             readlink debian/tmp/boot/xen)"
-verstring="${verstring##*/}"
+verstring="${verstring##*xen-}"
 verstring="${verstring%.?z*}"
 
 for f in `cd "$t/boot" && find * -type f -print`; do
-	cp -v "$t/boot/$f" "$dest/${f/$verstring/xen-$newvers}"
+	cp -v "$t/boot/$f" "$dest/${f/$verstring/$newvers}"
 done
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:50 UTC

head link

[Pkg-xen-devel] [PATCH 12/19] d/shuffle-boot-files: Simplify loop structure

Rather than changing into the work directory and then running find,
instead simply run find and strip off the directory name later.  Since
the directory name was being re-added to the source filename later, this
reduces complexity.  Instead the work directory name needs to be stripped
off the target filename.

Put another way, remove all processing of source filename and do more
processing of target filename.

While at it use structures which conform to POSIX, allowing the use of
/bin/sh instead of bash.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-boot-files | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index 88bcdd8d68..f4b11eddb6 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -25,6 +25,8 @@ verstring="$(readlink debian/tmp/boot/xen.?z* ||
 verstring="${verstring##*xen-}"
 verstring="${verstring%.?z*}"
 
-for f in `cd "$t/boot" && find * -type f -print`; do
-	cp -v "$t/boot/$f" "$dest/${f/$verstring/$newvers}"
+find "$t/boot" -type f -print | while read f; do
+	head="${f%$verstring*}"
+	tail="${f#*$verstring}"
+	cp -vl "$f" "$dest${head#$t/boot}$newvers${tail}"
 done
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:51 UTC

head link

[Pkg-xen-devel] [PATCH 13/19] d/shuffle-boot-files: Switch to POSIX shell, instead of Bash

While bash is great for an interactive shell, it is poor for scripting
use.  Loading many libraries means large overhead.  Plus its command
parser has to be slow to be user-friendly, instead of fast for scripts.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/shuffle-boot-files | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/shuffle-boot-files b/debian/shuffle-boot-files
index f4b11eddb6..7c0f53deff 100755
--- a/debian/shuffle-boot-files
+++ b/debian/shuffle-boot-files
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/sh
 
 set -e
 
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:51 UTC

head link

[Pkg-xen-devel] [PATCH 14/19] debian/rules: Set CC/LD to enable cross-building

Always set $(CC) and $(LD) when building.  This has no effect on native
builds, but is needed for cross-builds to use the right compiler.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/rules | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/debian/rules b/debian/rules
index 95f3cc4311..dffdef9a9d 100755
--- a/debian/rules
+++ b/debian/rules
@@ -135,6 +135,8 @@ dpkg_CPPFLAGS := $(shell $(dbmo) dpkg-buildflags --get
CPPFLAGS)
 dpkg_LDFLAGS  := $(shell $(dbmo) dpkg-buildflags --get LDFLAGS)
 
 make_args_common := \
+	CC=$(DEB_HOST_MULTIARCH)-gcc \
+	LD=$(DEB_HOST_MULTIARCH)-ld \
 	XEN_COMPILE_ARCH=$(xen_arch_$(DEB_BUILD_ARCH))
 
 make_args_xen := $(make_args_common) \
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:51 UTC

head link

[Pkg-xen-devel] [PATCH 15/19] debian/rules: Setup use of noOCAML profile

This adds support for use of the "noocaml" profile
(https://wiki.debian.org/BuildProfileSpec).  This allows disabling build
of OCAML portions of Xen.  Compatibility between non-OCAML and OCAML
builds needs to be maintained since updates might make cross-building of
OCAML possible or an installation might replace one with the other.

Presently the primary expected use is to allow cross-building of the Xen
packages.  At this time cross-building for OCAML is very broken.
Hopefully it may work in future, but at this time it does not.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/control |  4 ++--
 debian/rules   | 11 +++++++++++
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/debian/control b/debian/control
index 90007f6a02..4c25e95971 100644
--- a/debian/control
+++ b/debian/control
@@ -32,8 +32,8 @@ Build-Depends:
    zlib1g-dev,
    pandoc,
    markdown,
-   ocaml-native-compilers | ocaml-nox,
-   ocaml-findlib,
+   ocaml-native-compilers <!noocaml> | ocaml-nox <!noocaml>,
+   ocaml-findlib <!noocaml>,
 Homepage: https://xenproject.org/
 Vcs-Browser: https://salsa.debian.org/xen-team/debian-xen
 Vcs-Git: https://salsa.debian.org/xen-team/debian-xen.git
diff --git a/debian/rules b/debian/rules
index dffdef9a9d..e9f334f7d3 100755
--- a/debian/rules
+++ b/debian/rules
@@ -149,6 +149,11 @@ make_args_tools := $(make_args_common) \
 	EXTRA_CFLAGS_XEN_TOOLS='$(dpkg_CFLAGS) $(dpkg_CPPFLAGS)' \
 	PREPEND_LDFLAGS_XEN_TOOLS='$(dpkg_LDFLAGS)'
 
+# Disable Xen's OCAML build if specified
+ifneq ($(filter noocaml,$(DEB_BUILD_PROFILES)),)
+make_args_tools += OCAML_TOOLS=n
+endif
+
 export PYBUILD_NAME=xen
 export PYBUILD_DISABLE=test
 export PYBUILD_SYSTEM=distutils
@@ -223,6 +228,12 @@ override_dh_auto_install: $(TEMPLATED_FILES)
 	$(MAKE) $(make_args_tools) DESTDIR=$t \
 		install-{tools,docs} CONFIG_PV_SHIM=n
 	:
+	@# xen-utils-common needs these against another installation w/OCAML
+	if [ -n "$(filter noocaml,$(DEB_BUILD_PROFILES))" ]; then \
+		ln tools/ocaml/xenstored/oxenstored.conf $t/etc/xen/oxenstored.conf ; \
+		ln -s /usr/lib/xen-common/bin/xen-utils-wrapper $t/usr/sbin/oxenstored ; \
+	fi
+	:
 	@# shim install target needs to be run separately because we
 	@# need to pass it the make_args_xen settings, in particular
 	@# on i386 we need to pass x86_64 here to actually build it.
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:51 UTC

head link

[Pkg-xen-devel] [PATCH 16/19] Rework "debian/rules: Do not try to move EFI binaries on armhf"

What is actually needed is a move command which fails if the move
fails (i386/amd64, the EFI files were created, but the move failed), but
succeeds if no files are moved (armhf, no EFI files are created).  A
combination of find/xargs matches this.

This reworks commit 8ff478af61fa8a87806a89bbd618cd9da2354302.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/rules | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/debian/rules b/debian/rules
index e9f334f7d3..75afea0a95 100755
--- a/debian/rules
+++ b/debian/rules
@@ -245,10 +245,7 @@ override_dh_auto_install: $(TEMPLATED_FILES)
 	esac
 	:
 	@# Inexplicably, upstream puts the efi binares in usr/lib64
-	case $(flavour) in \
-		armhf) ;; \
-		*) mv $t/usr/lib64/efi/* $t/boot/. ;; \
-	esac
+	find "$t/usr/lib64/efi" -mindepth 1 -maxdepth 1 -print0 | xargs -r
-0 mv -t "$t/boot"
 	:
 	@# This file contains an arch-specific path and we put it
 	@# in xen-utils-common, an arch-all package.  But the
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:51 UTC

head link

[Pkg-xen-devel] [PATCH 17/19] debian/xen.init: Load xen_acpi_processor on boot

This allows more control of processor state, potentially resulting in
reduced power usage.  Alternatively simply more information on processor
use.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/xen-utils-common.xen.init | 1 +
 1 file changed, 1 insertion(+)

diff --git a/debian/xen-utils-common.xen.init b/debian/xen-utils-common.xen.init
index 1a64d6767e..8dac05f9de 100644
--- a/debian/xen-utils-common.xen.init
+++ b/debian/xen-utils-common.xen.init
@@ -67,6 +67,7 @@ modules_setup()
 	modprobe xenfs 2>/dev/null
 	modprobe xen-evtchn 2>/dev/null
 	modprobe xen-gntdev 2>/dev/null
+	modprobe xen-acpi-processor 2>/dev/null
 }
 
 xenfs_setup()
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:51 UTC

head link

[Pkg-xen-devel] [PATCH 18/19] debian/scripts: Optimize runtime scripts

Fewer fork()s and execve()s quickly add up to significant savings.  I'm
concerned Debian is slowly headed towards recreating SunOS^WSolaris
5.7^W2.7^W7 and the layers and layers of scripts which killed
performance.

As these runtime scripts are heavily used, avoid all uses of external
programs by them.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/scripts/xen-utils-wrapper | 43 +++++++++++++++++++++++++++++---
 debian/scripts/xen-version       |  6 +++--
 2 files changed, 43 insertions(+), 6 deletions(-)

diff --git a/debian/scripts/xen-utils-wrapper b/debian/scripts/xen-utils-wrapper
index 4d27a62d33..6247fd0b29 100755
--- a/debian/scripts/xen-utils-wrapper
+++ b/debian/scripts/xen-utils-wrapper
@@ -1,6 +1,41 @@
-#!/bin/sh -e
+#!/bin/sh
 
-COMMAND="$(basename $0)"
-DIR=$(/usr/lib/xen-common/bin/xen-dir)
+# This portion is fast-path, avoid external programs, NOTE: NOT -e HERE!
+
+COMMAND="${0##*/}"
+
+DIR=/sys/hypervisor/version
+# these return non-zero if hypervisor is absent, this is detected below
+read major 2> /dev/null < $DIR/major
+read minor 2> /dev/null < $DIR/minor
+VERSION="$major.$minor"
+
+DIR="/usr/lib/xen-$VERSION/bin"
+
+[ -x "$DIR/$COMMAND" ] && exec "$DIR/$COMMAND"
"$@"
+
+
+# Certainly didn't work, now report failures, slower-path
+set -e
+
+error() {
+	err="$1"
+	shift
+	echo "ERROR: " "$@" >&2
+	exit $err
+}
+
+[ -e "/sys/hypervisor/type" ] || error 1 "Can't find
hypervisor information in sysfs!"
+
+read type < /sys/hypervisor/type
+if [ "$type" != xen ]; then
+	[ -n "$type" ] || error 1 "Can't read hypervisor type from
sysfs!"
+	error 1 "Hypervisor is not xen but '$type'!"
+fi
+
+if [ ! -d "/usr/lib/xen-$VERSION" ]; then
+	error 127 "Can't find version $VERSION of xen utils (maybe
xen-utils-$VERSION was already removed before rebooting out of Xen $VERSION),
bailing out!"
+fi
+
+error 127 "Unable to execute $DIR/$COMMAND"
 
-exec "$DIR/bin/$COMMAND" "$@"
diff --git a/debian/scripts/xen-version b/debian/scripts/xen-version
index 492070a43b..5c42ad7351 100755
--- a/debian/scripts/xen-version
+++ b/debian/scripts/xen-version
@@ -6,10 +6,12 @@ error() {
 }
 
 if [ -e "/sys/hypervisor/type" ]; then
-    type="$(cat /sys/hypervisor/type)"
+    read type < /sys/hypervisor/type
     if [ "$type" = xen ]; then
         DIR=/sys/hypervisor/version
-        VERSION="$(cat $DIR/major).$(cat $DIR/minor)"
+        read major < $DIR/major
+        read minor < $DIR/minor
+        VERSION="$major.$minor"
     elif [ -z "$type" ]; then
         error "Can't read hypervisor type from sysfs!"
     else
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Elliott Mitchell

2020-Dec-02 19:52 UTC

head link

[Pkg-xen-devel] [PATCH 19/19] d/xendomains.init: Remove use of `xen` compat wrapper for `xl`

During the transition from `xm` to `xl` there may have been some benefit
to using a `xen` wrapper to call the appropriate one.  Now `xl` is the
only one in use and has been the only one for some time.  As such adjust
the `xendomains` script to use an internal wrapper instead.

I am highly tempted to purge the `xen` wrapper, but could understand
some users desiring to keep the compatibility around.  Perhaps for a
future where something replaces `xl`.

Signed-off-by: Elliott Mitchell <ehem+debian at m5p.com>
---
 debian/xen-utils-common.xendomains.init | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/debian/xen-utils-common.xendomains.init
b/debian/xen-utils-common.xendomains.init
index ca03ae4cfe..cd6902d26e 100644
--- a/debian/xen-utils-common.xendomains.init
+++ b/debian/xen-utils-common.xendomains.init
@@ -31,6 +31,11 @@ fi
 
 shopt -s nullglob
 
+xen()
+{
+	xl "$@"
+}
+
 check_config_name()
 {
   /usr/lib/xen-common/bin/xen-init-name "$1" 2>/dev/null
-- 


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg at m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

Hans van Kranenburg

2020-Dec-04 19:27 UTC

head link

[Pkg-xen-devel] [PATCH 00/19] Fixing cross-compilation, some script work, revised

On 12/2/20 6:34 PM, Elliott Mitchell wrote:> Hopefully this takes care of the comments which have been made.  Quite a
> few commit messages were adjusted.  Two scripting commits were broken
> into tiny pieces.
> 
> There are a number of small differences, but the end result will be the
> same.
> 
> Hopefully I've figured out the right set of flags for the way you
prefer
> patches being sent.
No. Use git send-email. You're still, even after asking many times, not
using git send-email.

git format-patch formats *patches*, not emails, even while they might
appear to look like emails.
>  Right level/type of threading, etc.  The dates may
> differ since Git uses the author date, NOT the commit date when sending
> messages.  These were mostly authored in July, those are the dates they
> get.  Since some were originally added in September, it gets that date
> header.  Your mailer might choose to sort them this way, which differs
> from sequence number.
Use git send-email.
> Elliott Mitchell (19):
>   debian/rules: Correct shim install step for current Xen
>   d/shuffle-binaries: Remove useless extra argument being passed in
>   d/shuffle-binaries: Make error detection/message overt
>   d/shuffle-binaries: Add quoting for potentially changeable variables
>   d/shuffle-binaries: Switch loop from for to while
>   d/shuffle-binaries: Fix binary shuffling script for cross-building
>   d/shuffle-binaries: Switch to POSIX shell, instead of Bash
>   d/shuffle-boot-files: The Great Quotification
>   d/shuffle-boot-files: Merge arguments together
>   d/shuffle-boot-files: Support filenames besides .gz
>   d/shuffle-boot-files: Rework version string handling
>   d/shuffle-boot-files: Simplify loop structure
>   d/shuffle-boot-files: Switch to POSIX shell, instead of Bash
>   debian/rules: Set CC/LD to enable cross-building
>   debian/rules: Setup use of noOCAML profile
>   Rework "debian/rules: Do not try to move EFI binaries on armhf"
>   debian/xen.init: Load xen_acpi_processor on boot
>   debian/scripts: Optimize runtime scripts
>   d/xendomains.init: Remove use of `xen` compat wrapper for `xl`
> 
>  debian/control                          |  5 +--
>  debian/rules                            | 34 ++++++++++++------
>  debian/scripts/xen-utils-wrapper        | 43 +++++++++++++++++++---
>  debian/scripts/xen-version              |  6 ++--
>  debian/shuffle-binaries                 | 48 +++++++++++++------------
>  debian/shuffle-boot-files               | 25 ++++++-------
>  debian/xen-utils-common.xen.init        |  1 +
>  debian/xen-utils-common.xendomains.init |  5 +++
>  8 files changed, 114 insertions(+), 53 deletions(-)
>

Pkg xen devel - Dec 2020 - [PATCH 00/19] Fixing cross-compilation, some script work, revised

[Pkg-xen-devel] [PATCH 00/19] Fixing cross-compilation, some script work, revised

[Pkg-xen-devel] [PATCH 01/19] debian/rules: Correct shim install step for current Xen

[Pkg-xen-devel] [PATCH 02/19] d/shuffle-binaries: Remove useless extra argument being passed in

[Pkg-xen-devel] [PATCH 03/19] d/shuffle-binaries: Make error detection/message overt

[Pkg-xen-devel] [PATCH 04/19] d/shuffle-binaries: Add quoting for potentially changeable variables

[Pkg-xen-devel] [PATCH 05/19] d/shuffle-binaries: Switch loop from for to while

[Pkg-xen-devel] [PATCH 06/19] d/shuffle-binaries: Fix binary shuffling script for cross-building

[Pkg-xen-devel] [PATCH 07/19] d/shuffle-binaries: Switch to POSIX shell, instead of Bash

[Pkg-xen-devel] [PATCH 08/19] d/shuffle-boot-files: The Great Quotification

[Pkg-xen-devel] [PATCH 09/19] d/shuffle-boot-files: Merge arguments together

[Pkg-xen-devel] [PATCH 10/19] d/shuffle-boot-files: Support filenames besides .gz

[Pkg-xen-devel] [PATCH 11/19] d/shuffle-boot-files: Rework version string handling

[Pkg-xen-devel] [PATCH 12/19] d/shuffle-boot-files: Simplify loop structure

[Pkg-xen-devel] [PATCH 13/19] d/shuffle-boot-files: Switch to POSIX shell, instead of Bash

[Pkg-xen-devel] [PATCH 14/19] debian/rules: Set CC/LD to enable cross-building

[Pkg-xen-devel] [PATCH 15/19] debian/rules: Setup use of noOCAML profile

[Pkg-xen-devel] [PATCH 16/19] Rework "debian/rules: Do not try to move EFI binaries on armhf"

[Pkg-xen-devel] [PATCH 17/19] debian/xen.init: Load xen_acpi_processor on boot

[Pkg-xen-devel] [PATCH 18/19] debian/scripts: Optimize runtime scripts

[Pkg-xen-devel] [PATCH 19/19] d/xendomains.init: Remove use of `xen` compat wrapper for `xl`

[Pkg-xen-devel] [PATCH 00/19] Fixing cross-compilation, some script work, revised