Pkg xen devel - May 2015 - Bug#785187: Bug#785187: xen-hypervisor-4.5-amd64: Option ucode=scan is not working

On Tue, 2015-05-19 at 13:31 +0200, Stephan Seitz wrote:
> On Fri, May 15, 2015 at 08:26:42AM +0100, Ian Campbell wrote:
> >Here's an idea. First extract the real initrd from the back half of
the
> >initrd:
> >        ijc at dagon:tmp$ cat /boot/initrd.img | ( cpio -t >&2 ;
cat ) > initrd.real
> >        kernel
> >        kernel/x86
> >        kernel/x86/microcode
> >        kernel/x86/microcode/GenuineIntel.bin
> >        16 blocks
> >        ijc at dagon:tmp$ file initrd.real
> >        initrd.real: gzip compressed data, last modified: Sun Apr 19
16:31:17 2015, from Unix
> >        ijc at dagon:tmp$ zcat initrd.real | cpio -t | head -n 5
> >        .
> >        etc
> >        etc/ld.so.conf.d
> >        etc/ld.so.conf.d/fakeroot-x86_64-linux-gnu.conf
> >        etc/ld.so.conf.d/x86_64-linux-gnu.conf
> >
> >Now you can experiment with regenerating the ucode bit with whatever
> >commands are recommended and reconcatenating the two. If that works
then
> >we know there is some incompatibility between iucode-tools's output
and
> >Xen's cpio reader (could be a bug in either or both I think).
> 
> All right. Can you please provide some commands how to create and 
> reconcatenate the two initrds? I don?t have any experience with cpio.
The reconcatenate bit is easy, just:
   $ cat initrd.ucode initrd.real > initrd.img
With initrd.real extracted above.

initrd.ucode creation I'm a little unsure about but something like:

$ mkdir initrd.ucode.tree
$ cd initrd.ucode.tree
$ cat /boot/initrd.img | cpio -id
$ find 
.
./kernel
./kernel/x86
./kernel/x86/microcode
./kernel/x86/microcode/GenuineIntel.bin
$ find | cpio -o -H newc > ../initrd.ucode

Is what I think is required/expected.

BTW, could you please post the output "xl dmesg" for the failing case,
I
should have asked for that right at the start in case it contains
anything of use.

Ian.

On Tue, May 19, 2015 at 12:47:51PM +0100, Ian Campbell
wrote:
>The reconcatenate bit is easy, just:
>   $ cat initrd.ucode initrd.real > initrd.img
>With initrd.real extracted above.
>
>initrd.ucode creation I'm a little unsure about but something like:
>
>$ mkdir initrd.ucode.tree
>$ cd initrd.ucode.tree
>$ cat /boot/initrd.img | cpio -id
>$ find
>.
>./kernel
>./kernel/x86
>./kernel/x86/microcode
>./kernel/x86/microcode/GenuineIntel.bin
>$ find | cpio -o -H newc > ../initrd.ucode
That interesting. Your steps lead to an initrd with the same size as the 
initrd created by the Debian tools. But the new one is working with 
ucode=scan, the old one isn?t.
>BTW, could you please post the output "xl dmesg" for the failing
case, I
>should have asked for that right at the start in case it contains
>anything of use.
I can do this, but neither ?xl dmesg? nor ?dmesg? are telling anything 
about microcode. Are there ways to make the hypervisor more verbose?

Shade and sweet water!
	
	Stephan

-- 
| Stephan Seitz          E-Mail: stse at fsing.rootsland.net |
| Public Keys: http://fsing.rootsland.net/~stse/keys.html |
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3735 bytes
Desc: not available
URL:
<http://lists.alioth.debian.org/pipermail/pkg-xen-devel/attachments/20150527/59a1d4a6/attachment.bin>

Hi Ian & Co!

Any new ideas about this bug? I?m not sure if this is a bug in the 
hypervisor or in the tools generating the initrd.

The manual steps provided by Ian generate a working initrd.

Many greetings,

	Stephan

-- 
| Stephan Seitz          E-Mail: stse at fsing.rootsland.net |
| Public Keys: http://fsing.rootsland.net/~stse/keys.html |
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3735 bytes
Desc: not available
URL:
<http://lists.alioth.debian.org/pipermail/pkg-xen-devel/attachments/20150701/dc68af4b/attachment.bin>

AFAICT our current version does not correspond to any version in the
Linux history. This commit resynchronised to the state in Linux
commit 598bae70c2a8e35c8d39b610cca2b32afcf047af.

Differences from upstream: find_cpio_data is __init, printk instead of
pr_*.

This appears to fix Debian bug #785187. "Appears" because my test box
happens to be AMD and the issue is that the (valid) cpio generated by
the Intel ucode is not liked by the old Xen code. I've tested by
hacking the hypervisor to look for the Intel path.

Reported-by: Stephan Seitz <stse+debianbugs at fsing.rootsland.net>
Signed-off-by: Ian Campbell <ian.campbell at citrix.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk at oracle.com>
Cc: Jan Beulich <jbeulich at suse.com>
Cc: Stephan Seitz <stse+debianbugs at fsing.rootsland.net>
Cc: 785187 at bugs.debian.org
---
This should be backported.
---
 xen/common/earlycpio.c      |   39 ++++++++++++++++++++-------------------
 xen/include/xen/earlycpio.h |    1 +
 2 files changed, 21 insertions(+), 19 deletions(-)

diff --git a/xen/common/earlycpio.c b/xen/common/earlycpio.c
index 5e54142..f6b1a9e 100644
--- a/xen/common/earlycpio.c
+++ b/xen/common/earlycpio.c
@@ -54,25 +54,26 @@ enum cpio_fields {
 
 /**
  * cpio_data find_cpio_data - Search for files in an uncompressed cpio
- * @path:   The directory to search for, including a slash at the end
- * @data:   Pointer to the the cpio archive or a header inside
- * @len:    Remaining length of the cpio based on data pointer
- * @offset: When a matching file is found, this is the offset to the
- *          beginning of the cpio. It can be used to iterate through
- *          the cpio to find all files inside of a directory path
+ * @path:       The directory to search for, including a slash at the end
+ * @data:       Pointer to the the cpio archive or a header inside
+ * @len:        Remaining length of the cpio based on data pointer
+ * @nextoff:    When a matching file is found, this is the offset from the
+ *              beginning of the cpio to the beginning of the next file, not
the
+ *              matching file itself. It can be used to iterate through the
cpio
+ *              to find all files inside of a directory path.
  *
- * @return: struct cpio_data containing the address, length and
- *          filename (with the directory path cut off) of the found file.
- *          If you search for a filename and not for files in a directory,
- *          pass the absolute path of the filename in the cpio and make sure
- *          the match returned an empty filename string.
+ * @return:     struct cpio_data containing the address, length and
+ *              filename (with the directory path cut off) of the found file.
+ *              If you search for a filename and not for files in a directory,
+ *              pass the absolute path of the filename in the cpio and make
sure
+ *              the match returned an empty filename string.
  */
 
 struct cpio_data __init find_cpio_data(const char *path, void *data,
-					  size_t len,  long *offset)
+				       size_t len,  long *nextoff)
 {
 	const size_t cpio_header_len = 8*C_NFIELDS - 2;
-	struct cpio_data cd = { NULL, 0 };
+	struct cpio_data cd = { NULL, 0, "" };
 	const char *p, *dptr, *nptr;
 	unsigned int ch[C_NFIELDS], *chp, v;
 	unsigned char c, x;
@@ -129,17 +130,17 @@ struct cpio_data __init find_cpio_data(const char *path,
void *data,
 		if ((ch[C_MODE] & 0170000) == 0100000 &&
 		    ch[C_NAMESIZE] >= mypathsize &&
 		    !memcmp(p, path, mypathsize)) {
-			*offset = (long)nptr - (long)data;
+			*nextoff = (long)nptr - (long)data;
 			if (ch[C_NAMESIZE] - mypathsize >= MAX_CPIO_FILE_NAME) {
 				printk(
 				"File %s exceeding MAX_CPIO_FILE_NAME [%d]\n",
 				p, MAX_CPIO_FILE_NAME);
 			}
-			if (ch[C_NAMESIZE] - 1 /* includes \0 */ == mypathsize) {
-				cd.data = (void *)dptr;
-				cd.size = ch[C_FILESIZE];
-				return cd; /* Found it! */
-			}
+			strlcpy(cd.name, p + mypathsize, MAX_CPIO_FILE_NAME);
+
+			cd.data = (void *)dptr;
+			cd.size = ch[C_FILESIZE];
+			return cd; /* Found it! */
 		}
 		len -= (nptr - p);
 		p = nptr;
diff --git a/xen/include/xen/earlycpio.h b/xen/include/xen/earlycpio.h
index 85d144a..16d9404 100644
--- a/xen/include/xen/earlycpio.h
+++ b/xen/include/xen/earlycpio.h
@@ -6,6 +6,7 @@
 struct cpio_data {
 	void *data;
 	size_t size;
+	char name[MAX_CPIO_FILE_NAME];
 };
 
 struct cpio_data find_cpio_data(const char *path, void *data, size_t len,
-- 
1.7.10.4