Pkg xen devel - Sep 2022 - Bug#1020787: Bug#1020787: linux-image-5.19.0-2-amd64: After updating to 5.19 kernel the VMs are started without XSAVE CPU flags

On Tue, Sep 27, 2022 at 01:39:46AM +0200, Diederik de Haas
wrote:
> Control: tag -1 moreinfo
> 
> On Monday, 26 September 2022 20:04:10 CEST patrick wrote:
> > I have a XEN hypervisor running with some VMs (everything based on
Debian).
> 
> Which version of Xen are you using?
> 
> > After updating to the Kernel to 5.19 (from 5.18.0-4), some binaries
are
> > falling to execute. If the VMs are started with Kernel 5.18.0-4 they
are
> > running fine.
> 
> Is this all about the dom0 kernel or is it all/some about using 5.19 as 
> domU kernel? Are the issues happing on dom0 or inside domU?
> 
> If the issues happen inside (a) domU, can you share a (minimal) domU 
> configuration file so it becomes easier to replicate?
> 
> > And indeed there is some difference in /proc/cpuinfo:
> > The flags for "fma xsave avx2 bmi2 xsaveopt xsavec xgetbv1
md_clear" are
> > missing, which might result in gnutls failures.
> 
> In kernel 5.19 the following commits were added under
``arch/x86/kernel/fpu/``:
> 
> b91c0922bf1ed15b67a6faa404bc64e3ed532ec2 x86/fpu: Cleanup variable
shadowing
> 8ad7e8f696951f192c6629a0cbda9ac94c773159 x86/fpu/xsave: Support XSAVEC in
the kernel
> f5c0b4f30416c670408a77be94703d04d22b57df x86/prctl: Remove pointless task
argument
> 
> Of these, the first 2 seem like possible candidates that caused the issue.
>
https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s4.2.2
> describes a way to apply a simple patch to a kernel.
> What you could try, is creating a patch from reverting one of the earlier
> mentioned commits and use that with 'test-patches'.
> 
> It's probably also useful to know what CPU(s) are in the machine
(dom0).
> 
> On Monday, 26 September 2022 20:31:17 CEST Ps Ps wrote:
> > On Xen Hypervisor I just found this logs:
> 
> So this is on dom0? In which log file did you find it?
> 
> Generally: be as specific as you can be and describe *exactly* what you did
> and the exact results (if any). Also try to make it as easy as possible for
> others to reproduce what you're experiencing.
> 
> FTR: I did not see this issue on my dom0 (Xen 4.16.2-1; kernel 5.19.11-1):
> 
> root at dom0:~# dmesg
> [    0.000000] Linux version 5.19.0-2-amd64 (debian-kernel at
lists.debian.org) (gcc-11 (Debian 11.3.0-6) 11.3.0, GNU ld (GNU Binutils for
Debian) 2.38.90.20220713) #1 SMP PREEMPT_DYNAMIC Debian 5.19.11-1 (2022-09-24)
> [    0.000000] Command line: placeholder
root=UUID=8008723b-668f-43f6-b432-8c56ed53f48a ro quiet net.ifnames=0
> [    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating
point registers'
> [    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE
registers'
> [    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX
registers'
> [    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
> [    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832
bytes, using 'standard' format.
> [    0.000000] signal: max sigframe size: 1776
> [    0.000000] Released 0 page(s)
> 
> root at dom0:~# grep flag /proc/cpuinfo | uniq
> flags           : fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush
acpi mmx fxsr sse sse2 ss ht syscall nx rdtscp lm constant_tsc rep_good nopl
nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor est ssse3 fma cx16 sse4_1
sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm
3dnowprefetch cpuid_fault ssbd ibrs ibpb stibp fsgsbase bmi1 avx2 bmi2 erms rtm
rdseed adx xsaveopt md_clear
> 
> 
> Also found this patch which should make the error msg more informative ...
> https://lore.kernel.org/all/20220810221909.12768-1-andrew.cooper3 at
citrix.com/
> Even though I haven't experienced it (yet?), the language of this patch
> seems to indicate you're not alone with it.
There is also Xen patch fixing the issue:
https://lore.kernel.org/xen-devel/a4ec41e6-16cd-4452-19c1-5d6d9e3bddf8 at
suse.com/


-- 
Best Regards,
Marek Marczykowski-G?recki
Invisible Things Lab
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL:
<http://alioth-lists.debian.net/pipermail/pkg-xen-devel/attachments/20220927/29e2fd32/attachment.sig>

Am Dienstag, dem 27.09.2022 um 02:16 +0200 schrieb Marek
Marczykowski-G?recki:
> There is also Xen patch fixing the issue:
> https://lore.kernel.org/xen-devel/a4ec41e6-16cd-4452-19c1-5d6d9e3bddf8 at
suse.com/
> 
> 
Interessting. I will give it a try. Thanks.

Hi Marek,

On dinsdag 27 september 2022 02:16:31 CEST Marek Marczykowski-G?recki
wrote:
> On Tue, Sep 27, 2022 at 01:39:46AM +0200, Diederik de Haas wrote:
> > In kernel 5.19 the following commits were added under
> > ``arch/x86/kernel/fpu/``:
> > 
> > b91c0922bf1ed15b67a6faa404bc64e3ed532ec2 x86/fpu: Cleanup variable
> > shadowing 
> > 8ad7e8f696951f192c6629a0cbda9ac94c773159 x86/fpu/xsave: Support
> > XSAVEC in the kernel 
> > f5c0b4f30416c670408a77be94703d04d22b57df x86/prctl:
> > Remove pointless task argument
8ad7e8f696951f192c6629a0cbda9ac94c773159 seems to be the 'offending'
commit ...
> > Also found this patch which should make the error msg more informative
...
> > https://lore.kernel.org/all/20220810221909.12768-1-andrew.cooper3 at
citrix.c
> > om/ Even though I haven't experienced it (yet?), the language of
this
> > patch seems to indicate you're not alone with it.
> 
> There is also Xen patch fixing the issue:
> https://lore.kernel.org/xen-devel/a4ec41e6-16cd-4452-19c1-5d6d9e3bddf8 at
suse.
> com/
http://xenbits.xen.org/gitweb/?
p=xen.git;a=commit;h=c3bd0b83ea5b7c0da6542687436042eeea1e7909 is where it's 
committed in Xen's master branch. I haven't seen a backport to 4.16
(yet?)

Looking at the patch thread you linked, I get the impression that the change 
in the Linux kernel exposed the problem on the Xen side, but the Linux kernel 
change didn't introduce a bug (Andrew's patch seems to make it more 
informative, not correct a bug).

But I'm not sure that assesment is correct. What's your take on this?

Cheers,
  Diederik
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: This is a digitally signed message part.
URL:
<http://alioth-lists.debian.net/pipermail/pkg-xen-devel/attachments/20220927/ff2c170c/attachment.sig>