thr3ads.net - Xen devel - [Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt [Jul 2011]

If this information is useful, please help other people find it:
Share via:

Olaf Hering

2011-Jul-22 18:49 UTC

[Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt

What is the best way to reregister the timer interrupt in the kexec
kernel in a pv-on-hvm guest? Right now the crash shown below happens
because EVTCHNOP_bind_virq returns -EEXIST. Can the timer on the cpus be
unregistered during shutdown? xen_teardown_timer() does not allow that
for cpu #0.  I added some silly loop which tries to match vcpu/cpu in
bind_virq_to_irq() using EVTCHNOP_status to find the currently used port
number. This helps and booting proceeds, but I feel that cant be the
right approach.

Olaf



....
[    0.000000] No AGP bridge found
[    0.000000] Memory: 486288k/524288k available (2812k kernel code, 452k
absent, 37548k reserved, 3557k data, 480k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=8,
Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:4352 nr_irqs:1152 16
[    0.000000] Xen HVM callback vector for event delivery is enabled
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] Detected 2793.044 MHz processor.
[    0.004000] Calibrating delay loop (skipped), value calculated using timer
frequency.. 5586.08 BogoMIPS (lpj=11172176)
[    0.004000] pid_max: default: 32768 minimum: 301
[    0.004000] Mount-cache hash table entries: 256
[    0.004231] CPU: Physical Processor ID: 0
[    0.005459] CPU: Processor Core ID: 0
[    0.008671] ACPI: Core revision 20110413
[    0.020172] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=0 pin2=0
[    0.061644] CPU0: Intel(R) Core(TM) i7 CPU         860  @ 2.80GHz stepping 05
[    0.063779] Xen: using vcpuop timer interface
[    0.064006] installing Xen timer for CPU 0
[    0.065166] ------------[ cut here ]------------
[    0.066468] kernel BUG at drivers/xen/events.c:902!
[    0.067855] invalid opcode: 0000 [#1] SMP
[    0.068003] CPU 0
[    0.068003] Modules linked in:
[    0.068003]
[    0.068003] Pid: 1, comm: swapper Not tainted
3.0.0-20110722.1852.20-kernel-linux-3_0 #1 Xen HVM domU
[    0.068003] RIP: 0010:[<ffffffff811b41dc>]  [<ffffffff811b41dc>]
bind_virq_to_irq+0xac/0x170
[    0.068003] RSP: 0018:ffff88001d89fe20  EFLAGS: 00010282
[    0.068003] RAX: ffffffffffffffef RBX: 0000000000000040 RCX: 0000000000000246
[    0.068003] RDX: 00000000ffffffef RSI: ffff88001d89fe40 RDI: 0000000000000001
[    0.068003] RBP: 0000000000000000 R08: ffff88001d807500 R09: ffffffff818ee2e8
[    0.068003] R10: 000000000000005a R11: 0000000000000000 R12: 0000000000000000
[    0.068003] R13: 0000000000000000 R14: 000000000000ebe0 R15: 0000000000000000
[    0.068003] FS:  0000000000000000(0000) GS:ffff88001ea00000(0000)
knlGS:0000000000000000
[    0.068003] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[    0.068003] CR2: 00007f7c9b6e0ae0 CR3: 0000000001603000 CR4: 00000000000006b0
[    0.068003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.068003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[    0.068003] Process swapper (pid: 1, threadinfo ffff88001d89e000, task
ffff88001d8a0000)
[    0.068003] Stack:
[    0.068003]  ffff88001d89fe80 0000000000000000 0000000000000000
0000000000000000
[    0.068003]  0000000000000000 0000000000000000 ffffffff81636120
0000000000000000
[    0.068003]  0000000000000000 ffff88001d830050 000000000001ce20
ffffffff81007870
[    0.068003] Call Trace:
[    0.068003]  [<ffffffff81007870>] ?
xen_hvm_setup_cpu_clockevents+0x20/0x20
[    0.068003]  [<ffffffff811b479d>] ? bind_virq_to_irqhandler+0x2d/0x80
[    0.068003]  [<ffffffff81007801>] ? xen_setup_timer+0x51/0xa0
[    0.068003]  [<ffffffff816500e2>] ? xen_time_init+0x8a/0xaf
[    0.068003]  [<ffffffff81659031>] ? native_smp_prepare_cpus+0x216/0x374
[    0.068003]  [<ffffffff816508bc>] ? xen_hvm_smp_prepare_cpus+0x9/0x2e
[    0.068003]  [<ffffffff8164c551>] ? kernel_init+0x3f/0x128
[    0.068003]  [<ffffffff812bc604>] ? kernel_thread_helper+0x4/0x10
[    0.068003]  [<ffffffff8164c512>] ? parse_early_options+0x1a/0x1a
[    0.068003]  [<ffffffff812bc600>] ? gs_change+0x13/0x13
[    0.068003] Code: 63 81 e8 f8 dc ec ff 44 89 64 24 20 44 89 6c 24 24 bf 01 00
00 00 48 8d 74 24 20 e8 2f d2 e4 ff 83 f8 da 89 c2 74 6e 85 d2 74 04 <0f>
0b eb fe 89 df 8b 6c 24 28 e8 b5 e5 ff ff 48 89 c1 8b 40 10
[    0.068003] RIP  [<ffffffff811b41dc>] bind_virq_to_irq+0xac/0x170
[    0.068003]  RSP <ffff88001d89fe20>
[    0.132010] ---[ end trace 4eaa2a86a8e2da22 ]---


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2011-Jul-23 06:57 UTC

head link

Re: [Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt

On 22/07/2011 19:49, "Olaf Hering" <olaf@aepfle.de> wrote:
> What is the best way to reregister the timer interrupt in the kexec
> kernel in a pv-on-hvm guest? Right now the crash shown below happens
> because EVTCHNOP_bind_virq returns -EEXIST. Can the timer on the cpus be
> unregistered during shutdown? xen_teardown_timer() does not allow that
> for cpu #0.  I added some silly loop which tries to match vcpu/cpu in
> bind_virq_to_irq() using EVTCHNOP_status to find the currently used port
> number. This helps and booting proceeds, but I feel that cant be the
> right approach.
Sounbds like it''s simply some Linux-guest issue here to be untangled.
All
Xen requires you to do is EVTCHNOP_close the old virq-evtchn binding.

 -- Keir
> Olaf
> 
> 
> 
> ....
> [    0.000000] No AGP bridge found
> [    0.000000] Memory: 486288k/524288k available (2812k kernel code, 452k
> absent, 37548k reserved, 3557k data, 480k init)
> [    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=8,
> Nodes=1
> [    0.000000] Hierarchical RCU implementation.
> [    0.000000] NR_IRQS:4352 nr_irqs:1152 16
> [    0.000000] Xen HVM callback vector for event delivery is enabled
> [    0.000000] Console: colour VGA+ 80x25
> [    0.000000] console [ttyS0] enabled
> [    0.000000] Detected 2793.044 MHz processor.
> [    0.004000] Calibrating delay loop (skipped), value calculated using
timer
> frequency.. 5586.08 BogoMIPS (lpj=11172176)
> [    0.004000] pid_max: default: 32768 minimum: 301
> [    0.004000] Mount-cache hash table entries: 256
> [    0.004231] CPU: Physical Processor ID: 0
> [    0.005459] CPU: Processor Core ID: 0
> [    0.008671] ACPI: Core revision 20110413
> [    0.020172] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=0 pin2=0
> [    0.061644] CPU0: Intel(R) Core(TM) i7 CPU         860  @ 2.80GHz
stepping
> 05
> [    0.063779] Xen: using vcpuop timer interface
> [    0.064006] installing Xen timer for CPU 0
> [    0.065166] ------------[ cut here ]------------
> [    0.066468] kernel BUG at drivers/xen/events.c:902!
> [    0.067855] invalid opcode: 0000 [#1] SMP
> [    0.068003] CPU 0
> [    0.068003] Modules linked in:
> [    0.068003]
> [    0.068003] Pid: 1, comm: swapper Not tainted
> 3.0.0-20110722.1852.20-kernel-linux-3_0 #1 Xen HVM domU
> [    0.068003] RIP: 0010:[<ffffffff811b41dc>] 
[<ffffffff811b41dc>]
> bind_virq_to_irq+0xac/0x170
> [    0.068003] RSP: 0018:ffff88001d89fe20  EFLAGS: 00010282
> [    0.068003] RAX: ffffffffffffffef RBX: 0000000000000040 RCX:
> 0000000000000246
> [    0.068003] RDX: 00000000ffffffef RSI: ffff88001d89fe40 RDI:
> 0000000000000001
> [    0.068003] RBP: 0000000000000000 R08: ffff88001d807500 R09:
> ffffffff818ee2e8
> [    0.068003] R10: 000000000000005a R11: 0000000000000000 R12:
> 0000000000000000
> [    0.068003] R13: 0000000000000000 R14: 000000000000ebe0 R15:
> 0000000000000000
> [    0.068003] FS:  0000000000000000(0000) GS:ffff88001ea00000(0000)
> knlGS:0000000000000000
> [    0.068003] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [    0.068003] CR2: 00007f7c9b6e0ae0 CR3: 0000000001603000 CR4:
> 00000000000006b0
> [    0.068003] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [    0.068003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [    0.068003] Process swapper (pid: 1, threadinfo ffff88001d89e000, task
> ffff88001d8a0000)
> [    0.068003] Stack:
> [    0.068003]  ffff88001d89fe80 0000000000000000 0000000000000000
> 0000000000000000
> [    0.068003]  0000000000000000 0000000000000000 ffffffff81636120
> 0000000000000000
> [    0.068003]  0000000000000000 ffff88001d830050 000000000001ce20
> ffffffff81007870
> [    0.068003] Call Trace:
> [    0.068003]  [<ffffffff81007870>] ?
xen_hvm_setup_cpu_clockevents+0x20/0x20
> [    0.068003]  [<ffffffff811b479d>] ?
bind_virq_to_irqhandler+0x2d/0x80
> [    0.068003]  [<ffffffff81007801>] ? xen_setup_timer+0x51/0xa0
> [    0.068003]  [<ffffffff816500e2>] ? xen_time_init+0x8a/0xaf
> [    0.068003]  [<ffffffff81659031>] ?
native_smp_prepare_cpus+0x216/0x374
> [    0.068003]  [<ffffffff816508bc>] ?
xen_hvm_smp_prepare_cpus+0x9/0x2e
> [    0.068003]  [<ffffffff8164c551>] ? kernel_init+0x3f/0x128
> [    0.068003]  [<ffffffff812bc604>] ? kernel_thread_helper+0x4/0x10
> [    0.068003]  [<ffffffff8164c512>] ? parse_early_options+0x1a/0x1a
> [    0.068003]  [<ffffffff812bc600>] ? gs_change+0x13/0x13
> [    0.068003] Code: 63 81 e8 f8 dc ec ff 44 89 64 24 20 44 89 6c 24 24 bf
01
> 00 00 00 48 8d 74 24 20 e8 2f d2 e4 ff 83 f8 da 89 c2 74 6e 85 d2 74 04
<0f>
> 0b eb fe 89 df 8b 6c 24 28 e8 b5 e5 ff ff 48 89 c1 8b 40 10
> [    0.068003] RIP  [<ffffffff811b41dc>] bind_virq_to_irq+0xac/0x170
> [    0.068003]  RSP <ffff88001d89fe20>
> [    0.132010] ---[ end trace 4eaa2a86a8e2da22 ]---
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Campbell

2011-Jul-23 08:01 UTC

head link

Re: [Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt

On Sat, 2011-07-23 at 07:57 +0100, Keir Fraser wrote:> On 22/07/2011 19:49, "Olaf Hering" <olaf@aepfle.de> wrote:
> 
> > What is the best way to reregister the timer interrupt in the kexec
> > kernel in a pv-on-hvm guest? Right now the crash shown below happens
> > because EVTCHNOP_bind_virq returns -EEXIST. Can the timer on the cpus
be
> > unregistered during shutdown? xen_teardown_timer() does not allow that
> > for cpu #0.  I added some silly loop which tries to match vcpu/cpu in
> > bind_virq_to_irq() using EVTCHNOP_status to find the currently used
port
> > number. This helps and booting proceeds, but I feel that cant be the
> > right approach.
> 
> Sounbds like it''s simply some Linux-guest issue here to be
untangled. All
> Xen requires you to do is EVTCHNOP_close the old virq-evtchn binding.
That might be tricky to do in the target kernel, which may not know what
needs closing. There''s EVTCHNOP_reset which looks like it would be a
sensible thing to call early on in the target but seems like it would
close to much -- e.g. the xenstore evtchn?

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Olaf Hering

2011-Jul-23 08:52 UTC

head link

Re: [Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt

On Sat, Jul 23, Ian Campbell wrote:
> On Sat, 2011-07-23 at 07:57 +0100, Keir Fraser wrote:
> > On 22/07/2011 19:49, "Olaf Hering" <olaf@aepfle.de>
wrote:
> > 
> > > What is the best way to reregister the timer interrupt in the
kexec
> > > kernel in a pv-on-hvm guest? Right now the crash shown below
happens
> > > because EVTCHNOP_bind_virq returns -EEXIST. Can the timer on the
cpus be
> > > unregistered during shutdown? xen_teardown_timer() does not allow
that
> > > for cpu #0.  I added some silly loop which tries to match
vcpu/cpu in
> > > bind_virq_to_irq() using EVTCHNOP_status to find the currently
used port
> > > number. This helps and booting proceeds, but I feel that cant be
the
> > > right approach.
> > 
> > Sounbds like it''s simply some Linux-guest issue here to be
untangled. All
> > Xen requires you to do is EVTCHNOP_close the old virq-evtchn binding.
> 
> That might be tricky to do in the target kernel, which may not know what
> needs closing. There''s EVTCHNOP_reset which looks like it would be
a
> sensible thing to call early on in the target but seems like it would
> close to much -- e.g. the xenstore evtchn?
After some more poking around in the reboot code path I have figured it
out, a syscore_ops has to be registerd to shutdown the irqs.
I will post some patches for review.

Olaf

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2011-Jul-23 09:55 UTC

head link

Re: [Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt

On 23/07/2011 09:01, "Ian Campbell" <Ian.Campbell@citrix.com>
wrote:
>> Sounbds like it''s simply some Linux-guest issue here to be
untangled. All
>> Xen requires you to do is EVTCHNOP_close the old virq-evtchn binding.
> 
> That might be tricky to do in the target kernel, which may not know what
> needs closing. There''s EVTCHNOP_reset which looks like it would be
a
> sensible thing to call early on in the target but seems like it would
> close to much -- e.g. the xenstore evtchn?
If you can''t do it in the source kernel (e.g., because it''s
crashed) the
best bet is to call EVTCHN_status on every port to find the channels you
want to clobber. Which is what Olaf hacked up already I believe, just needs
integrating cleanly.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Jul 2011 - pv-on-hvm kexec, howto reregister timer interrupt

[Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt

Re: [Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt

Re: [Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt

Re: [Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt

Re: [Xen-devel] pv-on-hvm kexec, howto reregister timer interrupt