Masroor Vettuparambil
2008-Feb-07 13:59 UTC
[Xen-devel] save/restore pass-through enabled domU
Hi all, Can I save a pass-through enabled domU and restore it later on the same machine? I tried this with no success. In my case, saving was successful only when I removed corresponding driver from domU. Also when I restored it, all pci associated commands (ie lspci, insmod driver) produce the following error, pcifront pci-0: pciback not responding!!! BUG: soft lockup detected on CPU#0! Call Trace: <IRQ> [<ffffffff80257c0c>] softlockup_tick+0xd8/0xea [<ffffffff8020f0ee>] timer_interrupt+0x3a1/0x3ff [<ffffffff80257ef8>] handle_IRQ_event+0x4e/0x96 [<ffffffff80257fe4>] __do_IRQ+0xa4/0x105 [<ffffffff8020b0e0>] call_softirq+0x1c/0x28 [<ffffffff8020ceb7>] do_IRQ+0x65/0x73 [<ffffffff80377106>] evtchn_do_upcall+0xac/0x12d [<ffffffff8020ac16>] do_hypervisor_callback+0x1e/0x2c <EOI> [<ffffffff8020622a>] hypercall_page+0x22a/0x1000 [<ffffffff8020622a>] hypercall_page+0x22a/0x1000 [<ffffffff80214297>] xen_send_IPI_mask+0x0/0xf5 [<ffffffff8037664e>] force_evtchn_callback+0xa/0xb [<ffffffff8031b2b8>] pci_user_read_config_dword+0x86/0x9d [<ffffffff8031fd7c>] pci_read_config+0x114/0x1ae [<ffffffff802bef00>] read+0x84/0xc1 [<ffffffff8027fff9>] vfs_read+0xcb/0x171 [<ffffffff802804bf>] sys_pread64+0x50/0x70 [<ffffffff8020ab6b>] error_exit+0x0/0x71 [<ffffffff8020a42e>] system_call+0x86/0x8b [<ffffffff8020a3a8>] system_call+0x0/0x8b If this works, I can think abt migration of pass-through enabled domU between machines having identical hardware. regards, Masroor _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
No, it certainly will not work. In fact I thought we disabled it in the tools, but maybe I¹m mistaken. -- Keir On 7/2/08 13:59, "Masroor Vettuparambil" <Masroor.Vettuparambil@neterion.com> wrote:> Hi all, > > Can I save a pass-through enabled domU and restore it later on the same > machine? > I tried this with no success. In my case, saving was successful only when I > removed corresponding driver from domU. > Also when I restored it, all pci associated commands (ie lspci, insmod driver) > produce the following error, > > pcifront pci-0: pciback not responding!!! > BUG: soft lockup detected on CPU#0! > > Call Trace: > <IRQ> [<ffffffff80257c0c>] softlockup_tick+0xd8/0xea > [<ffffffff8020f0ee>] timer_interrupt+0x3a1/0x3ff > [<ffffffff80257ef8>] handle_IRQ_event+0x4e/0x96 > [<ffffffff80257fe4>] __do_IRQ+0xa4/0x105 > [<ffffffff8020b0e0>] call_softirq+0x1c/0x28 > [<ffffffff8020ceb7>] do_IRQ+0x65/0x73 > [<ffffffff80377106>] evtchn_do_upcall+0xac/0x12d > [<ffffffff8020ac16>] do_hypervisor_callback+0x1e/0x2c > <EOI> [<ffffffff8020622a>] hypercall_page+0x22a/0x1000 > [<ffffffff8020622a>] hypercall_page+0x22a/0x1000 > [<ffffffff80214297>] xen_send_IPI_mask+0x0/0xf5 > [<ffffffff8037664e>] force_evtchn_callback+0xa/0xb > [<ffffffff8031b2b8>] pci_user_read_config_dword+0x86/0x9d > [<ffffffff8031fd7c>] pci_read_config+0x114/0x1ae > [<ffffffff802bef00>] read+0x84/0xc1 > [<ffffffff8027fff9>] vfs_read+0xcb/0x171 > [<ffffffff802804bf>] sys_pread64+0x50/0x70 > [<ffffffff8020ab6b>] error_exit+0x0/0x71 > [<ffffffff8020a42e>] system_call+0x86/0x8b > [<ffffffff8020a3a8>] system_call+0x0/0x8b > If this works, I can think abt migration of pass-through enabled domU between > machines having identical hardware. > > regards, > Masroor > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Kieran Mansley
2008-Feb-07 14:18 UTC
Re: [Xen-devel] save/restore pass-through enabled domU
On Thu, 2008-02-07 at 08:59 -0500, Masroor Vettuparambil wrote:> Hi all, > > Can I save a pass-through enabled domU and restore it later on the > same machine? > I tried this with no success. In my case, saving was successful only > when I removed corresponding driver from domU. > > If this works, I can think abt migration of pass-through enabled domU > between machines having identical hardware.I''m not sure about PCI pass-through support, but the network acceleration plugin architecture we contributed to 3.2 makes solving the problems of save/restore and migration between servers (even with different hardware) much easier for those things that use it. Migration using PCI pass-through will be tricky to say the least, even between identical hardware, and particularly if you have any state on your NIC. Kieran _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Masroor Vettuparambil
2008-Feb-07 14:33 UTC
RE: [Xen-devel] save/restore pass-through enabled domU
Keir, I was trying this on v3.1. Probably this has been disabled after that. by the way, just curious to know whats the bottleneck not letting Xen to do so? regards Masroor ________________________________ From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] Sent: Thursday, February 07, 2008 7:41 PM To: Masroor Vettuparambil; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] save/restore pass-through enabled domU No, it certainly will not work. In fact I thought we disabled it in the tools, but maybe I''m mistaken. -- Keir On 7/2/08 13:59, "Masroor Vettuparambil" <Masroor.Vettuparambil@neterion.com> wrote: Hi all, Can I save a pass-through enabled domU and restore it later on the same machine? I tried this with no success. In my case, saving was successful only when I removed corresponding driver from domU. Also when I restored it, all pci associated commands (ie lspci, insmod driver) produce the following error, pcifront pci-0: pciback not responding!!! BUG: soft lockup detected on CPU#0! Call Trace: <IRQ> [<ffffffff80257c0c>] softlockup_tick+0xd8/0xea [<ffffffff8020f0ee>] timer_interrupt+0x3a1/0x3ff [<ffffffff80257ef8>] handle_IRQ_event+0x4e/0x96 [<ffffffff80257fe4>] __do_IRQ+0xa4/0x105 [<ffffffff8020b0e0>] call_softirq+0x1c/0x28 [<ffffffff8020ceb7>] do_IRQ+0x65/0x73 [<ffffffff80377106>] evtchn_do_upcall+0xac/0x12d [<ffffffff8020ac16>] do_hypervisor_callback+0x1e/0x2c <EOI> [<ffffffff8020622a>] hypercall_page+0x22a/0x1000 [<ffffffff8020622a>] hypercall_page+0x22a/0x1000 [<ffffffff80214297>] xen_send_IPI_mask+0x0/0xf5 [<ffffffff8037664e>] force_evtchn_callback+0xa/0xb [<ffffffff8031b2b8>] pci_user_read_config_dword+0x86/0x9d [<ffffffff8031fd7c>] pci_read_config+0x114/0x1ae [<ffffffff802bef00>] read+0x84/0xc1 [<ffffffff8027fff9>] vfs_read+0xcb/0x171 [<ffffffff802804bf>] sys_pread64+0x50/0x70 [<ffffffff8020ab6b>] error_exit+0x0/0x71 [<ffffffff8020a42e>] system_call+0x86/0x8b [<ffffffff8020a3a8>] system_call+0x0/0x8b If this works, I can think abt migration of pass-through enabled domU between machines having identical hardware. regards, Masroor ________________________________ _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Daniel Stodden
2008-Feb-07 16:18 UTC
Re: [Xen-devel] save/restore pass-through enabled domU
On Thu, 2008-02-07 at 14:18 +0000, Kieran Mansley wrote:> Migration using PCI pass-through will be tricky to say the least, even > between identical hardware, and particularly if you have any state on > your NIC.What happened to the idea of saving/resuming via PCI D3 power-off? Assuming that the device map could be restored. Always thought this would be rather straightforward, iff someone implements control over that. regards, daniel -- Daniel Stodden LRR - Lehrstuhl für Rechnertechnik und Rechnerorganisation Institut für Informatik der TU München D-85748 Garching http://www.lrr.in.tum.de/~stodden mailto:stodden@cs.tum.edu PGP Fingerprint: F5A4 1575 4C56 E26A 0B33 3D80 457E 82AE B0D8 735B _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel