Tom Rotenberg
2009-Sep-23 10:05 UTC
[Xen-devel] Should ''xm save'' work, when using PCI pass through?
Hi, I have a domain with PCI pass-through, and i would like to save it, and restore it later (maybe even after rebooting the dom0). Is it possible to do this using ''xm save'' & ''xm restore'', although my domain has PCI pass through? Tom _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt
2009-Sep-23 10:45 UTC
RE: [Xen-devel] Should ''xm save'' work, when using PCI pass through?
> I have a domain with PCI pass-through, and i would like to save it, > and restore it later (maybe even after rebooting the dom0). Is it > possible to do this using ''xm save'' & ''xm restore'', although my domain > has PCI pass through?You''ll need to make sure the physical devices are shutdown. The best way of doing this is to hot unplug them. You could also achieve this by putting the VM into S3 sleep before doing the save. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Sep-23 11:16 UTC
Re: [Xen-devel] Should ''xm save'' work, when using PCI pass through?
Well, i just tried sending the domain to S3, and then saving it''s state, but i got an error saying, that ''migration isn''t allowed with pass-through devices''. How can i workaround this? On Wed, Sep 23, 2009 at 1:45 PM, Ian Pratt <Ian.Pratt@eu.citrix.com> wrote:>> I have a domain with PCI pass-through, and i would like to save it, >> and restore it later (maybe even after rebooting the dom0). Is it >> possible to do this using ''xm save'' & ''xm restore'', although my domain >> has PCI pass through? > > You''ll need to make sure the physical devices are shutdown. The best way of doing this is to hot unplug them. You could also achieve this by putting the VM into S3 sleep before doing the save. > > Ian >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2009-Sep-23 12:49 UTC
Re: [Xen-devel] Should ''xm save'' work, when using PCI pass through?
I''m not sure xend can save/restore passthrough state. In which case your only option is to hot-unplug the PCI devices from the guest first. -- Keir On 23/09/2009 12:16, "Tom Rotenberg" <tom.rotenberg@gmail.com> wrote:> Well, i just tried sending the domain to S3, and then saving it''s > state, but i got an error saying, that ''migration isn''t allowed with > pass-through devices''. > How can i workaround this? > > On Wed, Sep 23, 2009 at 1:45 PM, Ian Pratt <Ian.Pratt@eu.citrix.com> wrote: >>> I have a domain with PCI pass-through, and i would like to save it, >>> and restore it later (maybe even after rebooting the dom0). Is it >>> possible to do this using ''xm save'' & ''xm restore'', although my domain >>> has PCI pass through? >> >> You''ll need to make sure the physical devices are shutdown. The best way of >> doing this is to hot unplug them. You could also achieve this by putting the >> VM into S3 sleep before doing the save. >> >> Ian >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Sep-23 12:51 UTC
Re: [Xen-devel] Should ''xm save'' work, when using PCI pass through?
Why? if i send the domain to S3, and then save the PCI config space - isn''t it enough? On Wed, Sep 23, 2009 at 3:49 PM, Keir Fraser <keir.fraser@eu.citrix.com> wrote:> I''m not sure xend can save/restore passthrough state. In which case your > only option is to hot-unplug the PCI devices from the guest first. > > -- Keir > > On 23/09/2009 12:16, "Tom Rotenberg" <tom.rotenberg@gmail.com> wrote: > >> Well, i just tried sending the domain to S3, and then saving it''s >> state, but i got an error saying, that ''migration isn''t allowed with >> pass-through devices''. >> How can i workaround this? >> >> On Wed, Sep 23, 2009 at 1:45 PM, Ian Pratt <Ian.Pratt@eu.citrix.com> wrote: >>>> I have a domain with PCI pass-through, and i would like to save it, >>>> and restore it later (maybe even after rebooting the dom0). Is it >>>> possible to do this using ''xm save'' & ''xm restore'', although my domain >>>> has PCI pass through? >>> >>> You''ll need to make sure the physical devices are shutdown. The best way of >>> doing this is to hot unplug them. You could also achieve this by putting the >>> VM into S3 sleep before doing the save. >>> >>> Ian >>> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2009-Sep-23 13:02 UTC
Re: [Xen-devel] Should ''xm save'' work, when using PCI pass through?
Well, you could try removing the warning check from xend and see what happens... -- Keir On 23/09/2009 13:51, "Tom Rotenberg" <tom.rotenberg@gmail.com> wrote:> Why? if i send the domain to S3, and then save the PCI config space - > isn''t it enough? > > On Wed, Sep 23, 2009 at 3:49 PM, Keir Fraser <keir.fraser@eu.citrix.com> > wrote: >> I''m not sure xend can save/restore passthrough state. In which case your >> only option is to hot-unplug the PCI devices from the guest first. >> >> -- Keir >> >> On 23/09/2009 12:16, "Tom Rotenberg" <tom.rotenberg@gmail.com> wrote: >> >>> Well, i just tried sending the domain to S3, and then saving it''s >>> state, but i got an error saying, that ''migration isn''t allowed with >>> pass-through devices''. >>> How can i workaround this? >>> >>> On Wed, Sep 23, 2009 at 1:45 PM, Ian Pratt <Ian.Pratt@eu.citrix.com> wrote: >>>>> I have a domain with PCI pass-through, and i would like to save it, >>>>> and restore it later (maybe even after rebooting the dom0). Is it >>>>> possible to do this using ''xm save'' & ''xm restore'', although my domain >>>>> has PCI pass through? >>>> >>>> You''ll need to make sure the physical devices are shutdown. The best way of >>>> doing this is to hot unplug them. You could also achieve this by putting >>>> the >>>> VM into S3 sleep before doing the save. >>>> >>>> Ian >>>> >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >> >> >>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Sep-23 13:03 UTC
Re: [Xen-devel] Should ''xm save'' work, when using PCI pass through?
I''ll think i''ll give it a try... On Wed, Sep 23, 2009 at 4:02 PM, Keir Fraser <keir.fraser@eu.citrix.com> wrote:> Well, you could try removing the warning check from xend and see what > happens... > > -- Keir > > On 23/09/2009 13:51, "Tom Rotenberg" <tom.rotenberg@gmail.com> wrote: > >> Why? if i send the domain to S3, and then save the PCI config space - >> isn''t it enough? >> >> On Wed, Sep 23, 2009 at 3:49 PM, Keir Fraser <keir.fraser@eu.citrix.com> >> wrote: >>> I''m not sure xend can save/restore passthrough state. In which case your >>> only option is to hot-unplug the PCI devices from the guest first. >>> >>> -- Keir >>> >>> On 23/09/2009 12:16, "Tom Rotenberg" <tom.rotenberg@gmail.com> wrote: >>> >>>> Well, i just tried sending the domain to S3, and then saving it''s >>>> state, but i got an error saying, that ''migration isn''t allowed with >>>> pass-through devices''. >>>> How can i workaround this? >>>> >>>> On Wed, Sep 23, 2009 at 1:45 PM, Ian Pratt <Ian.Pratt@eu.citrix.com> wrote: >>>>>> I have a domain with PCI pass-through, and i would like to save it, >>>>>> and restore it later (maybe even after rebooting the dom0). Is it >>>>>> possible to do this using ''xm save'' & ''xm restore'', although my domain >>>>>> has PCI pass through? >>>>> >>>>> You''ll need to make sure the physical devices are shutdown. The best way of >>>>> doing this is to hot unplug them. You could also achieve this by putting >>>>> the >>>>> VM into S3 sleep before doing the save. >>>>> >>>>> Ian >>>>> >>>> >>>> _______________________________________________ >>>> Xen-devel mailing list >>>> Xen-devel@lists.xensource.com >>>> http://lists.xensource.com/xen-devel >>> >>> >>> > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tom Rotenberg
2009-Sep-23 15:35 UTC
Re: [Xen-devel] Should ''xm save'' work, when using PCI pass through?
Just tried it... it doesn''t work. It looks like, because we have
pass-through devices, the restore is trying to send signals to qemu,
while the qemu instance is None...
The errors i see in the xend.log:
"
...
[2009-09-23 11:43:44 4345] DEBUG (XendDomainInfo:1120)
XendDomainInfo.destroyDevice: deviceClass = console, device console/0
[2009-09-23 11:43:44 4345] DEBUG (XendDomainInfo:2196) Removing pci/0
[2009-09-23 11:43:44 4345] DEBUG (XendDomainInfo:1120)
XendDomainInfo.destroyDevice: deviceClass = pci, device = pci/0
[2009-09-23 11:43:44 4345] DEBUG (pciif:418) XendDomainInfo.handleAerStateWatch
[2009-09-23 11:43:44 4345] DEBUG (pciif:537) pci: unregister aer watch
[2009-09-23 11:43:44 4345] ERROR (XendDomain:1155) Restore failed
Traceback (most recent call last):
File "usr/lib/python2.5/site-packages/xen/xend/XendDomain.py", line
1147, in domain_restore_fd
return XendCheckpoint.restore(self, fd, paused=paused,
relocating=relocating)
File "usr/lib/python2.5/site-packages/xen/xend/XendCheckpoint.py",
line 213, in restore
dominfo = xd.restore_(vmconfig)
File "usr/lib/python2.5/site-packages/xen/xend/XendDomain.py", line
509, in restore_
dominfo = XendDomainInfo.restore(config)
File "usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py",
line 234, in restore
vm.resume()
File "usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py",
line 501, in resume
self._createDevices()
File "usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py",
line 2170, in _createDevices
self.pci_device_configure_boot()
File "usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py",
line 594, in pci_device_configure_boot
self.pci_device_configure(dev_sxp)
File "usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py",
line 830, in pci_device_configure
vdevfn = self.hvm_pci_device_insert(dev_config)
File "usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py",
line 722, in hvm_pci_device_insert
return self.hvm_pci_device_insert_dev(new_dev)
File "usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py",
line 736, in hvm_pci_device_insert_dev
self.image.signalDeviceModel(''pci-ins'',
''pci-inserted'', bdf_str)
AttributeError: ''NoneType'' object has no attribute
''signalDeviceModel''
"
Shouldn''t we develop basic save/restore functionality for domains with
pass-through devices (without migration to a different machine)?
Tom
On Wed, Sep 23, 2009 at 4:02 PM, Keir Fraser <keir.fraser@eu.citrix.com>
wrote:> Well, you could try removing the warning check from xend and see what
> happens...
>
> -- Keir
>
> On 23/09/2009 13:51, "Tom Rotenberg"
<tom.rotenberg@gmail.com> wrote:
>
>> Why? if i send the domain to S3, and then save the PCI config space -
>> isn''t it enough?
>>
>> On Wed, Sep 23, 2009 at 3:49 PM, Keir Fraser
<keir.fraser@eu.citrix.com>
>> wrote:
>>> I''m not sure xend can save/restore passthrough state. In
which case your
>>> only option is to hot-unplug the PCI devices from the guest first.
>>>
>>> -- Keir
>>>
>>> On 23/09/2009 12:16, "Tom Rotenberg"
<tom.rotenberg@gmail.com> wrote:
>>>
>>>> Well, i just tried sending the domain to S3, and then saving
it''s
>>>> state, but i got an error saying, that ''migration
isn''t allowed with
>>>> pass-through devices''.
>>>> How can i workaround this?
>>>>
>>>> On Wed, Sep 23, 2009 at 1:45 PM, Ian Pratt
<Ian.Pratt@eu.citrix.com> wrote:
>>>>>> I have a domain with PCI pass-through, and i would like
to save it,
>>>>>> and restore it later (maybe even after rebooting the
dom0). Is it
>>>>>> possible to do this using ''xm save''
& ''xm restore'', although my domain
>>>>>> has PCI pass through?
>>>>>
>>>>> You''ll need to make sure the physical devices are
shutdown. The best way of
>>>>> doing this is to hot unplug them. You could also achieve
this by putting
>>>>> the
>>>>> VM into S3 sleep before doing the save.
>>>>>
>>>>> Ian
>>>>>
>>>>
>>>> _______________________________________________
>>>> Xen-devel mailing list
>>>> Xen-devel@lists.xensource.com
>>>> http://lists.xensource.com/xen-devel
>>>
>>>
>>>
>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Keir Fraser
2009-Sep-23 15:50 UTC
Re: [Xen-devel] Should ''xm save'' work, when using PCI pass through?
On 23/09/2009 16:35, "Tom Rotenberg" <tom.rotenberg@gmail.com> wrote:> Shouldn''t we develop basic save/restore functionality for domains with > pass-through devices (without migration to a different machine)?Extending xend in this way would be a great project for someone to pick up. In the meantime you could try the XCI project http://xenbits.xensource.com/XCI. Its toolstack should support this already. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel