Hi, all, Current domU suspend/resume infrastructure doesn''t satisfy physical PM activity like Xen S3, as I posted before: http://lists.xensource.com/archives/html/xen-devel/2007-01/msg00647.html Basically this patch set adds two features: - Driver domain suspend/resume support - Light-weight suspend/resume, with all old domain context kept For the driver domain, new added logic like process freeze is skipped for non-driver domain to avoid impact. One important note here is, driver domain suspend will only work correctly for the new added commands, where old domain context are still kept intact after resume. For save/restore and migration, there''s no easy way to do when domain is destroyed and re-created. The reason is the machine address saved within private driver structures. Unless domainU itself takes IOMMU as the default DMA interface for all drivers, we may need to explicitly prevent user from save/restore/migration on driver domain later. For the new added commands "sleep/wake", I''m not sure whether it''s preferred way. Since it''s purpose is very simple by sending suspend notification to domU, I choose to add instead of intervening complex logic of save/restore. Anyway, their names do explain different meaning indeed. Verified on Cset 13803. Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 6/2/07 08:12, "Tian, Kevin" <kevin.tian@intel.com> wrote:> For the new added commands "sleep/wake", I''m not sure whether > it''s preferred way. Since it''s purpose is very simple by sending suspend > > notification to domU, I choose to add instead of intervening complex > logic of save/restore. Anyway, their names do explain different meaning > indeed. > > Verified on Cset 13803.Have you considered destroying and recreating driver domains rather than sleeping them? They are stateless after all, and should be quick (enough) to boot. We want to support driver domain restart anyway for driver-bug isolation. The question mark there is whether we have the xenbus protocol sorted out yet to allow frontends to automatically reconnect. You don''t even have to worry about that if you''re prepared to do lightweight suspend/resume of all non-driver domains across S3. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] >Sent: 2007年2月7日 5:45 >On 6/2/07 08:12, "Tian, Kevin" <kevin.tian@intel.com> wrote: > >> For the new added commands "sleep/wake", I''m not sure whether >> it''s preferred way. Since it''s purpose is very simple by sending suspend >> >> notification to domU, I choose to add instead of intervening complex >> logic of save/restore. Anyway, their names do explain different >meaning >> indeed. >> >> Verified on Cset 13803. > >Have you considered destroying and recreating driver domains rather >than >sleeping them? They are stateless after all, and should be quick (enough) >to >boot.Sorry that I don''t quite understand you. For suspend-to-disk, maybe we can use that policy to recreate driver domains. But for suspend-to-ram, that process is still slow and we just need suspend domain itself. Also I''m not sure why it''s stateless... How do you recover workloads after re-creation?> >We want to support driver domain restart anyway for driver-bug isolation. >The question mark there is whether we have the xenbus protocol sorted >out >yet to allow frontends to automatically reconnect. You don''t even have to >worry about that if you''re prepared to do lightweight suspend/resume of >all >non-driver domains across S3. > > -- KeirAnyway driver domain restart should be another issue. BTW, I can destroy and re-create a driver domain manually now. So, could you be more specific which part you''d like to change in my patch sets? :-) Thanks. Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 7/2/07 01:25, "Tian, Kevin" <kevin.tian@intel.com> wrote:>> Have you considered destroying and recreating driver domains rather >> than >> sleeping them? They are stateless after all, and should be quick (enough) >> to >> boot. > > Sorry that I don''t quite understand you. For suspend-to-disk, maybe > we can use that policy to recreate driver domains. But for > suspend-to-ram, that process is still slow and we just need suspend > domain itself. Also I''m not sure why it''s stateless... How do you > recover workloads after re-creation?A proper driver domain would be little more than a device driver wrapped in minios-style Xen-interfacing code. They should be quick to start, and restart would lose no workload. I suppose there are people running device drivers as part of a full guest however... Is it necessary to sleep/wake non driver domains? Could we check in S3 support that would work for systems with all drivers in dom0, and then consider this patchset as an extension? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] >Sent: 2007年2月7日 9:57 >> Sorry that I don''t quite understand you. For suspend-to-disk, maybe >> we can use that policy to recreate driver domains. But for >> suspend-to-ram, that process is still slow and we just need suspend >> domain itself. Also I''m not sure why it''s stateless... How do you >> recover workloads after re-creation? > >A proper driver domain would be little more than a device driver wrapped >in >minios-style Xen-interfacing code. They should be quick to start, and >restart would lose no workload. I suppose there are people running >device >drivers as part of a full guest however...So this is the point about definition about driver domains. If we consider driver domain as the part of I/O virtualization service provider like dom0, minios-style is perfectly right and quick. However we have to consider the cases where user just wants device passthrough for performance. Actually most requirements on the list before driver domain was back to xen3.0, are all related to a full guest which doesn''t provide backend service to others. Then actually we have two scenarios: - Minios-driver domains created automatically to balance I/O virtualization, as part of increasing overall xen performance - Full guest with direct device assignment just for performance, which is created by user manually I think we need cover both cases.> >Is it necessary to sleep/wake non driver domains? Could we check in S3 >support that would work for systems with all drivers in dom0, and then >consider this patchset as an extension? > > -- KeirIt''s still necessary to sleep/wake non driver domains, or else simple pause/unpause out of domain''s knowledge is not enough. For example, at least time_resume is necessary after a long sleep/wake period. If you prefer to check in basic S3 first, OK I can hold it after that. Currently I sent out this patch set because it''s self-contained. Anyway, we''re doing final cleanup and rebase for S3, and should be able to send out soon. Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 7/2/07 02:21, "Tian, Kevin" <kevin.tian@intel.com> wrote:> It''s still necessary to sleep/wake non driver domains, or else simple > pause/unpause out of domain''s knowledge is not enough. For example, > at least time_resume is necessary after a long sleep/wake period. > > If you prefer to check in basic S3 first, OK I can hold it after that. > Currently I sent out this patch set because it''s self-contained. Anyway, > we''re doing final cleanup and rebase for S3, and should be able to > send out soon.Sounds good. With feature freeze for 3.0.5 now, all these patches are likely to be held pending for the next few weeks at least anyway. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Puthiyaparambil, Aravindh
2007-Feb-07 05:01 UTC
RE: [Xen-devel] [PATCH 0/6] PM extension to domU
> So this is the point about definition about driver domains. If we > consider driver domain as the part of I/O virtualization service > provider like dom0, minios-style is perfectly right and quick. > However we have to consider the cases where user just wants > device passthrough for performance. Actually most requirements > on the list before driver domain was back to xen3.0, are all related > to a full guest which doesn''t provide backend service to others. > > Then actually we have two scenarios: > - Minios-driver domains created automatically to balance I/O > virtualization, as part of increasing overall xen performance > - Full guest with direct device assignment just for performance, > which is created by user manuallyKevin, Keir, Are Mini-OS based driver domains possible today? Is there a group working on this? Thanks, Aravindh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 7/2/07 05:01, "Puthiyaparambil, Aravindh" <aravindh.puthiyaparambil@unisys.com> wrote:>> Then actually we have two scenarios: >> - Minios-driver domains created automatically to balance I/O >> virtualization, as part of increasing overall xen performance >> - Full guest with direct device assignment just for performance, >> which is created by user manually > > Kevin, Keir, > > Are Mini-OS based driver domains possible today? Is there a group > working on this?Not really. The best you can do right now is a stripped-down Linux kernel. If you don''t boot user space (e.g., have a very small init) then that''s actually not bad. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel