Jiri Denemark
2008-Nov-13 08:40 UTC
[Xen-devel] [PATCH] CPU affinity reset during save/restore
Hi, This is the same patch I sent earlier this week with Signed-off-by-line added. Even though there is a quite old cs 14588 (http://xenbits.xensource.com/xen-unstable.hg?rev/853853686147) described as a fix to this issue, the CPU affinity is still reset after dom-U being saved and restored in current xen-unstable. I found two reasons for this. First, although XendConfig._parse_sxp() tries to parse the affinity from ''cpus'', the list is not stored in domain''s checkpoint file. This is because cs 15152 (http://xenbits.xensource.com/xen-unstable.hg?rev/681ed46676a6) prevents ''cpus'' list to be generated in XendConfig.to_sxp(). Second, even though ''cpus'' is correctly parsed in XendConfig._parse_sxp(), it is not used for setting CPU affinity in XendDomainInfo.resume(). That said, after reverting cs 14588 and applying the attached patch, the CPU affinity is correctly restored after restoring the domain. The attached patch just copies the code related to CPU affinity from XendDomainInfo._initDomain() into XendDomainInfo._setCPUAffinity() and calls this method from _initDomain() and resume(). What do you think about this way to fix the CPU affinity reset? If it is not the right way, what way do you prefer this issue to be fixed? Any comments how the patch could be improved are most welcome. Signed-off-by: Jiri Denemark <jdenemar@redhat.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dan Magenheimer
2008-Nov-13 15:14 UTC
RE: [Xen-devel] [PATCH] CPU affinity reset during save/restore
I looked at this problem a few months ago. Your solution looks great for save/restore on the same machine. But if the restore occurs on another machine that doesn''t have the same set of CPUs, does the right thing happen? (And more importantly, what is "the right thing" for a very different set of CPUs, for example saving on a machine with no hyperthreading and restoring on one with hyper- threading?) Even on the same machine, affinity decisions for domain X that are made between save of domain Y and restore of domain Y might imply that the same affinities for domain Y should not be applied on restore of Y. So after thinking about it, I decided not to implement this as IMHO affinity decisions are best left to higher level management tools.> -----Original Message----- > From: Jiri Denemark [mailto:jdenemar@redhat.com] > Sent: Thursday, November 13, 2008 1:40 AM > To: xen-devel@lists.xensource.com > Subject: [Xen-devel] [PATCH] CPU affinity reset during save/restore > > > Hi, > > This is the same patch I sent earlier this week with > Signed-off-by-line added. > > Even though there is a quite old cs 14588 > (http://xenbits.xensource.com/xen-unstable.hg?rev/853853686147) described as a fix to this issue, the CPU affinity is still reset after dom-U being saved and restored in current xen-unstable. I found two reasons for this. First, although XendConfig._parse_sxp() tries to parse the affinity from ''cpus'', the list is not stored in domain''s checkpoint file. This is because cs 15152 (http://xenbits.xensource.com/xen-unstable.hg?rev/681ed46676a6) prevents ''cpus'' list to be generated in XendConfig.to_sxp(). Second, even though ''cpus'' is correctly parsed in XendConfig._parse_sxp(), it is not used for setting CPU affinity in XendDomainInfo.resume(). That said, after reverting cs 14588 and applying the attached patch, the CPU affinity is correctly restored after restoring the domain. The attached patch just copies the code related to CPU affinity from XendDomainInfo._initDomain() into XendDomainInfo._setCPUAffinity() and calls this method from _initDomain() and resume(). What do you think about this way to fix the CPU affinity reset? If it is not the right way, what way do you prefer this issue to be fixed? Any comments how the patch could be improved are most welcome. Signed-off-by: Jiri Denemark <jdenemar@redhat.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiri Denemark
2008-Nov-14 11:12 UTC
Re: [Xen-devel] [PATCH] CPU affinity reset during save/restore
> So after thinking about it, I decided not to implement > this as IMHO affinity decisions are best left to higher > level management tools.Well, yes, you are right affinity decisions should be made by management tools. However, I think this is more targeted into a static world, where a static set of domains is running on a host, affinity is set in each domain''s configuration file and when dom0 is rebooted, domUs are automatically saved and restored. Which is when the affinity should be restored too. Moreover, such behavior does not really interfere with any higher level management tool, as it can always repin cpus after restoring or migrating a domain, as this process would probably be under that tool''s control anyway. Of course, if domains are migrated manually, one would have to repin cpus manually, too. Jirka -- Experience is what causes a person to make new mistakes instead of old ones. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiri Denemark
2008-Nov-14 16:19 UTC
Re: [Xen-devel] [PATCH] CPU affinity reset during save/restore
Hi, This is an improved patch for restoring CPU affinity on domain resume.> The attached patch just copies the code related to CPU affinity from > XendDomainInfo._initDomain() into XendDomainInfo._setCPUAffinity() and calls > this method from _initDomain() and resume().I''ve added a try-except around self._setCPUAffinity() in resum() so that the domain can be resumed on different HW, where CPUs mentioned in the ''cpus'' list are not available. In that case, the affinity is just ignored. Without this improvement, such domain would fail to resume with invalid argument. Signed-off-by: Jiri Denemark <jdenemar@redhat.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel