I brought this up with a few people at the summit, but I thought I''d gauge interest here. The current format used for save files and migration has a number of problems: 1) it''s not self-identifying (word size, machine type) 2) it''s unversioned 3) it misses some useful meta information (e.g. xen version) 4) it''s not easily extendable 5) it needlessly uses a different format from core files, requiring a debugger to implement both formats as a backend if support for both is wanted 1) and 5) makes implementing a debugger that works sanely very difficult. 2) and 4) are likely to significantly complicate things in the future, whilst 3) makes debugging a lot less useful. As the migration format needs modifying for HVM anyway as Intel presented, and presumably for XML config changes, I suggest that a new format be put in place during the 3.0.4 timeline. Are there other people interested in such a change? I was thinking of a simple ELF-like format. One problem is that of migration''s streaming format, but this could be dealt with as a special case (e.g. that section''s size is explicitly labelled as ''0'' to mean "special case"). There also remains the question of supporting older Xens. I don''t believe that anything other than migration back-compatibility is interesting in this case, and presumably this can be handled reasonably easily by a slight refactoring of xc_linux_{save,restore} (on the presumption that people will want to migrate both to and from such older dom0''s), and falling back to compat mode when the new negotation API fails. regards john _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> There also remains the question of supporting older Xens. I don''t > believe that anything other than migration back-compatibility is > interesting in this case, and presumably this can be handled reasonably > easily by a slight refactoring of xc_linux_{save,restore} (on the > presumption that people will want to migrate both to and from such older > dom0''s), and falling back to compat mode when the new negotation API > fails.I wonder how necessary even this is; there''s no strict guarantee that domains running on newer Xens will even be able to run on older Xens if migrated there. I guess you *might* want to be able to migrate older->newer (that''s probably that you meant anyway) so that you could perform an upgrade. This does run the risk of confusing users with non-symmetric migration, however. Just wondering if it''s worth the hassle, but I guess it''d be nice not to require a whole cluster to have the same Xen/dom0/tools configuration. Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> I brought this up with a few people at the summit, but I > thought I''d gauge interest here. > > The current format used for save files and migration has a number of > problems: > > 1) it''s not self-identifying (word size, machine type) > 2) it''s unversioned > 3) it misses some useful meta information (e.g. xen version) > 4) it''s not easily extendable > 5) it needlessly uses a different format from core files, > requiring a debugger to implement both formats as a backend > if support for both is wantedThe plan is to fix much of this post 3.0.3 while integrating HVM save/restore. Would be good to have your help on this. save/restore is quite different from a core, so I''m not sure its worth doing #5. For the sake of optimizing IO, keeping the data pages aligned is worthwhile (I also want to make support for RDMA during live relo easy as well). For PV guest live relo we have to transfer page type info, and there is some advantage to transferring this ''as we go'' rather than in a big batch at the end.> There also remains the question of supporting older Xens. I > don''t believe that anything other than migration > back-compatibility is interesting in this case, and > presumably this can be handled reasonably easily by a slight > refactoring of xc_linux_{save,restore} (on the presumption > that people will want to migrate both to and from such older > dom0''s), and falling back to compat mode when the new > negotation API fails.It''s an interesting question whether we can get away with breaking old saved images. We''ve never guaranteed stability of this ABI, but it would be nice to retain backward compatibility if it''s not too hard. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, Sep 14, 2006 at 02:04:40PM +0100, Ian Pratt wrote:> The plan is to fix much of this post 3.0.3 while integrating HVM > save/restore. Would be good to have your help on this.Does this mean somebody is working on it already? I saw Jun''s proposed format changes briefly during his presentation and they seemed inadequate in terms of fixing the problems I see in the current format. But certainly since Jun has a vested interest I''d be happy to help come up with a reasonable format.> save/restore is quite different from a core, so I''m not sure its worth > doing #5.Can you explain why you think so? In particular what difficulties do you expect with using the same format for both? Both contain VCPU context(s), PFN tables and their associated page contents. The places where they differ are essentially minor. I think (from the Linux POV) getting ptrace support on save files for nearly free is certainly worth it.> For the sake of optimizing IO, keeping the data pages aligned is > worthwhile (I also want to make support for RDMA during live relo easy > as well).There''s no reason this can''t be done.> For PV guest live relo we have to transfer page type info, and > there is some advantage to transferring this ''as we go'' rather than in a > big batch at the end.Absolutely, as I mentioned, any format would need to have enough leeway to support streaming (and, after all, page resends). It''s perfectly possible to have a slightly modified ELF-style format for this...> > There also remains the question of supporting older Xens. I > > don''t believe that anything other than migration > > back-compatibility is interesting in this case, and > > presumably this can be handled reasonably easily by a slight > > refactoring of xc_linux_{save,restore} (on the presumption > > that people will want to migrate both to and from such older > > dom0''s), and falling back to compat mode when the new > > negotation API fails. > > It''s an interesting question whether we can get away with breaking old > saved images. We''ve never guaranteed stability of this ABI, but it would > be nice to retain backward compatibility if it''s not too hard.It''s probably easy enough to do. regards john _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Sep 14, 2006, at 8:14 AM, John Levon wrote:> > I brought this up with a few people at the summit, but I thought I''d > gauge interest here. > > The current format used for save files and migration has a number of > problems: > > 1) it''s not self-identifying (word size, machine type) > 2) it''s unversioned > 3) it misses some useful meta information (e.g. xen version) > 4) it''s not easily extendable > 5) it needlessly uses a different format from core files, requiring a > debugger to implement both formats as a backend if support for both is > wanted >I would like us to seriously consider a directory structure that contains as much information as possible. This would address 1-4 and we can develop a tools that can can make vmcore''s gcore''s for the plethora of debugging tools (gdb, crash, dbx, adb) used for 5. The result could be used for saving, debugging, cloning, move to simulators and can simply be tarballed for storage and sharing. There is so much the regular core files do not address. The PPC team is just about to go deep on this and will slowly begin to discover all that happens here and how different we are from the current arch-users. -JX _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, Sep 14, 2006 at 09:37:07AM -0400, Jimi Xenidis wrote:> >I brought this up with a few people at the summit, but I thought I''d > >gauge interest here. > > > >The current format used for save files and migration has a number of > >problems: > > > >1) it''s not self-identifying (word size, machine type) > >2) it''s unversioned > >3) it misses some useful meta information (e.g. xen version) > >4) it''s not easily extendable > >5) it needlessly uses a different format from core files, requiring a > >debugger to implement both formats as a backend if support for both is > >wanted > > > > I would like us to seriously consider a directory structure that > contains as much information as possible.That''s fine. We could, for example, use this container format as return data from an "xm export"[1]. But the question of what format the memory image takes will still remain, so I think this is a useful starting point too. regards john [1] we''ll need this for at least the config files once the lifecycle changes are in _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel