Hi All, I saw someone had done patches for this and run benchmarks a while ago but still no mention of support within Xen. Is support for RDMA based migration planned? (Or even better is it already present?) Thanks, Rob The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ SAQ is the trading name of SEMTEC Limited. Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. ISPA Member Find us in http://www.thebestof.co.uk/petersfield _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> I saw someone had done patches for this and run benchmarks a while ago > but still no mention of support within Xen. Is support for RDMA based > migration planned? (Or even better is it already present?)I don''t think I''ve seen any patches for doing live relocation using RDMA, but it wouldn''t be hard to create them. However, the benefits may not actually be that great: when doing a live relocation you''re usually moving a VM from a loaded machine to a less-loaded one, hence it''s the sending machine that you care most about CPU usage. Assuming the NIC you''re using has TSO, it should be relatively efficient to send the data over TCP. However, this could definitely be optimized by adding a zero copy TX mechanism to avoid the copy from user space to the kernel, and this would benefit all NICs, not just RDMA. It requires a little bit of work to do this as sendfile isn''t a good fit and the foreign pages will likely need page structs which isn''t the case at present. Ian> > > Thanks, > > Rob > > The SAQ Group > > Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ > SAQ is the trading name of SEMTEC Limited. Registered in England & > Wales > Company Number: 06481952 > > http://www.saqnet.co.uk AS29219 > > SAQ Group Delivers high quality, honestly priced communication andI.T.> services to UK Business. > > Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : > Transit : Backups : Managed Networks : Remote Support. > > ISPA Member > > Find us in http://www.thebestof.co.uk/petersfield > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Here are some results comparing Ininiband with IPOIB TCP/IP migration to Infiniband native RDMA migration: http://nowlab.cse.ohio-state.edu/publications/conf-papers/2007/huang-clu ster07.pdf 4 to 5 times quicker is about the average so it seems it could bring big advantages for those that use Infiniband or IWarp NICs. Any chance of it being considered for a future version? Rob -----Original Message----- From: Ian Pratt [mailto:Ian.Pratt@eu.citrix.com] Sent: 02 January 2009 12:07 To: Robert Dunkley; xen-devel@lists.xensource.com Cc: Ian Pratt Subject: RE: [Xen-devel] RDMA based Live Migration> I saw someone had done patches for this and run benchmarks a while ago > but still no mention of support within Xen. Is support for RDMA based > migration planned? (Or even better is it already present?)I don''t think I''ve seen any patches for doing live relocation using RDMA, but it wouldn''t be hard to create them. However, the benefits may not actually be that great: when doing a live relocation you''re usually moving a VM from a loaded machine to a less-loaded one, hence it''s the sending machine that you care most about CPU usage. Assuming the NIC you''re using has TSO, it should be relatively efficient to send the data over TCP. However, this could definitely be optimized by adding a zero copy TX mechanism to avoid the copy from user space to the kernel, and this would benefit all NICs, not just RDMA. It requires a little bit of work to do this as sendfile isn''t a good fit and the foreign pages will likely need page structs which isn''t the case at present. Ian> > > Thanks, > > Rob > > The SAQ Group > > Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ > SAQ is the trading name of SEMTEC Limited. Registered in England & > Wales > Company Number: 06481952 > > http://www.saqnet.co.uk AS29219 > > SAQ Group Delivers high quality, honestly priced communication andI.T.> services to UK Business. > > Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : > Transit : Backups : Managed Networks : Remote Support. > > ISPA Member > > Find us in http://www.thebestof.co.uk/petersfield > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel