Here''s a question about FE driver and log dirty, when live migration is concerned. Log dirty mode is used in live migration, which works for those polluted pages from CPU issued accesses, but not for DMA path (as BE driver talked here which access from another domain like dom0). Most frontend drivers don''t implement their own suspend interface (netfront implements when accelerator is enabled). That seems to make sense, e.g. blkfront shadows pending reqs, and requeue them after resume. However considering log dirty mode, there''s an assumption for this style: **There must be some CPU issued accesses to target data page before xen_suspend is invoked, if that page is serviced by BE driver and has been dequeued from FE queues within the loop copy process. Or else log dirty mode is not triggered to re-transimit that page in next loop. ** Does that assumption always hold true for existing FE drivers? Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 14/7/08 08:18, "Tian, Kevin" <kevin.tian@intel.com> wrote:> **There must be some CPU issued accesses to target data page > before xen_suspend is invoked, if that page is serviced by BE driver > and has been dequeued from FE queues within the loop copy > process. Or else log dirty mode is not triggered to re-transimit that > page in next loop. ** > > Does that assumption always hold true for existing FE drivers?Not sure what you mean? There can be requests which the FE re-queues after resume which have in fact already been satisfied (pre-suspend) and even the response queued up in the old shared request/response ring. The key is that replaying them is idempotent. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] >Sent: 2008年7月14日 15:28 > >On 14/7/08 08:18, "Tian, Kevin" <kevin.tian@intel.com> wrote: > >> **There must be some CPU issued accesses to target data page >> before xen_suspend is invoked, if that page is serviced by BE driver >> and has been dequeued from FE queues within the loop copy >> process. Or else log dirty mode is not triggered to re-transimit that >> page in next loop. ** >> >> Does that assumption always hold true for existing FE drivers? > >Not sure what you mean? There can be requests which the FE >re-queues after >resume which have in fact already been satisfied (pre-suspend) >and even the >response queued up in the old shared request/response ring. >The key is that >replaying them is idempotent. >I can understand the replay trick here. My question is whether there''re some requests/responses got dequeued from FE driver and already sent to up level component, which however has not been ever accessed by CPU (e.g. only descriptor is accessed) before __xen_suspend is entered. Take network receive for example (I''m not familar with this path), is it possible that some data page is already queued in up level protocol, and then suspend watch is triggered before receive process is scheduled, and unfortunately within that window the page has not been accessed yet? In this case, page becomes dirty in live migration process, but not get recoded by log dirty logic since it''s only BE to access it... Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 14/7/08 08:39, "Tian, Kevin" <kevin.tian@intel.com> wrote:> I can understand the replay trick here. My question is whether there''re > some requests/responses got dequeued from FE driver and already > sent to up level component, which however has not been ever accessed > by CPU (e.g. only descriptor is accessed) before __xen_suspend is > entered. Take network receive for example (I''m not familar with this > path), is it possible that some data page is already queued in up level > protocol, and then suspend watch is triggered before receive process > is scheduled, and unfortunately within that window the page has not > been accessed yet? In this case, page becomes dirty in live migration > process, but not get recoded by log dirty logic since it''s only BE to > access it...Granted mappings cause the mapped page to become dirtied when the BE relinquishes the grant. This happens after the I/O transfer but before the response has been queued for the FE. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] >Sent: 2008年7月14日 17:15 > >On 14/7/08 08:39, "Tian, Kevin" <kevin.tian@intel.com> wrote: > >> I can understand the replay trick here. My question is >whether there''re >> some requests/responses got dequeued from FE driver and already >> sent to up level component, which however has not been ever accessed >> by CPU (e.g. only descriptor is accessed) before __xen_suspend is >> entered. Take network receive for example (I''m not familar with this >> path), is it possible that some data page is already queued >in up level >> protocol, and then suspend watch is triggered before receive process >> is scheduled, and unfortunately within that window the page has not >> been accessed yet? In this case, page becomes dirty in live migration >> process, but not get recoded by log dirty logic since it''s only BE to >> access it... > >Granted mappings cause the mapped page to become dirtied when the BE >relinquishes the grant. This happens after the I/O transfer >but before the >response has been queued for the FE. >Got it. Thanks, - Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel