James Harper
2010-Dec-10 05:23 UTC
[Xen-devel] common time reference between domU and dom0
I would like to be able to measure front/back ring message latency between a Windows DomU and Dom0. To do this I need a common time reference between the two, preferably with at least microsecond precision (latency <1us is irrelevant for my purposes). Does such a thing exist? A small amount of overhead in measuring this latency is acceptable. Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2010-Dec-10 07:19 UTC
Re: [Xen-devel] common time reference between domU and dom0
System time, as exposed to guests by Xen, should be good enough. -- Keir On 10/12/2010 05:23, "James Harper" <james.harper@bendigoit.com.au> wrote:> I would like to be able to measure front/back ring message latency > between a Windows DomU and Dom0. To do this I need a common time > reference between the two, preferably with at least microsecond > precision (latency <1us is irrelevant for my purposes). Does such a > thing exist? A small amount of overhead in measuring this latency is > acceptable. > > Thanks > > James > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
James Harper
2010-Dec-10 08:23 UTC
RE: [Xen-devel] common time reference between domU and dom0
> > System time, as exposed to guests by Xen, should be good enough. >How is this exposed for HVM systems? James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Henry Pepper
2010-Dec-10 08:26 UTC
Re: [Xen-devel] common time reference between domU and dom0
I was under the impression that none of the domains has the correct time. Neither Dom0 nor DomU receives a timer interrupt, but only a timer event. Events that could be missed. So timing in the millisecond can have jitter and drift. Is there actually a way to keep a precise timer in a domain? On top of this I would think that both domains would have to have exclusive access to their own CPU in order to avoid being switched off of the CPU in periods of time(quantums of 30 ms using the default scheduler). Henry On Fri, Dec 10, 2010 at 8:19 AM, Keir Fraser <keir@xen.org> wrote:> System time, as exposed to guests by Xen, should be good enough. > > -- Keir > > > On 10/12/2010 05:23, "James Harper" <james.harper@bendigoit.com.au> wrote: > >> I would like to be able to measure front/back ring message latency >> between a Windows DomU and Dom0. To do this I need a common time >> reference between the two, preferably with at least microsecond >> precision (latency <1us is irrelevant for my purposes). Does such a >> thing exist? A small amount of overhead in measuring this latency is >> acceptable. >> >> Thanks >> >> James >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2010-Dec-10 10:00 UTC
Re: [Xen-devel] common time reference between domU and dom0
On 10/12/2010 08:23, "James Harper" <james.harper@bendigoit.com.au> wrote:>> >> System time, as exposed to guests by Xen, should be good enough. >> > > How is this exposed for HVM systems?Given all HVM guest time sources are trapped to the hypervisor, including TSC, you may as well just use the HVMOP_get_time hypercall. -- Keir> James_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2010-Dec-10 10:03 UTC
Re: [Xen-devel] common time reference between domU and dom0
All domains have access to Xen system time, either via hypercall or by calculating it themselves directly given information Xen provides in shared memory. Time interrupt/events can''t really be missed - they are delivered synchronously to the guest. -- Keir On 10/12/2010 08:26, "Henry Pepper" <henrypepr@gmail.com> wrote:> I was under the impression that none of the domains has the correct time. > Neither Dom0 nor DomU receives a timer interrupt, but only a timer > event. Events that could be missed. > So timing in the millisecond can have jitter and drift. > Is there actually a way to keep a precise timer in a domain? > > On top of this I would think that both domains would have to have > exclusive access to their own CPU in order to avoid being switched off > of the CPU in periods of time(quantums of 30 ms using the default > scheduler). > > Henry > > On Fri, Dec 10, 2010 at 8:19 AM, Keir Fraser <keir@xen.org> wrote: >> System time, as exposed to guests by Xen, should be good enough. >> >> -- Keir >> >> >> On 10/12/2010 05:23, "James Harper" <james.harper@bendigoit.com.au> wrote: >> >>> I would like to be able to measure front/back ring message latency >>> between a Windows DomU and Dom0. To do this I need a common time >>> reference between the two, preferably with at least microsecond >>> precision (latency <1us is irrelevant for my purposes). Does such a >>> thing exist? A small amount of overhead in measuring this latency is >>> acceptable. >>> >>> Thanks >>> >>> James >>> >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
James Harper
2010-Dec-10 10:20 UTC
RE: [Xen-devel] common time reference between domU and dom0
> On 10/12/2010 08:23, "James Harper" <james.harper@bendigoit.com.au>wrote:> > >> > >> System time, as exposed to guests by Xen, should be good enough. > >> > > > > How is this exposed for HVM systems? > > Given all HVM guest time sources are trapped to the hypervisor,including> TSC, you may as well just use the HVMOP_get_time hypercall. >Is that a new call? I don''t see it mentioned in 4.0-testing anywhere. Is there a fallback if HVMOP_get_time doesn''t exist? James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Paolo Bonzini
2010-Dec-10 10:22 UTC
[Xen-devel] Re: common time reference between domU and dom0
On 12/10/2010 11:20 AM, James Harper wrote:> Is that a new call? I don''t see it mentioned in 4.0-testing anywhere. > > Is there a fallback if HVMOP_get_time doesn''t exist?Not really, but it is really an easy backport to your hypervisor. It will even be in RHEL5.6/CentOS 5.6. Paolo _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2010-Dec-10 10:45 UTC
Re: [Xen-devel] common time reference between domU and dom0
On 10/12/2010 10:20, "James Harper" <james.harper@bendigoit.com.au> wrote:>> On 10/12/2010 08:23, "James Harper" <james.harper@bendigoit.com.au> > wrote: >> >>>> >>>> System time, as exposed to guests by Xen, should be good enough. >>>> >>> >>> How is this exposed for HVM systems? >> >> Given all HVM guest time sources are trapped to the hypervisor, > including >> TSC, you may as well just use the HVMOP_get_time hypercall. >> > > Is that a new call? I don''t see it mentioned in 4.0-testing anywhere.Yeah I think it''s in 4.1 only.> Is there a fallback if HVMOP_get_time doesn''t exist?Switch on vtsc (might be default) and just use RDTSC? Might be good enough. I''m not sure that computing system time yourself from fields in shared info works well for HVM guests. HVMOP_get_time I think had been in XenServer patch queue a while and has just been pulled across to upstream. -- Keir> James >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2010-Dec-10 10:45 UTC
Re: [Xen-devel] Re: common time reference between domU and dom0
On 10/12/2010 10:22, "Paolo Bonzini" <pbonzini@redhat.com> wrote:> On 12/10/2010 11:20 AM, James Harper wrote: >> Is that a new call? I don''t see it mentioned in 4.0-testing anywhere. >> >> Is there a fallback if HVMOP_get_time doesn''t exist? > > Not really, but it is really an easy backport to your hypervisor. It > will even be in RHEL5.6/CentOS 5.6.It''s probably worth backporting for 4.0.2 too. -- Keir> Paolo > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tim Deegan
2010-Dec-10 10:51 UTC
Re: [Xen-devel] common time reference between domU and dom0
At 10:45 +0000 on 10 Dec (1291977908), Keir Fraser wrote:> On 10/12/2010 10:20, "James Harper" <james.harper@bendigoit.com.au> wrote: > > >> On 10/12/2010 08:23, "James Harper" <james.harper@bendigoit.com.au> > > wrote: > >> > >>>> > >>>> System time, as exposed to guests by Xen, should be good enough. > >>>> > >>> > >>> How is this exposed for HVM systems? > >> > >> Given all HVM guest time sources are trapped to the hypervisor, > > including > >> TSC, you may as well just use the HVMOP_get_time hypercall. > >> > > > > Is that a new call? I don''t see it mentioned in 4.0-testing anywhere. > > Yeah I think it''s in 4.1 only. > > > Is there a fallback if HVMOP_get_time doesn''t exist? > > Switch on vtsc (might be default) and just use RDTSC? Might be good enough.IIRC HVMOP_get_time was added because from Windows drivers you can''t tell what the Windows kernel has done to the TSC so you can''t use it to get system time yourself. It seems to me that exposing the TSC offset alongside the rest of the shared info would fix that but maybe you''d be as well off backporting HVMOP_get_time instead. Tim. -- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, Xen Platform Team Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dan Magenheimer
2010-Dec-10 20:24 UTC
RE: [Xen-devel] common time reference between domU and dom0
> >>> How is this exposed for HVM systems? > >> > >> Given all HVM guest time sources are trapped to the hypervisor, > > including > >> TSC, you may as well just use the HVMOP_get_time hypercall. > >> > > > > Is that a new call? I don''t see it mentioned in 4.0-testing anywhere. > > Yeah I think it''s in 4.1 only. > > > Is there a fallback if HVMOP_get_time doesn''t exist? > > Switch on vtsc (might be default) and just use RDTSC? Might be good > enough. > I''m not sure that computing system time yourself from fields in shared > info > works well for HVM guests. HVMOP_get_time I think had been in XenServer > patch queue a while and has just been pulled across to upstream.I believe vtsc will be different in different domains as I believe Xen adjusts it so tsc in any guest effectively starts at zero at time of domain launch. Also, an HVM domain may "write" to tsc and change its effective value. Also, I''m not sure IIRC, but I think vtsc is never on for dom0. And IIRC the default for vtsc (vtsc=0) is dependent on the underlying hardware and events (such as migration/save/restore) that transpire. It can be forced on in a vm config file (but again IIRC not for dom0). See xen-XXX.hg/docs/misc/tscmode.txt for XXX >= 4.0 Dan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dan Magenheimer
2010-Dec-10 20:27 UTC
RE: [Xen-devel] Re: common time reference between domU and dom0
Urk. Paolo, I hope Redhat isn''t using HVMOP_get_time now as a basis for pvops time? That was one of the purposes of all the tsc work in 4.0 so selecting TSC as the clocksource should have the same result (and IIRC the trap is faster than the hypercall).> -----Original Message----- > From: Keir Fraser [mailto:keir@xen.org] > Sent: Friday, December 10, 2010 3:46 AM > To: Paolo Bonzini; James Harper > Cc: xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Re: common time reference between domU and > dom0 > > On 10/12/2010 10:22, "Paolo Bonzini" <pbonzini@redhat.com> wrote: > > > On 12/10/2010 11:20 AM, James Harper wrote: > >> Is that a new call? I don''t see it mentioned in 4.0-testing > anywhere. > >> > >> Is there a fallback if HVMOP_get_time doesn''t exist? > > > > Not really, but it is really an easy backport to your hypervisor. It > > will even be in RHEL5.6/CentOS 5.6. > > It''s probably worth backporting for 4.0.2 too. > > -- Keir > > > Paolo > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Paolo Bonzini
2010-Dec-10 21:41 UTC
Re: [Xen-devel] Re: common time reference between domU and dom0
On 12/10/2010 09:27 PM, Dan Magenheimer wrote:> Urk. Paolo, I hope Redhat isn''t using HVMOP_get_time now as a basis > for pvops time?No. I only plan to use it in Windows guests. That said...> That was one of the purposes of all the tsc work > in 4.0 so selecting TSC as the clocksource should have the same > result (and IIRC the trap is faster than the hypercall).... this has not been backported to RHEL5 though. Paolo _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel