Xen devel - Oct 2010 - [PATCH] time-xen : Reset monotonic time when sync up time from dom0 to domU

Signed-off-by: Shunli Yi <shyi@websense.com>
Signed-off-by: Hang Du <hdu@websense.com>

When sync up time from dom0 to domU, the monotonic time should be reset or domU
cannot sync up backward time from dom0.

diff -urN a/arch/i386/kernel/time-xen.c   b/arch/i386/kernel/time-xen.c

--- a/arch/i386/kernel/time-xen.c   2010-10-11 10:41:06.000000000 +0800
+++ b/arch/i386/kernel/time-xen.c   2010-10-11 10:43:32.000000000 +0800
@@ -715,6 +715,8 @@
    }

    if (shadow_tv_version != HYPERVISOR_shared_info->wc_version) {
+        if (!independent_wallclock)
+            monotonic_reset();
        update_wallclock();
        schedule_clock_was_set_work = 1;
    }



 Protected by Websense Hosted Email Security -- www.websense.com 



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> On 12.10.10 at 12:19, "Du, Hang" <hdu@websense.com>
wrote:
> Signed-off-by: Shunli Yi <shyi@websense.com>
> Signed-off-by: Hang Du <hdu@websense.com>
> 
> When sync up time from dom0 to domU, the monotonic time should be reset or 
> domU cannot sync up backward time from dom0.
Without a better description of what you try to do and why there
is no is_initial_xendomain() check in this code despite talk being
about sync-ing into a single direction (admittedly the word "backward"
above is confusing me quite a bit), I would ask that this patch not be
applied.

Jan
> diff -urN a/arch/i386/kernel/time-xen.c   b/arch/i386/kernel/time-xen.c
> 
> --- a/arch/i386/kernel/time-xen.c   2010-10-11 10:41:06.000000000 +0800
> +++ b/arch/i386/kernel/time-xen.c   2010-10-11 10:43:32.000000000 +0800
> @@ -715,6 +715,8 @@
>     }
> 
>     if (shadow_tv_version != HYPERVISOR_shared_info->wc_version) {
> +        if (!independent_wallclock)
> +            monotonic_reset();
>         update_wallclock();
>         schedule_clock_was_set_work = 1;
>     }
> 
> 
> 
>  Protected by Websense Hosted Email Security -- www.websense.com 



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I''m not sure what kernel tree you are proposing this against, but as I
said in the previous thread, I think this is a bad idea for general use.  It is
too easy for a dom0 system administrator to unwittingly wreak subtle and
mysterious havoc on possibly dozens of guests each with dozens of users.  The
only advantage is that it saves time for that dom0 administrator who would
otherwise have to go into each guest individually and change the time
(backwards, which is never recommended in any running system).

 

From: Du, Hang [mailto:hdu@websense.com] 
Sent: Tuesday, October 12, 2010 4:19 AM
To: xen-devel@lists.xensource.com
Cc: Liu, Saipu; Yi, Shunli
Subject: [Xen-devel] [PATCH] time-xen : Reset monotonic time when sync up time
from dom0 to domU

 

Signed-off-by: Shunli Yi <shyi@websense.com>

Signed-off-by: Hang Du <hdu@websense.com>

 

When sync up time from dom0 to domU, the monotonic time should be reset or domU
cannot sync up backward time from dom0.

 

diff -urN a/arch/i386/kernel/time-xen.c   b/arch/i386/kernel/time-xen.c   

 

--- a/arch/i386/kernel/time-xen.c   2010-10-11 10:41:06.000000000 +0800

+++ b/arch/i386/kernel/time-xen.c   2010-10-11 10:43:32.000000000 +0800

@@ -715,6 +715,8 @@

    }   

 

    if (shadow_tv_version != HYPERVISOR_shared_info->wc_version) {

+        if (!independent_wallclock)

+            monotonic_reset();

        update_wallclock();

        schedule_clock_was_set_work = 1;

    }

 

 

Protected by Websense Hosted Email Security - www.websense.com


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jan,

Sorry for my brief description in previous mail and missing is_initial_xendomain
check. The kernel I submit this patch is linux-2.6.18-xen-3.4.2, I submit the
patch again with in_initial-xendomain check.

In this patch, we support the backward time changing sync to all domUs which
configured to use "dependent wall clock".
Currently, without the backward time syncing, when we change the time backward
in Dom0, the time in DomU would be froze.
And this cause some commands hang and don''t executed until the time
catch up with the domU time.
For example:
"rpm -q kernel-xen"
"sleep 1"
Monotonic time should be reset when sync up time from dom0 to domU to support
domU backward time syncing.

diff -urN a/arch/i386/kernel/time-xen.c   b/arch/i386/kernel/time-xen.c   

--- a/arch/i386/kernel/time-xen.c   2010-10-11 10:41:06.000000000 +0800
+++ b/arch/i386/kernel/time-xen.c   2010-10-11 10:43:32.000000000 +0800
@@ -715,6 +715,8 @@
    }   
 
    if (shadow_tv_version != HYPERVISOR_shared_info->wc_version) {
+        if (!is_initial_xendomain() && !independent_wallclock)
+            monotonic_reset();
        update_wallclock();
        schedule_clock_was_set_work = 1;
    }


 Protected by Websense Hosted Email Security -- www.websense.com 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> On 13.10.10 at 05:24, "Du, Hang" <hdu@websense.com>
wrote:
> Sorry for my brief description in previous mail and missing 
> is_initial_xendomain check. The kernel I submit this patch is 
> linux-2.6.18-xen-3.4.2, I submit the patch again with in_initial-xendomain 
> check.
Actually, I didn''t expect you to blindly insert that check, but rather
either explain why only DomU-s need the changed behavior (as your
original description suggested), or refine the description of the
problem you''re trying to solve.
> In this patch, we support the backward time changing sync to all domUs
which
> configured to use "dependent wall clock".
> Currently, without the backward time syncing, when we change the time 
> backward in Dom0, the time in DomU would be froze. 
> And this cause some commands hang and don''t executed until the
time catch up
> with the domU time.
> For example:
> "rpm -q kernel-xen"
> "sleep 1"
> Monotonic time should be reset when sync up time from dom0 to domU to 
> support domU backward time syncing.
I can see your point, however ...
> diff -urN a/arch/i386/kernel/time-xen.c   b/arch/i386/kernel/time-xen.c   
> 
> --- a/arch/i386/kernel/time-xen.c   2010-10-11 10:41:06.000000000 +0800
> +++ b/arch/i386/kernel/time-xen.c   2010-10-11 10:43:32.000000000 +0800
> @@ -715,6 +715,8 @@
>     }   
>  
>     if (shadow_tv_version != HYPERVISOR_shared_info->wc_version) {
> +        if (!is_initial_xendomain() && !independent_wallclock)
> +            monotonic_reset();
>         update_wallclock();
... you definitely need to call monotonic_reset() *after* the
update_wallclock() (or else another vCPU, preempted in
do_gettimeofday() between the end of the xtime read loop
and the acquiring of the monotonic lock, would be able to
overwrite monotonic.tv with values obtained before the wall
clock update - similar to the secondary problem addressed by
c/s 1021).

Further, blindly running a reset here doesn''t seem nice in
the (general) case of the time getting updated forwards.
>         schedule_clock_was_set_work = 1;
>     }
You''ll also want to address Dan''s concerns, and you will want
to
get your patch formatted so that it would actually apply (read:
no tab -> space conversion) if you expect it to be committed
by Keir at some point.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

(bringing the topic up to a more theoretical level and including
Keir and Jeremy)

I wonder if the "bug" here is that "dependent wall clock" is
an incredibly poor replacement for NTP (or a Windows Time Server)
and a hack and really shouldn''t even exist?  I suppose one could
argue that it makes some sense in a XCP environment, and maybe
in a server environment where a single physical server is extremely
isolated, but does it ever make sense in a real world server
environment?

In other words, I''m arguing that the correct "fix" here is:
Don''t use dependent wallclock.
> -----Original Message-----
> From: Jan Beulich [mailto:JBeulich@novell.com]
> Sent: Wednesday, October 13, 2010 6:38 AM
> To: Hang Du
> Cc: Saipu Liu; Shunli Yi; xen-devel@lists.xensource.com
> Subject: RE: [Xen-devel] [PATCH] time-xen : Reset monotonic time when
> sync up time from dom0 to domU
> 
> >>> On 13.10.10 at 05:24, "Du, Hang"
<hdu@websense.com> wrote:
> > Sorry for my brief description in previous mail and missing
> > is_initial_xendomain check. The kernel I submit this patch is
> > linux-2.6.18-xen-3.4.2, I submit the patch again with in_initial-
> xendomain
> > check.
> 
> Actually, I didn''t expect you to blindly insert that check, but
rather
> either explain why only DomU-s need the changed behavior (as your
> original description suggested), or refine the description of the
> problem you''re trying to solve.
> 
> > In this patch, we support the backward time changing sync to all
> domUs which
> > configured to use "dependent wall clock".
> > Currently, without the backward time syncing, when we change the time
> > backward in Dom0, the time in DomU would be froze.
> > And this cause some commands hang and don''t executed until
the time
> catch up
> > with the domU time.
> > For example:
> > "rpm -q kernel-xen"
> > "sleep 1"
> > Monotonic time should be reset when sync up time from dom0 to domU to
> > support domU backward time syncing.
> 
> I can see your point, however ...
> 
> > diff -urN a/arch/i386/kernel/time-xen.c   b/arch/i386/kernel/time-
> xen.c
> >
> > --- a/arch/i386/kernel/time-xen.c   2010-10-11 10:41:06.000000000
> +0800
> > +++ b/arch/i386/kernel/time-xen.c   2010-10-11 10:43:32.000000000
> +0800
> > @@ -715,6 +715,8 @@
> >     }
> >
> >     if (shadow_tv_version != HYPERVISOR_shared_info->wc_version) {
> > +        if (!is_initial_xendomain() &&
!independent_wallclock)
> > +            monotonic_reset();
> >         update_wallclock();
> 
> ... you definitely need to call monotonic_reset() *after* the
> update_wallclock() (or else another vCPU, preempted in
> do_gettimeofday() between the end of the xtime read loop
> and the acquiring of the monotonic lock, would be able to
> overwrite monotonic.tv with values obtained before the wall
> clock update - similar to the secondary problem addressed by
> c/s 1021).
> 
> Further, blindly running a reset here doesn''t seem nice in
> the (general) case of the time getting updated forwards.
> 
> >         schedule_clock_was_set_work = 1;
> >     }
> 
> You''ll also want to address Dan''s concerns, and you will
want to
> get your patch formatted so that it would actually apply (read:
> no tab -> space conversion) if you expect it to be committed
> by Keir at some point.
> 
> Jan
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/13/2010 08:56 AM, Dan Magenheimer wrote:
> (bringing the topic up to a more theoretical level and including
> Keir and Jeremy)
>
> I wonder if the "bug" here is that "dependent wall
clock" is
> an incredibly poor replacement for NTP (or a Windows Time Server)
> and a hack and really shouldn''t even exist?  I suppose one could
> argue that it makes some sense in a XCP environment, and maybe
> in a server environment where a single physical server is extremely
> isolated, but does it ever make sense in a real world server
> environment?
>
> In other words, I''m arguing that the correct "fix" here
is:
> Don''t use dependent wallclock.
Yes, that''s always been my view.  pvops kernels don''t
implement it.

    J
>> -----Original Message-----
>> From: Jan Beulich [mailto:JBeulich@novell.com]
>> Sent: Wednesday, October 13, 2010 6:38 AM
>> To: Hang Du
>> Cc: Saipu Liu; Shunli Yi; xen-devel@lists.xensource.com
>> Subject: RE: [Xen-devel] [PATCH] time-xen : Reset monotonic time when
>> sync up time from dom0 to domU
>>
>>>>> On 13.10.10 at 05:24, "Du, Hang"
<hdu@websense.com> wrote:
>>> Sorry for my brief description in previous mail and missing
>>> is_initial_xendomain check. The kernel I submit this patch is
>>> linux-2.6.18-xen-3.4.2, I submit the patch again with in_initial-
>> xendomain
>>> check.
>> Actually, I didn''t expect you to blindly insert that check,
but rather
>> either explain why only DomU-s need the changed behavior (as your
>> original description suggested), or refine the description of the
>> problem you''re trying to solve.
>>
>>> In this patch, we support the backward time changing sync to all
>> domUs which
>>> configured to use "dependent wall clock".
>>> Currently, without the backward time syncing, when we change the
time
>>> backward in Dom0, the time in DomU would be froze.
>>> And this cause some commands hang and don''t executed until
the time
>> catch up
>>> with the domU time.
>>> For example:
>>> "rpm -q kernel-xen"
>>> "sleep 1"
>>> Monotonic time should be reset when sync up time from dom0 to domU
to
>>> support domU backward time syncing.
>> I can see your point, however ...
>>
>>> diff -urN a/arch/i386/kernel/time-xen.c   b/arch/i386/kernel/time-
>> xen.c
>>> --- a/arch/i386/kernel/time-xen.c   2010-10-11 10:41:06.000000000
>> +0800
>>> +++ b/arch/i386/kernel/time-xen.c   2010-10-11 10:43:32.000000000
>> +0800
>>> @@ -715,6 +715,8 @@
>>>     }
>>>
>>>     if (shadow_tv_version != HYPERVISOR_shared_info->wc_version)
{
>>> +        if (!is_initial_xendomain() &&
!independent_wallclock)
>>> +            monotonic_reset();
>>>         update_wallclock();
>> ... you definitely need to call monotonic_reset() *after* the
>> update_wallclock() (or else another vCPU, preempted in
>> do_gettimeofday() between the end of the xtime read loop
>> and the acquiring of the monotonic lock, would be able to
>> overwrite monotonic.tv with values obtained before the wall
>> clock update - similar to the secondary problem addressed by
>> c/s 1021).
>>
>> Further, blindly running a reset here doesn''t seem nice in
>> the (general) case of the time getting updated forwards.
>>
>>>         schedule_clock_was_set_work = 1;
>>>     }
>> You''ll also want to address Dan''s concerns, and you
will want to
>> get your patch formatted so that it would actually apply (read:
>> no tab -> space conversion) if you expect it to be committed
>> by Keir at some point.
>>
>> Jan
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 13/10/2010 16:56, "Dan Magenheimer"
<dan.magenheimer@oracle.com> wrote:
> (bringing the topic up to a more theoretical level and including
> Keir and Jeremy)
> 
> I wonder if the "bug" here is that "dependent wall
clock" is
> an incredibly poor replacement for NTP (or a Windows Time Server)
> and a hack and really shouldn''t even exist?  I suppose one could
> argue that it makes some sense in a XCP environment, and maybe
> in a server environment where a single physical server is extremely
> isolated, but does it ever make sense in a real world server
> environment?
Yes it''s a bad replacement for ntp. Pv_ops doesn''t bother to
implement it,
syncs with Xen wallclock once during boot and thereafter expects ntp to be
used within the guest.

 -- Keir
> In other words, I''m arguing that the correct "fix" here
is:
> Don''t use dependent wallclock.
> 
>> -----Original Message-----
>> From: Jan Beulich [mailto:JBeulich@novell.com]
>> Sent: Wednesday, October 13, 2010 6:38 AM
>> To: Hang Du
>> Cc: Saipu Liu; Shunli Yi; xen-devel@lists.xensource.com
>> Subject: RE: [Xen-devel] [PATCH] time-xen : Reset monotonic time when
>> sync up time from dom0 to domU
>> 
>>>>> On 13.10.10 at 05:24, "Du, Hang"
<hdu@websense.com> wrote:
>>> Sorry for my brief description in previous mail and missing
>>> is_initial_xendomain check. The kernel I submit this patch is
>>> linux-2.6.18-xen-3.4.2, I submit the patch again with in_initial-
>> xendomain
>>> check.
>> 
>> Actually, I didn''t expect you to blindly insert that check,
but rather
>> either explain why only DomU-s need the changed behavior (as your
>> original description suggested), or refine the description of the
>> problem you''re trying to solve.
>> 
>>> In this patch, we support the backward time changing sync to all
>> domUs which
>>> configured to use "dependent wall clock".
>>> Currently, without the backward time syncing, when we change the
time
>>> backward in Dom0, the time in DomU would be froze.
>>> And this cause some commands hang and don''t executed until
the time
>> catch up
>>> with the domU time.
>>> For example:
>>> "rpm -q kernel-xen"
>>> "sleep 1"
>>> Monotonic time should be reset when sync up time from dom0 to domU
to
>>> support domU backward time syncing.
>> 
>> I can see your point, however ...
>> 
>>> diff -urN a/arch/i386/kernel/time-xen.c   b/arch/i386/kernel/time-
>> xen.c
>>> 
>>> --- a/arch/i386/kernel/time-xen.c   2010-10-11 10:41:06.000000000
>> +0800
>>> +++ b/arch/i386/kernel/time-xen.c   2010-10-11 10:43:32.000000000
>> +0800
>>> @@ -715,6 +715,8 @@
>>>     }
>>> 
>>>     if (shadow_tv_version != HYPERVISOR_shared_info->wc_version)
{
>>> +        if (!is_initial_xendomain() &&
!independent_wallclock)
>>> +            monotonic_reset();
>>>         update_wallclock();
>> 
>> ... you definitely need to call monotonic_reset() *after* the
>> update_wallclock() (or else another vCPU, preempted in
>> do_gettimeofday() between the end of the xtime read loop
>> and the acquiring of the monotonic lock, would be able to
>> overwrite monotonic.tv with values obtained before the wall
>> clock update - similar to the secondary problem addressed by
>> c/s 1021).
>> 
>> Further, blindly running a reset here doesn''t seem nice in
>> the (general) case of the time getting updated forwards.
>> 
>>>         schedule_clock_was_set_work = 1;
>>>     }
>> 
>> You''ll also want to address Dan''s concerns, and you
will want to
>> get your patch formatted so that it would actually apply (read:
>> no tab -> space conversion) if you expect it to be committed
>> by Keir at some point.
>> 
>> Jan
>> 
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 16:56 +0100 on 13 Oct (1286988987), Dan Magenheimer
wrote:
> I wonder if the "bug" here is that "dependent wall
clock" is
> an incredibly poor replacement for NTP (or a Windows Time Server)
> and a hack and really shouldn''t even exist?
I''ve seen Windows Time server do some pretty odd things, but leaving
that aside - the problem with dependent wallclock is that it distributes
only the wallclock time and not the drift parameters, so you get 
sawtoothing.

Running NTP in guests has its own problems, particularly around live
migration, where NTP can lose stability quite badly.

There was a paper about this at OSDI last week:
http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

Tim.
>  I suppose one could
> argue that it makes some sense in a XCP environment, and maybe
> in a server environment where a single physical server is extremely
> isolated, but does it ever make sense in a real world server
> environment?
> 
> In other words, I''m arguing that the correct "fix" here
is:
> Don''t use dependent wallclock.
> 
> > -----Original Message-----
> > From: Jan Beulich [mailto:JBeulich@novell.com]
> > Sent: Wednesday, October 13, 2010 6:38 AM
> > To: Hang Du
> > Cc: Saipu Liu; Shunli Yi; xen-devel@lists.xensource.com
> > Subject: RE: [Xen-devel] [PATCH] time-xen : Reset monotonic time when
> > sync up time from dom0 to domU
> > 
> > >>> On 13.10.10 at 05:24, "Du, Hang"
<hdu@websense.com> wrote:
> > > Sorry for my brief description in previous mail and missing
> > > is_initial_xendomain check. The kernel I submit this patch is
> > > linux-2.6.18-xen-3.4.2, I submit the patch again with in_initial-
> > xendomain
> > > check.
> > 
> > Actually, I didn''t expect you to blindly insert that check,
but rather
> > either explain why only DomU-s need the changed behavior (as your
> > original description suggested), or refine the description of the
> > problem you''re trying to solve.
> > 
> > > In this patch, we support the backward time changing sync to all
> > domUs which
> > > configured to use "dependent wall clock".
> > > Currently, without the backward time syncing, when we change the
time
> > > backward in Dom0, the time in DomU would be froze.
> > > And this cause some commands hang and don''t executed
until the time
> > catch up
> > > with the domU time.
> > > For example:
> > > "rpm -q kernel-xen"
> > > "sleep 1"
> > > Monotonic time should be reset when sync up time from dom0 to
domU to
> > > support domU backward time syncing.
> > 
> > I can see your point, however ...
> > 
> > > diff -urN a/arch/i386/kernel/time-xen.c  
b/arch/i386/kernel/time-
> > xen.c
> > >
> > > --- a/arch/i386/kernel/time-xen.c   2010-10-11 10:41:06.000000000
> > +0800
> > > +++ b/arch/i386/kernel/time-xen.c   2010-10-11 10:43:32.000000000
> > +0800
> > > @@ -715,6 +715,8 @@
> > >     }
> > >
> > >     if (shadow_tv_version !=
HYPERVISOR_shared_info->wc_version) {
> > > +        if (!is_initial_xendomain() &&
!independent_wallclock)
> > > +            monotonic_reset();
> > >         update_wallclock();
> > 
> > ... you definitely need to call monotonic_reset() *after* the
> > update_wallclock() (or else another vCPU, preempted in
> > do_gettimeofday() between the end of the xtime read loop
> > and the acquiring of the monotonic lock, would be able to
> > overwrite monotonic.tv with values obtained before the wall
> > clock update - similar to the secondary problem addressed by
> > c/s 1021).
> > 
> > Further, blindly running a reset here doesn''t seem nice in
> > the (general) case of the time getting updated forwards.
> > 
> > >         schedule_clock_was_set_work = 1;
> > >     }
> > 
> > You''ll also want to address Dan''s concerns, and you
will want to
> > get your patch formatted so that it would actually apply (read:
> > no tab -> space conversion) if you expect it to be committed
> > by Keir at some point.
> > 
> > Jan
> > 
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/13/2010 09:16 AM, Tim Deegan wrote:
> At 16:56 +0100 on 13 Oct (1286988987), Dan Magenheimer wrote:
>> I wonder if the "bug" here is that "dependent wall
clock" is
>> an incredibly poor replacement for NTP (or a Windows Time Server)
>> and a hack and really shouldn''t even exist?
> I''ve seen Windows Time server do some pretty odd things, but
leaving
> that aside - the problem with dependent wallclock is that it distributes
> only the wallclock time and not the drift parameters, so you get 
> sawtoothing.
>
> Running NTP in guests has its own problems, particularly around live
> migration, where NTP can lose stability quite badly.
There seems to be some work on replacements for NTP which converge
quicker and are shinier and better.

I''ve thought that having some kind of lightweight dom0<->domU
sync
protocol would be a better replacement for dependent time than NTP
(unless full NTP is required).
> There was a paper about this at OSDI last week:
> http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf
Ooh, look, RADclock, just what I was thinking about.

    J
> Tim.
>
>>  I suppose one could
>> argue that it makes some sense in a XCP environment, and maybe
>> in a server environment where a single physical server is extremely
>> isolated, but does it ever make sense in a real world server
>> environment?
>>
>> In other words, I''m arguing that the correct "fix"
here is:
>> Don''t use dependent wallclock.
>>
>>> -----Original Message-----
>>> From: Jan Beulich [mailto:JBeulich@novell.com]
>>> Sent: Wednesday, October 13, 2010 6:38 AM
>>> To: Hang Du
>>> Cc: Saipu Liu; Shunli Yi; xen-devel@lists.xensource.com
>>> Subject: RE: [Xen-devel] [PATCH] time-xen : Reset monotonic time
when
>>> sync up time from dom0 to domU
>>>
>>>>>> On 13.10.10 at 05:24, "Du, Hang"
<hdu@websense.com> wrote:
>>>> Sorry for my brief description in previous mail and missing
>>>> is_initial_xendomain check. The kernel I submit this patch is
>>>> linux-2.6.18-xen-3.4.2, I submit the patch again with
in_initial-
>>> xendomain
>>>> check.
>>> Actually, I didn''t expect you to blindly insert that
check, but rather
>>> either explain why only DomU-s need the changed behavior (as your
>>> original description suggested), or refine the description of the
>>> problem you''re trying to solve.
>>>
>>>> In this patch, we support the backward time changing sync to
all
>>> domUs which
>>>> configured to use "dependent wall clock".
>>>> Currently, without the backward time syncing, when we change
the time
>>>> backward in Dom0, the time in DomU would be froze.
>>>> And this cause some commands hang and don''t executed
until the time
>>> catch up
>>>> with the domU time.
>>>> For example:
>>>> "rpm -q kernel-xen"
>>>> "sleep 1"
>>>> Monotonic time should be reset when sync up time from dom0 to
domU to
>>>> support domU backward time syncing.
>>> I can see your point, however ...
>>>
>>>> diff -urN a/arch/i386/kernel/time-xen.c  
b/arch/i386/kernel/time-
>>> xen.c
>>>> --- a/arch/i386/kernel/time-xen.c   2010-10-11
10:41:06.000000000
>>> +0800
>>>> +++ b/arch/i386/kernel/time-xen.c   2010-10-11
10:43:32.000000000
>>> +0800
>>>> @@ -715,6 +715,8 @@
>>>>     }
>>>>
>>>>     if (shadow_tv_version !=
HYPERVISOR_shared_info->wc_version) {
>>>> +        if (!is_initial_xendomain() &&
!independent_wallclock)
>>>> +            monotonic_reset();
>>>>         update_wallclock();
>>> ... you definitely need to call monotonic_reset() *after* the
>>> update_wallclock() (or else another vCPU, preempted in
>>> do_gettimeofday() between the end of the xtime read loop
>>> and the acquiring of the monotonic lock, would be able to
>>> overwrite monotonic.tv with values obtained before the wall
>>> clock update - similar to the secondary problem addressed by
>>> c/s 1021).
>>>
>>> Further, blindly running a reset here doesn''t seem nice in
>>> the (general) case of the time getting updated forwards.
>>>
>>>>         schedule_clock_was_set_work = 1;
>>>>     }
>>> You''ll also want to address Dan''s concerns, and
you will want to
>>> get your patch formatted so that it would actually apply (read:
>>> no tab -> space conversion) if you expect it to be committed
>>> by Keir at some point.
>>>
>>> Jan
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: Dan Magenheimer [mailto:dan.magenheimer@oracle.com]
> Sent: 2010年10月13日 23:56
> To: Jan Beulich; Du, Hang; Keir Fraser; Jeremy Fitzhardinge
> Cc: Liu, Saipu; Yi, Shunli; xen-devel@lists.xensource.com
> Subject: RE: [Xen-devel] [PATCH] time-xen : Reset monotonic time when sync
up
> time from dom0 to domU
> 
> (bringing the topic up to a more theoretical level and including Keir and
Jeremy)
> 
> I wonder if the "bug" here is that "dependent wall
clock" is an incredibly poor
> replacement for NTP (or a Windows Time Server) and a hack and really
shouldn't
> even exist?  I suppose one could argue that it makes some sense in a XCP
> environment, and maybe in a server environment where a single physical
server
> is extremely isolated, but does it ever make sense in a real world server
> environment?
> 
[Shunli] We encounter the time-syncing issue when deploy enterprise
applications.
We suppose the "dependent wall clock" feature is for some enterprise
environment (or products) which have uniform administrator and manager tool.
So, they can maintain the time on Dom0 only, and Xen would maintain the time in
DomainUs.

Why we think this defect should be fixed?
1. We implemented a forward time syncing from Dom0 to DomU, also the backward
time syncing should be supported.
2. Currently, backward time setting in Dom0 would cause some DomU applications
hang (when set "independent_wallclock=0").
     For example : rpm
3. We don't think the backward time setting is risk for DomUs, this only
impacts the DomUs who set "independent_wallclock=0", and the
administrator should always know what they are doing.
> In other words, I'm arguing that the correct "fix" here is:
> Don't use dependent wallclock.
[Shunli] Of cause, we can, don't use the dependent wallclock. 
But, we still want to do something for those peoples who want the
"dependent wallclock".
It's easy to fix, and we didn't hear known risks about changing time
backward in DomU.

> 
> > -----Original Message-----
> > From: Jan Beulich [mailto:JBeulich@novell.com]
> > Sent: Wednesday, October 13, 2010 6:38 AM
> > To: Hang Du
> > Cc: Saipu Liu; Shunli Yi; xen-devel@lists.xensource.com
> > Subject: RE: [Xen-devel] [PATCH] time-xen : Reset monotonic time when
> > sync up time from dom0 to domU
> >
> > >>> On 13.10.10 at 05:24, "Du, Hang"
<hdu@websense.com> wrote:
> > > Sorry for my brief description in previous mail and missing
> > > is_initial_xendomain check. The kernel I submit this patch is
> > > linux-2.6.18-xen-3.4.2, I submit the patch again with in_initial-
> > xendomain
> > > check.
> >
> > Actually, I didn't expect you to blindly insert that check, but
rather
> > either explain why only DomU-s need the changed behavior (as your
> > original description suggested), or refine the description of the
> > problem you're trying to solve.
> >
> > > In this patch, we support the backward time changing sync to all
> > domUs which
> > > configured to use "dependent wall clock".
> > > Currently, without the backward time syncing, when we change the
> > > time backward in Dom0, the time in DomU would be froze.
> > > And this cause some commands hang and don't executed until
the time
> > catch up
> > > with the domU time.
> > > For example:
> > > "rpm -q kernel-xen"
> > > "sleep 1"
> > > Monotonic time should be reset when sync up time from dom0 to
domU
> > > to support domU backward time syncing.
> >
> > I can see your point, however ...
> >
> > > diff -urN a/arch/i386/kernel/time-xen.c  
b/arch/i386/kernel/time-
> > xen.c
> > >
> > > --- a/arch/i386/kernel/time-xen.c   2010-10-11 10:41:06.000000000
> > +0800
> > > +++ b/arch/i386/kernel/time-xen.c   2010-10-11 10:43:32.000000000
> > +0800
> > > @@ -715,6 +715,8 @@
> > >     }
> > >
> > >     if (shadow_tv_version !=
HYPERVISOR_shared_info->wc_version) {
> > > +        if (!is_initial_xendomain() &&
!independent_wallclock)
> > > +            monotonic_reset();
> > >         update_wallclock();
> >
> > ... you definitely need to call monotonic_reset() *after* the
> > update_wallclock() (or else another vCPU, preempted in
> > do_gettimeofday() between the end of the xtime read loop and the
> > acquiring of the monotonic lock, would be able to overwrite
> > monotonic.tv with values obtained before the wall clock update -
> > similar to the secondary problem addressed by c/s 1021).
> >
> > Further, blindly running a reset here doesn't seem nice in the
> > (general) case of the time getting updated forwards.
> >
> > >         schedule_clock_was_set_work = 1;
> > >     }
> >
> > You'll also want to address Dan's concerns, and you will want
to get
> > your patch formatted so that it would actually apply (read:
> > no tab -> space conversion) if you expect it to be committed by
Keir
> > at some point.
> >
> > Jan
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> 
>  To report this as spam, please forward to spam@websense.com.  Thank you.

 Protected by Websense Hosted Email Security -- www.websense.com 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 17:48 +0100 on 13 Oct (1286992083), Jeremy Fitzhardinge
wrote:
> > There was a paper about this at OSDI last week:
> > http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf
> 
> Ooh, look, RADclock, just what I was thinking about.
Yes, it looks pretty good.  Also they can use Xen stime as the local
oscillator and distribute drift numbers from xenstore, so no hypervisor
patches (and no hypervisor-interface changes) required. :)

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/14/2010 02:07 AM, Tim Deegan wrote:
> At 17:48 +0100 on 13 Oct (1286992083), Jeremy Fitzhardinge wrote:
>>> There was a paper about this at OSDI last week:
>>> http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf
>> Ooh, look, RADclock, just what I was thinking about.
> Yes, it looks pretty good.  Also they can use Xen stime as the local
> oscillator and distribute drift numbers from xenstore, so no hypervisor
> patches (and no hypervisor-interface changes) required. :)
Neat!  Is it something we should encourage them to submit for xen-unstable?

    J


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 00:35 +0100 on 15 Oct (1287102942), Jeremy Fitzhardinge
wrote:
>  On 10/14/2010 02:07 AM, Tim Deegan wrote:
> > At 17:48 +0100 on 13 Oct (1286992083), Jeremy Fitzhardinge wrote:
> >>> There was a paper about this at OSDI last week:
> >>>
http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf
> >> Ooh, look, RADclock, just what I was thinking about.
> > Yes, it looks pretty good.  Also they can use Xen stime as the local
> > oscillator and distribute drift numbers from xenstore, so no
hypervisor
> > patches (and no hypervisor-interface changes) required. :)
> 
> Neat!  Is it something we should encourage them to submit for xen-unstable?
Well I did, but there are no hypervisor patches. :) 

CC''ing the speaker: are there any dom0 tools or kernel patches we could
be looking at for xen-unstable?

Regards,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> From: Tim Deegan [mailto:Tim.Deegan@citrix.com]
> Sent: Thursday, October 14, 2010 3:08 AM
> To: Jeremy Fitzhardinge
> Cc: Dan Magenheimer; Jan Beulich; Hang Du; Keir Fraser; Saipu Liu;
> Shunli Yi; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] [PATCH] time-xen : Reset monotonic time when
> sync up time from dom0 to domU
> 
> At 17:48 +0100 on 13 Oct (1286992083), Jeremy Fitzhardinge wrote:
> > > There was a paper about this at OSDI last week:
> > >
http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf
> >
> > Ooh, look, RADclock, just what I was thinking about.
> 
> Yes, it looks pretty good.  Also they can use Xen stime as the local
> oscillator and distribute drift numbers from xenstore, so no hypervisor
> patches (and no hypervisor-interface changes) required. :)
Maybe I''m misunderstanding the paper, but isn''t it required
that Xen stime be directly readable for every attempt to
sample time (e.g. requiring at least some small interface
change such as adding a hypercall to obtain Xen stime)?
The current pvclock mechanism still depends on TSC to
interpolate between "Xen stime samples periodically written
to memory" to get adequate precision.

Also, for those certain enterprise applications that want
to sample time 10000+ samples/second/processor, and need
to know "immediately" when a sample might be bad (due
to, for example, live migration), I think each sample would
need to check xenstore.  Is xenstore up to that kind of
pounding (and is it fast enough)?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 15:09 +0100 on 15 Oct (1287155382), Dan Magenheimer
wrote:
> > At 17:48 +0100 on 13 Oct (1286992083), Jeremy Fitzhardinge wrote:
> > > > There was a paper about this at OSDI last week:
> > > >
http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf
> > >
> > > Ooh, look, RADclock, just what I was thinking about.
> > 
> > Yes, it looks pretty good.  Also they can use Xen stime as the local
> > oscillator and distribute drift numbers from xenstore, so no
hypervisor
> > patches (and no hypervisor-interface changes) required. :)
> 
> Maybe I''m misunderstanding the paper, but isn''t it
required
> that Xen stime be directly readable for every attempt to
> sample time (e.g. requiring at least some small interface
> change such as adding a hypercall to obtain Xen stime)?
You can obtain xen stime directly from the shared-into page and RDTSC.
There might need to be _kernel_ changes to make that available to
userspace, though my impression is that they push packet-timestamping
into the kernel.
> Also, for those certain enterprise applications that want
> to sample time 10000+ samples/second/processor, and need
> to know "immediately" when a sample might be bad (due
> to, for example, live migration), I think each sample would
> need to check xenstore.  Is xenstore up to that kind of
> pounding (and is it fast enough)?
No it''s certainly not (either of those things), but:
 - the problem of turning a distributed wallclock into something suitable
   for timestamping that aggressively is orthogonal to the problem of
   distributing that wallclock in the first place; and
 - all the user really needs is a generation counter to know that the 
   clock correction values are stale, and the kernel can provide that 
   alongside the stime.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > Also, for those certain enterprise applications that want
> > to sample time 10000+ samples/second/processor, and need
> > to know "immediately" when a sample might be bad (due
> > to, for example, live migration), I think each sample would
> > need to check xenstore.  Is xenstore up to that kind of
> > pounding (and is it fast enough)?
> 
> No it''s certainly not (either of those things), but:
>  - the problem of turning a distributed wallclock into something
> suitable
>    for timestamping that aggressively is orthogonal to the problem of
>    distributing that wallclock in the first place; and
Agreed that this is a better solution to a very important problem.
I''m just trying to determine if it helps solve another time-related
real world problem.
>  - all the user really needs is a generation counter to know that the
>    clock correction values are stale, and the kernel can provide that
>    alongside the stime.
Agreed that a generation counter solves this.  BUT...

(1) afaik there is no generation counter in any time-related
kernel API; and

(2) afaict this generation counter would need to be "pushed"
from the dom0 kernel so would either (a) require domU kernels
to read from xenstore, or (b) require some kind of privileged
hypercall from dom0 to the hypervisor telling the hypervisor
to change the generation counter (and scaling values?) for all
domUs.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

(RADclock authors cc''ed in attempt to re-merge the two threads...
Tim, if it''s not too late, please reply to this email rather
than the previous one without the full cc list)
> From: Dan Magenheimer
> Sent: Friday, October 15, 2010 8:47 AM
> To: Tim Deegan
> Cc: Jeremy Fitzhardinge; Jan Beulich; Hang Du; Keir Fraser; Saipu Liu;
> Shunli Yi; xen-devel@lists.xensource.com
> Subject: RE: [Xen-devel] [PATCH] time-xen : Reset monotonic time when
> sync up time from dom0 to domU
> 
> > > Also, for those certain enterprise applications that want
> > > to sample time 10000+ samples/second/processor, and need
> > > to know "immediately" when a sample might be bad (due
> > > to, for example, live migration), I think each sample would
> > > need to check xenstore.  Is xenstore up to that kind of
> > > pounding (and is it fast enough)?
> >
> > No it''s certainly not (either of those things), but:
> >  - the problem of turning a distributed wallclock into something
> > suitable
> >    for timestamping that aggressively is orthogonal to the problem of
> >    distributing that wallclock in the first place; and
> 
> Agreed that this is a better solution to a very important problem.
> I''m just trying to determine if it helps solve another
time-related
> real world problem.
> 
> >  - all the user really needs is a generation counter to know that the
> >    clock correction values are stale, and the kernel can provide that
> >    alongside the stime.
> 
> Agreed that a generation counter solves this.  BUT...
> 
> (1) afaik there is no generation counter in any time-related
> kernel API; and
> 
> (2) afaict this generation counter would need to be "pushed"
> from the dom0 kernel so would either (a) require domU kernels
> to read from xenstore, or (b) require some kind of privileged
> hypercall from dom0 to the hypervisor telling the hypervisor
> to change the generation counter (and scaling values?) for all
> domUs.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 15:46 +0100 on 15 Oct (1287157606), Dan Magenheimer
wrote:
> Agreed that this is a better solution to a very important problem.
> I''m just trying to determine if it helps solve another
time-related
> real world problem.
Yep.  AFAICS it doesn''t really affect that at all. :|
> >  - all the user really needs is a generation counter to know that the
> >    clock correction values are stale, and the kernel can provide that
> >    alongside the stime.
> 
> Agreed that a generation counter solves this.  BUT...
> 
> (1) afaik there is no generation counter in any time-related
> kernel API; and
OK, but one could be added wherever the stime is exposed.  This line of
reasoning runs into the earlier discussions about PV time protocols fopr
userspace (inluding process migration &c), and I''ve nothing useful
to
add to that now. :)
> (2) afaict this generation counter would need to be "pushed"
> from the dom0 kernel so would either (a) require domU kernels
> to read from xenstore, or (b) require some kind of privileged
> hypercall from dom0 to the hypervisor telling the hypervisor
> to change the generation counter (and scaling values?) for all
> domUs.
I was under the impression that the only time you''d really need to
invalidate the xenstore values was on live migration, which the domU
kernel can certainly know about.  One reason they use xenstore is that
the correction numbers are quite stable and don''t need to be polled
every time.

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/15/2010 05:10 AM, Darryl Veitch wrote:
> Hi Tim,
>
> good to hear from you.  Dom0 will need the usual RADclock patches
> which now integrate the Xen support. These supply two main things:  
> i)  kernel level raw timestamping (ie reading of the chosen counter)
> of packet arrivals and departures in `good but safe'' locations and
an
> ability to pass those timestamps up (works transparently with pcap,
> and doesn''t affect its normal operation)
Are you trying to get these changes into mainline Linux?
> ii)  the ability to access the raw counter of choice from both kernel
> and user context
The kernel already has this info, and I have patches for mapping it into
userspace.  Or would a syscall suffice?
> In the Xen context, the choice of counter will end up being Xen
> Clocksource which is already accessible to guests as is already a
> suitable counter of the right size (64-bit).
>
> Take a look at the RADclock downloads
> <http://www.cubinlab.ee.unimelb.edu.au/radclock/download.php> page.
> There have been a number of changes recently in particular in the
> integration of the Xen support so I will direct you to Julien Ridoux
> for finer-grained answers as he and Tim Broomhead actually wrote the
> code. 
>
> We''ll be quizzing you shortly about our needs to make this work
under
> HV using that way to access the Xenstore you mentioned, and we''d
like
> to talk to you about how to improve packet timestamping under Dom0. 
OK.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Darryl -

 

So how would a userland enterprise app go about doing this?


For example, the Oracle DB (and other Oracle enterprise apps) collects
timestamps at a high frequency, sometimes more than 10K timestamps/second. 
(These are used primarily for collecting transaction statistics, but these
statistics are collected by default and must be explicitly disabled by a DB
administrator.)   As one might expect, the difference between any two of these
timestamps is the key result and the two timestamps may differ by seconds or
minutes so, as  a result, the two timestamps of interest may cross a live
migration boundary.

 

I am guessing that to use the approach you suggest, both the counter and some
form of "generation indicator" must be stored for each timestamp, then
when the difference between the two is to be calculated, the RADclock history is
consulted, essentially to lookup, if they are different, the two generation
indicators and use them to adjust the two counters.

 

Ideally, this would be entirely hidden from userland in the kernel, but the cost
of a "real" system call is too high to be incurred 10K/sec, and
I''m not sure how RADclock history could be consulted in a vsyscall.

 

Thanks,

Dan

 

P.S. Since you probably don''t closely follow the xen-devel list, some
discussion has occurred on other threads without cc''ing you.  If you
are interested, search for RADclock in xen.markmail.org.

 

From: Darryl Veitch [mailto:dveitch@unimelb.edu.au] 
Sent: Friday, October 15, 2010 4:57 PM
To: Dan Magenheimer
Cc: Tim Deegan; Jeremy Fitzhardinge; Jan Beulich; Hang Du; Keir Fraser; Saipu
Liu; Shunli Yi; xen-devel@lists.xensource.com; Julien Ridoux; Timothy Broomhead
Subject: Re: [Xen-devel] [PATCH] time-xen : Reset monotonic time when sync up
time from dom0 to domU

 

Hi All,

 

in terms of pounding on xenstore, one way to get around this is to note that raw
timestamps (that is what we call counter readings) can be taken at the required
events but only converted to walltime later. This could be done in batches
asynchronously, and perhaps even much later off-line, depending on the
application.  This is one of the advantages of the feedforward approach. The
RADclock keeps historical data which enables raw timestamps to be converted to
walltime at any time with identically zero loss of precision- they would be
exactly the same wallclock values as if they were calculated immediately.

 

Darryl

 

 

On 15/10/2010, at 4:53 PM, Dan Magenheimer wrote:





(RADclock authors cc''ed in attempt to re-merge the two threads...
Tim, if it''s not too late, please reply to this email rather
than the previous one without the full cc list)




From: Dan Magenheimer

Sent: Friday, October 15, 2010 8:47 AM

To: Tim Deegan

Cc: Jeremy Fitzhardinge; Jan Beulich; Hang Du; Keir Fraser; Saipu Liu;

Shunli Yi; HYPERLINK
"mailto:xen-devel@lists.xensource.com"xen-devel@lists.xensource.com

Subject: RE: [Xen-devel] [PATCH] time-xen : Reset monotonic time when

sync up time from dom0 to domU

 

Also, for those certain enterprise applications that want

to sample time 10000+ samples/second/processor, and need

to know "immediately" when a sample might be bad (due

to, for example, live migration), I think each sample would

need to check xenstore.  Is xenstore up to that kind of

pounding (and is it fast enough)?

 

No it''s certainly not (either of those things), but:

- the problem of turning a distributed wallclock into something

suitable

  for timestamping that aggressively is orthogonal to the problem of

  distributing that wallclock in the first place; and

 

Agreed that this is a better solution to a very important problem.

I''m just trying to determine if it helps solve another time-related

real world problem.

 

- all the user really needs is a generation counter to know that the

  clock correction values are stale, and the kernel can provide that

  alongside the stime.

 

Agreed that a generation counter solves this.  BUT...

 

(1) afaik there is no generation counter in any time-related

kernel API; and

 

(2) afaict this generation counter would need to be "pushed"

from the dom0 kernel so would either (a) require domU kernels

to read from xenstore, or (b) require some kind of privileged

hypercall from dom0 to the hypervisor telling the hypervisor

to change the generation counter (and scaling values?) for all

domUs.

 

-----------------------------------------------

Darryl Veitch

Sabbatical Address:

TREC INRIA

23 Avenue D''Italie

75214 Paris Cedex 13, France 

Tel:  +33 (0)1 39 63 55 40

&

Technicolor Paris Research Lab

1, rue Jeanne d''Arc

92443 Issy-les-Moulineaux Cedex, France

Tel:  +33 (0)1 41 86 52 19

 

Global:

dveitch AT unimelb edu au

http://www.cubinlab.ee.unimelb.edu.au/~darryl/

AIM id:  darrylveitch

Skype:  darrylveitch

 

 





 

 

 

 





 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 16/10/2010, at 10:21 AM, Jeremy Fitzhardinge wrote:
> On 10/15/2010 05:10 AM, Darryl Veitch wrote:
>> Hi Tim,
>> 
>> good to hear from you.  Dom0 will need the usual RADclock patches
>> which now integrate the Xen support. These supply two main things:  
>> i)  kernel level raw timestamping (ie reading of the chosen counter)
>> of packet arrivals and departures in `good but safe'' locations
and an
>> ability to pass those timestamps up (works transparently with pcap,
>> and doesn''t affect its normal operation)
> 
> Are you trying to get these changes into mainline Linux?
We are intending to do this, but have not pushed comprehensively yet, though we
hope to start soon after we have the adoption into FreeBSD finalised
(theoretically around end-2010).  We have had some discussions with John Stultz
in preparation.   Would you be interested in being involved in encouraging
adoption here ? :-)

> 
>> ii)  the ability to access the raw counter of choice from both kernel
>> and user context
> 
> The kernel already has this info, and I have patches for mapping it into
> userspace.  Or would a syscall suffice?
We use syscall''s currently.

Darryl



-----------------------------------------------
Darryl Veitch
Sabbatical Address:
TREC INRIA
23 Avenue D''Italie
75214 Paris Cedex 13, France 
Tel:  +33 (0)1 39 63 55 40
&
Technicolor Paris Research Lab
1, rue Jeanne d''Arc
92443 Issy-les-Moulineaux Cedex, France
Tel:  +33 (0)1 41 86 52 19

Global:
dveitch AT unimelb edu au
http://www.cubinlab.ee.unimelb.edu.au/~darryl/
AIM id:  darrylveitch
Skype:  darrylveitch












_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/20/2010 08:36 PM, Darryl Veitch wrote:
>
> On 16/10/2010, at 10:21 AM, Jeremy Fitzhardinge wrote:
>
>> On 10/15/2010 05:10 AM, Darryl Veitch wrote:
>>> Hi Tim,
>>>
>>> good to hear from you.  Dom0 will need the usual RADclock patches
>>> which now integrate the Xen support. These supply two main things:
>>> i)  kernel level raw timestamping (ie reading of the chosen
counter)
>>> of packet arrivals and departures in `good but safe''
locations and an
>>> ability to pass those timestamps up (works transparently with pcap,
>>> and doesn''t affect its normal operation)
>>
>> Are you trying to get these changes into mainline Linux?
>
> We are intending to do this, but have not pushed comprehensively yet,
> though we hope to start soon after we have the adoption into FreeBSD
> finalised (theoretically around end-2010).  We have had some
> discussions with John Stultz in preparation.   Would you be interested
> in being involved in encouraging adoption here ? :-) 
I''m happy to give you help out where I can.  John is definitely a
person
who should be involved.  Where''s the current set of patches?
>
>>
>>> ii)  the ability to access the raw counter of choice from both
kernel
>>> and user context
>>
>> The kernel already has this info, and I have patches for mapping it
into
>> userspace.  Or would a syscall suffice?
>
> We use syscall''s currently.
OK, that''s simple.   What do you add?  A new lock time for
clock_gettime?

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 22/10/2010, at 7:49 AM, Jeremy Fitzhardinge wrote:
>>>  
>>> Are you trying to get these changes into mainline Linux?
>> 
>> We are intending to do this, but have not pushed comprehensively yet,
>> though we hope to start soon after we have the adoption into FreeBSD
>> finalised (theoretically around end-2010).  We have had some
>> discussions with John Stultz in preparation.   Would you be interested
>> in being involved in encouraging adoption here ? :-) 
> 
> I''m happy to give you help out where I can.  John is definitely a
person
> who should be involved.  Where''s the current set of patches?

http://www.cubinlab.ee.unimelb.edu.au/radclock/download.php



--------------------------------------------------------
Darryl Veitch
Principal Research Fellow
Dept. of E&E Engineering
University of Melbourne
Rm 5.13, EEE building
Tel:      +61(0)3 8344 3817
Fax:     +61(0)3 8344 3823
Skype:       darrylveitch

dveitch AT unimelb.edu.au
www.cubinlab.ee.unimelb.edu.au/~darryl/






_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel