Xen devel - Oct 2013 - [PATCH 0/2] Viridian MSRs

This set of two patches advertises 3 constant, read-only MSRs of timing
information to a viridian capable VM.

There is an as-yet-unidentified issue when running Windows 8.1 / Server 2012r2
under Xen where it will periodically (1 in 10 attempt) appear to fall into an
idle loop rather than schedule userspace processes (such as failing to run a
login session).

I am still investigating the underlying cause.  One possibility is an
interaction of TSC time calibration interacting poorly with the Xen scheduler.

Unfortunately, attempting to divine what windows is unhappy about with its
environment is rather tricky (even a BSOD would be more useful than the
current symptoms), but providing these MSRs causes Windows to prefer rdtsc
over the HPET main counter as a source of time, and ''fixes''
the above issue.

CC: Paul Durrant <paul.durrant@citrix.com>
CC: Keir Fraser <keir@xen.org>
CC: Jan Beulich <JBeulich@suse.com>

--
1.7.10.4

This viridian MSR is a read-only source of time (in units of 100ns) since the
domain started.

From: Paul Durrant <paul.durrant@citrix.com>
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Keir Fraser <keir@xen.org>
CC: Jan Beulich <JBeulich@suse.com>

---
---
 xen/arch/x86/hvm/viridian.c      |   24 ++++++++++++++++--------
 xen/include/asm-x86/perfc_defn.h |    1 +
 2 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/hvm/viridian.c b/xen/arch/x86/hvm/viridian.c
index a6721c3..f363037 100644
--- a/xen/arch/x86/hvm/viridian.c
+++ b/xen/arch/x86/hvm/viridian.c
@@ -17,13 +17,14 @@
 #include <public/hvm/hvm_op.h>
 
 /* Viridian MSR numbers. */
-#define VIRIDIAN_MSR_GUEST_OS_ID 0x40000000
-#define VIRIDIAN_MSR_HYPERCALL   0x40000001
-#define VIRIDIAN_MSR_VP_INDEX    0x40000002
-#define VIRIDIAN_MSR_EOI         0x40000070
-#define VIRIDIAN_MSR_ICR         0x40000071
-#define VIRIDIAN_MSR_TPR         0x40000072
-#define VIRIDIAN_MSR_APIC_ASSIST 0x40000073
+#define VIRIDIAN_MSR_GUEST_OS_ID                0x40000000
+#define VIRIDIAN_MSR_HYPERCALL                  0x40000001
+#define VIRIDIAN_MSR_VP_INDEX                   0x40000002
+#define VIRIDIAN_MSR_TIME_REF_COUNT             0x40000020
+#define VIRIDIAN_MSR_EOI                        0x40000070
+#define VIRIDIAN_MSR_ICR                        0x40000071
+#define VIRIDIAN_MSR_TPR                        0x40000072
+#define VIRIDIAN_MSR_APIC_ASSIST                0x40000073
 
 /* Viridian Hypercall Status Codes. */
 #define HV_STATUS_SUCCESS                       0x0000
@@ -33,6 +34,7 @@
 #define HvNotifyLongSpinWait    8
 
 /* Viridian CPUID 4000003, Viridian MSR availability. */
+#define CPUID3A_MSR_REF_COUNT   (1 << 1)
 #define CPUID3A_MSR_APIC_ACCESS (1 << 4)
 #define CPUID3A_MSR_HYPERCALL   (1 << 5)
 #define CPUID3A_MSR_VP_INDEX    (1 << 6)
@@ -83,7 +85,8 @@ int cpuid_viridian_leaves(unsigned int leaf, unsigned int
*eax,
         break;
     case 3:
         /* Which hypervisor MSRs are available to the guest */
-        *eax = (CPUID3A_MSR_APIC_ACCESS |
+        *eax = (CPUID3A_MSR_REF_COUNT   |
+                CPUID3A_MSR_APIC_ACCESS |
                 CPUID3A_MSR_HYPERCALL   |
                 CPUID3A_MSR_VP_INDEX);
         break;
@@ -305,6 +308,11 @@ int rdmsr_viridian_regs(uint32_t idx, uint64_t *val)
         *val = v->vcpu_id;
         break;
 
+    case VIRIDIAN_MSR_TIME_REF_COUNT:
+        perfc_incr(mshv_rdmsr_time_ref_count);
+        *val = hvm_get_guest_time(v) / 100;
+        break;
+
     case VIRIDIAN_MSR_ICR:
         perfc_incr(mshv_rdmsr_icr);
         *val = (((uint64_t)vlapic_get_reg(vcpu_vlapic(v), APIC_ICR2) <<
32) |
diff --git a/xen/include/asm-x86/perfc_defn.h b/xen/include/asm-x86/perfc_defn.h
index 5eaa417..bd251f5 100644
--- a/xen/include/asm-x86/perfc_defn.h
+++ b/xen/include/asm-x86/perfc_defn.h
@@ -118,6 +118,7 @@ PERFCOUNTER(mshv_call_long_wait,        "MS Hv Notify
long wait")
 PERFCOUNTER(mshv_rdmsr_osid,            "MS Hv rdmsr Guest OS ID")
 PERFCOUNTER(mshv_rdmsr_hc_page,         "MS Hv rdmsr hypercall page")
 PERFCOUNTER(mshv_rdmsr_vp_index,        "MS Hv rdmsr vp index")
+PERFCOUNTER(mshv_rdmsr_time_ref_count,  "MS Hv rdmsr time reference
count")
 PERFCOUNTER(mshv_rdmsr_icr,             "MS Hv rdmsr icr")
 PERFCOUNTER(mshv_rdmsr_tpr,             "MS Hv rdmsr tpr")
 PERFCOUNTER(mshv_rdmsr_apic_assist,     "MS Hv rdmsr APIC assist")
-- 
1.7.10.4

These viridian MSRs are read-only sources of the TSC and APIC frequency (in
units of Hz)

From: Paul Durrant <paul.durrant@citrix.com>
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Keir Fraser <keir@xen.org>
CC: Jan Beulich <JBeulich@suse.com>
---
 xen/arch/x86/hvm/viridian.c      |   16 +++++++++++++++-
 xen/arch/x86/hvm/vlapic.c        |    3 ---
 xen/include/asm-x86/hvm/vlapic.h |    3 +++
 xen/include/asm-x86/perfc_defn.h |    2 ++
 4 files changed, 20 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/hvm/viridian.c b/xen/arch/x86/hvm/viridian.c
index f363037..2b86d66 100644
--- a/xen/arch/x86/hvm/viridian.c
+++ b/xen/arch/x86/hvm/viridian.c
@@ -21,6 +21,8 @@
 #define VIRIDIAN_MSR_HYPERCALL                  0x40000001
 #define VIRIDIAN_MSR_VP_INDEX                   0x40000002
 #define VIRIDIAN_MSR_TIME_REF_COUNT             0x40000020
+#define VIRIDIAN_MSR_TSC_FREQUENCY              0x40000022
+#define VIRIDIAN_MSR_APIC_FREQUENCY             0x40000023
 #define VIRIDIAN_MSR_EOI                        0x40000070
 #define VIRIDIAN_MSR_ICR                        0x40000071
 #define VIRIDIAN_MSR_TPR                        0x40000072
@@ -38,6 +40,7 @@
 #define CPUID3A_MSR_APIC_ACCESS (1 << 4)
 #define CPUID3A_MSR_HYPERCALL   (1 << 5)
 #define CPUID3A_MSR_VP_INDEX    (1 << 6)
+#define CPUID3A_MSR_FREQ        (1 << 11)
 
 /* Viridian CPUID 4000004, Implementation Recommendations. */
 #define CPUID4A_MSR_BASED_APIC  (1 << 3)
@@ -88,7 +91,8 @@ int cpuid_viridian_leaves(unsigned int leaf, unsigned int
*eax,
         *eax = (CPUID3A_MSR_REF_COUNT   |
                 CPUID3A_MSR_APIC_ACCESS |
                 CPUID3A_MSR_HYPERCALL   |
-                CPUID3A_MSR_VP_INDEX);
+                CPUID3A_MSR_VP_INDEX    |
+                CPUID3A_MSR_FREQ);
         break;
     case 4:
         /* Recommended hypercall usage. */
@@ -313,6 +317,16 @@ int rdmsr_viridian_regs(uint32_t idx, uint64_t *val)
         *val = hvm_get_guest_time(v) / 100;
         break;
 
+    case VIRIDIAN_MSR_TSC_FREQUENCY:
+        perfc_incr(mshv_rdmsr_tsc_frequency);
+        *val = (uint64_t)d->arch.tsc_khz * 1000ull;
+        break;
+
+    case VIRIDIAN_MSR_APIC_FREQUENCY:
+        perfc_incr(mshv_rdmsr_apic_frequency);
+        *val = 1000000000ull / APIC_BUS_CYCLE_NS;
+        break;
+
     case VIRIDIAN_MSR_ICR:
         perfc_incr(mshv_rdmsr_icr);
         *val = (((uint64_t)vlapic_get_reg(vcpu_vlapic(v), APIC_ICR2) <<
32) |
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index 5c33d3a..bc06010 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -44,9 +44,6 @@
 #define VLAPIC_VERSION                  0x00050014
 #define VLAPIC_LVT_NUM                  6
 
-/* vlapic''s frequence is 100 MHz */
-#define APIC_BUS_CYCLE_NS               10
-
 #define LVT_MASK \
     APIC_LVT_MASKED | APIC_SEND_PENDING | APIC_VECTOR_MASK
 
diff --git a/xen/include/asm-x86/hvm/vlapic.h b/xen/include/asm-x86/hvm/vlapic.h
index eb6dec9..66f0aff 100644
--- a/xen/include/asm-x86/hvm/vlapic.h
+++ b/xen/include/asm-x86/hvm/vlapic.h
@@ -80,6 +80,9 @@ struct vlapic {
     } init_sipi;
 };
 
+/* vlapic''s frequence is 100 MHz */
+#define APIC_BUS_CYCLE_NS               10
+
 static inline uint32_t vlapic_get_reg(struct vlapic *vlapic, uint32_t reg)
 {
     return *((uint32_t *)(&vlapic->regs->data[reg]));
diff --git a/xen/include/asm-x86/perfc_defn.h b/xen/include/asm-x86/perfc_defn.h
index bd251f5..6c590aa 100644
--- a/xen/include/asm-x86/perfc_defn.h
+++ b/xen/include/asm-x86/perfc_defn.h
@@ -119,6 +119,8 @@ PERFCOUNTER(mshv_rdmsr_osid,            "MS Hv rdmsr
Guest OS ID")
 PERFCOUNTER(mshv_rdmsr_hc_page,         "MS Hv rdmsr hypercall page")
 PERFCOUNTER(mshv_rdmsr_vp_index,        "MS Hv rdmsr vp index")
 PERFCOUNTER(mshv_rdmsr_time_ref_count,  "MS Hv rdmsr time reference
count")
+PERFCOUNTER(mshv_rdmsr_tsc_frequency,   "MS Hv rdmsr TSC frequency")
+PERFCOUNTER(mshv_rdmsr_apic_frequency,  "MS Hv rdmsr APIC frequency")
 PERFCOUNTER(mshv_rdmsr_icr,             "MS Hv rdmsr icr")
 PERFCOUNTER(mshv_rdmsr_tpr,             "MS Hv rdmsr tpr")
 PERFCOUNTER(mshv_rdmsr_apic_assist,     "MS Hv rdmsr APIC assist")
-- 
1.7.10.4

>>> On 15.10.13 at 20:12, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> This set of two patches advertises 3 constant, read-only MSRs of timing
> information to a viridian capable VM.
> 
> There is an as-yet-unidentified issue when running Windows 8.1 / Server
2012r2
> under Xen where it will periodically (1 in 10 attempt) appear to fall into 
> an
> idle loop rather than schedule userspace processes (such as failing to run
a
> login session).
> 
> I am still investigating the underlying cause.  One possibility is an
> interaction of TSC time calibration interacting poorly with the Xen 
> scheduler.
> 
> Unfortunately, attempting to divine what windows is unhappy about with its
> environment is rather tricky (even a BSOD would be more useful than the
> current symptoms), but providing these MSRs causes Windows to prefer rdtsc
> over the HPET main counter as a source of time, and
''fixes'' the above issue.
I''m curious whether you would have put any consideration into
the growing use of Hyper-V features when available - they had
to play tricks in the past to avoid using them when in fact running
on Xen. In particular in the case here I''m not certain your change
won''t interact badly with https://lkml.org/lkml/2013/9/3/417.

Jan

On 16/10/13 11:12, Jan Beulich wrote:
>>>> On 15.10.13 at 20:12, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
>> This set of two patches advertises 3 constant, read-only MSRs of timing
>> information to a viridian capable VM.
>>
>> There is an as-yet-unidentified issue when running Windows 8.1 / Server
2012r2
>> under Xen where it will periodically (1 in 10 attempt) appear to fall
into
>> an
>> idle loop rather than schedule userspace processes (such as failing to
run a
>> login session).
>>
>> I am still investigating the underlying cause.  One possibility is an
>> interaction of TSC time calibration interacting poorly with the Xen 
>> scheduler.
>>
>> Unfortunately, attempting to divine what windows is unhappy about with
its
>> environment is rather tricky (even a BSOD would be more useful than the
>> current symptoms), but providing these MSRs causes Windows to prefer
rdtsc
>> over the HPET main counter as a source of time, and
''fixes'' the above issue.
> I''m curious whether you would have put any consideration into
> the growing use of Hyper-V features when available - they had
> to play tricks in the past to avoid using them when in fact running
> on Xen. In particular in the case here I''m not certain your change
> won''t interact badly with https://lkml.org/lkml/2013/9/3/417.
>
> Jan
>
On Xen, viridian extensions is still an opt-in feature using an hvm param.

I don''t see how this would interact badly with that change?  If Linux
or
indeed anything else is unable to tell the difference between running on
Xen and running on hyperV, that is a but in the guest, not a bug in Xen
for providing viridian according to the specification.

~Andrew

>>> On 16.10.13 at 13:05, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> On 16/10/13 11:12, Jan Beulich wrote:
>>>>> On 15.10.13 at 20:12, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
>>> This set of two patches advertises 3 constant, read-only MSRs of
timing
>>> information to a viridian capable VM.
>>>
>>> There is an as-yet-unidentified issue when running Windows 8.1 /
Server 2012r2
>>> under Xen where it will periodically (1 in 10 attempt) appear to
fall into
>>> an
>>> idle loop rather than schedule userspace processes (such as failing
to run a
>>> login session).
>>>
>>> I am still investigating the underlying cause.  One possibility is
an
>>> interaction of TSC time calibration interacting poorly with the Xen
>>> scheduler.
>>>
>>> Unfortunately, attempting to divine what windows is unhappy about
with its
>>> environment is rather tricky (even a BSOD would be more useful than
the
>>> current symptoms), but providing these MSRs causes Windows to
prefer rdtsc
>>> over the HPET main counter as a source of time, and
''fixes'' the above issue.
>> I''m curious whether you would have put any consideration into
>> the growing use of Hyper-V features when available - they had
>> to play tricks in the past to avoid using them when in fact running
>> on Xen. In particular in the case here I''m not certain your
change
>> won''t interact badly with https://lkml.org/lkml/2013/9/3/417.
> 
> On Xen, viridian extensions is still an opt-in feature using an hvm param.
> 
> I don''t see how this would interact badly with that change?  If
Linux or
> indeed anything else is unable to tell the difference between running on
> Xen and running on hyperV, that is a but in the guest, not a bug in Xen
> for providing viridian according to the specification.
Iirc the main problem originally was that the Viridian check was
done before the Xen check (or was it with on upstream kernels
having CONFIG_XEN disabled, which is a valid configuration and
ought to work without a contrived check for Xen), and the Viridian
emulation done by Xen wasn''t good enough to actually run Linux
on top.

With any changes like the one here, the question ought to not
only be whether it helps Viridian, but also whether it doesn''t
break Linux.

Jan

On 16/10/13 12:21, Jan Beulich wrote:
>>>> On 16.10.13 at 13:05, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
>> On 16/10/13 11:12, Jan Beulich wrote:
>>>>>> On 15.10.13 at 20:12, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
>>>> This set of two patches advertises 3 constant, read-only MSRs
of timing
>>>> information to a viridian capable VM.
>>>>
>>>> There is an as-yet-unidentified issue when running Windows 8.1
/ Server 2012r2
>>>> under Xen where it will periodically (1 in 10 attempt) appear
to fall into
>>>> an
>>>> idle loop rather than schedule userspace processes (such as
failing to run a
>>>> login session).
>>>>
>>>> I am still investigating the underlying cause.  One possibility
is an
>>>> interaction of TSC time calibration interacting poorly with the
Xen
>>>> scheduler.
>>>>
>>>> Unfortunately, attempting to divine what windows is unhappy
about with its
>>>> environment is rather tricky (even a BSOD would be more useful
than the
>>>> current symptoms), but providing these MSRs causes Windows to
prefer rdtsc
>>>> over the HPET main counter as a source of time, and
''fixes'' the above issue.
>>> I''m curious whether you would have put any consideration
into
>>> the growing use of Hyper-V features when available - they had
>>> to play tricks in the past to avoid using them when in fact running
>>> on Xen. In particular in the case here I''m not certain
your change
>>> won''t interact badly with
https://lkml.org/lkml/2013/9/3/417.
>> On Xen, viridian extensions is still an opt-in feature using an hvm
param.
>>
>> I don''t see how this would interact badly with that change? 
If Linux or
>> indeed anything else is unable to tell the difference between running
on
>> Xen and running on hyperV, that is a but in the guest, not a bug in Xen
>> for providing viridian according to the specification.
> Iirc the main problem originally was that the Viridian check was
> done before the Xen check (or was it with on upstream kernels
> having CONFIG_XEN disabled, which is a valid configuration and
> ought to work without a contrived check for Xen), and the Viridian
> emulation done by Xen wasn''t good enough to actually run Linux
> on top.
>
> With any changes like the one here, the question ought to not
> only be whether it helps Viridian, but also whether it doesn''t
> break Linux.
>
> Jan
>
I disagree.  There is a perfectly good mechanism for advertising which
viridian extensions are available, which was being blindly ignored by
Linux (The specific bug was the HyperV drivers assuming a HyperV timer
without checking that it was actually present, leading to an hang when
waiting for a timer interrupt).

This is a Linux bug; Xen should not be functionally crippled because a
guest can''t enumerate support correctly.

And anyway - the entire set of viridian extensions is an off-by-default,
opt-in configuration option in the first place.  Anyone who decided to
try Linux with viridan can turn it off if it doesn''t work.

~Andrew

> -----Original Message-----
> From: Andrew Cooper [mailto:andrew.cooper3@citrix.com]
> Sent: 16 October 2013 14:37
> To: Jan Beulich
> Cc: Paul Durrant; xen-devel; Keir (Xen.org)
> Subject: Re: [PATCH 0/2] Viridian MSRs
> 
> On 16/10/13 12:21, Jan Beulich wrote:
> >>>> On 16.10.13 at 13:05, Andrew Cooper
<andrew.cooper3@citrix.com>
> wrote:
> >> On 16/10/13 11:12, Jan Beulich wrote:
> >>>>>> On 15.10.13 at 20:12, Andrew Cooper
<andrew.cooper3@citrix.com>
> wrote:
> >>>> This set of two patches advertises 3 constant, read-only
MSRs of timing
> >>>> information to a viridian capable VM.
> >>>>
> >>>> There is an as-yet-unidentified issue when running Windows
8.1 /
> Server 2012r2
> >>>> under Xen where it will periodically (1 in 10 attempt)
appear to fall into
> >>>> an
> >>>> idle loop rather than schedule userspace processes (such
as failing to
> run a
> >>>> login session).
> >>>>
> >>>> I am still investigating the underlying cause.  One
possibility is an
> >>>> interaction of TSC time calibration interacting poorly
with the Xen
> >>>> scheduler.
> >>>>
> >>>> Unfortunately, attempting to divine what windows is
unhappy about
> with its
> >>>> environment is rather tricky (even a BSOD would be more
useful than
> the
> >>>> current symptoms), but providing these MSRs causes Windows
to
> prefer rdtsc
> >>>> over the HPET main counter as a source of time, and
''fixes'' the above
> issue.
> >>> I''m curious whether you would have put any
consideration into
> >>> the growing use of Hyper-V features when available - they had
> >>> to play tricks in the past to avoid using them when in fact
running
> >>> on Xen. In particular in the case here I''m not
certain your change
> >>> won''t interact badly with
https://lkml.org/lkml/2013/9/3/417.
> >> On Xen, viridian extensions is still an opt-in feature using an
hvm param.
> >>
> >> I don''t see how this would interact badly with that
change?  If Linux or
> >> indeed anything else is unable to tell the difference between
running on
> >> Xen and running on hyperV, that is a but in the guest, not a bug
in Xen
> >> for providing viridian according to the specification.
> > Iirc the main problem originally was that the Viridian check was
> > done before the Xen check (or was it with on upstream kernels
> > having CONFIG_XEN disabled, which is a valid configuration and
> > ought to work without a contrived check for Xen), and the Viridian
> > emulation done by Xen wasn''t good enough to actually run
Linux
> > on top.
> >
> > With any changes like the one here, the question ought to not
> > only be whether it helps Viridian, but also whether it
doesn''t
> > break Linux.
> >
> > Jan
> >
> 
> I disagree.  There is a perfectly good mechanism for advertising which
> viridian extensions are available, which was being blindly ignored by
> Linux (The specific bug was the HyperV drivers assuming a HyperV timer
> without checking that it was actually present, leading to an hang when
> waiting for a timer interrupt).
> 
> This is a Linux bug; Xen should not be functionally crippled because a
> guest can''t enumerate support correctly.
> 
> And anyway - the entire set of viridian extensions is an off-by-default,
> opt-in configuration option in the first place.  Anyone who decided to
> try Linux with viridan can turn it off if it doesn''t work.
> 
The set of functionality we need to provide to be compliant with Windows is
documented here:

http://msdn.microsoft.com/en-us/library/windows/hardware/hh975392.aspx

Linux should assume no more.

  Paul

On 16/10/13 14:36, Andrew Cooper wrote:
> 
> I disagree.  There is a perfectly good mechanism for advertising which
> viridian extensions are available, which was being blindly ignored by
> Linux (The specific bug was the HyperV drivers assuming a HyperV timer
> without checking that it was actually present, leading to an hang when
> waiting for a timer interrupt).
> 
> This is a Linux bug; Xen should not be functionally crippled because a
> guest can''t enumerate support correctly.
> 
> And anyway - the entire set of viridian extensions is an off-by-default,
> opt-in configuration option in the first place.  Anyone who decided to
> try Linux with viridan can turn it off if it doesn''t work.
I think Jan is suggesting that you pre-emptively check that it doesn''t
cause breakage in Linux.  So that any Linux bugs (if there are any) can
be fixed sooner.

David

>>> On 15.10.13 at 20:12, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> This set of two patches advertises 3 constant, read-only MSRs of timing
> information to a viridian capable VM.
> 
> There is an as-yet-unidentified issue when running Windows 8.1 / Server
2012r2
> under Xen where it will periodically (1 in 10 attempt) appear to fall into 
> an
> idle loop rather than schedule userspace processes (such as failing to run
a
> login session).
> 
> I am still investigating the underlying cause.  One possibility is an
> interaction of TSC time calibration interacting poorly with the Xen 
> scheduler.
> 
> Unfortunately, attempting to divine what windows is unhappy about with its
> environment is rather tricky (even a BSOD would be more useful than the
> current symptoms), but providing these MSRs causes Windows to prefer rdtsc
> over the HPET main counter as a source of time, and
''fixes'' the above issue.
> 
> CC: Paul Durrant <paul.durrant@citrix.com>
I was kind of expecting to get a Reviewed-by (or other comments)
from you on this - is there any hope for this to be happening at
some point?

Thanks, Jan

>>> On 15.10.13 at 20:12, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> This set of two patches advertises 3 constant, read-only MSRs of timing
> information to a viridian capable VM.
> 
> There is an as-yet-unidentified issue when running Windows 8.1 / Server
2012r2
> under Xen where it will periodically (1 in 10 attempt) appear to fall into 
> an
> idle loop rather than schedule userspace processes (such as failing to run
a
> login session).
> 
> I am still investigating the underlying cause.  One possibility is an
> interaction of TSC time calibration interacting poorly with the Xen 
> scheduler.
> 
> Unfortunately, attempting to divine what windows is unhappy about with its
> environment is rather tricky (even a BSOD would be more useful than the
> current symptoms), but providing these MSRs causes Windows to prefer rdtsc
> over the HPET main counter as a source of time, and
''fixes'' the above issue.
So even now reading it again after a couple of weeks I''m still
uncertain whether the issue you describe is with or without these
patches applied, or independent of them.

Jan

> -----Original Message-----
> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: 05 November 2013 15:16
> To: Paul Durrant
> Cc: Andrew Cooper; xen-devel; Keir (Xen.org)
> Subject: Re: [PATCH 0/2] Viridian MSRs
> 
> >>> On 15.10.13 at 20:12, Andrew Cooper
<andrew.cooper3@citrix.com>
> wrote:
> > This set of two patches advertises 3 constant, read-only MSRs of
timing
> > information to a viridian capable VM.
> >
> > There is an as-yet-unidentified issue when running Windows 8.1 /
Server
> 2012r2
> > under Xen where it will periodically (1 in 10 attempt) appear to fall
into
> > an
> > idle loop rather than schedule userspace processes (such as failing to
run a
> > login session).
> >
> > I am still investigating the underlying cause.  One possibility is an
> > interaction of TSC time calibration interacting poorly with the Xen
> > scheduler.
> >
> > Unfortunately, attempting to divine what windows is unhappy about with
> its
> > environment is rather tricky (even a BSOD would be more useful than
the
> > current symptoms), but providing these MSRs causes Windows to prefer
> rdtsc
> > over the HPET main counter as a source of time, and
''fixes'' the above issue.
> >
> > CC: Paul Durrant <paul.durrant@citrix.com>
> 
> I was kind of expecting to get a Reviewed-by (or other comments)
> from you on this - is there any hope for this to be happening at
> some point?
> 
Since I originated the code and both patches carry my s-o-b I didn''t
feel it was appropriate for me to review or ack.

  Paul

On 05/11/13 15:17, Jan Beulich wrote:
>>>> On 15.10.13 at 20:12, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
>> This set of two patches advertises 3 constant, read-only MSRs of timing
>> information to a viridian capable VM.
>>
>> There is an as-yet-unidentified issue when running Windows 8.1 / Server
2012r2
>> under Xen where it will periodically (1 in 10 attempt) appear to fall
into
>> an
>> idle loop rather than schedule userspace processes (such as failing to
run a
>> login session).
>>
>> I am still investigating the underlying cause.  One possibility is an
>> interaction of TSC time calibration interacting poorly with the Xen 
>> scheduler.
>>
>> Unfortunately, attempting to divine what windows is unhappy about with
its
>> environment is rather tricky (even a BSOD would be more useful than the
>> current symptoms), but providing these MSRs causes Windows to prefer
rdtsc
>> over the HPET main counter as a source of time, and
''fixes'' the above issue.
> So even now reading it again after a couple of weeks I''m still
> uncertain whether the issue you describe is with or without these
> patches applied, or independent of them.
>
> Jan
>
It turns out that the problem appears to be independent, and appears to
be in the USB stack.

~Andrew

On Tue, 2013-10-15 at 19:12 +0100, Andrew Cooper wrote:
> This viridian MSR is a read-only source of time (in units of 100ns) since
the
> domain started.
> 
> From: Paul Durrant <paul.durrant@citrix.com>
> Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> CC: Keir Fraser <keir@xen.org>
> CC: Jan Beulich <JBeulich@suse.com>
BTW, to have the affect you probably intended (overriding the author on
commit with git am) the From should have been the first line, followed
by a blank line (the so-called pseudo-header area).

Ian.

>>> On 05.11.13 at 16:21, Paul Durrant <Paul.Durrant@citrix.com>
wrote:
>> I was kind of expecting to get a Reviewed-by (or other comments)
>> from you on this - is there any hope for this to be happening at
>> some point?
> 
> Since I originated the code and both patches carry my s-o-b I
didn''t feel it
> was appropriate for me to review or ack.
I''m sorry for not noticing - I only paid attention to the Cc: list
in the mail header.

Jan