Xen devel - Aug 2009 - [PATCH] Fixed legacy issues when extends number of vcpus > 32

Hi, Keir
  Attached two patches fixed the legacy issues after extending number of vcpus
to more for HVM domain.  The first one fixed vcpu bitmask issue for interrupt
delivery, and the second one fixed vpid shortage issue after extending number of
vcpus.
Xiantao


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Let me think about these. For patch 1 I think we can perhaps do more work in
the loop which matches vlapic identifiers, and thus avoid needing a
"temporary cpumask" to remember matches. For patch 2 I''ve
been intending to
throw away the VMX VPID logic and share the SVM logic, as it flushes TLBs no
more than the VMX logic and doesn''t suffer the same problems with
VPID/ASID
exhaustion.

 Thanks,
 Keir

On 16/08/2009 03:17, "Zhang, Xiantao" <xiantao.zhang@intel.com>
wrote:
> Hi, Keir
>   Attached two patches fixed the legacy issues after extending number of
vcpus
> to more for HVM domain.  The first one fixed vcpu bitmask issue for
interrupt
> delivery, and the second one fixed vpid shortage issue after extending
number
> of vcpus.  
> Xiantao


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
> Let me think about these. For patch 1 I think we can perhaps do more
> work in the loop which matches vlapic identifiers, and thus avoid
> needing a "temporary cpumask" to remember matches. For patch 2
I''ve
> been intending to throw away the VMX VPID logic and share the SVM
> logic, as it flushes TLBs no more than the VMX logic and doesn''t
> suffer the same problems with VPID/ASID exhaustion.
We have 2^16 vpids after removing the limit, so it should support 65535 vcpus
runing concurrently in a system, so we don''t need to consider the
exhaustion case from this point of view ?
Xiantao
>  Thanks,
>  Keir
> 
> On 16/08/2009 03:17, "Zhang, Xiantao"
<xiantao.zhang@intel.com> wrote:
> 
>> Hi, Keir
>>   Attached two patches fixed the legacy issues after extending
>> number of vcpus to more for HVM domain.  The first one fixed vcpu
>> bitmask issue for interrupt delivery, and the second one fixed vpid
>> shortage issue after extending number of vcpus. Xiantao

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 16/08/2009 09:55, "Zhang, Xiantao" <xiantao.zhang@intel.com>
wrote:
> Keir Fraser wrote:
>> Let me think about these. For patch 1 I think we can perhaps do more
>> work in the loop which matches vlapic identifiers, and thus avoid
>> needing a "temporary cpumask" to remember matches. For patch
2 I''ve
>> been intending to throw away the VMX VPID logic and share the SVM
>> logic, as it flushes TLBs no more than the VMX logic and
doesn''t
>> suffer the same problems with VPID/ASID exhaustion.
> 
> We have 2^16 vpids after removing the limit, so it should support 65535
vcpus
> runing concurrently in a system, so we don''t need to consider the
exhaustion
> case from this point of view ?
Why have two sets of logic when one is superior to the other? It
doesn''t
make sense. I''ll take a look at your patch and apply it for now,
however.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>> Keir Fraser wrote:
>>> Let me think about these. For patch 1 I think we can perhaps do
more
>>> work in the loop which matches vlapic identifiers, and thus avoid
>>> needing a "temporary cpumask" to remember matches. For
patch 2 I''ve
>>> been intending to throw away the VMX VPID logic and share the SVM
>>> logic, as it flushes TLBs no more than the VMX logic and
doesn''t
>>> suffer the same problems with VPID/ASID exhaustion.
>>
>> We have 2^16 vpids after removing the limit, so it should support 65535
vcpus
>> runing concurrently in a system, so we don''t need to consider
the exhaustion
>> case from this point of view ?
>
>Why have two sets of logic when one is superior to the other? It
doesn''t
>make sense. I''ll take a look at your patch and apply it for now,
however.
On hardware side, the key difference is that VMX VPID space is very large:
2^16 VPIDs, and 0 is reserved for VMX root mode. So 65535 VPIDs can be
assigned to VMX vCPUs. We use a bitmap to manage the VMX VPID space:
we reclaim freed VPIDs and reuse them later globally.

If I understand correctly, Xen manages SVM ASIDs per LP. So Xen needs to
allocate a new ASID on the target LP after each vCPU migration. To accelerate
ASID allocation after each vCPU migration, Xen doesn''t use a bitmap to
claim
freed ASIDs, while just performs a TLB flush and forces each vCPU on this LP to
regenerate a ASID when ASID exhaustion happens on a LP.

I''d agree that as VPID space is per LP, it''s not necessary to
be globally managed.
If we manage such a big VPID space using a bitmap on each LP, it will require
quite
a few memory and be inefficient on VPID allocation and reclaim. So probably we
can apply the current ASID allocation approach to VPID assuming VPID exhaustion
will be much less.

On the other side, I can''t understand why we need to consider the
overflow of
generations?

Thanks!
-Xin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 17/08/2009 07:40, "Li, Xin" <xin.li@intel.com> wrote:
> I''d agree that as VPID space is per LP, it''s not
necessary to be globally
> managed.
> If we manage such a big VPID space using a bitmap on each LP, it will
require
> quite
> a few memory and be inefficient on VPID allocation and reclaim. So probably
we
> can apply the current ASID allocation approach to VPID assuming VPID
> exhaustion
> will be much less.
> 
> On the other side, I can''t understand why we need to consider the
overflow of
> generations?
Well, the guy who developed it wanted to consider that case. But it is
pretty pointless of course since a 64-bit generation counter will never
overflow.

Anyhow, your current patch is clearly an improvement on the existing VPID
code, so we''ll go with that for the time being.

 Thanks,
 Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel