Linux Virtualization - Nov 2017 - [PATCH] x86/paravirt: Add kernel parameter to choose paravirt lock type

On 11/01/2017 11:51 AM, Juergen Gross wrote:
> On 01/11/17 16:32, Waiman Long wrote:
>> Currently, there are 3 different lock types that can be chosen for
>> the x86 architecture:
>>
>>  - qspinlock
>>  - pvqspinlock
>>  - unfair lock
>>
>> One of the above lock types will be chosen at boot time depending on
>> a number of different factors.
>>
>> Ideally, the hypervisors should be able to pick the best performing
>> lock type for the current VM configuration. That is not currently
>> the case as the performance of each lock type are affected by many
>> different factors like the number of vCPUs in the VM, the amount vCPU
>> overcommitment, the CPU type and so on.
>>
>> Generally speaking, unfair lock performs well for VMs with a small
>> number of vCPUs. Native qspinlock may perform better than pvqspinlock
>> if there is vCPU pinning and there is no vCPU over-commitment.
>>
>> This patch adds a new kernel parameter to allow administrator to
>> choose the paravirt spinlock type to be used. VM administrators can
>> experiment with the different lock types and choose one that can best
>> suit their need, if they want to. Hypervisor developers can also use
>> that to experiment with different lock types so that they can come
>> up with a better algorithm to pick the best lock type.
>>
>> The hypervisor paravirt spinlock code will override this new parameter
>> in determining if pvqspinlock should be used. The parameter, however,
>> will override Xen's xen_nopvspin in term of disabling unfair lock.
> Hmm, I'm not sure we need pvlock_type _and_ xen_nopvspin. What do
others
> think?
I don't think we need xen_nopvspin, but I don't want to remove that
without agreement from the community.
>>  DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
>>  
>>  void __init native_pv_lock_init(void)
>>  {
>> -	if (!static_cpu_has(X86_FEATURE_HYPERVISOR))
>> +	if (pv_spinlock_type == locktype_unfair)
>> +		return;
>> +
>> +	if (!static_cpu_has(X86_FEATURE_HYPERVISOR) ||
>> +	   (pv_spinlock_type != locktype_auto))
>>  		static_branch_disable(&virt_spin_lock_key);
> Really? I don't think locktype_paravirt should disable the static key.
With paravirt spinlock, it doesn't matter if the static key is disabled
or not. Without CONFIG_PARAVIRT_SPINLOCKS, however, it does degenerate
into the native qspinlock. So you are right, I should check for paravirt
type as well.

Cheers,
Longman

On 11/01/2017 12:28 PM, Waiman Long wrote:
> On 11/01/2017 11:51 AM, Juergen Gross wrote:
>> On 01/11/17 16:32, Waiman Long wrote:
>>> Currently, there are 3 different lock types that can be chosen for
>>> the x86 architecture:
>>>
>>>  - qspinlock
>>>  - pvqspinlock
>>>  - unfair lock
>>>
>>> One of the above lock types will be chosen at boot time depending
on
>>> a number of different factors.
>>>
>>> Ideally, the hypervisors should be able to pick the best performing
>>> lock type for the current VM configuration. That is not currently
>>> the case as the performance of each lock type are affected by many
>>> different factors like the number of vCPUs in the VM, the amount
vCPU
>>> overcommitment, the CPU type and so on.
>>>
>>> Generally speaking, unfair lock performs well for VMs with a small
>>> number of vCPUs. Native qspinlock may perform better than
pvqspinlock
>>> if there is vCPU pinning and there is no vCPU over-commitment.
>>>
>>> This patch adds a new kernel parameter to allow administrator to
>>> choose the paravirt spinlock type to be used. VM administrators can
>>> experiment with the different lock types and choose one that can
best
>>> suit their need, if they want to. Hypervisor developers can also
use
>>> that to experiment with different lock types so that they can come
>>> up with a better algorithm to pick the best lock type.
>>>
>>> The hypervisor paravirt spinlock code will override this new
parameter
>>> in determining if pvqspinlock should be used. The parameter,
however,
>>> will override Xen's xen_nopvspin in term of disabling unfair
lock.
>> Hmm, I'm not sure we need pvlock_type _and_ xen_nopvspin. What do
others
>> think?
> I don't think we need xen_nopvspin, but I don't want to remove that
> without agreement from the community.
I also don't think xen_nopvspin will be needed after pvlock_type is
introduced.

-boris

On 11/01/2017 03:01 PM, Boris Ostrovsky wrote:
> On 11/01/2017 12:28 PM, Waiman Long wrote:
>> On 11/01/2017 11:51 AM, Juergen Gross wrote:
>>> On 01/11/17 16:32, Waiman Long wrote:
>>>> Currently, there are 3 different lock types that can be chosen
for
>>>> the x86 architecture:
>>>>
>>>>  - qspinlock
>>>>  - pvqspinlock
>>>>  - unfair lock
>>>>
>>>> One of the above lock types will be chosen at boot time
depending on
>>>> a number of different factors.
>>>>
>>>> Ideally, the hypervisors should be able to pick the best
performing
>>>> lock type for the current VM configuration. That is not
currently
>>>> the case as the performance of each lock type are affected by
many
>>>> different factors like the number of vCPUs in the VM, the
amount vCPU
>>>> overcommitment, the CPU type and so on.
>>>>
>>>> Generally speaking, unfair lock performs well for VMs with a
small
>>>> number of vCPUs. Native qspinlock may perform better than
pvqspinlock
>>>> if there is vCPU pinning and there is no vCPU over-commitment.
>>>>
>>>> This patch adds a new kernel parameter to allow administrator
to
>>>> choose the paravirt spinlock type to be used. VM administrators
can
>>>> experiment with the different lock types and choose one that
can best
>>>> suit their need, if they want to. Hypervisor developers can
also use
>>>> that to experiment with different lock types so that they can
come
>>>> up with a better algorithm to pick the best lock type.
>>>>
>>>> The hypervisor paravirt spinlock code will override this new
parameter
>>>> in determining if pvqspinlock should be used. The parameter,
however,
>>>> will override Xen's xen_nopvspin in term of disabling
unfair lock.
>>> Hmm, I'm not sure we need pvlock_type _and_ xen_nopvspin. What
do others
>>> think?
>> I don't think we need xen_nopvspin, but I don't want to remove
that
>> without agreement from the community.
> I also don't think xen_nopvspin will be needed after pvlock_type is
> introduced.
>
> -boris
Another reason that I didn't try to remove xen_nopvspin is backward 
compatibility concern. One way to handle it is to depreciate it and
treat it as an alias to pvlock_type=queued.

Cheers,
Longman