Linux Virtualization - Jun 2014 - [RFC PATCH 2/2] block: virtio-blk: support multi virt queues per virtio-blk device

Il 17/06/2014 17:50, Ming Lei ha scritto:
>> > It would be nice to allocate virtqueues dynamically instead of
>> > hardcoding the limit.  virtio-scsi also allocates virtqueues
>> > dynamically.
> virtio-scsi may have lots of LUN, but virtio-blk only has one disk
> which needn't lots of hardware queues.
If you want to do queue steering based on the guest VCPU number, the 
number of queues must be = to the number of VCPUs shouldn't it?

I tried using a divisor of the number of VCPUs, but couldn't get the 
block layer to deliver interrupts to the right VCPU.

Paolo

On Tue, Jun 17, 2014 at 11:53 PM, Paolo Bonzini <pbonzini at redhat.com>
wrote:
> Il 17/06/2014 17:50, Ming Lei ha scritto:
>
>>> > It would be nice to allocate virtqueues dynamically instead of
>>> > hardcoding the limit.  virtio-scsi also allocates virtqueues
>>> > dynamically.
>>
>> virtio-scsi may have lots of LUN, but virtio-blk only has one disk
>> which needn't lots of hardware queues.
>
>
> If you want to do queue steering based on the guest VCPU number, the number
> of queues must be = to the number of VCPUs shouldn't it?
>
> I tried using a divisor of the number of VCPUs, but couldn't get the
block
> layer to deliver interrupts to the right VCPU.
For blk-mq's hardware queue, that won't be necessary to equal to
VCPUs number, and irq affinity per hw queue can be simply set as
blk_mq_hw_ctx->cpumask.

Thanks,
--
Ming Lei

Il 17/06/2014 18:00, Ming Lei ha scritto:
>> > If you want to do queue steering based on the guest VCPU number,
the number
>> > of queues must be = to the number of VCPUs shouldn't it?
>> >
>> > I tried using a divisor of the number of VCPUs, but couldn't
get the block
>> > layer to deliver interrupts to the right VCPU.
> For blk-mq's hardware queue, that won't be necessary to equal to
> VCPUs number, and irq affinity per hw queue can be simply set as
> blk_mq_hw_ctx->cpumask.
Yes, but on top of that you want to have each request processed exactly 
by the CPU that sent it.  Unless the cpumasks are singletons, most of 
the benefit went away in my virtio-scsi tests.  Perhaps blk-mq is smarter.

Can you try benchmarking a 16 VCPU guest with 8 and 16 queues?

Paolo