search for: nr_cpu_ids

When tag_set->nr_maps is 1, the block layer limits the number of hw queues by nr_cpu_ids. No matter how many hw queues are use by virtio-blk/virtio-scsi, as they both have (tag_set->nr_maps == 1), they can use at most nr_cpu_ids hw queues. In addition, specifically for pci scenario, when the 'num-queues' specified by qemu is more than maxcpus, virtio-blk/virtio-scsi would n...

...question I am asking in this email thread is for a difference scenario. The issue is not we are not having enough vectors (although this is why only 1 vector is allocated for all virtio-blk queues). As so far virtio-blk has (set->nr_maps == 1), block layer would limit the number of hw queues by nr_cpu_ids, we indeed do not need more than nr_cpu_ids hw queues in virtio-blk. That's why I ask why not change the flow as below options when the number of supported hw queues is more than nr_cpu_ids (and set->nr_maps == 1. virtio-blk does not set nr_maps and block layer would set it to 1 when the dr...

...question I am asking in this email thread is for a difference scenario. The issue is not we are not having enough vectors (although this is why only 1 vector is allocated for all virtio-blk queues). As so far virtio-blk has (set->nr_maps == 1), block layer would limit the number of hw queues by nr_cpu_ids, we indeed do not need more than nr_cpu_ids hw queues in virtio-blk. That's why I ask why not change the flow as below options when the number of supported hw queues is more than nr_cpu_ids (and set->nr_maps == 1. virtio-blk does not set nr_maps and block layer would set it to 1 when the dr...

When tag_set->nr_maps is 1, the block layer limits the number of hw queues by nr_cpu_ids. No matter how many hw queues are used by virtio-blk, as it has (tag_set->nr_maps == 1), it can use at most nr_cpu_ids hw queues. In addition, specifically for pci scenario, when the 'num-queues' specified by qemu is more than maxcpus, virtio-blk would not be able to allocate more than...

When tag_set->nr_maps is 1, the block layer limits the number of hw queues by nr_cpu_ids. No matter how many hw queues are used by virtio-scsi, as it has (tag_set->nr_maps == 1), it can use at most nr_cpu_ids hw queues. In addition, specifically for pci scenario, when the 'num_queues' specified by qemu is more than maxcpus, virtio-scsi would not be able to allocate more tha...

From: Dongli Zhang <dongli.zhang at oracle.com> [ Upstream commit bf348f9b78d413e75bb079462751a1d86b6de36c ] When tag_set->nr_maps is 1, the block layer limits the number of hw queues by nr_cpu_ids. No matter how many hw queues are used by virtio-blk, as it has (tag_set->nr_maps == 1), it can use at most nr_cpu_ids hw queues. In addition, specifically for pci scenario, when the 'num-queues' specified by qemu is more than maxcpus, virtio-blk would not be able to allocate more than...

From: Dongli Zhang <dongli.zhang at oracle.com> [ Upstream commit bf348f9b78d413e75bb079462751a1d86b6de36c ] When tag_set->nr_maps is 1, the block layer limits the number of hw queues by nr_cpu_ids. No matter how many hw queues are used by virtio-blk, as it has (tag_set->nr_maps == 1), it can use at most nr_cpu_ids hw queues. In addition, specifically for pci scenario, when the 'num-queues' specified by qemu is more than maxcpus, virtio-blk would not be able to allocate more than...

From: Dongli Zhang <dongli.zhang at oracle.com> [ Upstream commit bf348f9b78d413e75bb079462751a1d86b6de36c ] When tag_set->nr_maps is 1, the block layer limits the number of hw queues by nr_cpu_ids. No matter how many hw queues are used by virtio-blk, as it has (tag_set->nr_maps == 1), it can use at most nr_cpu_ids hw queues. In addition, specifically for pci scenario, when the 'num-queues' specified by qemu is more than maxcpus, virtio-blk would not be able to allocate more than...

From: Dongli Zhang <dongli.zhang at oracle.com> [ Upstream commit bf348f9b78d413e75bb079462751a1d86b6de36c ] When tag_set->nr_maps is 1, the block layer limits the number of hw queues by nr_cpu_ids. No matter how many hw queues are used by virtio-blk, as it has (tag_set->nr_maps == 1), it can use at most nr_cpu_ids hw queues. In addition, specifically for pci scenario, when the 'num-queues' specified by qemu is more than maxcpus, virtio-blk would not be able to allocate more than...

...free IDs */ unsigned nr_free; unsigned freelist[]; }; @@ -1209,21 +1215,21 @@ static inline void steal_tags(struct percpu_ida *pool, unsigned cpus_have_tags, cpu = pool->cpu_last_stolen; struct percpu_ida_cpu *remote; - for (cpus_have_tags = bitmap_weight(pool->cpus_have_tags, nr_cpu_ids); + for (cpus_have_tags = cpumask_weight(&pool->cpus_have_tags); cpus_have_tags * IDA_PCPU_SIZE > pool->nr_tags / 2; cpus_have_tags--) { - cpu = find_next_bit(pool->cpus_have_tags, nr_cpu_ids, cpu); + cpu = cpumask_next(cpu, &pool->cpus_have_tags); - if (cp...

...this email thread is for a difference scenario. > > The issue is not we are not having enough vectors (although this is why only 1 > vector is allocated for all virtio-blk queues). As so far virtio-blk has > (set->nr_maps == 1), block layer would limit the number of hw queues by > nr_cpu_ids, we indeed do not need more than nr_cpu_ids hw queues in virtio-blk. > > That's why I ask why not change the flow as below options when the number of > supported hw queues is more than nr_cpu_ids (and set->nr_maps == 1. virtio-blk > does not set nr_maps and block layer would set...

...fic > queues. Would need some device<->transport interface, obviously. > This sounds a little bit similar to multiple hctx maps? So far, as virtio-blk only supports set->nr_maps = 1, no matter how many hw queues are assigned for virtio-blk, blk_mq_alloc_tag_set() would use at most nr_cpu_ids hw queues. 2981 int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set) ... ... 3021 /* 3022 * There is no use for more h/w queues than cpus if we just have 3023 * a single map 3024 */ 3025 if (set->nr_maps == 1 && set->nr_hw_queues > nr_cpu...

...fic > queues. Would need some device<->transport interface, obviously. > This sounds a little bit similar to multiple hctx maps? So far, as virtio-blk only supports set->nr_maps = 1, no matter how many hw queues are assigned for virtio-blk, blk_mq_alloc_tag_set() would use at most nr_cpu_ids hw queues. 2981 int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set) ... ... 3021 /* 3022 * There is no use for more h/w queues than cpus if we just have 3023 * a single map 3024 */ 3025 if (set->nr_maps == 1 && set->nr_hw_queues > nr_cpu...

On Fri, 15 Mar 2019 12:50:11 +0800 Jason Wang <jasowang at redhat.com> wrote: > Or something like I proposed several years ago? > https://do-db2.lkml.org/lkml/2014/12/25/169 > > Btw, for virtio-net, I think we actually want to go for having a maximum > number of supported queues like what hardware did. This would be useful > for e.g cpu hotplug or XDP (requires per cpu

On Fri, 15 Mar 2019 12:50:11 +0800 Jason Wang <jasowang at redhat.com> wrote: > Or something like I proposed several years ago? > https://do-db2.lkml.org/lkml/2014/12/25/169 > > Btw, for virtio-net, I think we actually want to go for having a maximum > number of supported queues like what hardware did. This would be useful > for e.g cpu hotplug or XDP (requires per cpu

...> +{ > + struct smp_call_on_cpu_struct sscs = { > + .work = __WORK_INITIALIZER(sscs.work, smp_call_on_cpu_callback), > + .done = COMPLETION_INITIALIZER_ONSTACK(sscs.done), > + .func = func, > + .data = par, > + .cpu = pin ? cpu : -1, > + }; > + > + if (cpu >= nr_cpu_ids) You might want to also include cpu_online(). if (cpu >= nr_cpu_ids || !cpu_online(cpu)) > + return -ENXIO; Seeing how its fairly hard to schedule work on a cpu that's not actually there. > + > + queue_work_on(cpu, system_wq, &sscs.work); > + wait_for_completion(&ss...

...> +{ > + struct smp_call_on_cpu_struct sscs = { > + .work = __WORK_INITIALIZER(sscs.work, smp_call_on_cpu_callback), > + .done = COMPLETION_INITIALIZER_ONSTACK(sscs.done), > + .func = func, > + .data = par, > + .cpu = pin ? cpu : -1, > + }; > + > + if (cpu >= nr_cpu_ids) You might want to also include cpu_online(). if (cpu >= nr_cpu_ids || !cpu_online(cpu)) > + return -ENXIO; Seeing how its fairly hard to schedule work on a cpu that's not actually there. > + > + queue_work_on(cpu, system_wq, &sscs.work); > + wait_for_completion(&ss...

...gt;freelist = (void *) __get_free_pages(GFP_KERNEL, order); > + if (!pool->freelist) > + return -ENOMEM; > + > + for (i = 0; i < nr_tags; i++) > + pool->freelist[i] = i; > + > + pool->nr_free = nr_tags; > + > + pool->cpus_have_tags = kzalloc(BITS_TO_LONGS(nr_cpu_ids) * > + sizeof(unsigned long), GFP_KERNEL); > + if (!pool->cpus_have_tags) > + goto err; > + > + pool->tag_cpu = __alloc_percpu(sizeof(struct percpu_ida_cpu) + > + IDA_PCPU_SIZE * sizeof(unsigned), > + sizeof(unsigned)); > + if (!pool->...

...gt;freelist = (void *) __get_free_pages(GFP_KERNEL, order); > + if (!pool->freelist) > + return -ENOMEM; > + > + for (i = 0; i < nr_tags; i++) > + pool->freelist[i] = i; > + > + pool->nr_free = nr_tags; > + > + pool->cpus_have_tags = kzalloc(BITS_TO_LONGS(nr_cpu_ids) * > + sizeof(unsigned long), GFP_KERNEL); > + if (!pool->cpus_have_tags) > + goto err; > + > + pool->tag_cpu = __alloc_percpu(sizeof(struct percpu_ida_cpu) + > + IDA_PCPU_SIZE * sizeof(unsigned), > + sizeof(unsigned)); > + if (!pool->...

...ce<->transport interface, obviously. > > > > This sounds a little bit similar to multiple hctx maps? > > So far, as virtio-blk only supports set->nr_maps = 1, no matter how many hw > queues are assigned for virtio-blk, blk_mq_alloc_tag_set() would use at most > nr_cpu_ids hw queues. > > 2981 int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set) > ... ... > 3021 /* > 3022 * There is no use for more h/w queues than cpus if we just have > 3023 * a single map > 3024 */ > 3025 if (set->nr_maps == 1 &am...