Jason Wang
2022-Nov-17 03:36 UTC
[PATCH 3/6] vduse: Add sysfs interface for irq affinity setup
On Wed, Nov 16, 2022 at 3:46 PM Yongji Xie <xieyongji at bytedance.com> wrote:> > On Wed, Nov 16, 2022 at 3:11 PM Jason Wang <jasowang at redhat.com> wrote: > > > > On Tue, Nov 15, 2022 at 10:49 AM Yongji Xie <xieyongji at bytedance.com> wrote: > > > > > > On Mon, Nov 14, 2022 at 4:55 PM Jason Wang <jasowang at redhat.com> wrote: > > > > > > > > On Mon, Nov 14, 2022 at 4:20 PM Yongji Xie <xieyongji at bytedance.com> wrote: > > > > > > > > > > On Mon, Nov 14, 2022 at 3:58 PM Jason Wang <jasowang at redhat.com> wrote: > > > > > > > > > > > > On Mon, Nov 14, 2022 at 3:16 PM Xie Yongji <xieyongji at bytedance.com> wrote: > > > > > > > > > > > > > > Add sysfs interface for each vduse virtqueue to setup > > > > > > > irq affinity. This would be useful for performance > > > > > > > tuning, e.g., mitigate the virtqueue lock contention > > > > > > > in virtio block driver. > > > > > > > > > > > > Do we have any perforamnce numbers for this? > > > > > > > > > > > > > > > > Almost 50% improvement (600k iops -> 900k iops) in the high iops > > > > > workloads. I have mentioned it in the cover-letter. > > > > > > > > For some reason, I miss that. > > > > > > > > I also wonder if we can do this automatically, then there's no need to > > > > play with sysfs which is kind of a burden for the management layer. > > > > > > > > > > This is hard to do since vduse doesn't know which cpu should be bound > > > for a certain virtqueue. > > > > Probably via the kick_vq()? It probably won't work when notification > > is disabled. But we need to think a little bit more about this. > > Yes, another problem is that this way can only work when the cpu and > virtqueue are 1:1 mapping. It's still hard to decide which cpu to bind > in the N:1 mapping case.This is the same situation as what you propose here. I think it would be better to use cpumask instead of cpu id here.> > So I think it could be an optimization, but the sysfs interface is still needed. > > > Requiring management software to do ad-hoc running just for VDUSE > > seems not easy. > > > > I'm not sure. In the kubernetes environment, something like a CSI/CNI > plugin can do it.Only works when the process is bound to a specific cpu. If a process is migrated to another CPU, it would be hard to track. Thanks> > Thanks, > Yongji >