Virtualization - May 2018 - [RFC v3 4/5] virtio_ring: add event idx support in packed ring

On 2018?05?03? 21:54, Tiwei Bie wrote:
> On Thu, May 03, 2018 at 03:25:29PM +0800, Jason Wang wrote:
>> On 2018?05?03? 10:09, Tiwei Bie wrote:
>>>>>> So how about we use the straightforward way then?
>>>>> You mean we do new += vq->vring_packed.num instead
>>>>> of event_idx -= vq->vring_packed.num before calling
>>>>> vring_need_event()?
>>>>>
>>>>> The problem is that, the second param (new_idx) of
>>>>> vring_need_event() will be used for:
>>>>>
>>>>> (__u16)(new_idx - event_idx - 1)
>>>>> (__u16)(new_idx - old)
>>>>>
>>>>> So if we change new, we will need to change old too.
>>>> I think that since we have a branch there anyway,
>>>> we are better off just special-casing if (wrap_counter !=
vq->wrap_counter).
>>>> Treat is differenty and avoid casts.
>>>>
>>>>> And that would be an ugly hack..
>>>>>
>>>>> Best regards,
>>>>> Tiwei Bie
>>>> I consider casts and huge numbers with two's complement
>>>> games even uglier.
>>> The dependency on two's complement game is introduced
>>> since the split ring.
>>>
>>> In packed ring, old is calculated via:
>>>
>>> old = vq->next_avail_idx - vq->num_added;
>>>
>>> In split ring, old is calculated via:
>>>
>>> old = vq->avail_idx_shadow - vq->num_added;
>>>
>>> In both cases, when vq->num_added is bigger, old will
>>> be a big number.
>>>
>>> Best regards,
>>> Tiwei Bie
>>>
>> How about just do something like vhost:
>>
>> static u16 vhost_idx_diff(struct vhost_virtqueue *vq, u16 old, u16 new)
>> {
>>  ??? if (new > old)
>>  ??? ??? return new - old;
>>  ??? return? (new + vq->num - old);
>> }
>>
>> static bool vhost_vring_packed_need_event(struct vhost_virtqueue *vq,
>>  ??? ??? ??? ??? ??? ? __u16 event_off, __u16 new,
>>  ??? ??? ??? ??? ??? ? __u16 old)
>> {
>>  ??? return (__u16)(vhost_idx_diff(vq, new, event_off) - 1) <
>>  ??? ?????? (__u16)vhost_idx_diff(vq, new, old);
>> }
>>
>> ?
> It seems that there is a typo in above code. The second
> param of vhost_idx_diff() is `old`, but when calling this
> function in vhost_vring_packed_need_event(), `new` is
> passed as the second param.
Right.
>
> If we assume the second param of vhost_idx_diff() is new
> and the third one is old, i.e.:
>
> static u16 vhost_idx_diff(struct vhost_virtqueue *vq, u16 new, u16 old)
> {
>  ??? if (new > old)
>  ??? ??? return new - old;
>  ??? return? (new + vq->num - old);
> }
>
> I think it's still not right.
>
> Because in virtqueue_enable_cb_delayed(), we may set an
> event_off which is bigger than new and both of them have
> wrapped. And in this case, although new is smaller than
> event_off (i.e. the third param -- old), new shouldn't
> add vq->num, and actually we are expecting a very big
> idx diff.
Yes, so to calculate distance correctly between event and new, we just 
need to compare the warp counter and return false if it doesn't match 
without the need to try to add vq.num here.

Thanks
>
> Best regards,
> Tiwei Bie

On 2018?05?08? 11:05, Jason Wang wrote:
>>
>> Because in virtqueue_enable_cb_delayed(), we may set an
>> event_off which is bigger than new and both of them have
>> wrapped. And in this case, although new is smaller than
>> event_off (i.e. the third param -- old), new shouldn't
>> add vq->num, and actually we are expecting a very big
>> idx diff.
>
> Yes, so to calculate distance correctly between event and new, we just 
> need to compare the warp counter and return false if it doesn't match 
> without the need to try to add vq.num here.
>
> Thanks 
Sorry, looks like the following should work, we need add vq.num if 
used_wrap_counter does not match:

static bool vhost_vring_packed_need_event(struct vhost_virtqueue *vq,
 ??? ??? ??? ??? ??? ? __u16 off_wrap, __u16 new,
 ??? ??? ??? ??? ??? ? __u16 old)
{
 ??? bool wrap = off_wrap >> 15;
 ??? int off = off_wrap & ~(1 << 15);
 ??? __u16 d1, d2;

 ??? if (wrap != vq->used_wrap_counter)
 ??? ??? d1 = new + vq->num - off - 1;
 ??? else
 ??? ??? d1 = new - off - 1;

 ??? if (new > old)
 ??? ??? d2 = new - old;
 ??? else
 ??? ??? d2 = new + vq->num - old;

 ??? return d1 < d2;
}

Thanks

On Tue, May 08, 2018 at 01:40:40PM +0800, Jason Wang
wrote:
> On 2018?05?08? 11:05, Jason Wang wrote:
> > > 
> > > Because in virtqueue_enable_cb_delayed(), we may set an
> > > event_off which is bigger than new and both of them have
> > > wrapped. And in this case, although new is smaller than
> > > event_off (i.e. the third param -- old), new shouldn't
> > > add vq->num, and actually we are expecting a very big
> > > idx diff.
> > 
> > Yes, so to calculate distance correctly between event and new, we just
> > need to compare the warp counter and return false if it doesn't
match
> > without the need to try to add vq.num here.
> > 
> > Thanks
> 
> Sorry, looks like the following should work, we need add vq.num if
> used_wrap_counter does not match:
> 
> static bool vhost_vring_packed_need_event(struct vhost_virtqueue *vq,
> ??? ??? ??? ??? ??? ? __u16 off_wrap, __u16 new,
> ??? ??? ??? ??? ??? ? __u16 old)
> {
> ??? bool wrap = off_wrap >> 15;
> ??? int off = off_wrap & ~(1 << 15);
> ??? __u16 d1, d2;
> 
> ??? if (wrap != vq->used_wrap_counter)
> ??? ??? d1 = new + vq->num - off - 1;
Just to draw your attention (maybe you have already
noticed this).

In this case (i.e. wrap != vq->used_wrap_counter),
it's also possible that (off < new) is true. Because,

when virtqueue_enable_cb_delayed_packed() is used,
`off` is calculated in driver in a way like this:

	off = vq->last_used_idx + bufs;
	if (off >= vq->vring_packed.num) {
		off -= vq->vring_packed.num;
		wrap_counter ^= 1;
	}

And when `new` (in vhost) is close to vq->num. The
vq->last_used_idx + bufs (in driver) can be bigger
than vq->vring_packed.num, and:

1. `off` will wrap;
2. wrap counters won't match;
3. off < new;

And d1 (i.e. new + vq->num - off - 1) will be a value
bigger than vq->num. I'm okay with this, although it's
a bit weird.

Best regards,
Tiwei Bie
> ??? else
> ??? ??? d1 = new - off - 1;
> 
> ??? if (new > old)
> ??? ??? d2 = new - old;
> ??? else
> ??? ??? d2 = new + vq->num - old;
> 
> ??? return d1 < d2;
> }
> 
> Thanks
>