Linux Virtualization - May 2018 - [PATCH net-next v11 2/5] netvsc: refactor notifier/event handling code to use the failover framework

On 5/22/2018 9:12 AM, Jiri Pirko wrote:
> Fixing the subj, sorry about that.
>
> Tue, May 22, 2018 at 05:46:21PM CEST, mst at redhat.com wrote:
>> On Tue, May 22, 2018 at 05:36:14PM +0200, Jiri Pirko wrote:
>>> Tue, May 22, 2018 at 05:28:42PM CEST, sridhar.samudrala at
intel.com wrote:
>>>> On 5/22/2018 2:08 AM, Jiri Pirko wrote:
>>>>> Tue, May 22, 2018 at 11:06:37AM CEST, jiri at resnulli.us
wrote:
>>>>>> Tue, May 22, 2018 at 04:06:18AM CEST, sridhar.samudrala
at intel.com wrote:
>>>>>>> Use the registration/notification framework
supported by the generic
>>>>>>> failover infrastructure.
>>>>>>>
>>>>>>> Signed-off-by: Sridhar Samudrala
<sridhar.samudrala at intel.com>
>>>>>> In previous patchset versions, the common code did
>>>>>> netdev_rx_handler_register() and
netdev_upper_dev_link() etc
>>>>>> (netvsc_vf_join()). Now, this is still done in netvsc.
Why?
>>>>>>
>>>>>> This should be part of the common "failover"
code.
>>>> Based on Stephen's feedback on earlier patches, i tried to
minimize the changes to
>>>> netvsc and only commonize the notifier and the main event
handler routine.
>>>> Another complication is that netvsc does part of registration
in a delayed workqueue.
>>> :( This kind of degrades the whole efford of having single solution
>>> in "failover" module. I think that common parts, as
>>> netdev_rx_handler_register() and others certainly is should be
inside
>>> the common module. This is not a good time to minimize changes.
Let's do
>>> the thing properly and fix the netvsc mess now.
>>>
>>>
>>>> It should be possible to move some of the code from
net_failover.c to generic
>>>> failover.c in future if Stephen is ok with it.
>>>>
>>>>
>>>>> Also note that in the current patchset you use IFF_FAILOVER
flag for
>>>>> master, yet for the slave you use IFF_SLAVE. That is wrong.
>>>>> IFF_FAILOVER_SLAVE should be used.
>>>> Not sure which code you are referring to.? I only set
IFF_FAILOVER_SLAVE
>>>> in patch 3.
>>> The existing netvsc driver.
>> We really can't change netvsc's flags now, even if it's
interface is
>> messy, it's being used in the field. We can add a flag that makes
netvsc
>> behave differently, and if this flag also allows enhanced functionality
>> userspace will gradually switch.
> Okay, although in this case, it really does not make much sense, so be
> it. Leave the netvsc set the ->priv flag to IFF_SLAVE as it is doing
> now. (This once-wrong-forever-wrong policy is flustrating me).
>
> But since this patchset introduces private flag IFF_FAILOVER and
> IFF_FAILOVER_SLAVE, and we set IFF_FAILOVER to the netvsc netdev
> instance, we should also set IFF_FAILOVER_SLAVE to the enslaved VF
> netdevice to get at least some consistency between virtio_net and
> netvsc.
OK. I can make this change to set/unset IFF_FAILOVER_SLAVE in the netvsc
register/unregister routines so that it is consistent with virtio_net.

Based on your discussion with mst, i think we can even remove IFF_SLAVE
setting on netvsc as it should not impact userspace.  If Stephen is OK
we can make this change too.

Do you see any other items that need to be resolved for this series to go in
this merge window?


>
>> Anything breaking userspace I fully expect Stephen to nack and
>> IMO with good reason.
>>
>> -- 
>> MST

Tue, May 22, 2018 at 10:54:29PM CEST, sridhar.samudrala at intel.com
wrote:
>
>
>On 5/22/2018 9:12 AM, Jiri Pirko wrote:
>> Fixing the subj, sorry about that.
>> 
>> Tue, May 22, 2018 at 05:46:21PM CEST, mst at redhat.com wrote:
>> > On Tue, May 22, 2018 at 05:36:14PM +0200, Jiri Pirko wrote:
>> > > Tue, May 22, 2018 at 05:28:42PM CEST, sridhar.samudrala at
intel.com wrote:
>> > > > On 5/22/2018 2:08 AM, Jiri Pirko wrote:
>> > > > > Tue, May 22, 2018 at 11:06:37AM CEST, jiri at
resnulli.us wrote:
>> > > > > > Tue, May 22, 2018 at 04:06:18AM CEST,
sridhar.samudrala at intel.com wrote:
>> > > > > > > Use the registration/notification
framework supported by the generic
>> > > > > > > failover infrastructure.
>> > > > > > > 
>> > > > > > > Signed-off-by: Sridhar Samudrala
<sridhar.samudrala at intel.com>
>> > > > > > In previous patchset versions, the common code
did
>> > > > > > netdev_rx_handler_register() and
netdev_upper_dev_link() etc
>> > > > > > (netvsc_vf_join()). Now, this is still done in
netvsc. Why?
>> > > > > > 
>> > > > > > This should be part of the common
"failover" code.
>> > > > Based on Stephen's feedback on earlier patches, i
tried to minimize the changes to
>> > > > netvsc and only commonize the notifier and the main
event handler routine.
>> > > > Another complication is that netvsc does part of
registration in a delayed workqueue.
>> > > :( This kind of degrades the whole efford of having single
solution
>> > > in "failover" module. I think that common parts, as
>> > > netdev_rx_handler_register() and others certainly is should
be inside
>> > > the common module. This is not a good time to minimize
changes. Let's do
>> > > the thing properly and fix the netvsc mess now.
>> > > 
>> > > 
>> > > > It should be possible to move some of the code from
net_failover.c to generic
>> > > > failover.c in future if Stephen is ok with it.
>> > > > 
>> > > > 
>> > > > > Also note that in the current patchset you use
IFF_FAILOVER flag for
>> > > > > master, yet for the slave you use IFF_SLAVE. That
is wrong.
>> > > > > IFF_FAILOVER_SLAVE should be used.
>> > > > Not sure which code you are referring to.? I only set
IFF_FAILOVER_SLAVE
>> > > > in patch 3.
>> > > The existing netvsc driver.
>> > We really can't change netvsc's flags now, even if
it's interface is
>> > messy, it's being used in the field. We can add a flag that
makes netvsc
>> > behave differently, and if this flag also allows enhanced
functionality
>> > userspace will gradually switch.
>> Okay, although in this case, it really does not make much sense, so be
>> it. Leave the netvsc set the ->priv flag to IFF_SLAVE as it is doing
>> now. (This once-wrong-forever-wrong policy is flustrating me).
>> 
>> But since this patchset introduces private flag IFF_FAILOVER and
>> IFF_FAILOVER_SLAVE, and we set IFF_FAILOVER to the netvsc netdev
>> instance, we should also set IFF_FAILOVER_SLAVE to the enslaved VF
>> netdevice to get at least some consistency between virtio_net and
>> netvsc.
>
>OK. I can make this change to set/unset IFF_FAILOVER_SLAVE in the netvsc
>register/unregister routines so that it is consistent with virtio_net.
>
>Based on your discussion with mst, i think we can even remove IFF_SLAVE
>setting on netvsc as it should not impact userspace.  If Stephen is OK
>we can make this change too.
>
>Do you see any other items that need to be resolved for this series to go in
>this merge window?
As I wrote previously, the common code including rx_handler registration
and setting of flags and master link should be done in a common code,
moved away from netvsc code.

Thanks.

>
>
>
>> 
>> > Anything breaking userspace I fully expect Stephen to nack and
>> > IMO with good reason.
>> > 
>> > -- 
>> > MST
>

On 5/22/2018 11:27 PM, Jiri Pirko wrote:
> Tue, May 22, 2018 at 10:54:29PM CEST, sridhar.samudrala at intel.com wrote:
>>
>> On 5/22/2018 9:12 AM, Jiri Pirko wrote:
>>> Fixing the subj, sorry about that.
>>>
>>> Tue, May 22, 2018 at 05:46:21PM CEST, mst at redhat.com wrote:
>>>> On Tue, May 22, 2018 at 05:36:14PM +0200, Jiri Pirko wrote:
>>>>> Tue, May 22, 2018 at 05:28:42PM CEST, sridhar.samudrala at
intel.com wrote:
>>>>>> On 5/22/2018 2:08 AM, Jiri Pirko wrote:
>>>>>>> Tue, May 22, 2018 at 11:06:37AM CEST, jiri at
resnulli.us wrote:
>>>>>>>> Tue, May 22, 2018 at 04:06:18AM CEST,
sridhar.samudrala at intel.com wrote:
>>>>>>>>> Use the registration/notification framework
supported by the generic
>>>>>>>>> failover infrastructure.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Sridhar Samudrala
<sridhar.samudrala at intel.com>
>>>>>>>> In previous patchset versions, the common code
did
>>>>>>>> netdev_rx_handler_register() and
netdev_upper_dev_link() etc
>>>>>>>> (netvsc_vf_join()). Now, this is still done in
netvsc. Why?
>>>>>>>>
>>>>>>>> This should be part of the common
"failover" code.
>>>>>> Based on Stephen's feedback on earlier patches, i
tried to minimize the changes to
>>>>>> netvsc and only commonize the notifier and the main
event handler routine.
>>>>>> Another complication is that netvsc does part of
registration in a delayed workqueue.
>>>>> :( This kind of degrades the whole efford of having single
solution
>>>>> in "failover" module. I think that common parts,
as
>>>>> netdev_rx_handler_register() and others certainly is should
be inside
>>>>> the common module. This is not a good time to minimize
changes. Let's do
>>>>> the thing properly and fix the netvsc mess now.
>>>>>
>>>>>
>>>>>> It should be possible to move some of the code from
net_failover.c to generic
>>>>>> failover.c in future if Stephen is ok with it.
>>>>>>
>>>>>>
>>>>>>> Also note that in the current patchset you use
IFF_FAILOVER flag for
>>>>>>> master, yet for the slave you use IFF_SLAVE. That
is wrong.
>>>>>>> IFF_FAILOVER_SLAVE should be used.
>>>>>> Not sure which code you are referring to.? I only set
IFF_FAILOVER_SLAVE
>>>>>> in patch 3.
>>>>> The existing netvsc driver.
>>>> We really can't change netvsc's flags now, even if
it's interface is
>>>> messy, it's being used in the field. We can add a flag that
makes netvsc
>>>> behave differently, and if this flag also allows enhanced
functionality
>>>> userspace will gradually switch.
>>> Okay, although in this case, it really does not make much sense, so
be
>>> it. Leave the netvsc set the ->priv flag to IFF_SLAVE as it is
doing
>>> now. (This once-wrong-forever-wrong policy is flustrating me).
>>>
>>> But since this patchset introduces private flag IFF_FAILOVER and
>>> IFF_FAILOVER_SLAVE, and we set IFF_FAILOVER to the netvsc netdev
>>> instance, we should also set IFF_FAILOVER_SLAVE to the enslaved VF
>>> netdevice to get at least some consistency between virtio_net and
>>> netvsc.
>> OK. I can make this change to set/unset IFF_FAILOVER_SLAVE in the
netvsc
>> register/unregister routines so that it is consistent with virtio_net.
>>
>> Based on your discussion with mst, i think we can even remove IFF_SLAVE
>> setting on netvsc as it should not impact userspace.  If Stephen is OK
>> we can make this change too.
>>
>> Do you see any other items that need to be resolved for this series to
go in
>> this merge window?
> As I wrote previously, the common code including rx_handler registration
> and setting of flags and master link should be done in a common code,
> moved away from netvsc code.
>
This requires re-introducing the 2 additional ops pre_register and
pre_unregister
that i removed in the last couple of revisions to minimize netvsc changes and
the
indirect calls that Stephen expressed some concern.

But, as these calls don't happen in hot path, i guess it should not be a big
issue and the right way to go.
Will submit a v12 with these updates.