Linux Virtualization - Apr 2018 - [RFC PATCH net-next v5 3/4] virtio_net: Extend virtio to use VF datapath when available

Wed, Apr 11, 2018 at 08:24:43AM CEST, sridhar.samudrala at intel.com
wrote:
>On 4/10/2018 11:03 PM, Jiri Pirko wrote:
>> Tue, Apr 10, 2018 at 05:59:02PM CEST, sridhar.samudrala at intel.com
wrote:
>> > On 4/10/2018 8:43 AM, Jiri Pirko wrote:
>> > > Tue, Apr 10, 2018 at 05:27:48PM CEST, sridhar.samudrala at
intel.com wrote:
>> > > > On 4/10/2018 8:22 AM, Jiri Pirko wrote:
>> > > > > Tue, Apr 10, 2018 at 05:13:40PM CEST,
sridhar.samudrala at intel.com wrote:
>> > > > > > On 4/10/2018 3:55 AM, Jiri Pirko wrote:
>> > > > > > > Mon, Apr 09, 2018 at 08:47:06PM CEST,
sridhar.samudrala at intel.com wrote:
>> > > > > > > > On 4/9/2018 1:07 AM, Jiri Pirko
wrote:
>> > > > > > > > > Sat, Apr 07, 2018 at 12:59:14AM
CEST, sridhar.samudrala at intel.com wrote:
>> > > > > > > > > > On 4/6/2018 5:48 AM, Jiri
Pirko wrote:
>> > > > > > > > > > > Thu, Apr 05, 2018 at
11:08:22PM CEST, sridhar.samudrala at intel.com wrote:
>> > > > > > > > > [...]
>> > > > > > > > > 
>> > > > > > > > > > > > +static int
virtnet_bypass_join_child(struct net_device *bypass_netdev,
>> > > > > > > > > > > > +				     struct
net_device *child_netdev)
>> > > > > > > > > > > > +{
>> > > > > > > > > > > > +	struct
virtnet_bypass_info *vbi;
>> > > > > > > > > > > > +	bool backup;
>> > > > > > > > > > > > +
>> > > > > > > > > > > > +	vbi =
netdev_priv(bypass_netdev);
>> > > > > > > > > > > > +	backup =
(child_netdev->dev.parent == bypass_netdev->dev.parent);
>> > > > > > > > > > > > +	if (backup ?
rtnl_dereference(vbi->backup_netdev) :
>> > > > > > > > > > > > +		
rtnl_dereference(vbi->active_netdev)) {
>> > > > > > > > > > > > +	
netdev_info(bypass_netdev,
>> > > > > > > > > > > > +			    "%s
attempting to join bypass dev when %s already present\n",
>> > > > > > > > > > > > +			   
child_netdev->name, backup ? "backup" : "active");
>> > > > > > > > > > > Bypass module should
check if there is already some other netdev
>> > > > > > > > > > > enslaved and refuse
right there.
>> > > > > > > > > > This will work for
virtio-net with 3 netdev model, but this check has to be done by netvsc
>> > > > > > > > > > as its bypass_netdev is
same as the backup_netdev.
>> > > > > > > > > > Will add a flag while
registering with the bypass module to indicate if the driver is doing
>> > > > > > > > > > a 2 netdev or 3 netdev
model and based on that flag this check can be done in bypass module
>> > > > > > > > > > for 3 netdev scenario.
>> > > > > > > > > Just let me undestand it
clearly. What I expect the difference would be
>> > > > > > > > > between 2netdev and3 netdev
model is this:
>> > > > > > > > > 2netdev:
>> > > > > > > > >        bypass_master
>> > > > > > > > >           /
>> > > > > > > > >          /
>> > > > > > > > > VF_slave
>> > > > > > > > > 
>> > > > > > > > > 3netdev:
>> > > > > > > > >        bypass_master
>> > > > > > > > >           /     \
>> > > > > > > > >          /       \
>> > > > > > > > > VF_slave   backup_slave
>> > > > > > > > > 
>> > > > > > > > > Is that correct? If not, how
does it look like?
>> > > > > > > > > 
>> > > > > > > > > 
>> > > > > > > > Looks correct.
>> > > > > > > > VF_slave and backup_slave are the
original netdevs and are present in both the models.
>> > > > > > > > In the 3 netdev model, bypass_master
netdev is created and VF_slave and backup_slave are
>> > > > > > > > marked as the 2 slaves of this new
netdev.
>> > > > > > > You say it looks correct and in another
sentence you provide completely
>> > > > > > > different description. Could you please
look again?
>> > > > > > > 
>> > > > > > To be exact, 2 netdev model with netvsc looks
like this.
>> > > > > > 
>> > > > > >       netvsc_netdev
>> > > > > >         /
>> > > > > >        /
>> > > > > > VF_slave
>> > > > > > 
>> > > > > > With virtio_net, 3 netdev model
>> > > > > > 
>> > > > > >     bypass_netdev
>> > > > > >         /     \
>> > > > > >        /       \
>> > > > > > VF_slave   virtio_net netdev
>> > > > > Could you also mark the original netdev which is
there now? is it
>> > > > > bypass_netdev or virtio_net_netdev ?
>> > > > bypass_netdev
>> > > >       /     \
>> > > >      /       \
>> > > > VF_slave   virtio_net netdev (original)
>> > > That does not make sense.
>> > > 1) You diverge from the behaviour of the netvsc, where the
original
>> > >      netdev is a master of the VF
>> > > 2) If the original netdev is a slave, you cannot have any IP
address
>> > >      configured on it (well you could, but the rx_handler
would eat every
>> > >      incoming packet). So you will break the user bacause he
would have to
>> > >      move the configuration to the new master device.
>> > > This only makes sense that the original netdev becomes the
master for both
>> > > netvsc and virtio_net.
>> > Forgot to mention that bypass_netdev takes over the name of the
original
>> > netdev and
>> > virtio_net netdev will get the backup name.
>> What do you mean by "name"?
>
>bypass_netdev also is associated with the same pci device as the original
virtio_net
>netdev via SET_NETDEV_DEV().? Also, we added ndo_get_phys_port_name() to
virtio_net
>that will return _bkup when BACKUP feature is enabled.
Okay.
>
>So for ex: if virtio_net inteface was getting 'ens12' as the name
assigned by udev
>without BACKUP feature,? when BACKUP feature is enabled,? the? bypass_netdev
will be
>named 'ens12' and the original virtio_net will get named as
ens12n_bkup.
Got it.

I don't like the bypass_master to look differently in netvsc and
virtio_net :/ The best would be to convert netvsc to 3 netdev model and
treat them the same. The more I think about it, the more the 2 netdev
model feels wrong.

>
>
>> 
>> > So the userspace network configuration doesn't need to change.
>> > 
>> > 
>

On Wed, Apr 11, 2018 at 10:03:32AM +0200, Jiri Pirko
wrote:
> Wed, Apr 11, 2018 at 08:24:43AM CEST, sridhar.samudrala at intel.com wrote:
> >On 4/10/2018 11:03 PM, Jiri Pirko wrote:
> >> Tue, Apr 10, 2018 at 05:59:02PM CEST, sridhar.samudrala at
intel.com wrote:
> >> > On 4/10/2018 8:43 AM, Jiri Pirko wrote:
> >> > > Tue, Apr 10, 2018 at 05:27:48PM CEST, sridhar.samudrala
at intel.com wrote:
> >> > > > On 4/10/2018 8:22 AM, Jiri Pirko wrote:
> >> > > > > Tue, Apr 10, 2018 at 05:13:40PM CEST,
sridhar.samudrala at intel.com wrote:
> >> > > > > > On 4/10/2018 3:55 AM, Jiri Pirko wrote:
> >> > > > > > > Mon, Apr 09, 2018 at 08:47:06PM
CEST, sridhar.samudrala at intel.com wrote:
> >> > > > > > > > On 4/9/2018 1:07 AM, Jiri Pirko
wrote:
> >> > > > > > > > > Sat, Apr 07, 2018 at
12:59:14AM CEST, sridhar.samudrala at intel.com wrote:
> >> > > > > > > > > > On 4/6/2018 5:48 AM,
Jiri Pirko wrote:
> >> > > > > > > > > > > Thu, Apr 05,
2018 at 11:08:22PM CEST, sridhar.samudrala at intel.com wrote:
> >> > > > > > > > > [...]
> >> > > > > > > > > 
> >> > > > > > > > > > > > +static int
virtnet_bypass_join_child(struct net_device *bypass_netdev,
> >> > > > > > > > > > > > +				    
struct net_device *child_netdev)
> >> > > > > > > > > > > > +{
> >> > > > > > > > > > > > +	struct
virtnet_bypass_info *vbi;
> >> > > > > > > > > > > > +	bool
backup;
> >> > > > > > > > > > > > +
> >> > > > > > > > > > > > +	vbi =
netdev_priv(bypass_netdev);
> >> > > > > > > > > > > > +	backup =
(child_netdev->dev.parent == bypass_netdev->dev.parent);
> >> > > > > > > > > > > > +	if
(backup ? rtnl_dereference(vbi->backup_netdev) :
> >> > > > > > > > > > > > +		
rtnl_dereference(vbi->active_netdev)) {
> >> > > > > > > > > > > > +	
netdev_info(bypass_netdev,
> >> > > > > > > > > > > > +			   
"%s attempting to join bypass dev when %s already present\n",
> >> > > > > > > > > > > > +			   
child_netdev->name, backup ? "backup" : "active");
> >> > > > > > > > > > > Bypass module
should check if there is already some other netdev
> >> > > > > > > > > > > enslaved and
refuse right there.
> >> > > > > > > > > > This will work for
virtio-net with 3 netdev model, but this check has to be done by netvsc
> >> > > > > > > > > > as its bypass_netdev
is same as the backup_netdev.
> >> > > > > > > > > > Will add a flag while
registering with the bypass module to indicate if the driver is doing
> >> > > > > > > > > > a 2 netdev or 3
netdev model and based on that flag this check can be done in bypass module
> >> > > > > > > > > > for 3 netdev
scenario.
> >> > > > > > > > > Just let me undestand it
clearly. What I expect the difference would be
> >> > > > > > > > > between 2netdev and3
netdev model is this:
> >> > > > > > > > > 2netdev:
> >> > > > > > > > >        bypass_master
> >> > > > > > > > >           /
> >> > > > > > > > >          /
> >> > > > > > > > > VF_slave
> >> > > > > > > > > 
> >> > > > > > > > > 3netdev:
> >> > > > > > > > >        bypass_master
> >> > > > > > > > >           /     \
> >> > > > > > > > >          /       \
> >> > > > > > > > > VF_slave   backup_slave
> >> > > > > > > > > 
> >> > > > > > > > > Is that correct? If not,
how does it look like?
> >> > > > > > > > > 
> >> > > > > > > > > 
> >> > > > > > > > Looks correct.
> >> > > > > > > > VF_slave and backup_slave are
the original netdevs and are present in both the models.
> >> > > > > > > > In the 3 netdev model,
bypass_master netdev is created and VF_slave and backup_slave are
> >> > > > > > > > marked as the 2 slaves of this
new netdev.
> >> > > > > > > You say it looks correct and in
another sentence you provide completely
> >> > > > > > > different description. Could you
please look again?
> >> > > > > > > 
> >> > > > > > To be exact, 2 netdev model with netvsc
looks like this.
> >> > > > > > 
> >> > > > > >       netvsc_netdev
> >> > > > > >         /
> >> > > > > >        /
> >> > > > > > VF_slave
> >> > > > > > 
> >> > > > > > With virtio_net, 3 netdev model
> >> > > > > > 
> >> > > > > >     bypass_netdev
> >> > > > > >         /     \
> >> > > > > >        /       \
> >> > > > > > VF_slave   virtio_net netdev
> >> > > > > Could you also mark the original netdev which
is there now? is it
> >> > > > > bypass_netdev or virtio_net_netdev ?
> >> > > > bypass_netdev
> >> > > >       /     \
> >> > > >      /       \
> >> > > > VF_slave   virtio_net netdev (original)
> >> > > That does not make sense.
> >> > > 1) You diverge from the behaviour of the netvsc, where
the original
> >> > >      netdev is a master of the VF
> >> > > 2) If the original netdev is a slave, you cannot have
any IP address
> >> > >      configured on it (well you could, but the
rx_handler would eat every
> >> > >      incoming packet). So you will break the user
bacause he would have to
> >> > >      move the configuration to the new master device.
> >> > > This only makes sense that the original netdev becomes
the master for both
> >> > > netvsc and virtio_net.
> >> > Forgot to mention that bypass_netdev takes over the name of
the original
> >> > netdev and
> >> > virtio_net netdev will get the backup name.
> >> What do you mean by "name"?
> >
> >bypass_netdev also is associated with the same pci device as the
original virtio_net
> >netdev via SET_NETDEV_DEV().? Also, we added ndo_get_phys_port_name()
to virtio_net
> >that will return _bkup when BACKUP feature is enabled.
> 
> Okay.
> 
> >
> >So for ex: if virtio_net inteface was getting 'ens12' as the
name assigned by udev
> >without BACKUP feature,? when BACKUP feature is enabled,? the?
bypass_netdev will be
> >named 'ens12' and the original virtio_net will get named as
ens12n_bkup.
> 
> Got it.
> 
> I don't like the bypass_master to look differently in netvsc and
> virtio_net :/ The best would be to convert netvsc to 3 netdev model and
> treat them the same. The more I think about it, the more the 2 netdev
> model feels wrong.
If you believe that, then this patchset is a step in the right
direction.

With something like this patchset applied, converting netvsc to a 3
device model will presumably be just a flag flip away. Afterwards
we'll be able to drop dead code handling the bypass_master flag.
> 
> >
> >
> >> 
> >> > So the userspace network configuration doesn't need to
change.
> >> > 
> >> > 
> >

Wed, Apr 11, 2018 at 04:45:07PM CEST, mst at redhat.com
wrote:
>On Wed, Apr 11, 2018 at 10:03:32AM +0200, Jiri Pirko wrote:
>> Wed, Apr 11, 2018 at 08:24:43AM CEST, sridhar.samudrala at intel.com
wrote:
>> >On 4/10/2018 11:03 PM, Jiri Pirko wrote:
>> >> Tue, Apr 10, 2018 at 05:59:02PM CEST, sridhar.samudrala at
intel.com wrote:
>> >> > On 4/10/2018 8:43 AM, Jiri Pirko wrote:
>> >> > > Tue, Apr 10, 2018 at 05:27:48PM CEST,
sridhar.samudrala at intel.com wrote:
>> >> > > > On 4/10/2018 8:22 AM, Jiri Pirko wrote:
>> >> > > > > Tue, Apr 10, 2018 at 05:13:40PM CEST,
sridhar.samudrala at intel.com wrote:
>> >> > > > > > On 4/10/2018 3:55 AM, Jiri Pirko
wrote:
>> >> > > > > > > Mon, Apr 09, 2018 at 08:47:06PM
CEST, sridhar.samudrala at intel.com wrote:
>> >> > > > > > > > On 4/9/2018 1:07 AM, Jiri
Pirko wrote:
>> >> > > > > > > > > Sat, Apr 07, 2018 at
12:59:14AM CEST, sridhar.samudrala at intel.com wrote:
>> >> > > > > > > > > > On 4/6/2018 5:48
AM, Jiri Pirko wrote:
>> >> > > > > > > > > > > Thu, Apr 05,
2018 at 11:08:22PM CEST, sridhar.samudrala at intel.com wrote:
>> >> > > > > > > > > [...]
>> >> > > > > > > > > 
>> >> > > > > > > > > > > > +static
int virtnet_bypass_join_child(struct net_device *bypass_netdev,
>> >> > > > > > > > > > > > +				  
struct net_device *child_netdev)
>> >> > > > > > > > > > > > +{
>> >> > > > > > > > > > > > +
struct virtnet_bypass_info *vbi;
>> >> > > > > > > > > > > > +	bool
backup;
>> >> > > > > > > > > > > > +
>> >> > > > > > > > > > > > +	vbi =
netdev_priv(bypass_netdev);
>> >> > > > > > > > > > > > +
backup = (child_netdev->dev.parent == bypass_netdev->dev.parent);
>> >> > > > > > > > > > > > +	if
(backup ? rtnl_dereference(vbi->backup_netdev) :
>> >> > > > > > > > > > > > +		
rtnl_dereference(vbi->active_netdev)) {
>> >> > > > > > > > > > > > +	
netdev_info(bypass_netdev,
>> >> > > > > > > > > > > > +			   
"%s attempting to join bypass dev when %s already present\n",
>> >> > > > > > > > > > > > +			   
child_netdev->name, backup ? "backup" : "active");
>> >> > > > > > > > > > > Bypass
module should check if there is already some other netdev
>> >> > > > > > > > > > > enslaved and
refuse right there.
>> >> > > > > > > > > > This will work
for virtio-net with 3 netdev model, but this check has to be done by netvsc
>> >> > > > > > > > > > as its
bypass_netdev is same as the backup_netdev.
>> >> > > > > > > > > > Will add a flag
while registering with the bypass module to indicate if the driver is doing
>> >> > > > > > > > > > a 2 netdev or 3
netdev model and based on that flag this check can be done in bypass module
>> >> > > > > > > > > > for 3 netdev
scenario.
>> >> > > > > > > > > Just let me undestand
it clearly. What I expect the difference would be
>> >> > > > > > > > > between 2netdev and3
netdev model is this:
>> >> > > > > > > > > 2netdev:
>> >> > > > > > > > >        bypass_master
>> >> > > > > > > > >           /
>> >> > > > > > > > >          /
>> >> > > > > > > > > VF_slave
>> >> > > > > > > > > 
>> >> > > > > > > > > 3netdev:
>> >> > > > > > > > >        bypass_master
>> >> > > > > > > > >           /     \
>> >> > > > > > > > >          /       \
>> >> > > > > > > > > VF_slave  
backup_slave
>> >> > > > > > > > > 
>> >> > > > > > > > > Is that correct? If
not, how does it look like?
>> >> > > > > > > > > 
>> >> > > > > > > > > 
>> >> > > > > > > > Looks correct.
>> >> > > > > > > > VF_slave and backup_slave
are the original netdevs and are present in both the models.
>> >> > > > > > > > In the 3 netdev model,
bypass_master netdev is created and VF_slave and backup_slave are
>> >> > > > > > > > marked as the 2 slaves of
this new netdev.
>> >> > > > > > > You say it looks correct and in
another sentence you provide completely
>> >> > > > > > > different description. Could you
please look again?
>> >> > > > > > > 
>> >> > > > > > To be exact, 2 netdev model with
netvsc looks like this.
>> >> > > > > > 
>> >> > > > > >       netvsc_netdev
>> >> > > > > >         /
>> >> > > > > >        /
>> >> > > > > > VF_slave
>> >> > > > > > 
>> >> > > > > > With virtio_net, 3 netdev model
>> >> > > > > > 
>> >> > > > > >     bypass_netdev
>> >> > > > > >         /     \
>> >> > > > > >        /       \
>> >> > > > > > VF_slave   virtio_net netdev
>> >> > > > > Could you also mark the original netdev
which is there now? is it
>> >> > > > > bypass_netdev or virtio_net_netdev ?
>> >> > > > bypass_netdev
>> >> > > >       /     \
>> >> > > >      /       \
>> >> > > > VF_slave   virtio_net netdev (original)
>> >> > > That does not make sense.
>> >> > > 1) You diverge from the behaviour of the netvsc,
where the original
>> >> > >      netdev is a master of the VF
>> >> > > 2) If the original netdev is a slave, you cannot
have any IP address
>> >> > >      configured on it (well you could, but the
rx_handler would eat every
>> >> > >      incoming packet). So you will break the user
bacause he would have to
>> >> > >      move the configuration to the new master
device.
>> >> > > This only makes sense that the original netdev
becomes the master for both
>> >> > > netvsc and virtio_net.
>> >> > Forgot to mention that bypass_netdev takes over the name
of the original
>> >> > netdev and
>> >> > virtio_net netdev will get the backup name.
>> >> What do you mean by "name"?
>> >
>> >bypass_netdev also is associated with the same pci device as the
original virtio_net
>> >netdev via SET_NETDEV_DEV().? Also, we added
ndo_get_phys_port_name() to virtio_net
>> >that will return _bkup when BACKUP feature is enabled.
>> 
>> Okay.
>> 
>> >
>> >So for ex: if virtio_net inteface was getting 'ens12' as
the name assigned by udev
>> >without BACKUP feature,? when BACKUP feature is enabled,? the?
bypass_netdev will be
>> >named 'ens12' and the original virtio_net will get named as
ens12n_bkup.
>> 
>> Got it.
>> 
>> I don't like the bypass_master to look differently in netvsc and
>> virtio_net :/ The best would be to convert netvsc to 3 netdev model and
>> treat them the same. The more I think about it, the more the 2 netdev
>> model feels wrong.
>
>If you believe that, then this patchset is a step in the right
>direction.
>
>With something like this patchset applied, converting netvsc to a 3
>device model will presumably be just a flag flip away. Afterwards
If done properly. Yes.
>we'll be able to drop dead code handling the bypass_master flag.
>
>> 
>> >
>> >
>> >> 
>> >> > So the userspace network configuration doesn't need
to change.
>> >> > 
>> >> > 
>> >