Willem de Bruijn
2023-Oct-09 10:06 UTC
[RFC PATCH 5/7] tun: Introduce virtio-net hashing feature
On Mon, Oct 9, 2023 at 3:02?AM Akihiko Odaki <akihiko.odaki at daynix.com> wrote:> > On 2023/10/09 18:57, Willem de Bruijn wrote: > > On Mon, Oct 9, 2023 at 3:57?AM Akihiko Odaki <akihiko.odaki at daynix.com> wrote: > >> > >> On 2023/10/09 17:04, Willem de Bruijn wrote: > >>> On Sun, Oct 8, 2023 at 3:46?PM Akihiko Odaki <akihiko.odaki at daynix.com> wrote: > >>>> > >>>> On 2023/10/09 5:08, Willem de Bruijn wrote: > >>>>> On Sun, Oct 8, 2023 at 10:04?PM Akihiko Odaki <akihiko.odaki at daynix.com> wrote: > >>>>>> > >>>>>> On 2023/10/09 4:07, Willem de Bruijn wrote: > >>>>>>> On Sun, Oct 8, 2023 at 7:22?AM Akihiko Odaki <akihiko.odaki at daynix.com> wrote: > >>>>>>>> > >>>>>>>> virtio-net have two usage of hashes: one is RSS and another is hash > >>>>>>>> reporting. Conventionally the hash calculation was done by the VMM. > >>>>>>>> However, computing the hash after the queue was chosen defeats the > >>>>>>>> purpose of RSS. > >>>>>>>> > >>>>>>>> Another approach is to use eBPF steering program. This approach has > >>>>>>>> another downside: it cannot report the calculated hash due to the > >>>>>>>> restrictive nature of eBPF. > >>>>>>>> > >>>>>>>> Introduce the code to compute hashes to the kernel in order to overcome > >>>>>>>> thse challenges. An alternative solution is to extend the eBPF steering > >>>>>>>> program so that it will be able to report to the userspace, but it makes > >>>>>>>> little sense to allow to implement different hashing algorithms with > >>>>>>>> eBPF since the hash value reported by virtio-net is strictly defined by > >>>>>>>> the specification. > >>>>>>>> > >>>>>>>> The hash value already stored in sk_buff is not used and computed > >>>>>>>> independently since it may have been computed in a way not conformant > >>>>>>>> with the specification. > >>>>>>>> > >>>>>>>> Signed-off-by: Akihiko Odaki <akihiko.odaki at daynix.com> > >>>>>>>> --- > >>>>>>> > >>>>>>>> +static const struct tun_vnet_hash_cap tun_vnet_hash_cap = { > >>>>>>>> + .max_indirection_table_length > >>>>>>>> + TUN_VNET_HASH_MAX_INDIRECTION_TABLE_LENGTH, > >>>>>>>> + > >>>>>>>> + .types = VIRTIO_NET_SUPPORTED_HASH_TYPES > >>>>>>>> +}; > >>>>>>> > >>>>>>> No need to have explicit capabilities exchange like this? Tun either > >>>>>>> supports all or none. > >>>>>> > >>>>>> tun does not support VIRTIO_NET_RSS_HASH_TYPE_IP_EX, > >>>>>> VIRTIO_NET_RSS_HASH_TYPE_TCP_EX, and VIRTIO_NET_RSS_HASH_TYPE_UDP_EX. > >>>>>> > >>>>>> It is because the flow dissector does not support IPv6 extensions. The > >>>>>> specification is also vague, and does not tell how many TLVs should be > >>>>>> consumed at most when interpreting destination option header so I chose > >>>>>> to avoid adding code for these hash types to the flow dissector. I doubt > >>>>>> anyone will complain about it since nobody complains for Linux. > >>>>>> > >>>>>> I'm also adding this so that we can extend it later. > >>>>>> max_indirection_table_length may grow for systems with 128+ CPUs, or > >>>>>> types may have other bits for new protocols in the future. > >>>>>> > >>>>>>> > >>>>>>>> case TUNSETSTEERINGEBPF: > >>>>>>>> - ret = tun_set_ebpf(tun, &tun->steering_prog, argp); > >>>>>>>> + bpf_ret = tun_set_ebpf(tun, &tun->steering_prog, argp); > >>>>>>>> + if (IS_ERR(bpf_ret)) > >>>>>>>> + ret = PTR_ERR(bpf_ret); > >>>>>>>> + else if (bpf_ret) > >>>>>>>> + tun->vnet_hash.flags &= ~TUN_VNET_HASH_RSS; > >>>>>>> > >>>>>>> Don't make one feature disable another. > >>>>>>> > >>>>>>> TUNSETSTEERINGEBPF and TUNSETVNETHASH are mutually exclusive > >>>>>>> functions. If one is enabled the other call should fail, with EBUSY > >>>>>>> for instance. > >>>>>>> > >>>>>>>> + case TUNSETVNETHASH: > >>>>>>>> + len = sizeof(vnet_hash); > >>>>>>>> + if (copy_from_user(&vnet_hash, argp, len)) { > >>>>>>>> + ret = -EFAULT; > >>>>>>>> + break; > >>>>>>>> + } > >>>>>>>> + > >>>>>>>> + if (((vnet_hash.flags & TUN_VNET_HASH_REPORT) && > >>>>>>>> + (tun->vnet_hdr_sz < sizeof(struct virtio_net_hdr_v1_hash) || > >>>>>>>> + !tun_is_little_endian(tun))) || > >>>>>>>> + vnet_hash.indirection_table_mask >> >>>>>>>> + TUN_VNET_HASH_MAX_INDIRECTION_TABLE_LENGTH) { > >>>>>>>> + ret = -EINVAL; > >>>>>>>> + break; > >>>>>>>> + } > >>>>>>>> + > >>>>>>>> + argp = (u8 __user *)argp + len; > >>>>>>>> + len = (vnet_hash.indirection_table_mask + 1) * 2; > >>>>>>>> + if (copy_from_user(vnet_hash_indirection_table, argp, len)) { > >>>>>>>> + ret = -EFAULT; > >>>>>>>> + break; > >>>>>>>> + } > >>>>>>>> + > >>>>>>>> + argp = (u8 __user *)argp + len; > >>>>>>>> + len = virtio_net_hash_key_length(vnet_hash.types); > >>>>>>>> + > >>>>>>>> + if (copy_from_user(vnet_hash_key, argp, len)) { > >>>>>>>> + ret = -EFAULT; > >>>>>>>> + break; > >>>>>>>> + } > >>>>>>> > >>>>>>> Probably easier and less error-prone to define a fixed size control > >>>>>>> struct with the max indirection table size. > >>>>>> > >>>>>> I made its size variable because the indirection table and key may grow > >>>>>> in the future as I wrote above. > >>>>>> > >>>>>>> > >>>>>>> Btw: please trim the CC: list considerably on future patches. > >>>>>> > >>>>>> I'll do so in the next version with the TUNSETSTEERINGEBPF change you > >>>>>> proposed. > >>>>> > >>>>> To be clear: please don't just resubmit with that one change. > >>>>> > >>>>> The skb and cb issues are quite fundamental issues that need to be resolved. > >>>>> > >>>>> I'd like to understand why adjusting the existing BPF feature for this > >>>>> exact purpose cannot be amended to return the key it produced. > >>>> > >>>> eBPF steering program is not designed for this particular problem in my > >>>> understanding. It was introduced to derive hash values with an > >>>> understanding of application-specific semantics of packets instead of > >>>> generic IP/TCP/UDP semantics. > >>>> > >>>> This problem is rather different in terms that the hash derivation is > >>>> strictly defined by virtio-net. I don't think it makes sense to > >>>> introduce the complexity of BPF when you always run the same code. > >>>> > >>>> It can utilize the existing flow dissector and also make it easier to > >>>> use for the userspace by implementing this in the kernel. > >>> > >>> Ok. There does appear to be overlap in functionality. But it might be > >>> easier to deploy to just have standard Toeplitz available without > >>> having to compile and load an eBPF program. > >>> > >>> As for the sk_buff and cb[] changes. The first is really not needed. > >>> sk_buff simply would not scale if every edge case needs a few bits. > >> > >> An alternative is to move the bit to cb[] and clear it for every code > >> paths that lead to ndo_start_xmit(), but I'm worried that it is error-prone. > >> > >> I think we can put the bit in sk_buff for now. We can implement the > >> alternative when we are short of bits. > > > > I disagree. sk_buff fields add a cost to every code path. They cannot > > be added for every edge case. > > It only takes an unused bit and does not grow the sk_buff size so I > think it has practically no cost for now.The problem is that that thinking leads to death by a thousand cuts. "for now" forces the cost of having to think hard how to avoid growing sk_buff onto the next person. Let's do it right from the start.