Stefano Garzarella
2022-Mar-02 16:19 UTC
[virtio-comment] [PATCH v5 2/2] virtio-vsock: add mergeable buffer feature bit
On Thu, Feb 24, 2022 at 10:15:47PM +0000, beshleman.devbox at gmail.com wrote:>From: Jiang Wang <jiang.wang at bytedance.com> > >Add support for mergeable buffers for virtio-vsock. Mergeable buffers >allow individual large packets to be spread across multiple buffers >while still using only a single packet header. This avoids >artificially restraining packet size to the size of a single >buffer and offers a performant fragmentation/defragmentation >scheme.We need this new functionality because we want to fragment a datagram packet over multiple buffers, right? This reminded me that we don't have a maximum size for now, in this case what would it be? Maybe it would be helpful to define it as Laura is planning to do here: https://lists.oasis-open.org/archives/virtio-comment/202202/msg00053.html> >Signed-off-by: Jiang Wang <jiang.wang at bytedance.com> >Signed-off-by: Bobby Eshleman <bobby.eshleman at bytedance.com> >--- > virtio-vsock.tex | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 76 insertions(+) > >diff --git a/virtio-vsock.tex b/virtio-vsock.tex >index 1a66a1b..bf44d5d 100644 >--- a/virtio-vsock.tex >+++ b/virtio-vsock.tex >@@ -39,6 +39,7 @@ \subsection{Feature bits}\label{sec:Device Types / Socket Device / Feature bits} > \item[VIRTIO_VSOCK_F_STREAM (0)] stream socket type is supported. > \item[VIRTIO_VSOCK_F_SEQPACKET (1)] seqpacket socket type is supported. > \item[VIRTIO_VSOCK_F_DGRAM (2)] datagram socket type is supported. >+\item[VIRTIO_VSOCK_F_MRG_RXBUF (3)] driver can merge receive buffers. > \end{description} > > \subsection{Device configuration layout}\label{sec:Device Types / Socket Device / Device configuration layout} >@@ -87,6 +88,8 @@ \subsection{Device Operation}\label{sec:Device Types / Socket Device / Device Op > > Packets transmitted or received contain a header before the payload: > >+If feature VIRTIO_VSOCK_F_MRG_RXBUF is not negotiated, use the following header. >+ > \begin{lstlisting} > struct virtio_vsock_hdr { > le64 src_cid; >@@ -102,6 +105,15 @@ \subsection{Device Operation}\label{sec:Device Types / Socket Device / Device Op > }; > \end{lstlisting} > >+If feature VIRTIO_VSOCK_F_MRG_RXBUF is negotiated, use the following header. >+\begin{lstlisting} >+struct virtio_vsock_hdr_mrg_rxbuf { >+ struct virtio_vsock_hdr hdr; >+ le16 num_buffers; >+}; >+\end{lstlisting} >+ >+ > The upper 32 bits of src_cid and dst_cid are reserved and zeroed. > > Most packets simply transfer data but control packets are also used for >@@ -207,6 +219,25 @@ \subsubsection{Buffer Space Management}\label{sec:Device Types / Socket Device / > virtqueue is full. For receivers, the packet is dropped if there is no space > in the receive buffer. > >+\drivernormative{\paragraph}{Device Operation: Buffer Space Management}{Device Types / Socket Device / Device Operation / Setting Up Receive Buffers} >+\begin{itemize} >+\item If VIRTIO_VSOCK_F_MRG_RXBUF is not negotiated, the driver SHOULD populate the datagram rx queue >+ with buffers of at least 4096 bytes. >+\item If VIRTIO_VSOCK_F_MRG_RXBUF is negotiated, each buffer MUST be at >+ least the size of the struct virtio_vsock_hdr_mgr_rxbuf. >+\end{itemize} >+ >+\begin{note} >+Each buffer may be split across multiple descriptor elements. >+\end{note} >+ >+\devicenormative{\paragraph}{Device Operation: Buffer Space Management}{Device Types / Socket Device / Device Operation / Setting Up Receive Buffers} >+The device MUST set \field{num_buffers} to the number of descriptors used when >+transmitting the packet. >+ >+The device MUST use only a single descriptor if VIRTIO_VSOCK_F_MRG_RXBUF >+is not negotiated. >+ > \drivernormative{\paragraph}{Device Operation: Buffer Space Management}{Device Types / Socket Device / Device Operation / Buffer Space Management} > For stream sockets, VIRTIO_VSOCK_OP_RW data packets MUST only be transmitted when the peer has > sufficient free buffer space for the payload. For dgram sockets, VIRTIO_VSOCK_OP_RW data packets >@@ -229,6 +260,7 @@ \subsubsection{Receive and Transmit}\label{sec:Device Types / Socket Device / De > The driver queues outgoing packets on the tx virtqueue and allocates incoming packet > receive buffers on the rx virtqueue. Packets are of the following form: > >+If VIRTIO_VSOCK_F_MRG_RXBUF is not negotiated, use the following. > \begin{lstlisting} > struct virtio_vsock_packet { > struct virtio_vsock_hdr hdr; >@@ -236,11 +268,42 @@ \subsubsection{Receive and >Transmit}\label{sec:Device Types / Socket Device / De > }; > \end{lstlisting} > >+If VIRTIO_VSOCK_F_MRG_RXBUF is negotiated, use the following form: >+\begin{lstlisting} >+struct virtio_vsock_packet_mrg_rxbuf { >+ struct virtio_vsock_hdr_mrg_rxbuf hdr; >+ u8 data[]; >+}; >+\end{lstlisting} >+ >+ > Virtqueue buffers for outgoing packets are read-only. Virtqueue buffers for > incoming packets are write-only. > > When transmitting packets to the device, \field{num_buffers} is not used. > >+\begin{enumerate} >+\item \field{num_buffers} indicates how many descriptors >+ this packet is spread over (including this one). >+ This is valid only if VIRTIO_VSOCK_F_MRG_RXBUF is negotiated. >+ This allows receipt of large packets without having to allocate large >+ buffers: a packet that does not fit in a single buffer can flow >+ over to the next buffer, and so on. In this case, there will be >+ at least \field{num_buffers} used buffers in the virtqueue, and the device >+ chains them together to form a single packet in a way similar to >+ how it would store it in a single buffer spread over multiple >+ descriptors. >+ The other buffers will not begin with a struct virtio_vsock_hdr. >+ >+ If VIRTIO_VSOCK_F_MRG_RXBUF is not negotiated, then only one >+ descriptor is used. >+ >+\item If >+ \field{num_buffers} is one, then the entire packet will be >+ contained within this buffer, immediately following the struct >+ virtio_vsock_hdr.^ Should it be virtio_vsock_hdr_mrg_rxbuf?>+\end{enumerate} >+ > \drivernormative{\paragraph}{Device Operation: Receive and Transmit}{Device Types / Socket Device / Device Operation / Receive and Transmit} > > The \field{guest_cid} configuration field MUST be used as the source CID when >@@ -256,6 +319,19 @@ \subsubsection{Receive and Transmit}\label{sec:Device Types / Socket Device / De > A VIRTIO_VSOCK_OP_RST reply MUST be sent if a packet is received with an > unknown \field{type} value. > >+If VIRTIO_VSOCK_F_MRG_RXBUF has been negotiated, the device MUST set >+\field{num_buffers} to indicate the number of buffers >+the packet (including the header) is spread over. >+ >+If a receive packet is spread over multiple buffers, the device >+MUST use all buffers but the last (i.e. the first $\field{num_buffers} - >+1$ buffers) completely up to the full length of each buffer >+supplied by the driver. >+ >+The device MUST use all buffers used by a single receive >+packet together, such that at least \field{num_buffers} are >+observed by driver as used. >+ > \subsubsection{Stream Sockets}\label{sec:Device Types / Socket Device / Device Operation / Stream Sockets} > > Connections are established by sending a VIRTIO_VSOCK_OP_REQUEST packet. If a >-- >2.11.0 > > >This publicly archived list offers a means to provide input to the >OASIS Virtual I/O Device (VIRTIO) TC. > >In order to verify user consent to the Feedback License terms and >to minimize spam in the list archive, subscription is required >before posting. > >Subscribe: virtio-comment-subscribe at lists.oasis-open.org >Unsubscribe: virtio-comment-unsubscribe at lists.oasis-open.org >List help: virtio-comment-help at lists.oasis-open.org >List archive: https://lists.oasis-open.org/archives/virtio-comment/ >Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf >List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists >Committee: https://www.oasis-open.org/committees/virtio/ >Join OASIS: https://www.oasis-open.org/join/ >