libvirt users - Oct 2017 - Re: Need to increase the rx and tx buffer size of my interface

Dear Users/Developers,

I am using a KVM Ubuntu VM as a degrader to apply specific delays to
incoming packets. As the delay for my packets can be higher than 7.5
seconds, there is not enough buffer on my interface to buffer all the
packets. Therefore those overflowing packets are dropped in the machine and
not forwarded.

When I tried to use the command  ethtool -G ens8 rx 512 to increase the
buffer size, I get the following error.

Cannot set device ring parameters: Operation not permitted

I have kept the VM xml files as specified in the link :
https://libvirt.org/formatdomain.html. The value that I kept in my xml file
is as follows.

<interface type='direct'>
      <mac address='52:54:00:72:f9:eb'/>
      <source dev='enp7s0f0' mode='vepa'/>
      <model type='virtio'/>
      <driver name='vhost' queues='5'
rx_queue_size='512'
tx_queue_size='512'>
      <host csum='off' gso='off' tso4='off'
tso6='off' ecn='off' ufo='off'
mrg_rxbuf='off'/>
      <guest csum='off' tso4='off' tso6='off'
ecn='off' ufo='off'/>
      </driver>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x08'
function='0x0'/>
    </interface>
    <interface type='direct'>
      <mac address='52:54:00:00:b5:99'/>
      <source dev='enp7s0f1' mode='vepa'/>
      <model type='virtio'/>
      <driver name='vhost' queues='5'
rx_queue_size='512'
tx_queue_size='512'>
      <host csum='off' gso='off' tso4='off'
tso6='off' ecn='off' ufo='off'
mrg_rxbuf='off'/>
      <guest csum='off' tso4='off' tso6='off'
ecn='off' ufo='off'/>
      </driver>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x09'
function='0x0'/>

After I saved the xml and redefined the machine and started it up, I
checked to see if the buffers are increased. I still see that the buffers
are as follows.

Ring parameters for ens9:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: *256*
RX Mini: 0
RX Jumbo: 0
TX: *256*


How can I increase the buffer on my KVM VM?

Best Regards,
Ashish Kurian

On 10/25/2017 01:53 PM, Ashish Kurian wrote:
> Dear Users/Developers,
> 
> I am using a KVM Ubuntu VM as a degrader to apply specific delays to
> incoming packets. As the delay for my packets can be higher than 7.5
> seconds, there is not enough buffer on my interface to buffer all the
> packets. Therefore those overflowing packets are dropped in the machine and
> not forwarded.
> 
> When I tried to use the command  ethtool -G ens8 rx 512 to increase the
> buffer size, I get the following error.
> 
> Cannot set device ring parameters: Operation not permitted
> 
> I have kept the VM xml files as specified in the link :
> https://libvirt.org/formatdomain.html. The value that I kept in my xml file
> is as follows.
> 
> <interface type='direct'>
>       <mac address='52:54:00:72:f9:eb'/>
>       <source dev='enp7s0f0' mode='vepa'/>
>       <model type='virtio'/>
>       <driver name='vhost' queues='5'
rx_queue_size='512'
> tx_queue_size='512'>
>       <host csum='off' gso='off' tso4='off'
tso6='off' ecn='off' ufo='off'
> mrg_rxbuf='off'/>
>       <guest csum='off' tso4='off' tso6='off'
ecn='off' ufo='off'/>
>       </driver>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x08'
> function='0x0'/>
>     </interface>
>     <interface type='direct'>
>       <mac address='52:54:00:00:b5:99'/>
>       <source dev='enp7s0f1' mode='vepa'/>
>       <model type='virtio'/>
>       <driver name='vhost' queues='5'
rx_queue_size='512'
> tx_queue_size='512'>
>       <host csum='off' gso='off' tso4='off'
tso6='off' ecn='off' ufo='off'
> mrg_rxbuf='off'/>
>       <guest csum='off' tso4='off' tso6='off'
ecn='off' ufo='off'/>
>       </driver>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x09'
> function='0x0'/>
So what does the qemu command line look like? You can find it in either
libvirtd log or domain log.

http://wiki.libvirt.org/page/DebugLogs

Michal

Hi Michal,

What I found was that when I restarted the machine and did a virsh edit
command to see the xml config, I see that it is was not actually changed.
This suggests why I saw 256 again after restarting.

So now I tried again to edit the xml via virsh edit command and used the
following to set the parameters.

<driver name='qemu' txmode='iothread' ioeventfd='on'
event_idx='off'
queues='1' rx_queue_size='512' tx_queue_size='512'>
</driver>

It was not accepted and I got the error saying :


error: XML document failed to validate against schema: Unable to validate
doc against /usr/share/libvirt/schemas/domain.rng
Extra element devices in interleave
Element domain failed to validate content

What does this imply? I have two more other interfaces and do I have to the
same to them also?

Btw, there are now logs generated now in the domain log or libvirtd log





Best Regards,
Ashish Kurian

On Wed, Oct 25, 2017 at 2:50 PM, Michal Privoznik <mprivozn@redhat.com>
wrote:
> On 10/25/2017 01:53 PM, Ashish Kurian wrote:
> > Dear Users/Developers,
> >
> > I am using a KVM Ubuntu VM as a degrader to apply specific delays to
> > incoming packets. As the delay for my packets can be higher than 7.5
> > seconds, there is not enough buffer on my interface to buffer all the
> > packets. Therefore those overflowing packets are dropped in the
machine
> and
> > not forwarded.
> >
> > When I tried to use the command  ethtool -G ens8 rx 512 to increase
the
> > buffer size, I get the following error.
> >
> > Cannot set device ring parameters: Operation not permitted
> >
> > I have kept the VM xml files as specified in the link :
> > https://libvirt.org/formatdomain.html. The value that I kept in my xml
> file
> > is as follows.
> >
> > <interface type='direct'>
> >       <mac address='52:54:00:72:f9:eb'/>
> >       <source dev='enp7s0f0' mode='vepa'/>
> >       <model type='virtio'/>
> >       <driver name='vhost' queues='5'
rx_queue_size='512'
> > tx_queue_size='512'>
> >       <host csum='off' gso='off' tso4='off'
tso6='off' ecn='off'
> ufo='off'
> > mrg_rxbuf='off'/>
> >       <guest csum='off' tso4='off'
tso6='off' ecn='off' ufo='off'/>
> >       </driver>
> >       <address type='pci' domain='0x0000'
bus='0x00' slot='0x08'
> > function='0x0'/>
> >     </interface>
> >     <interface type='direct'>
> >       <mac address='52:54:00:00:b5:99'/>
> >       <source dev='enp7s0f1' mode='vepa'/>
> >       <model type='virtio'/>
> >       <driver name='vhost' queues='5'
rx_queue_size='512'
> > tx_queue_size='512'>
> >       <host csum='off' gso='off' tso4='off'
tso6='off' ecn='off'
> ufo='off'
> > mrg_rxbuf='off'/>
> >       <guest csum='off' tso4='off'
tso6='off' ecn='off' ufo='off'/>
> >       </driver>
> >       <address type='pci' domain='0x0000'
bus='0x00' slot='0x09'
> > function='0x0'/>
>
> So what does the qemu command line look like? You can find it in either
> libvirtd log or domain log.
>
> http://wiki.libvirt.org/page/DebugLogs
>
> Michal
>