Hi, Seeing a drop-off in iops when more vcpu''s are added:- 3.8.2 kernel/xen-4.2.1/single domU/LVM backend/8GB RAM domU/2GB RAM dom0 dom0_max_vcpus=2 dom0_vcpus_pin domU 8 cores fio result 145k iops domU 10 cores fio result 99k iops domU 12 cores fio result 89k iops domU 14 cores fio result 81k iops ioping . -c 3 4096 bytes from . (ext4 /dev/xvda1): request=1 time=0.1 ms 4096 bytes from . (ext4 /dev/xvda1): request=2 time=0.7 ms 4096 bytes from . (ext4 /dev/xvda1): request=3 time=0.8 ms --- . (ext4 /dev/xvda1) ioping statistics --- 3 requests completed in 2002.0 ms, 1836 iops, 7.2 mb/s min/avg/max/mdev = 0.1/0.5/0.8/0.3 ms The initial ioping response is good, then a lot of latency with later ones. Any ideas ? Thanks John
On Mon, Mar 18, 2013 at 11:23:00PM +0000, John Morrison wrote:> Hi, > > Seeing a drop-off in iops when more vcpu''s are added:- > > 3.8.2 kernel/xen-4.2.1/single domU/LVM backend/8GB RAM domU/2GB RAM dom0 > > dom0_max_vcpus=2 dom0_vcpus_pin > > domU 8 cores fio result 145k iops > domU 10 cores fio result 99k iops > domU 12 cores fio result 89k iops > domU 14 cores fio result 81k iops > > ioping . -c 3 > 4096 bytes from . (ext4 /dev/xvda1): request=1 time=0.1 ms > 4096 bytes from . (ext4 /dev/xvda1): request=2 time=0.7 ms > 4096 bytes from . (ext4 /dev/xvda1): request=3 time=0.8 ms > > --- . (ext4 /dev/xvda1) ioping statistics --- > 3 requests completed in 2002.0 ms, 1836 iops, 7.2 mb/s > min/avg/max/mdev = 0.1/0.5/0.8/0.3 ms > > The initial ioping response is good, then a lot of latency with later ones. Any ideas ?Is your guest compiled with PARAVIRT_SPINLOCK and are you using a PV or PVHVM guest?> > Thanks > > John > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >
Hi, Yes PARAVIRT_SPINLOCK and its a PV guest. iops are consistent now as I had a typo in the config file (cpus=) ioping still showing first 0.1s the rest 0.7/0.8ms Thanks John On 19 Mar 2013, at 12:58, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:> On Mon, Mar 18, 2013 at 11:23:00PM +0000, John Morrison wrote: >> Hi, >> >> Seeing a drop-off in iops when more vcpu''s are added:- >> >> 3.8.2 kernel/xen-4.2.1/single domU/LVM backend/8GB RAM domU/2GB RAM dom0 >> >> dom0_max_vcpus=2 dom0_vcpus_pin >> >> domU 8 cores fio result 145k iops >> domU 10 cores fio result 99k iops >> domU 12 cores fio result 89k iops >> domU 14 cores fio result 81k iops >> >> ioping . -c 3 >> 4096 bytes from . (ext4 /dev/xvda1): request=1 time=0.1 ms >> 4096 bytes from . (ext4 /dev/xvda1): request=2 time=0.7 ms >> 4096 bytes from . (ext4 /dev/xvda1): request=3 time=0.8 ms >> >> --- . (ext4 /dev/xvda1) ioping statistics --- >> 3 requests completed in 2002.0 ms, 1836 iops, 7.2 mb/s >> min/avg/max/mdev = 0.1/0.5/0.8/0.3 ms >> >> The initial ioping response is good, then a lot of latency with later ones. Any ideas ? > > Is your guest compiled with PARAVIRT_SPINLOCK and are you using a PV or PVHVM guest? > >> >> Thanks >> >> John >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel >>