search for: iodepth

Excellent description, thank you. With performance.write-behind-trickling-writes ON (default): ## 4k randwrite # fio --randrepeat=1 --ioengine=libaio --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=32 --size=256MB --readwrite=randwrite test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32 fio-3.1 Starting 1 process Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=17.3MiB/s][r=0,w=4422 IOPS][eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=4270...

On Tue, Mar 20, 2018 at 8:57 AM, Sam McLeod <mailinglists at smcleod.net> wrote: > Hi Raghavendra, > > > On 20 Mar 2018, at 1:55 pm, Raghavendra Gowdappa <rgowdapp at redhat.com> > wrote: > > Aggregating large number of small writes by write-behind into large writes > has been merged on master: > https://github.com/gluster/glusterfs/issues/364 > >

...from dom0 like this blktrace -w 60 -d /dev/disk/vbd/21-920 -o - > test.trace /dev/disk/vbd/21-920 is the software raid contains 2 lv volumes , each lv volume created in big srp attached disk Inside vm i try to do some work via fio: [global] rw=randread size=128m directory=/tmp ioengine=libaio iodepth=4 invalidate=1 direct=1 [bgwriter] rw=randwrite iodepth=32 [queryA] iodepth=1 ioengine=mmap direct=0 thinktime=3 [queryB] iodepth=1 ioengine=mmap direct=0 thinktime=5 [bgupdater] rw=randrw iodepth=16 thinktime=40 size=128m After that i try to get graph like iowatcher -t test.trace -o trace.svg...

...ram0: rw=129, want=8452072, limit=4096000 ... --- [2] modprobe brd rd_size=2048000 (or boot with ramdisk_size=2048000) mkfs.btrfs -m raid0 /dev/ram0 /dev/ram1 mount /dev/ram0 /mnt -o discard fio testcase umount /mnt --- [3] testcase [global] directory=/mnt rw=randread size=256m ioengine=libaio iodepth=4 invalidate=1 direct=1 [bgwriter] rw=randwrite iodepth=32 [queryA] iodepth=1 ioengine=mmap thinktime=3 [queryB] iodepth=1 ioengine=mmap thinktime=1 [bgupdater] rw=randrw iodepth=16 thinktime=1 size=32m -- Daniel J Blueman -- To unsubscribe from this list: send the line "unsubscribe linux...

...ent: 172.16.0.12:/gv0 on /mnt2 type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) # fio --ioengine=libaio --filename=fio.test --size=256M --direct=1 --rw=randrw --refill_buffers --norandommap --bs=8k --rwmixread=70 --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=fio-test fio-test: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=16 ... fio-2.16 Starting 16 processes fio-test: Laying out IO file(s) (1 file(s) / 256MB) Jobs: 14 (f=13): [m(5),_(1),m(8),f(1),_(1)] [33.9% done] [1...

...se.glusterfs > (rw,relatime,user_id=0,group_id=0,default_permissions, > allow_other,max_read=131072) > > > > # fio --ioengine=libaio --filename=fio.test --size=256M > --direct=1 --rw=randrw --refill_buffers --norandommap > --bs=8k --rwmixread=70 --iodepth=16 --numjobs=16 > --runtime=60 --group_reporting --name=fio-test > fio-test: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, > iodepth=16 > ... > fio-2.16 > Starting 16 processes > fio-test: Laying out IO file(s) (1 file(s) / 256MB) > Jobs: 14 (f=13): [m...

...uests I can indeed see that "strict sync = not" doesn't honor the sync request, similarly to ZFS. So to summarize, this is the situation: 1) I run a fio benchmark requesting, small, random, async writes. Command is "fio --direct=1 --sync=0 --rw=randwrite --bs=4K --numjobs=1 --iodepth=1 --runtime=60 --group_reporting --name=sambatest --size=32k --time_based". I run this command on both the host, as on the Samba client, both on the same exact ZFS dataset 2) The ZFS dataset only writes async, converting sync to async writes at all times 3) That same dataset being shared thr...

...ld the lock and block other vCPUs. On arm64 quad core VM(qemu-kvm), the patch can increase I/O performance a lot with VIRTIO_RING_F_EVENT_IDX enabled: - without the patch: 14K IOPS - with the patch: 34K IOPS fio script: [global] direct=1 bsrange=4k-4k timeout=10 numjobs=4 ioengine=libaio iodepth=64 filename=/dev/vdc group_reporting=1 [f1] rw=randread Cc: Rusty Russell <rusty at rustcorp.com.au> Cc: "Michael S. Tsirkin" <mst at redhat.com> Cc: virtualization at lists.linux-foundation.org Signed-off-by: Ming Lei <ming.lei at canonical.com> --- drivers/bloc...

...ld the lock and block other vCPUs. On arm64 quad core VM(qemu-kvm), the patch can increase I/O performance a lot with VIRTIO_RING_F_EVENT_IDX enabled: - without the patch: 14K IOPS - with the patch: 34K IOPS fio script: [global] direct=1 bsrange=4k-4k timeout=10 numjobs=4 ioengine=libaio iodepth=64 filename=/dev/vdc group_reporting=1 [f1] rw=randread Cc: Rusty Russell <rusty at rustcorp.com.au> Cc: "Michael S. Tsirkin" <mst at redhat.com> Cc: virtualization at lists.linux-foundation.org Signed-off-by: Ming Lei <ming.lei at canonical.com> --- drivers/bloc...

...ocol SMB3_11. Kernel version 5.0.15. To illustrate, when I do a random sync write benchmark on the host on this dataset, it will use RAM to do the write, drastically speeding up random writes. The below benchmark command on the ZFS host: fio --direct=1 --sync=1 --rw=randwrite --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --group_reporting --name=sambatest --time_based Has an average speed of 520MB/s (which is the maximum speed of my SATA SSD). Despite requesting a sync write, ZFS turns it in an async write, dramatically speeding it up. Clearly the results are great when I directly benchmark from the...

...---------------------------------------------------------- 1 worker 160k 488k - - - - worker per vq 160k 310k 620k 1300k 1836k 2326k Notes: 0. This used a simple fio command: fio --filename=/dev/sdb --direct=1 --rw=randrw --bs=4k \ --ioengine=libaio --iodepth=128 --numjobs=$JOBS_ABOVE and I used a VM with 16 vCPUs and 16 virtqueues. 1. The patches were tested with LIO's emulate_pr=0 which drops the LIO PR lock use. This was a bottleneck at around 12 vqs/jobs. 2. Because we have a hard limit of 1024 cmds, if the num jobs * iodepth was greater tha...

...f=19349958.33 cpu sys=723.63 majf=27597.33 ctx=850199927.33 usr=35.35 minf=19092343.00 FIO config file: [global] exec_prerun="echo 3 > /proc/sys/vm/drop_caches" group_reporting norandommap ioscheduler=noop thread bs=512 size=4MB direct=1 filename=/dev/vdb numjobs=256 ioengine=aio iodepth=64 loops=3 Signed-off-by: Stefan Hajnoczi <stefanha at linux.vnet.ibm.com> --- Other block drivers (cciss, rbd, nbd) use spin_unlock_irq() so I followed that. To me this seems wrong: blk_run_queue() uses spin_lock_irqsave() but we enable irqs with spin_unlock_irq(). If the caller of blk_run...

...f=19349958.33 cpu sys=723.63 majf=27597.33 ctx=850199927.33 usr=35.35 minf=19092343.00 FIO config file: [global] exec_prerun="echo 3 > /proc/sys/vm/drop_caches" group_reporting norandommap ioscheduler=noop thread bs=512 size=4MB direct=1 filename=/dev/vdb numjobs=256 ioengine=aio iodepth=64 loops=3 Signed-off-by: Stefan Hajnoczi <stefanha at linux.vnet.ibm.com> --- Other block drivers (cciss, rbd, nbd) use spin_unlock_irq() so I followed that. To me this seems wrong: blk_run_queue() uses spin_lock_irqsave() but we enable irqs with spin_unlock_irq(). If the caller of blk_run...

...ture, 'num_queues=N' need to be added into >> '-device virtio-blk-pci ...' of qemu command line, and suggest to pass >> 'vectors=N+1' to keep one MSI irq vector per each vq, and the feature >> depends on x-data-plane. >> >> Fio(libaio, randread, iodepth=64, bs=4K, jobs=N) is run inside VM to >> verify the improvement. >> >> I just create a small quadcore VM and run fio inside the VM, and >> num_queues of the virtio-blk device is set as 2, but looks the >> improvement is still obvious. The host is 2 sockets, 8cores(16t...

...ture, 'num_queues=N' need to be added into >> '-device virtio-blk-pci ...' of qemu command line, and suggest to pass >> 'vectors=N+1' to keep one MSI irq vector per each vq, and the feature >> depends on x-data-plane. >> >> Fio(libaio, randread, iodepth=64, bs=4K, jobs=N) is run inside VM to >> verify the improvement. >> >> I just create a small quadcore VM and run fio inside the VM, and >> num_queues of the virtio-blk device is set as 2, but looks the >> improvement is still obvious. The host is 2 sockets, 8cores(16t...

For vhost-scsi with 3 vqs and a workload like that tries to use those vqs like: fio --filename=/dev/sdb --direct=1 --rw=randrw --bs=4k \ --ioengine=libaio --iodepth=128 --numjobs=3 the single vhost worker thread will become a bottlneck and we are stuck at around 500K IOPs no matter how many jobs, virtqueues, and CPUs are used. To better utilize virtqueues and available CPUs, this patch allows userspace to create workers and bind them to vqs. You can have N...

...t: size=21G on /mnt type tmpfs (rw,relatime,size=22020096k) TEST CONFIG =========== FIO (https://linux.die.net/man/1/fio) is being run with and without the patches. Read test config: [Sequential] direct=1 ioengine=libaio runtime=5m time_based filename=/dev/vda bs=4k numjobs=16 rw=read unlink=1 iodepth=256 Write test config: [Sequential] direct=1 ioengine=libaio runtime=5m time_based filename=/dev/vda bs=4k numjobs=16 rw=write unlink=1 iodepth=256 The virtio block device comes up as /dev/vda on the guest with /sys/block/vda/queue/nr_requests=128 TEST RESULTS ============ Without the patche...

...t: size=21G on /mnt type tmpfs (rw,relatime,size=22020096k) TEST CONFIG =========== FIO (https://linux.die.net/man/1/fio) is being run with and without the patches. Read test config: [Sequential] direct=1 ioengine=libaio runtime=5m time_based filename=/dev/vda bs=4k numjobs=16 rw=read unlink=1 iodepth=256 Write test config: [Sequential] direct=1 ioengine=libaio runtime=5m time_based filename=/dev/vda bs=4k numjobs=16 rw=write unlink=1 iodepth=256 The virtio block device comes up as /dev/vda on the guest with /sys/block/vda/queue/nr_requests=128 TEST RESULTS ============ Without the patche...

...-virtblk-mq.1 For enabling the multi-vq feature, 'num_queues=N' need to be added into '-device virtio-blk-pci ...' of qemu command line, and suggest to pass 'vectors=N+1' to keep one MSI irq vector per each vq, and the feature depends on x-data-plane. Fio(libaio, randread, iodepth=64, bs=4K, jobs=N) is run inside VM to verify the improvement. I just create a small quadcore VM and run fio inside the VM, and num_queues of the virtio-blk device is set as 2, but looks the improvement is still obvious. 1), about scalability - without mutli-vq feature -- jobs=2, thoughput: 145K...

...-virtblk-mq.1 For enabling the multi-vq feature, 'num_queues=N' need to be added into '-device virtio-blk-pci ...' of qemu command line, and suggest to pass 'vectors=N+1' to keep one MSI irq vector per each vq, and the feature depends on x-data-plane. Fio(libaio, randread, iodepth=64, bs=4K, jobs=N) is run inside VM to verify the improvement. I just create a small quadcore VM and run fio inside the VM, and num_queues of the virtio-blk device is set as 2, but looks the improvement is still obvious. 1), about scalability - without mutli-vq feature -- jobs=2, thoughput: 145K...