search for: pwritev

On Thu, Nov 16, 2017 at 02:47:46PM +0000, Stefan Hajnoczi wrote: > The threads you observed are the thread pool that performs > preadv(2)/pwritev(2) syscalls. The Linux AIO API could be used instead > and does not use threads for read and write operations. I guess if I used AIO then I wouldn't get any parallelism at all since Linux doesn't block on local file access (at least, it never used to)? > Interesting. Did you perfo...

On Thu, Nov 16, 2017 at 3:10 PM, Richard W.M. Jones <rjones@redhat.com> wrote: > On Thu, Nov 16, 2017 at 02:47:46PM +0000, Stefan Hajnoczi wrote: >> The threads you observed are the thread pool that performs >> preadv(2)/pwritev(2) syscalls. The Linux AIO API could be used instead >> and does not use threads for read and write operations. > > I guess if I used AIO then I wouldn't get any parallelism at all since > Linux doesn't block on local file access (at least, it never used to)? Even assuming...

[CC to qemu-devel since I'm obviously doing something wrong here, I'm just not sure what.] I was getting ready to add multiple threads to ‘qemu-img convert’ (the longest part of v2v conversions) when I noticed that it had them already! (To be fair this was only added in February this year so no wonder we didn't notice.) To enable parallel convert we would need to use the ‘qemu-img

...2017 at 05:30:48PM +0000, Stefan Hajnoczi wrote: > On Thu, Nov 16, 2017 at 3:10 PM, Richard W.M. Jones <rjones@redhat.com> wrote: > > Both local filesystems, but on different SATA devices. > > Okay. I'm curious what the strace -f output looks like (only the > preadv(2)/pwritev(2) syscalls are interesting at the moment). Here's the full strace from one run: http://oirase.annexia.org/tmp/strace.xz The command is (with the output.qcow2 file created in a previous run): $ strace -o /tmp/strace -f -s 1024 qemu-img convert -f raw fedora-27.img -O qcow2 /var/tmp/outp...

...the destination) and > ‘qemu-img convert -m <#num-coroutines>’ option to select the > parallelism (defaults to 8). The documentation refers to coroutines > but I verified from strace that it is using real threads. The threads you observed are the thread pool that performs preadv(2)/pwritev(2) syscalls. The Linux AIO API could be used instead and does not use threads for read and write operations. So these threads are just an implementation detail. The caller doing the reads and writes is not multi-threaded but a number of coroutines executing in a single thread. The qemu-img conv...

...>> ‘qemu-img convert -m <#num-coroutines>’ option to select the >> parallelism (defaults to 8). The documentation refers to coroutines >> but I verified from strace that it is using real threads. > The threads you observed are the thread pool that performs > preadv(2)/pwritev(2) syscalls. The Linux AIO API could be used instead > and does not use threads for read and write operations. So these > threads are just an implementation detail. The caller doing the reads > and writes is not multi-threaded but a number of coroutines executing in > a single thread...

...7 +49,7 @@ def open(readonly): flags = os.O_RDONLY else: flags = os.O_RDWR - fd = os.open(filename, flags) + fd = os.open(filename, flags | os.O_DIRECT) return { 'fd': fd } def get_size(h): @@ -65,3 +65,6 @@ def pwrite(h, buf, offset, flags): n = os.pwritev(h['fd'], [buf], offset) if n != len(buf): raise RuntimeError("short write") + +def flush(h, flags): + os.fsync(h['fd']) diff --git a/server/plugins.c b/server/plugins.c index d4364cd2..ce4700a3 100644 --- a/server/plugins.c +++ b/server/plugins.c @@ -631,6...

...7 +49,7 @@ def open(readonly): flags = os.O_RDONLY else: flags = os.O_RDWR - fd = os.open(filename, flags) + fd = os.open(filename, flags | os.O_DIRECT) return { 'fd': fd } def get_size(h): @@ -65,3 +65,6 @@ def pwrite(h, buf, offset, flags): n = os.pwritev(h['fd'], [buf], offset) if n != len(buf): raise RuntimeError("short write") + +def flush(h, flags): + os.fsync(h['fd']) diff --git a/server/plugins.c b/server/plugins.c index d4364cd2..ce4700a3 100644 --- a/server/plugins.c +++ b/server/plugins.c @@ -631,6...

...= os.O_RDONLY > else: > flags = os.O_RDWR > - fd = os.open(filename, flags) > + fd = os.open(filename, flags | os.O_DIRECT) > return { 'fd': fd } > > def get_size(h): > @@ -65,3 +65,6 @@ def pwrite(h, buf, offset, flags): > n = os.pwritev(h['fd'], [buf], offset) > if n != len(buf): > raise RuntimeError("short write") > + > +def flush(h, flags): > + os.fsync(h['fd']) > diff --git a/server/plugins.c b/server/plugins.c > index d4364cd2..ce4700a3 100644 > --- a/server/pl...

...ettime SYSCALL FOUND: ioctl SYSCALL FOUND: removexattr SYSCALL FOUND: inotify_init1 SYSCALL FOUND: nfsservctl SYSCALL FOUND: perf_event_open SYSCALL FOUND: io_destroy SYSCALL FOUND: prctl SYSCALL FOUND: rt_sigpending SYSCALL FOUND: rtas SYSCALL FOUND: fcntl64 SYSCALL FOUND: mq_notify SYSCALL FOUND: pwritev SYSCALL FOUND: quotactl SYSCALL FOUND: epoll_ctl SYSCALL FOUND: tuxcall SYSCALL FOUND: setresuid SYSCALL FOUND: mincore SYSCALL FOUND: rt_sigreturn SYSCALL FOUND: restart_syscall SYSCALL FOUND: process_vm_readv SYSCALL FOUND: vhangup SYSCALL FOUND: oldstat SYSCALL FOUND: query_module SYSCALL FOUND:...

On Tue, Aug 20, 2013 at 07:44:39AM +0200, leroy christophe wrote: > > > Find attached two patches I have in order to build klibc 2.0.2 > against kernel 3.8.13 > We had to introduce those patches when going from kernel 3.6 to kernel 3.7 > Hope it helps. > those patches are wrong and again very brittle. just use the way it is described in `make help': A) cd ~/src/linux