Konrad Rzeszutek Wilk
2013-May-28 14:44 UTC
Re: [Qemu-devel] use O_DIRECT to open disk images for IDE failed under xen-4.1.2 and qemu upstream
On Mon, May 27, 2013 at 02:44:19PM +0200, Stefan Hajnoczi wrote:> On Sat, May 25, 2013 at 07:32:55AM +0000, Gonglei (Arei) wrote: > > > > > > > -----Original Message----- > > > From: Stefan Hajnoczi [mailto:stefanha@gmail.com] > > > Sent: Friday, May 24, 2013 8:20 PM > > > To: Gonglei (Arei) > > > Cc: Stefano Stabellini; Hanweidong; Luonengjun; qemu-devel@nongnu.org; > > > Wangzhenguo; xen-devel@lists.xen.org; Huangweidong (Hardware) > > > Subject: Re: [Qemu-devel] use O_DIRECT to open disk images for IDE failed > > > under xen-4.1.2 and qemu upstream > > > > > > On Fri, May 24, 2013 at 02:59:05AM +0000, Gonglei (Arei) wrote: > > > > Hi, > > > > > > > > > > > > > > On Thu, 23 May 2013, Gonglei (Arei) wrote: > > > > > > Hi, all > > > > > > > > > > > > I use O_DIRECT to open disk images for IDE, but I'm failed. After debug, I > > > get > > > > > the below logs: > > > > > > [2013-05-22 23:25:46] ide: CMD=c8 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x00 : 0x08 > > > > > > [2013-05-22 23:25:46] bmdma: writeb 0x00 : 0x09 > > > > > > [2013-05-22 23:25:46] bmdma_cmd_writeb: 0x00000009 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:26:39] bmdma: writeb 0x00 : 0x08 > > > > > > [2013-05-22 23:26:39] bmdma_cmd_writeb: 0x00000008 > > > > > > [2013-05-22 23:26:56] ====== offset:0 buf:0x7ff100f21c00 count:512 > > > > > aio_offset:0 > > > > > > [2013-05-22 23:31:30] ====== offset:0 buf:0x7ff100f21c00 count:512 > > > > > aio_offset:0 > > > > > > [2013-05-22 23:31:30] ====== handle_aiocb_rw_linear errno: -14 > > > > > > [2013-05-22 23:31:30] ====== paio_complete errno=14 > > > > > > [2013-05-22 23:31:30] ====== ide_dma_error!!! > > > > > > [2013-05-22 23:31:30] ide: read status addr=0x3f6 val=41 > > > > > > > > > > > > QEMU command line : > > > > > > qemu-system-i386 -xen-domid 837 -chardev > > > > > socket,id=libxl-cmd,path=/var/run/xen/qmp-libxl-837,server,nowait -mon > > > > > chardev=libxl-cmd,mode=control -name suse11 -vnc 0.0.0.0:1 -serial pty > > > -boot > > > > > order=c -usb -usbdevice tablet -smp 2,maxcpus=2 -device > > > > > rtl8139,id=nic0,netdev=net0,mac=00:16:3e:13:d3:72 -netdev > > > > > type=tap,id=net0,ifname=tap837.0,downscript=no -M xenfv -m 2040 -drive > > > > > > > > file=/mnt/sdd/image/suse.image,if=ide,index=0,media=disk,format=raw,cache > > > > > =none > > > > > > > > > > > > errno 14 shows Bad Address. And I find QEMU_AIO_MISALIGNED flag bit > > > is > > > > > not set through debug. > > > > > > > > > > > > /* > > > > > > * If O_DIRECT is used the buffer needs to be aligned on a sector > > > > > > * boundary. Check if this is the case or tell the low-level > > > > > > * driver that it needs to copy the buffer. > > > > > > */ > > > > > > if ((bs->open_flags & BDRV_O_NOCACHE)) { > > > > > > if (!bdrv_qiov_is_aligned(bs, qiov)) { //if the address is > > > > > aligned-512, will no meet the conditions > > > > > > type |= QEMU_AIO_MISALIGNED; > > > > > > #ifdef CONFIG_LINUX_AIO > > > > > > } else if (s->use_aio) { > > > > > > return laio_submit(bs, s->aio_ctx, s->fd, sector_num, qiov, > > > > > > nb_sectors, cb, opaque, type); > > > > > > #endif > > > > > > > > > > > > Next process: > > > > > > static ssize_t handle_aiocb_rw(struct qemu_paiocb *aiocb) > > > > > > { > > > > > > ssize_t nbytes; > > > > > > char *buf; > > > > > > > > > > > > if (!(aiocb->aio_type & QEMU_AIO_MISALIGNED)) { > > > > > > /* > > > > > > * If there is just a single buffer, and it is properly aligned > > > > > > * we can just use plain pread/pwrite without any problems. > > > > > > */ > > > > > > if (aiocb->aio_niov == 1) > > > > > > return handle_aiocb_rw_linear(aiocb, > > > > > aiocb->aio_iov->iov_base); //this way, and reports errno 14 next > > > > > > > > > > > > Anyone have a good method to resolve this bug? Thanks! > > > > > > > > > > I know that this is not the answer you are looking for but why do you > > > > > want to use O_DIRECT with IDE? > > > > > It should be perfectly safe to use write-back. > > > > > > > > A few days ago, I asked a question about the IDE FLUSH time of guest： > > > > http://lists.nongnu.org/archive/html/qemu-devel/2013-05/msg02642.html > > > > > > > > finally I found that because Qemu use write-back flag to open disk images by > > > default. > > > > so I hope to use O_DIRECT to avoid meeting that problem, but I'm failed > > > under Xen platform with Qemu upstream. > > > > > > cache=none does not eliminate the need for flush. If you want to do > > > that, try cache=writethrough or cache=directsync. > > > > > I had tried it, but I got the same error. > > Perhaps that's because the busy bit also needs to be set correctly for > other IDE commands (like read/write requests). > > > > Regarding the EFAULT you are seeing, did you check if the I/O buffer > > > address is valid? Try breaking in gdb and inspecting /proc/<pid>/maps > > > or just x <buffer-address> in gdb to see if the buffer is readable. > > > > > > Stefan > > > > Hi, Stefan > > > > The follows are details in gdb: > > handle_aiocb_rw_linear (aiocb=0x13a0f80, buf=0x7f0a637d7c00 <Address 0x7f0a637d7c00 out of bounds>) > > at /mnt/sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:213 > > 213 { > > (gdb) bt > > #0 handle_aiocb_rw_linear (aiocb=0x13a0f80, buf=0x7f0a637d7c00 <Address 0x7f0a637d7c00 out of bounds>) > > at /mnt/sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:213 > > #1 0x000000000058ba4f in handle_aiocb_rw (aiocb=0x13a0f80) at /mnt/sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:254 > > #2 0x000000000058bde2 in aio_thread (unused=0x0) at /mnt/sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:351 > > #3 0x00007f0a6a1ec7b6 in start_thread () from /lib64/libpthread.so.0 > > #4 0x00007f0a68f0e9cd in clone () from /lib64/libc.so.6 > > #5 0x0000000000000000 in ?? () > > (gdb) x 0x7f0a637d7c00 > > 0x7f0a637d7c00: Cannot access memory at address 0x7f0a637d7c00 > > (gdb) n > > 214 ssize_t offset = 0; > > (gdb) > > 217 while (offset < aiocb->aio_nbytes) { > > (gdb) n > > 218 if (aiocb->aio_type & QEMU_AIO_WRITE) > > (gdb) > > 224 len = pread(aiocb->aio_fildes, > > (gdb) > > 229 if (len == -1 && errno == EINTR) > > (gdb) > > 231 else if (len == -1) { > > (gdb) > > 232 offset = -errno; > > (gdb) p errno > > $1 = 14 > > > > linux-CGRmYS:~ # cat /proc/21837/maps > > 00400000-00846000 r-xp 00000000 08:05 28601 /usr/lib/xen/bin/qemu-system-i386 > > 00a45000-00a46000 r--p 00445000 08:05 28601 /usr/lib/xen/bin/qemu-system-i386 > > 00a46000-00a92000 rw-p 00446000 08:05 28601 /usr/lib/xen/bin/qemu-system-i386 > > 00a92000-01344000 rw-p 00000000 00:00 0 [heap] > > 01344000-01345000 rw-p 00000000 00:00 0 [heap] > > 01345000-01393000 rw-p 00000000 00:00 0 [heap] > > 01393000-01394000 rw-p 00000000 00:00 0 [heap] > > 01394000-013ef000 rw-p 00000000 00:00 0 [heap] > > 7f0a636d0000-7f0a637d0000 rw-s 00000000 00:03 4026532176 /proc/xen/privcmd > > 7f0a637d0000-7f0a638d0000 rw-s 00000000 00:03 4026532176 /proc/xen/privcmd > > Interesting, the memory is mapped read/write here. But it's shared > memory that belongs to /proc/xen/privcmd and maybe doesn't support > O_DIRECT? > > I don't know the Xen side of things but why are we doing I/O to > /proc/xen/privcmd?ioctls are the only things one should be doing on that file. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel