Boris Derzhavets
2009-Oct-02 06:58 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
Dmesg report attached:- [ 18.088588] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 18.088653] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 18.338143] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 18.338214] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 18.338270] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 18.378688] ------------[ cut here ]------------ [ 18.378696] kernel BUG at mm/slab.c:521! [ 18.378699] invalid opcode: 0000 [#1] SMP [ 18.378706] last sysfs file: /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/enable [ 18.378709] CPU 0 [ 18.378713] Modules linked in: ppdev bnep video output lp parport snd_hda_codec_atihdmi snd_hda_codec_analog arc4 ecb snd_seq_dummy snd_seq_oss rtl8187 snd_seq_midi snd_hda_intel snd_hda_codec snd_rawmidi mac80211 snd_seq_midi_event snd_pcm_oss led_class snd_seq snd_mixer_oss eeprom_93cx6 snd_seq_device snd_pcm pcspkr iTCO_wdt iTCO_vendor_support intel_agp snd_timer snd soundcore snd_page_alloc cfg80211 usbhid ohci1394 ieee1394 r8169 mii sky2 [ 18.378786] Pid: 2927, comm: Xorg Not tainted 2.6.31.1 #5 P5K Premium [ 18.378789] RIP: e030:[<ffffffff81107afe>] [<ffffffff81107afe>] kfree+0xae/0x1d0 [ 18.378799] RSP: e02b:ffff8801e1469c78 EFLAGS: 00010046 [ 18.378802] RAX: ffffea0006906580 RBX: ffff8801e01cfff8 RCX: 80000000063cd063 [ 18.378805] RDX: 8000000000080000 RSI: 80000000063cd063 RDI: ffff8801e01d0000 [ 18.378808] RBP: ffff8801e1469cc8 R08: ffffffff8186cde0 R09: 0000000000000000 [ 18.378811] R10: 0000000000007ff0 R11: 0000000000000001 R12: ffffffff81315b94 [ 18.378814] R13: ffff8801e1469dc8 R14: 0000000000002000 R15: ffff8801e951d780 [ 18.378821] FS: 00007f2f9bd2a700(0000) GS:ffffc90000000000(0000) knlGS:0000000000000000 [ 18.378825] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 18.378828] CR2: 0000000000e2e608 CR3: 00000001e0d7c000 CR4: 0000000000002660 [ 18.378831] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 18.378834] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 18.378838] Process Xorg (pid: 2927, threadinfo ffff8801e1468000, task ffff8801e6094480) [ 18.378840] Stack: [ 18.378842] ffffc90012977fff ffff8801e1469d30 ffff8801dfb37800 ffff8801e01d0000 [ 18.378850] <0> 0000000000000200 ffff8801e01cfff8 ffff8801dfb37800 ffff8801e1469dc8 [ 18.378859] <0> 0000000000002000 ffff8801e951d780 ffff8801e1469d68 ffffffff81315b94 [ 18.378869] Call Trace: [ 18.378877] [<ffffffff81315b94>] drm_sg_alloc+0x424/0x590 [ 18.378882] [<ffffffff81315d09>] drm_sg_alloc_ioctl+0x9/0x10 [ 18.378887] [<ffffffff8130ee25>] drm_ioctl+0x175/0x3c0 [ 18.378892] [<ffffffff8119d805>] ? ext4_file_write+0x55/0x180 [ 18.378897] [<ffffffff81315d00>] ? drm_sg_alloc_ioctl+0x0/0x10 [ 18.378903] [<ffffffff8100fe61>] ? xen_clocksource_read+0x21/0x30 [ 18.378908] [<ffffffff81010ef7>] ? xen_spin_lock+0xa7/0x110 [ 18.378913] [<ffffffff8111e47d>] vfs_ioctl+0x7d/0xa0 [ 18.378917] [<ffffffff8111e92b>] do_vfs_ioctl+0x3fb/0x590 [ 18.378921] [<ffffffff8111eb59>] sys_ioctl+0x99/0xa0 [ 18.378926] [<ffffffff8110f980>] ? sys_write+0x50/0x90 [ 18.378931] [<ffffffff81014f02>] system_call_fastpath+0x16/0x1b [ 18.378933] Code: ba 00 00 00 00 00 ea ff ff 48 01 d0 48 8b 10 66 85 d2 79 13 48 8b 40 10 48 8b 10 66 85 d2 79 07 48 8b 40 10 48 8b 10 84 d2 78 0a <0f> 0b eb fe 66 0f 1f 44 00 00 4c 8b 78 28 65 8b 04 25 88 e0 00 [ 18.379033] RIP [<ffffffff81107afe>] kfree+0xae/0x1d0 [ 18.379038] RSP <ffff8801e1469c78> [ 18.379042] ---[ end trace f245dfcffec6c0b3 ]--- [ 18.392165] [drm:drm_release] *ERROR* Device busy: 1 [ 20.202543] sky2 eth0: disabling interface [ 20.288075] sky2 peth0: enabling interface [ 20.288720] ADDRCONF(NETDEV_UP): peth0: link is not ready [ 21.524894] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 21.524960] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 21.752539] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 21.752601] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 21.752654] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 21.795030] ------------[ cut here ]------------ [ 21.795037] kernel BUG at mm/slab.c:521! [ 21.795041] invalid opcode: 0000 [#2] SMP [ 21.795047] last sysfs file: /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/enable [ 21.795050] CPU 1 [ 21.795054] Modules linked in: binfmt_misc ppdev bnep video output lp parport snd_hda_codec_atihdmi snd_hda_codec_analog arc4 ecb snd_seq_dummy snd_seq_oss rtl8187 snd_seq_midi snd_hda_intel snd_hda_codec snd_rawmidi mac80211 snd_seq_midi_event snd_pcm_oss led_class snd_seq snd_mixer_oss eeprom_93cx6 snd_seq_device snd_pcm pcspkr iTCO_wdt iTCO_vendor_support intel_agp snd_timer snd soundcore snd_page_alloc cfg80211 usbhid ohci1394 ieee1394 r8169 mii sky2 [ 21.795128] Pid: 3317, comm: Xorg Tainted: G D 2.6.31.1 #5 P5K Premium [ 21.795131] RIP: e030:[<ffffffff81107afe>] [<ffffffff81107afe>] kfree+0xae/0x1d0 [ 21.795141] RSP: e02b:ffff8801e01fbc78 EFLAGS: 00010046 [ 21.795144] RAX: ffffea0006aee280 RBX: ffff8801e8d2fff8 RCX: 80000000043d0063 [ 21.795147] RDX: 8000000000080000 RSI: 80000000043d0063 RDI: ffff8801e8d30000 [ 21.795150] RBP: ffff8801e01fbcc8 R08: ffffffff8186cde0 R09: 0000000000000000 [ 21.795153] R10: 0000000000007ff0 R11: 0000000000000000 R12: ffffffff81315b94 [ 21.795156] R13: ffff8801e01fbdc8 R14: 0000000000002000 R15: ffff8801e54c04c0 [ 21.795163] FS: 00007f0dbb7de700(0000) GS:ffffc90000018000(0000) knlGS:0000000000000000 [ 21.795166] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 21.795169] CR2: 00000000013d1608 CR3: 00000001dec6f000 CR4: 0000000000002660 [ 21.795173] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 21.795176] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 21.795179] Process Xorg (pid: 3317, threadinfo ffff8801e01fa000, task ffff8801e0116200) [ 21.795182] Stack: [ 21.795184] ffffc90014979fff ffff8801e01fbd30 ffff8801dfb37800 ffff8801e8d30000 [ 21.795192] <0> 0000000000000200 ffff8801e8d2fff8 ffff8801dfb37800 ffff8801e01fbdc8 [ 21.795201] <0> 0000000000002000 ffff8801e54c04c0 ffff8801e01fbd68 ffffffff81315b94 [ 21.795211] Call Trace: [ 21.795219] [<ffffffff81315b94>] drm_sg_alloc+0x424/0x590 [ 21.795224] [<ffffffff81312995>] ? drm_lock+0x255/0x3d0 [ 21.795228] [<ffffffff81315d09>] drm_sg_alloc_ioctl+0x9/0x10 [ 21.795233] [<ffffffff8130ee25>] drm_ioctl+0x175/0x3c0 [ 21.795238] [<ffffffff8119d805>] ? ext4_file_write+0x55/0x180 [ 21.795243] [<ffffffff81315d00>] ? drm_sg_alloc_ioctl+0x0/0x10 [ 21.795249] [<ffffffff8100fe61>] ? xen_clocksource_read+0x21/0x30 [ 21.795254] [<ffffffff81010ef7>] ? xen_spin_lock+0xa7/0x110 [ 21.795259] [<ffffffff8111e47d>] vfs_ioctl+0x7d/0xa0 [ 21.795264] [<ffffffff8111e92b>] do_vfs_ioctl+0x3fb/0x590 [ 21.795268] [<ffffffff8111eb59>] sys_ioctl+0x99/0xa0 [ 21.795273] [<ffffffff8110f980>] ? sys_write+0x50/0x90 [ 21.795277] [<ffffffff81014f02>] system_call_fastpath+0x16/0x1b [ 21.795280] Code: ba 00 00 00 00 00 ea ff ff 48 01 d0 48 8b 10 66 85 d2 79 13 48 8b 40 10 48 8b 10 66 85 d2 79 07 48 8b 40 10 48 8b 10 84 d2 78 0a <0f> 0b eb fe 66 0f 1f 44 00 00 4c 8b 78 28 65 8b 04 25 88 e0 00 [ 21.795380] RIP [<ffffffff81107afe>] kfree+0xae/0x1d0 [ 21.795385] RSP <ffff8801e01fbc78> [ 21.795389] ---[ end trace f245dfcffec6c0b4 ]--- [ 21.798496] [drm:drm_release] *ERROR* Device busy: 1 [ 22.081538] sky2 peth0: Link is up at 100 Mbps, full duplex, flow control both [ 22.082157] ADDRCONF(NETDEV_CHANGE): peth0: link becomes ready [ 22.119000] r8169: eth1: link down [ 22.119550] ADDRCONF(NETDEV_UP): eth1: link is not ready [ 24.372040] ADDRCONF(NETDEV_UP): wlan0: link is not ready [ 24.444865] device peth0 entered promiscuous mode [ 24.477124] eth0: port 1(peth0) entering forwarding state [ 24.954106] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 24.954171] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 25.183847] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 25.183912] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 25.183966] mtrr: type mismatch for d0000000,10000000 old: write-back new: write-combining [ 26.994058] xenbus_probe wake_waiting [ 26.994071] xenbus_probe wake_waiting [ 26.997905] XENBUS: Unable to read cpu state [ 26.998001] XENBUS: Unable to read cpu state [ 26.998192] xenbus_probe_devices backend [ 26.998261] xenbus_probe_devices failed xenbus_directory [ 26.998264] backend_probe_and_watch devices probed ok [ 26.998350] backend_probe_and_watch watch add ok ok [ 26.998352] backend_probe_and_watch all done [ 26.998355] xenbus_probe_devices device [ 26.998438] xenbus_probe_devices failed xenbus_directory [ 26.998442] frontend_probe_and_watch devices probed ok [ 26.998525] frontend_probe_and_watch watch add ok ok [ 26.998528] frontend_probe_and_watch all done [ 32.510017] peth0: no IPv6 routers present [ 34.800163] eth0: no IPv6 routers present --- On Fri, 10/2/09, Boris Derzhavets <bderzhavets@yahoo.com> wrote: From: Boris Derzhavets <bderzhavets@yahoo.com> Subject: Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800) To: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>, "Jeremy Fitzhardinge" <jeremy@goop.org> Cc: dri-devel@lists.sourceforge.net, xen-devel@lists.xensource.com, JBeulich@novell.com Date: Friday, October 2, 2009, 1:02 AM Patch applied on the box with Radeon HD 4650 and seems to be working for 2.6.31.1 under Xen 3.4.1 on top Ubuntu 9.04 Server ( Ubuntu Desktop installed via tasksel). Before login to Gnome Desktop monitor shows up for 2-3 seconds "power saving" message, like it has got disconnected, the awakes back and normal login procedure resumes. Boris. --- On Thu, 10/1/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote: From: Jeremy Fitzhardinge <jeremy@goop.org> Subject: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800) To: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com> Cc: dri-devel@lists.sourceforge.net, xen-devel@lists.xensource.com, JBeulich@novell.com Date: Thursday, October 1, 2009, 3:37 PM On 10/01/09 12:07, Jeremy Fitzhardinge wrote:> Could modify drm_vmalloc_dma to do the vmalloc "manually": > > 1. call __get_vm_area to reserve a chunk of vmalloc address space > 2. allocate a bunch of individual pages with dma_alloc_coherent > 3. insert them into the vmalloc mapping with map_vm_area > > That will guarantee a normal-looking vmalloc area with device-friendly > pages that subsequent pci_map_page operations will use as-is. >Like this (untested): diff --git a/drivers/gpu/drm/drm_scatter.c b/drivers/gpu/drm/drm_scatter.c index c7823c8..73bfa63 100644 --- a/drivers/gpu/drm/drm_scatter.c +++ b/drivers/gpu/drm/drm_scatter.c @@ -32,16 +32,60 @@ */ #include <linux/vmalloc.h> +#include <linux/mm.h> #include "drmP.h" #define DEBUG_SCATTER 0 -static inline void *drm_vmalloc_dma(unsigned long size) +static inline void *drm_vmalloc_dma(struct drm_device *drmdev, unsigned long size) { #if defined(__powerpc__) && defined(CONFIG_NOT_COHERENT_CACHE) return __vmalloc(size, GFP_KERNEL, PAGE_KERNEL | _PAGE_NO_CACHE); #else - return vmalloc_32(size); + struct device *dev = &drmdev->pdev->dev; + struct vm_struct *vma; + struct page **pages; + const int npages = PFN_UP(size); + int i; + + pages = kmalloc(npages * sizeof(*pages), GFP_KERNEL); + if (!pages) + goto out_free_pagearr; + + vma = __get_vm_area(size, VM_ALLOC, VMALLOC_START, VMALLOC_END); + if (!vma) + goto out_release_vma; + + for (i = 0; i < npages; i++) { + dma_addr_t phys; + void *addr; + addr = dma_alloc_coherent(dev, PAGE_SIZE, &phys, GFP_KERNEL); + if (addr == NULL) + goto out_free_pages; + + pages[i] = virt_to_page(addr); + } + + if (map_vm_area(vma, PAGE_KERNEL, &pages)) + goto out_free_pages; + + kfree(pages); + + return vma->addr; + +out_free_pages: + while(i > 0) { + void *addr = page_address(pages[--i]); + dma_free_coherent(dev, PAGE_SIZE, addr, virt_to_bus(addr)); + } + +out_release_vma: + vunmap(vma->addr); + +out_free_pagearr: + kfree(pages); + + return NULL; #endif } @@ -107,7 +151,7 @@ int drm_sg_alloc(struct drm_device *dev, struct drm_scatter_gather * request) } memset((void *)entry->busaddr, 0, pages * sizeof(*entry->busaddr)); - entry->virtual = drm_vmalloc_dma(pages << PAGE_SHIFT); + entry->virtual = drm_vmalloc_dma(dev, pages << PAGE_SHIFT); if (!entry->virtual) { kfree(entry->busaddr); kfree(entry->pagelist); _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel -----Inline Attachment Follows----- _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Oct-02 17:18 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
On 10/01/09 23:58, Boris Derzhavets wrote:> Dmesg report attached:- > > [ 18.088588] mtrr: type mismatch for d0000000,10000000 old: > write-back new: write-combining > [ 18.088653] mtrr: type mismatch for d0000000,10000000 old: > write-back new: write-combining > [ 18.338143] mtrr: type mismatch for d0000000,10000000 old: > write-back new: write-combining > [ 18.338214] mtrr: type mismatch for d0000000,10000000 old: > write-back new: write-combining > [ 18.338270] mtrr: type mismatch for d0000000,10000000 old: > write-back new: write-combining > [ 18.378688] ------------[ cut here ]------------ > [ 18.378696] kernel BUG at mm/slab.c:521! >OK, I have a fix for this. I''ll commit it shortly. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Boris Derzhavets
2009-Oct-02 17:23 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
Jeremy, Please, be aware of bugzilla.xensource.com [1519] the most recent entries :- http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1519 Boris. --- On Fri, 10/2/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote: From: Jeremy Fitzhardinge <jeremy@goop.org> Subject: Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800) To: "Boris Derzhavets" <bderzhavets@yahoo.com> Cc: dri-devel@lists.sourceforge.net, xen-devel@lists.xensource.com, JBeulich@novell.com, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com> Date: Friday, October 2, 2009, 1:18 PM On 10/01/09 23:58, Boris Derzhavets wrote:> Dmesg report attached:- > > [ 18.088588] mtrr: type mismatch for d0000000,10000000 old: > write-back new: write-combining > [ 18.088653] mtrr: type mismatch for d0000000,10000000 old: > write-back new: write-combining > [ 18.338143] mtrr: type mismatch for d0000000,10000000 old: > write-back new: write-combining > [ 18.338214] mtrr: type mismatch for d0000000,10000000 old: > write-back new: write-combining > [ 18.338270] mtrr: type mismatch for d0000000,10000000 old: > write-back new: write-combining > [ 18.378688] ------------[ cut here ]------------ > [ 18.378696] kernel BUG at mm/slab.c:521! >OK, I have a fix for this. I''ll commit it shortly. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Oct-02 18:42 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
On 10/02/09 10:23, Boris Derzhavets wrote:> Jeremy, > Please, be aware of bugzilla.xensource.com [1519] the most recent > entries :- > > http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1519 >Ah, OK. I pushed a variant of Konrad''s patches. Could you try them out? Thanks, J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Boris Derzhavets
2009-Oct-02 19:25 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
I will have to make a fresh git clone. Tomorrow by the end of the day it''s gonna be done. Boris. --- On Fri, 10/2/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote: From: Jeremy Fitzhardinge <jeremy@goop.org> Subject: Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800) To: "Boris Derzhavets" <bderzhavets@yahoo.com> Cc: dri-devel@lists.sourceforge.net, xen-devel@lists.xensource.com, JBeulich@novell.com, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com> Date: Friday, October 2, 2009, 2:42 PM On 10/02/09 10:23, Boris Derzhavets wrote:> Jeremy, > Please, be aware of bugzilla.xensource.com [1519] the most recent > entries :- > > http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1519 >Ah, OK. I pushed a variant of Konrad''s patches. Could you try them out? Thanks, J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2009-Oct-05 10:32 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
>>> Jeremy Fitzhardinge <jeremy@goop.org> 02.10.09 20:42 >>> >On 10/02/09 10:23, Boris Derzhavets wrote: >> Jeremy, >> Please, be aware of bugzilla.xensource.com [1519] the most recent >> entries :- >> >> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1519 >> > >Ah, OK. I pushed a variant of Konrad''s patches. Could you try them out?Are you really convinced fixing this in DRM is the right thing to do? To me, the use of vmalloc_32() in drivers/ieee1394/ seems to make similar assumptions (pci_map_sg() not modifying the buffer addresses), and who knows where else such assumptions exist. After all, vmalloc_32() is *defined* to have the property assumed by both of the users, and other than for most kmalloc() cases using GFP_DMA{,32} we''re talking about double buffering generally large amounts of data here even in the cases where the DMA API is being used properly. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2009-Oct-05 14:01 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
On Mon, Oct 05, 2009 at 11:32:31AM +0100, Jan Beulich wrote:> >>> Jeremy Fitzhardinge <jeremy@goop.org> 02.10.09 20:42 >>> > >On 10/02/09 10:23, Boris Derzhavets wrote: > >> Jeremy, > >> Please, be aware of bugzilla.xensource.com [1519] the most recent > >> entries :- > >> > >> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1519 > >> > > > >Ah, OK. I pushed a variant of Konrad''s patches. Could you try them out? > > Are you really convinced fixing this in DRM is the right thing to do? To > me, the use of vmalloc_32() in drivers/ieee1394/ seems to make similar > assumptions (pci_map_sg() not modifying the buffer addresses), and > who knows where else such assumptions exist. After all, vmalloc_32() > is *defined* to have the property assumed by both of the users, and > other than for most kmalloc() cases using GFP_DMA{,32} we''re talking > about double buffering generally large amounts of data here even in > the cases where the DMA API is being used properly.I was chatting with Jeremy about this, and I realized that in some sense the radeon driver assumes that physical != bus addresses. Which is OK on x86, but if this card was ever moved to a Sun box it would fail. The patch here thwarts this by allocating memory from the IOMMU, so that even if bus != physical address from pci_map_page, that is OK. Another way to make this work right is to modify the drivers that use vmalloc_32, with pci_map_[sg|page], is for them to check if the physical != bus address, and if so, remap the virtual address (page_to_vmalloc) to point to the new bus addresses (and free the "old" page). Obviously this requires some book-keeping, but it does fix those drivers if they were to move to non-x86 architecture. Or on machines where the IOMMU hands out non-physical addresses. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2009-Oct-05 14:37 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
On Mon, Oct 05, 2009 at 10:01:52AM -0400, Konrad Rzeszutek Wilk wrote:> On Mon, Oct 05, 2009 at 11:32:31AM +0100, Jan Beulich wrote: > > >>> Jeremy Fitzhardinge <jeremy@goop.org> 02.10.09 20:42 >>> > > >On 10/02/09 10:23, Boris Derzhavets wrote: > > >> Jeremy, > > >> Please, be aware of bugzilla.xensource.com [1519] the most recent > > >> entries :- > > >> > > >> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1519 > > >> > > > > > >Ah, OK. I pushed a variant of Konrad''s patches. Could you try them out? > > > > Are you really convinced fixing this in DRM is the right thing to do? To > > me, the use of vmalloc_32() in drivers/ieee1394/ seems to make similar > > assumptions (pci_map_sg() not modifying the buffer addresses), and > > who knows where else such assumptions exist. After all, vmalloc_32() > > is *defined* to have the property assumed by both of the users, and > > other than for most kmalloc() cases using GFP_DMA{,32} we''re talking > > about double buffering generally large amounts of data here even in > > the cases where the DMA API is being used properly. > > I was chatting with Jeremy about this, and I realized that in some > sense the radeon driver assumes that physical != bus addresses. Which is > OK on x86, but if this card was ever moved to a Sun box it would fail. > > The patch here thwarts this by allocating memory from the IOMMU, so that > even if bus != physical address from pci_map_page, that is OK.That is, if the IOMMU, allocates memory from the coherent pool to be same type of memory that it allocates when calling pci_map_[sg|[page].> > Another way to make this work right is to modify the drivers > that use vmalloc_32, with pci_map_[sg|page], is for them to check if the physical > != bus address, and if so, remap the virtual address (page_to_vmalloc) to > point to the new bus addresses (and free the "old" page). Obviously this > requires some book-keeping, but it does fix those drivers if they were to move to > non-x86 architecture. Or on machines where the IOMMU hands out non-physical > addresses.Jeremy suggested another one. That is for the Xen IOMMU to fix the virtual mappings. Meaning we would swipe out the virtual address of the page to point to a virtual address (along with returning an 32-bit bus address) under the 4GB mark. That involves also changing the mappings of the old virtual address to the new one where ever it may be. Jeremy, did I get that right? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Alex Deucher
2009-Oct-05 14:41 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
On Mon, Oct 5, 2009 at 10:01 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:> On Mon, Oct 05, 2009 at 11:32:31AM +0100, Jan Beulich wrote: >> >>> Jeremy Fitzhardinge <jeremy@goop.org> 02.10.09 20:42 >>> >> >On 10/02/09 10:23, Boris Derzhavets wrote: >> >> Jeremy, >> >> Please, be aware of bugzilla.xensource.com [1519] the most recent >> >> entries :- >> >> >> >> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1519 >> >> >> > >> >Ah, OK. I pushed a variant of Konrad''s patches. Could you try them out? >> >> Are you really convinced fixing this in DRM is the right thing to do? To >> me, the use of vmalloc_32() in drivers/ieee1394/ seems to make similar >> assumptions (pci_map_sg() not modifying the buffer addresses), and >> who knows where else such assumptions exist. After all, vmalloc_32() >> is *defined* to have the property assumed by both of the users, and >> other than for most kmalloc() cases using GFP_DMA{,32} we''re talking >> about double buffering generally large amounts of data here even in >> the cases where the DMA API is being used properly. > > I was chatting with Jeremy about this, and I realized that in some > sense the radeon driver assumes that physical != bus addresses. Which is > OK on x86, but if this card was ever moved to a Sun box it would fail. >FWIW, the radeon drm has been working fine on both sparc and ppc for years. Alex> The patch here thwarts this by allocating memory from the IOMMU, so that > even if bus != physical address from pci_map_page, that is OK. > > Another way to make this work right is to modify the drivers > that use vmalloc_32, with pci_map_[sg|page], is for them to check if the physical > != bus address, and if so, remap the virtual address (page_to_vmalloc) to > point to the new bus addresses (and free the "old" page). Obviously this > requires some book-keeping, but it does fix those drivers if they were to move to > non-x86 architecture. Or on machines where the IOMMU hands out non-physical > addresses. > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9-12, 2009. Register now! > http://p.sf.net/sfu/devconf > -- > _______________________________________________ > Dri-devel mailing list > Dri-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/dri-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2009-Oct-05 14:49 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
On Mon, Oct 05, 2009 at 10:41:04AM -0400, Alex Deucher wrote:> On Mon, Oct 5, 2009 at 10:01 AM, Konrad Rzeszutek Wilk > <konrad.wilk@oracle.com> wrote: > > On Mon, Oct 05, 2009 at 11:32:31AM +0100, Jan Beulich wrote: > >> >>> Jeremy Fitzhardinge <jeremy@goop.org> 02.10.09 20:42 >>> > >> >On 10/02/09 10:23, Boris Derzhavets wrote: > >> >> Jeremy, > >> >> Please, be aware of bugzilla.xensource.com [1519] the most recent > >> >> entries :- > >> >> > >> >> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1519 > >> >> > >> > > >> >Ah, OK. I pushed a variant of Konrad''s patches. Could you try them out? > >> > >> Are you really convinced fixing this in DRM is the right thing to do? To > >> me, the use of vmalloc_32() in drivers/ieee1394/ seems to make similar > >> assumptions (pci_map_sg() not modifying the buffer addresses), and > >> who knows where else such assumptions exist. After all, vmalloc_32() > >> is *defined* to have the property assumed by both of the users, and > >> other than for most kmalloc() cases using GFP_DMA{,32} we''re talking > >> about double buffering generally large amounts of data here even in > >> the cases where the DMA API is being used properly. > > > > I was chatting with Jeremy about this, and I realized that in some > > sense the radeon driver assumes that physical != bus addresses. Which is > > OK on x86, but if this card was ever moved to a Sun box it would fail. > > > > FWIW, the radeon drm has been working fine on both sparc and ppc for years.Thank you for keeping me honest! I thought that the IOMMU on those boxes would return physical != bus addresses? Maybe those days are long gone? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Alex Deucher
2009-Oct-05 14:54 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
On Mon, Oct 5, 2009 at 10:49 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:> On Mon, Oct 05, 2009 at 10:41:04AM -0400, Alex Deucher wrote: >> On Mon, Oct 5, 2009 at 10:01 AM, Konrad Rzeszutek Wilk >> <konrad.wilk@oracle.com> wrote: >> > On Mon, Oct 05, 2009 at 11:32:31AM +0100, Jan Beulich wrote: >> >> >>> Jeremy Fitzhardinge <jeremy@goop.org> 02.10.09 20:42 >>> >> >> >On 10/02/09 10:23, Boris Derzhavets wrote: >> >> >> Jeremy, >> >> >> Please, be aware of bugzilla.xensource.com [1519] the most recent >> >> >> entries :- >> >> >> >> >> >> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1519 >> >> >> >> >> > >> >> >Ah, OK. I pushed a variant of Konrad''s patches. Could you try them out? >> >> >> >> Are you really convinced fixing this in DRM is the right thing to do? To >> >> me, the use of vmalloc_32() in drivers/ieee1394/ seems to make similar >> >> assumptions (pci_map_sg() not modifying the buffer addresses), and >> >> who knows where else such assumptions exist. After all, vmalloc_32() >> >> is *defined* to have the property assumed by both of the users, and >> >> other than for most kmalloc() cases using GFP_DMA{,32} we''re talking >> >> about double buffering generally large amounts of data here even in >> >> the cases where the DMA API is being used properly. >> > >> > I was chatting with Jeremy about this, and I realized that in some >> > sense the radeon driver assumes that physical != bus addresses. Which is >> > OK on x86, but if this card was ever moved to a Sun box it would fail. >> > >> >> FWIW, the radeon drm has been working fine on both sparc and ppc for years. > > Thank you for keeping me honest! > > I thought that the IOMMU on those boxes would return physical != bus addresses? > Maybe those days are long gone? >Not sure off hand. I''m not particularly familiar with either arch. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Oct-05 19:09 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
On 10/05/09 03:32, Jan Beulich wrote:> Are you really convinced fixing this in DRM is the right thing to do? To > me, the use of vmalloc_32() in drivers/ieee1394/ seems to make similar > assumptions (pci_map_sg() not modifying the buffer addresses), and > who knows where else such assumptions exist. After all, vmalloc_32() > is *defined* to have the property assumed by both of the users, and > other than for most kmalloc() cases using GFP_DMA{,32} we''re talking > about double buffering generally large amounts of data here even in > the cases where the DMA API is being used properly. >vmalloc_32 is a general problem. The only clean way I can see to make it work under Xen is to make sure all the pages in the DMA32 zone are really under 4G in machine addresses. But that doesn''t scale very well if you have more than one domain wanting to use vmalloc_32 for hardware access (or a few small domains). I''m wondering if the "proper" fix is to introduce a vmalloc_dma() call which allocates the pages with the proper DMA API calls them maps them together in the vmalloc space, and then start migrating drivers over to this new API (ie as needed when people report Xen problems). That would be more generally correct, but perhaps it would be a bit more cumbersome because it would have to return both the vaddr of the vmapping, and the dma_addr_t array, so that vfree_dma() can make the proper dma_free_coherent calls. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Oct-05 19:12 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
On 10/05/09 07:37, Konrad Rzeszutek Wilk wrote:> Jeremy suggested another one. That is for the Xen IOMMU to fix the virtual > mappings. Meaning we would swipe out the virtual address of the page to point > to a virtual address (along with returning an 32-bit bus address) under the 4GB mark. > That involves also changing the mappings of the old virtual address to the new one > where ever it may be. > > Jeremy, did I get that right? >Yeah. We''d make xen_dma_map_page() use xen_exchange_memory to replace the underlying page with something matching the device''s requirements. That would be fairly straightforward for kernel and vmapped pages, but it gets pretty complex if the page is mapped into userspace (partly because of the need to use rmap to find all the user mappings, and partly because I don''t know what the locking/atomicity requirements would be). J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2009-Oct-06 08:10 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
>>> Jeremy Fitzhardinge <jeremy@goop.org> 05.10.09 21:09 >>> >vmalloc_32 is a general problem. The only clean way I can see to make >it work under Xen is to make sure all the pages in the DMA32 zone are >really under 4G in machine addresses. But that doesn''t scale very wellWhy? Just adding a hook in vmalloc.c would do.>if you have more than one domain wanting to use vmalloc_32 for hardware >access (or a few small domains).That''s why noted that doing this for the whole zone is likely overkill.>I''m wondering if the "proper" fix is to introduce a vmalloc_dma() call >which allocates the pages with the proper DMA API calls them maps them >together in the vmalloc space, and then start migrating drivers over to >this new API (ie as needed when people report Xen problems). That would >be more generally correct, but perhaps it would be a bit more cumbersome >because it would have to return both the vaddr of the vmapping, and the >dma_addr_t array, so that vfree_dma() can make the proper >dma_free_coherent calls.Hmm, yes, but the more cumbersome an API is to use, the less likely it is that people would want to adopt using it (or that it would be mergable in the first place)). Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Oct-06 19:01 UTC
Re: [Xen-devel] Re: ATI radeon fails with "iommu=soft swiotlb=force" (seen on RV730/RV740 and RS780/RS800)
On 10/06/09 01:10, Jan Beulich wrote:>>>> Jeremy Fitzhardinge <jeremy@goop.org> 05.10.09 21:09 >>> >>>> >> vmalloc_32 is a general problem. The only clean way I can see to make >> it work under Xen is to make sure all the pages in the DMA32 zone are >> really under 4G in machine addresses. But that doesn''t scale very well >> > Why? Just adding a hook in vmalloc.c would do. >Yeah, but I''m a bit allergic to suggesting new hooks.> Hmm, yes, but the more cumbersome an API is to use, the less likely it > is that people would want to adopt using it (or that it would be mergable > in the first place)). >It would need a somewhat detailed survey to work out how many drivers would be affected, and some input from other arch/iommu people to see if it is at all helpful to them. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel