Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 use of swiotlb
Hi Ingo, Here''s some patches to clean up swiotlb to prepare for some Xen dom0 patches. These have been posted before, but undergone a round of cleanups to address various comments. Thanks, J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 01 of 14] x86: remove unused iommu_nr_pages
The last usage was removed by the patch set culminating in commit e3c449f526cebb8d287241c7e82faafd9709668b Author: Joerg Roedel <joerg.roedel@amd.com> Date: Wed Oct 15 22:02:11 2008 -0700 x86, AMD IOMMU: convert driver to generic iommu_num_pages function Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Cc: Joerg Roedel <joerg.roedel@amd.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-kernel@vger.kernel.orgdiff -r 9b89e3b4ca90 arch/x86/include/asm/iommu.h --- arch/x86/include/asm/iommu.h | 2 -- arch/x86/kernel/pci-dma.c | 7 ------- 2 files changed, 9 deletions(-) diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h --- a/arch/x86/include/asm/iommu.h +++ b/arch/x86/include/asm/iommu.h @@ -7,8 +7,6 @@ extern int force_iommu, no_iommu; extern int iommu_detected; -extern unsigned long iommu_nr_pages(unsigned long addr, unsigned long len); - /* 10 seconds */ #define DMAR_OPERATION_TIMEOUT ((cycles_t) tsc_khz*10*1000) diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c --- a/arch/x86/kernel/pci-dma.c +++ b/arch/x86/kernel/pci-dma.c @@ -121,13 +121,6 @@ pci_swiotlb_init(); } -unsigned long iommu_nr_pages(unsigned long addr, unsigned long len) -{ - unsigned long size = roundup((addr & ~PAGE_MASK) + len, PAGE_SIZE); - - return size >> PAGE_SHIFT; -} -EXPORT_SYMBOL(iommu_nr_pages); #endif void *dma_generic_alloc_coherent(struct device *dev, size_t size, _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 02 of 14] swiotlb: allow architectures to override swiotlb pool allocation
Architectures may need to allocate memory specially for use with the swiotlb. Create the weak function swiotlb_alloc_boot() and swiotlb_alloc() defaulting to the current behaviour. Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Cc: Joerg Roedel <joerg.roedel@amd.com> Cc: Jan Beulich <jbeulich@novell.com> Cc: Tony Luck <tony.luck@intel.com> Cc: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp> --- include/linux/swiotlb.h | 3 +++ lib/swiotlb.c | 16 +++++++++++++--- 2 files changed, 16 insertions(+), 3 deletions(-) diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h --- a/include/linux/swiotlb.h +++ b/include/linux/swiotlb.h @@ -10,6 +10,9 @@ extern void swiotlb_init(void); +extern void *swiotlb_alloc_boot(size_t bytes, unsigned long nslabs); +extern void *swiotlb_alloc(unsigned order, unsigned long nslabs); + extern void *swiotlb_alloc_coherent(struct device *hwdev, size_t size, dma_addr_t *dma_handle, gfp_t flags); diff --git a/lib/swiotlb.c b/lib/swiotlb.c --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -21,6 +21,7 @@ #include <linux/mm.h> #include <linux/module.h> #include <linux/spinlock.h> +#include <linux/swiotlb.h> #include <linux/string.h> #include <linux/types.h> #include <linux/ctype.h> @@ -126,6 +127,16 @@ __setup("swiotlb=", setup_io_tlb_npages); /* make io_tlb_overflow tunable too? */ +void * __weak swiotlb_alloc_boot(size_t size, unsigned long nslabs) +{ + return alloc_bootmem_low_pages(size); +} + +void * __weak swiotlb_alloc(unsigned order, unsigned long nslabs) +{ + return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order); +} + /* * Statically reserve bounce buffer space and initialize bounce buffer data * structures for the software IO TLB used to implement the DMA API. @@ -145,7 +156,7 @@ /* * Get IO TLB memory from the low pages */ - io_tlb_start = alloc_bootmem_low_pages(bytes); + io_tlb_start = swiotlb_alloc_boot(bytes, io_tlb_nslabs); if (!io_tlb_start) panic("Cannot allocate SWIOTLB buffer"); io_tlb_end = io_tlb_start + bytes; @@ -202,8 +213,7 @@ bytes = io_tlb_nslabs << IO_TLB_SHIFT; while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) { - io_tlb_start = (char *)__get_free_pages(GFP_DMA | __GFP_NOWARN, - order); + io_tlb_start = swiotlb_alloc(order, io_tlb_nslabs); if (io_tlb_start) break; order--; _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 03 of 14] swiotlb: move some definitions to header
From: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- include/linux/swiotlb.h | 14 ++++++++++++++ lib/swiotlb.c | 14 +------------- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h --- a/include/linux/swiotlb.h +++ b/include/linux/swiotlb.h @@ -7,6 +7,20 @@ struct dma_attrs; struct scatterlist; +/* + * Maximum allowable number of contiguous slabs to map, + * must be a power of 2. What is the appropriate value ? + * The complexity of {map,unmap}_single is linearly dependent on this value. + */ +#define IO_TLB_SEGSIZE 128 + + +/* + * log of the size of each IO TLB slab. The number of slabs is command line + * controllable. + */ +#define IO_TLB_SHIFT 11 + extern void swiotlb_init(void); diff --git a/lib/swiotlb.c b/lib/swiotlb.c --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -23,6 +23,7 @@ #include <linux/spinlock.h> #include <linux/swiotlb.h> #include <linux/string.h> +#include <linux/swiotlb.h> #include <linux/types.h> #include <linux/ctype.h> @@ -40,19 +41,6 @@ #define SG_ENT_VIRT_ADDRESS(sg) (sg_virt((sg))) #define SG_ENT_PHYS_ADDRESS(sg) virt_to_bus(SG_ENT_VIRT_ADDRESS(sg)) -/* - * Maximum allowable number of contiguous slabs to map, - * must be a power of 2. What is the appropriate value ? - * The complexity of {map,unmap}_single is linearly dependent on this value. - */ -#define IO_TLB_SEGSIZE 128 - -/* - * log of the size of each IO TLB slab. The number of slabs is command line - * controllable. - */ -#define IO_TLB_SHIFT 11 - #define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT)) /* _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 04 of 14] swiotlb: consistently use address_needs_mapping everywhere
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> --- lib/swiotlb.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/lib/swiotlb.c b/lib/swiotlb.c --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -465,13 +465,9 @@ dma_addr_t dev_addr; void *ret; int order = get_order(size); - u64 dma_mask = DMA_32BIT_MASK; - - if (hwdev && hwdev->coherent_dma_mask) - dma_mask = hwdev->coherent_dma_mask; ret = (void *)__get_free_pages(flags, order); - if (ret && !is_buffer_dma_capable(dma_mask, virt_to_bus(ret), size)) { + if (ret && !address_needs_mapping(hwdev, virt_to_bus(ret), size)) { /* * The allocated memory isn''t reachable by the device. * Fall back on swiotlb_map_single(). @@ -495,9 +491,9 @@ dev_addr = virt_to_bus(ret); /* Confirm address can be DMA''d by device */ - if (!is_buffer_dma_capable(dma_mask, dev_addr, size)) { + if (!address_needs_mapping(hwdev, dev_addr, size)) { printk("hwdev DMA mask = 0x%016Lx, dev_addr = 0x%016Lx\n", - (unsigned long long)dma_mask, + (unsigned long long)dma_get_mask(hwdev), (unsigned long long)dev_addr); /* DMA_TO_DEVICE to avoid memcpy in unmap_single */ _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 05 of 14] swiotlb: add comment where we handle the overflow of a dma mask on 32 bit
From: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- lib/swiotlb.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/swiotlb.c b/lib/swiotlb.c --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -301,6 +301,10 @@ start_dma_addr = virt_to_bus(io_tlb_start) & mask; offset_slots = ALIGN(start_dma_addr, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT; + + /* + * Carefully handle integer overflow which can occur when mask == ~0UL. + */ max_slots = mask + 1 ? ALIGN(mask + 1, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT : 1UL << (BITS_PER_LONG - IO_TLB_SHIFT); _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 06 of 14] swiotlb: allow architectures to override phys<->bus<->phys conversions
From: Ian Campbell <ian.campbell@citrix.com> Architectures may need to override these conversions. Implement a __weak hook point containing the default implementation. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- include/linux/swiotlb.h | 3 ++ lib/swiotlb.c | 52 ++++++++++++++++++++++++++++++++--------------- 2 files changed, 39 insertions(+), 16 deletions(-) diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h --- a/include/linux/swiotlb.h +++ b/include/linux/swiotlb.h @@ -27,6 +27,9 @@ extern void *swiotlb_alloc_boot(size_t bytes, unsigned long nslabs); extern void *swiotlb_alloc(unsigned order, unsigned long nslabs); +extern dma_addr_t swiotlb_phys_to_bus(phys_addr_t address); +extern phys_addr_t swiotlb_bus_to_phys(dma_addr_t address); + extern void *swiotlb_alloc_coherent(struct device *hwdev, size_t size, dma_addr_t *dma_handle, gfp_t flags); diff --git a/lib/swiotlb.c b/lib/swiotlb.c --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -125,6 +125,26 @@ return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order); } +dma_addr_t __weak swiotlb_phys_to_bus(phys_addr_t paddr) +{ + return paddr; +} + +phys_addr_t __weak swiotlb_bus_to_phys(dma_addr_t baddr) +{ + return baddr; +} + +static dma_addr_t swiotlb_virt_to_bus(volatile void *address) +{ + return swiotlb_phys_to_bus(virt_to_phys(address)); +} + +static void *swiotlb_bus_to_virt(dma_addr_t address) +{ + return phys_to_virt(swiotlb_bus_to_phys(address)); +} + /* * Statically reserve bounce buffer space and initialize bounce buffer data * structures for the software IO TLB used to implement the DMA API. @@ -168,7 +188,7 @@ panic("Cannot allocate SWIOTLB overflow buffer!\n"); printk(KERN_INFO "Placing software IO TLB between 0x%lx - 0x%lx\n", - virt_to_bus(io_tlb_start), virt_to_bus(io_tlb_end)); + swiotlb_virt_to_bus(io_tlb_start), swiotlb_virt_to_bus(io_tlb_end)); } void __init @@ -250,7 +270,7 @@ printk(KERN_INFO "Placing %luMB software IO TLB between 0x%lx - " "0x%lx\n", bytes >> 20, - virt_to_bus(io_tlb_start), virt_to_bus(io_tlb_end)); + swiotlb_virt_to_bus(io_tlb_start), swiotlb_virt_to_bus(io_tlb_end)); return 0; @@ -298,7 +318,7 @@ unsigned long max_slots; mask = dma_get_seg_boundary(hwdev); - start_dma_addr = virt_to_bus(io_tlb_start) & mask; + start_dma_addr = swiotlb_virt_to_bus(io_tlb_start) & mask; offset_slots = ALIGN(start_dma_addr, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT; @@ -471,7 +491,7 @@ int order = get_order(size); ret = (void *)__get_free_pages(flags, order); - if (ret && !address_needs_mapping(hwdev, virt_to_bus(ret), size)) { + if (ret && !address_needs_mapping(hwdev, swiotlb_virt_to_bus(ret), size)) { /* * The allocated memory isn''t reachable by the device. * Fall back on swiotlb_map_single(). @@ -492,7 +512,7 @@ } memset(ret, 0, size); - dev_addr = virt_to_bus(ret); + dev_addr = swiotlb_virt_to_bus(ret); /* Confirm address can be DMA''d by device */ if (!address_needs_mapping(hwdev, dev_addr, size)) { @@ -552,7 +572,7 @@ swiotlb_map_single_attrs(struct device *hwdev, void *ptr, size_t size, int dir, struct dma_attrs *attrs) { - dma_addr_t dev_addr = virt_to_bus(ptr); + dma_addr_t dev_addr = swiotlb_virt_to_bus(ptr); void *map; BUG_ON(dir == DMA_NONE); @@ -573,7 +593,7 @@ map = io_tlb_overflow_buffer; } - dev_addr = virt_to_bus(map); + dev_addr = swiotlb_virt_to_bus(map); /* * Ensure that the address returned is DMA''ble @@ -603,7 +623,7 @@ swiotlb_unmap_single_attrs(struct device *hwdev, dma_addr_t dev_addr, size_t size, int dir, struct dma_attrs *attrs) { - char *dma_addr = bus_to_virt(dev_addr); + char *dma_addr = swiotlb_bus_to_virt(dev_addr); BUG_ON(dir == DMA_NONE); if (is_swiotlb_buffer(dma_addr)) @@ -633,7 +653,7 @@ swiotlb_sync_single(struct device *hwdev, dma_addr_t dev_addr, size_t size, int dir, int target) { - char *dma_addr = bus_to_virt(dev_addr); + char *dma_addr = swiotlb_bus_to_virt(dev_addr); BUG_ON(dir == DMA_NONE); if (is_swiotlb_buffer(dma_addr)) @@ -664,7 +684,7 @@ unsigned long offset, size_t size, int dir, int target) { - char *dma_addr = bus_to_virt(dev_addr) + offset; + char *dma_addr = swiotlb_bus_to_virt(dev_addr) + offset; BUG_ON(dir == DMA_NONE); if (is_swiotlb_buffer(dma_addr)) @@ -720,7 +740,7 @@ for_each_sg(sgl, sg, nelems, i) { addr = SG_ENT_VIRT_ADDRESS(sg); - dev_addr = virt_to_bus(addr); + dev_addr = swiotlb_virt_to_bus(addr); if (swiotlb_force || address_needs_mapping(hwdev, dev_addr, sg->length)) { void *map = map_single(hwdev, addr, sg->length, dir); @@ -733,7 +753,7 @@ sgl[0].dma_length = 0; return 0; } - sg->dma_address = virt_to_bus(map); + sg->dma_address = swiotlb_virt_to_bus(map); } else sg->dma_address = dev_addr; sg->dma_length = sg->length; @@ -764,7 +784,7 @@ for_each_sg(sgl, sg, nelems, i) { if (sg->dma_address != SG_ENT_PHYS_ADDRESS(sg)) - unmap_single(hwdev, bus_to_virt(sg->dma_address), + unmap_single(hwdev, swiotlb_bus_to_virt(sg->dma_address), sg->dma_length, dir); else if (dir == DMA_FROM_DEVICE) dma_mark_clean(SG_ENT_VIRT_ADDRESS(sg), sg->dma_length); @@ -797,7 +817,7 @@ for_each_sg(sgl, sg, nelems, i) { if (sg->dma_address != SG_ENT_PHYS_ADDRESS(sg)) - sync_single(hwdev, bus_to_virt(sg->dma_address), + sync_single(hwdev, swiotlb_bus_to_virt(sg->dma_address), sg->dma_length, dir, target); else if (dir == DMA_FROM_DEVICE) dma_mark_clean(SG_ENT_VIRT_ADDRESS(sg), sg->dma_length); @@ -821,7 +841,7 @@ int swiotlb_dma_mapping_error(struct device *hwdev, dma_addr_t dma_addr) { - return (dma_addr == virt_to_bus(io_tlb_overflow_buffer)); + return (dma_addr == swiotlb_virt_to_bus(io_tlb_overflow_buffer)); } /* @@ -833,7 +853,7 @@ int swiotlb_dma_supported(struct device *hwdev, u64 mask) { - return virt_to_bus(io_tlb_end - 1) <= mask; + return swiotlb_virt_to_bus(io_tlb_end - 1) <= mask; } EXPORT_SYMBOL(swiotlb_map_single); _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 07 of 14] swiotlb: add arch hook to force mapping
From: Ian Campbell <ian.campbell@citrix.com> Some architectures require special rules to determine whether a range needs mapping or not. This adds a weak function for architectures to override. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- include/linux/swiotlb.h | 2 ++ lib/swiotlb.c | 15 +++++++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h --- a/include/linux/swiotlb.h +++ b/include/linux/swiotlb.h @@ -30,6 +30,8 @@ extern dma_addr_t swiotlb_phys_to_bus(phys_addr_t address); extern phys_addr_t swiotlb_bus_to_phys(dma_addr_t address); +extern int swiotlb_arch_range_needs_mapping(void *ptr, size_t size); + extern void *swiotlb_alloc_coherent(struct device *hwdev, size_t size, dma_addr_t *dma_handle, gfp_t flags); diff --git a/lib/swiotlb.c b/lib/swiotlb.c --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -145,6 +145,11 @@ return phys_to_virt(swiotlb_bus_to_phys(address)); } +int __weak swiotlb_arch_range_needs_mapping(void *ptr, size_t size) +{ + return 0; +} + /* * Statically reserve bounce buffer space and initialize bounce buffer data * structures for the software IO TLB used to implement the DMA API. @@ -297,6 +302,11 @@ return !is_buffer_dma_capable(dma_get_mask(hwdev), addr, size); } +static inline int range_needs_mapping(void *ptr, size_t size) +{ + return swiotlb_force || swiotlb_arch_range_needs_mapping(ptr, size); +} + static int is_swiotlb_buffer(char *addr) { return addr >= io_tlb_start && addr < io_tlb_end; @@ -581,7 +591,8 @@ * we can safely return the device addr and not worry about bounce * buffering it. */ - if (!address_needs_mapping(hwdev, dev_addr, size) && !swiotlb_force) + if (!address_needs_mapping(hwdev, dev_addr, size) && + !range_needs_mapping(ptr, size)) return dev_addr; /* @@ -741,7 +752,7 @@ for_each_sg(sgl, sg, nelems, i) { addr = SG_ENT_VIRT_ADDRESS(sg); dev_addr = swiotlb_virt_to_bus(addr); - if (swiotlb_force || + if (range_needs_mapping(sg_virt(sg), sg->length) || address_needs_mapping(hwdev, dev_addr, sg->length)) { void *map = map_single(hwdev, addr, sg->length, dir); if (!map) { _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 08 of 14] swiotlb: factor out copy to/from device
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- lib/swiotlb.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/lib/swiotlb.c b/lib/swiotlb.c --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -312,6 +312,15 @@ return addr >= io_tlb_start && addr < io_tlb_end; } +static void +__sync_single(char *buffer, char *dma_addr, size_t size, int dir) +{ + if (dir == DMA_TO_DEVICE) + memcpy(dma_addr, buffer, size); + else + memcpy(buffer, dma_addr, size); +} + /* * Allocates bounce buffer and returns its kernel virtual address. */ @@ -413,7 +422,7 @@ for (i = 0; i < nslots; i++) io_tlb_orig_addr[index+i] = buffer + (i << IO_TLB_SHIFT); if (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL) - memcpy(dma_addr, buffer, size); + __sync_single(buffer, dma_addr, size, DMA_TO_DEVICE); return dma_addr; } @@ -437,7 +446,7 @@ * bounce... copy the data back into the original buffer * and * delete the bounce buffer. */ - memcpy(buffer, dma_addr, size); + __sync_single(buffer, dma_addr, size, DMA_FROM_DEVICE); /* * Return the buffer to the free list by setting the corresponding @@ -477,13 +486,13 @@ switch (target) { case SYNC_FOR_CPU: if (likely(dir == DMA_FROM_DEVICE || dir == DMA_BIDIRECTIONAL)) - memcpy(buffer, dma_addr, size); + __sync_single(buffer, dma_addr, size, DMA_FROM_DEVICE); else BUG_ON(dir != DMA_TO_DEVICE); break; case SYNC_FOR_DEVICE: if (likely(dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL)) - memcpy(dma_addr, buffer, size); + __sync_single(buffer, dma_addr, size, DMA_TO_DEVICE); else BUG_ON(dir != DMA_FROM_DEVICE); break; _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 09 of 14] swiotlb: support bouncing of HighMem pages
This requires us to treat DMA regions in terms of page+offset rather than virtual addressing since a HighMem page may not have a mapping. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- lib/swiotlb.c | 120 +++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 88 insertions(+), 32 deletions(-) diff --git a/lib/swiotlb.c b/lib/swiotlb.c --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -26,6 +26,7 @@ #include <linux/swiotlb.h> #include <linux/types.h> #include <linux/ctype.h> +#include <linux/highmem.h> #include <asm/io.h> #include <asm/dma.h> @@ -38,9 +39,6 @@ #define OFFSET(val,align) ((unsigned long) \ ( (val) & ( (align) - 1))) -#define SG_ENT_VIRT_ADDRESS(sg) (sg_virt((sg))) -#define SG_ENT_PHYS_ADDRESS(sg) virt_to_bus(SG_ENT_VIRT_ADDRESS(sg)) - #define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT)) /* @@ -91,7 +89,10 @@ * We need to save away the original address corresponding to a mapped entry * for the sync operations. */ -static unsigned char **io_tlb_orig_addr; +static struct swiotlb_phys_addr { + struct page *page; + unsigned int offset; +} *io_tlb_orig_addr; /* * Protect the above data structures in the map and unmap calls @@ -150,6 +151,11 @@ return 0; } +static dma_addr_t swiotlb_sg_to_bus(struct scatterlist *sg) +{ + return swiotlb_phys_to_bus(page_to_phys(sg_page(sg)) + sg->offset); +} + /* * Statically reserve bounce buffer space and initialize bounce buffer data * structures for the software IO TLB used to implement the DMA API. @@ -183,7 +189,7 @@ for (i = 0; i < io_tlb_nslabs; i++) io_tlb_list[i] = IO_TLB_SEGSIZE - OFFSET(i, IO_TLB_SEGSIZE); io_tlb_index = 0; - io_tlb_orig_addr = alloc_bootmem(io_tlb_nslabs * sizeof(char *)); + io_tlb_orig_addr = alloc_bootmem(io_tlb_nslabs * sizeof(struct swiotlb_phys_addr)); /* * Get the overflow emergency buffer @@ -258,12 +264,12 @@ io_tlb_list[i] = IO_TLB_SEGSIZE - OFFSET(i, IO_TLB_SEGSIZE); io_tlb_index = 0; - io_tlb_orig_addr = (unsigned char **)__get_free_pages(GFP_KERNEL, - get_order(io_tlb_nslabs * sizeof(char *))); + io_tlb_orig_addr = (struct swiotlb_phys_addr *)__get_free_pages(GFP_KERNEL, + get_order(io_tlb_nslabs * sizeof(struct swiotlb_phys_addr))); if (!io_tlb_orig_addr) goto cleanup3; - memset(io_tlb_orig_addr, 0, io_tlb_nslabs * sizeof(char *)); + memset(io_tlb_orig_addr, 0, io_tlb_nslabs * sizeof(struct swiotlb_phys_addr)); /* * Get the overflow emergency buffer @@ -312,20 +318,59 @@ return addr >= io_tlb_start && addr < io_tlb_end; } +static struct swiotlb_phys_addr swiotlb_bus_to_phys_addr(char *dma_addr) +{ + int index = (dma_addr - io_tlb_start) >> IO_TLB_SHIFT; + struct swiotlb_phys_addr buffer = io_tlb_orig_addr[index]; + buffer.offset += (long)dma_addr & ((1 << IO_TLB_SHIFT) - 1); + buffer.page += buffer.offset >> PAGE_SHIFT; + buffer.offset &= PAGE_SIZE - 1; + return buffer; +} + static void -__sync_single(char *buffer, char *dma_addr, size_t size, int dir) +__sync_single(struct swiotlb_phys_addr buffer, char *dma_addr, size_t size, int dir) { - if (dir == DMA_TO_DEVICE) - memcpy(dma_addr, buffer, size); - else - memcpy(buffer, dma_addr, size); + if (PageHighMem(buffer.page)) { + size_t len, bytes; + char *dev, *host, *kmp; + + len = size; + while (len != 0) { + unsigned long flags; + + bytes = len; + if ((bytes + buffer.offset) > PAGE_SIZE) + bytes = PAGE_SIZE - buffer.offset; + local_irq_save(flags); /* protects KM_BOUNCE_READ */ + kmp = kmap_atomic(buffer.page, KM_BOUNCE_READ); + dev = dma_addr + size - len; + host = kmp + buffer.offset; + if (dir == DMA_FROM_DEVICE) + memcpy(host, dev, bytes); + else + memcpy(dev, host, bytes); + kunmap_atomic(kmp, KM_BOUNCE_READ); + local_irq_restore(flags); + len -= bytes; + buffer.page++; + buffer.offset = 0; + } + } else { + void *v = page_address(buffer.page) + buffer.offset; + + if (dir == DMA_TO_DEVICE) + memcpy(dma_addr, v, size); + else + memcpy(v, dma_addr, size); + } } /* * Allocates bounce buffer and returns its kernel virtual address. */ static void * -map_single(struct device *hwdev, char *buffer, size_t size, int dir) +map_single(struct device *hwdev, struct swiotlb_phys_addr buffer, size_t size, int dir) { unsigned long flags; char *dma_addr; @@ -335,6 +380,7 @@ unsigned long mask; unsigned long offset_slots; unsigned long max_slots; + struct swiotlb_phys_addr slot_buf; mask = dma_get_seg_boundary(hwdev); start_dma_addr = swiotlb_virt_to_bus(io_tlb_start) & mask; @@ -419,8 +465,13 @@ * This is needed when we sync the memory. Then we sync the buffer if * needed. */ - for (i = 0; i < nslots; i++) - io_tlb_orig_addr[index+i] = buffer + (i << IO_TLB_SHIFT); + slot_buf = buffer; + for (i = 0; i < nslots; i++) { + slot_buf.page += slot_buf.offset >> PAGE_SHIFT; + slot_buf.offset &= PAGE_SIZE - 1; + io_tlb_orig_addr[index+i] = slot_buf; + slot_buf.offset += 1 << IO_TLB_SHIFT; + } if (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL) __sync_single(buffer, dma_addr, size, DMA_TO_DEVICE); @@ -436,12 +487,12 @@ unsigned long flags; int i, count, nslots = ALIGN(size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT; int index = (dma_addr - io_tlb_start) >> IO_TLB_SHIFT; - char *buffer = io_tlb_orig_addr[index]; + struct swiotlb_phys_addr buffer = swiotlb_bus_to_phys_addr(dma_addr); /* * First, sync the memory before unmapping the entry */ - if (buffer && ((dir == DMA_FROM_DEVICE) || (dir == DMA_BIDIRECTIONAL))) + if ((dir == DMA_FROM_DEVICE) || (dir == DMA_BIDIRECTIONAL)) /* * bounce... copy the data back into the original buffer * and * delete the bounce buffer. @@ -478,10 +529,7 @@ sync_single(struct device *hwdev, char *dma_addr, size_t size, int dir, int target) { - int index = (dma_addr - io_tlb_start) >> IO_TLB_SHIFT; - char *buffer = io_tlb_orig_addr[index]; - - buffer += ((unsigned long)dma_addr & ((1 << IO_TLB_SHIFT) - 1)); + struct swiotlb_phys_addr buffer = swiotlb_bus_to_phys_addr(dma_addr); switch (target) { case SYNC_FOR_CPU: @@ -525,7 +573,10 @@ * swiotlb_map_single(), which will grab memory from * the lowest available address range. */ - ret = map_single(hwdev, NULL, size, DMA_FROM_DEVICE); + struct swiotlb_phys_addr buffer; + buffer.page = virt_to_page(NULL); + buffer.offset = 0; + ret = map_single(hwdev, buffer, size, DMA_FROM_DEVICE); if (!ret) return NULL; } @@ -593,6 +644,7 @@ { dma_addr_t dev_addr = swiotlb_virt_to_bus(ptr); void *map; + struct swiotlb_phys_addr buffer; BUG_ON(dir == DMA_NONE); /* @@ -607,7 +659,9 @@ /* * Oh well, have to allocate and map a bounce buffer. */ - map = map_single(hwdev, ptr, size, dir); + buffer.page = virt_to_page(ptr); + buffer.offset = (unsigned long)ptr & ~PAGE_MASK; + map = map_single(hwdev, buffer, size, dir); if (!map) { swiotlb_full(hwdev, size, dir, 1); map = io_tlb_overflow_buffer; @@ -752,18 +806,20 @@ int dir, struct dma_attrs *attrs) { struct scatterlist *sg; - void *addr; + struct swiotlb_phys_addr buffer; dma_addr_t dev_addr; int i; BUG_ON(dir == DMA_NONE); for_each_sg(sgl, sg, nelems, i) { - addr = SG_ENT_VIRT_ADDRESS(sg); - dev_addr = swiotlb_virt_to_bus(addr); + dev_addr = swiotlb_sg_to_bus(sg); if (range_needs_mapping(sg_virt(sg), sg->length) || address_needs_mapping(hwdev, dev_addr, sg->length)) { - void *map = map_single(hwdev, addr, sg->length, dir); + void *map; + buffer.page = sg_page(sg); + buffer.offset = sg->offset; + map = map_single(hwdev, buffer, sg->length, dir); if (!map) { /* Don''t panic here, we expect map_sg users to do proper error handling. */ @@ -803,11 +859,11 @@ BUG_ON(dir == DMA_NONE); for_each_sg(sgl, sg, nelems, i) { - if (sg->dma_address != SG_ENT_PHYS_ADDRESS(sg)) + if (sg->dma_address != swiotlb_sg_to_bus(sg)) unmap_single(hwdev, swiotlb_bus_to_virt(sg->dma_address), sg->dma_length, dir); else if (dir == DMA_FROM_DEVICE) - dma_mark_clean(SG_ENT_VIRT_ADDRESS(sg), sg->dma_length); + dma_mark_clean(swiotlb_bus_to_virt(sg->dma_address), sg->dma_length); } } EXPORT_SYMBOL(swiotlb_unmap_sg_attrs); @@ -836,11 +892,11 @@ BUG_ON(dir == DMA_NONE); for_each_sg(sgl, sg, nelems, i) { - if (sg->dma_address != SG_ENT_PHYS_ADDRESS(sg)) + if (sg->dma_address != swiotlb_sg_to_bus(sg)) sync_single(hwdev, swiotlb_bus_to_virt(sg->dma_address), sg->dma_length, dir, target); else if (dir == DMA_FROM_DEVICE) - dma_mark_clean(SG_ENT_VIRT_ADDRESS(sg), sg->dma_length); + dma_mark_clean(swiotlb_bus_to_virt(sg->dma_address), sg->dma_length); } } _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 10 of 14] swiotlb: consolidate swiotlb info message printing
From: Ian Campbell <ian.campbell@citrix.com> Remove duplicated swiotlb info printing, and make it more detailed. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- lib/swiotlb.c | 33 ++++++++++++++++++++++++++++----- 1 file changed, 28 insertions(+), 5 deletions(-) diff --git a/lib/swiotlb.c b/lib/swiotlb.c --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -156,6 +156,32 @@ return swiotlb_phys_to_bus(page_to_phys(sg_page(sg)) + sg->offset); } +static void swiotlb_print_info(unsigned long bytes) +{ + phys_addr_t pstart, pend; + dma_addr_t bstart, bend; + + pstart = virt_to_phys(io_tlb_start); + pend = virt_to_phys(io_tlb_end); + + bstart = swiotlb_phys_to_bus(pstart); + bend = swiotlb_phys_to_bus(pend); + + printk(KERN_INFO "Placing %luMB software IO TLB between %p - %p\n", + bytes >> 20, io_tlb_start, io_tlb_end); + if (pstart != bstart || pend != bend) + printk(KERN_INFO "software IO TLB at phys %#llx - %#llx" + " bus %#llx - %#llx\n", + (unsigned long long)pstart, + (unsigned long long)pend, + (unsigned long long)bstart, + (unsigned long long)bend); + else + printk(KERN_INFO "software IO TLB at phys %#llx - %#llx\n", + (unsigned long long)pstart, + (unsigned long long)pend); +} + /* * Statically reserve bounce buffer space and initialize bounce buffer data * structures for the software IO TLB used to implement the DMA API. @@ -198,8 +224,7 @@ if (!io_tlb_overflow_buffer) panic("Cannot allocate SWIOTLB overflow buffer!\n"); - printk(KERN_INFO "Placing software IO TLB between 0x%lx - 0x%lx\n", - swiotlb_virt_to_bus(io_tlb_start), swiotlb_virt_to_bus(io_tlb_end)); + swiotlb_print_info(bytes); } void __init @@ -279,9 +304,7 @@ if (!io_tlb_overflow_buffer) goto cleanup4; - printk(KERN_INFO "Placing %luMB software IO TLB between 0x%lx - " - "0x%lx\n", bytes >> 20, - swiotlb_virt_to_bus(io_tlb_start), swiotlb_virt_to_bus(io_tlb_end)); + swiotlb_print_info(bytes); return 0; _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 11 of 14] x86: add swiotlb allocation functions
Add x86-specific swiotlb allocation functions. These are purely default for the moment. Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- arch/x86/kernel/pci-swiotlb_64.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/x86/kernel/pci-swiotlb_64.c b/arch/x86/kernel/pci-swiotlb_64.c --- a/arch/x86/kernel/pci-swiotlb_64.c +++ b/arch/x86/kernel/pci-swiotlb_64.c @@ -3,6 +3,8 @@ #include <linux/pci.h> #include <linux/cache.h> #include <linux/module.h> +#include <linux/swiotlb.h> +#include <linux/bootmem.h> #include <linux/dma-mapping.h> #include <asm/iommu.h> @@ -11,6 +13,16 @@ int swiotlb __read_mostly; +void *swiotlb_alloc_boot(size_t size, unsigned long nslabs) +{ + return alloc_bootmem_low_pages(size); +} + +void *swiotlb_alloc(unsigned order, unsigned long nslabs) +{ + return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order); +} + static dma_addr_t swiotlb_map_single_phys(struct device *hwdev, phys_addr_t paddr, size_t size, int direction) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 12 of 14] x86: unify pci iommu setup and allow swiotlb to compile for 32 bit
swiotlb on 32 bit will be used by Xen domain 0 support. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- arch/x86/include/asm/dma-mapping.h | 2 +- arch/x86/include/asm/pci.h | 2 ++ arch/x86/include/asm/pci_64.h | 1 - arch/x86/kernel/Makefile | 3 ++- arch/x86/kernel/pci-dma.c | 6 ++++-- arch/x86/kernel/pci-swiotlb_64.c | 2 ++ arch/x86/mm/init_32.c | 3 +++ 7 files changed, 14 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h --- a/arch/x86/include/asm/dma-mapping.h +++ b/arch/x86/include/asm/dma-mapping.h @@ -66,7 +66,7 @@ return dma_ops; else return dev->archdata.dma_ops; -#endif /* _ASM_X86_DMA_MAPPING_H */ +#endif } /* Make sure we keep the same behaviour */ diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h --- a/arch/x86/include/asm/pci.h +++ b/arch/x86/include/asm/pci.h @@ -84,6 +84,8 @@ static inline void early_quirks(void) { } #endif +extern void pci_iommu_alloc(void); + #endif /* __KERNEL__ */ #ifdef CONFIG_X86_32 diff --git a/arch/x86/include/asm/pci_64.h b/arch/x86/include/asm/pci_64.h --- a/arch/x86/include/asm/pci_64.h +++ b/arch/x86/include/asm/pci_64.h @@ -23,7 +23,6 @@ int reg, int len, u32 value); extern void dma32_reserve_bootmem(void); -extern void pci_iommu_alloc(void); /* The PCI address space does equal the physical memory * address space. The networking and block device layers use diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -110,6 +110,8 @@ obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o +obj-$(CONFIG_SWIOTLB) += pci-swiotlb_64.o # NB rename without _64 + ### # 64 bit specific files ifeq ($(CONFIG_X86_64),y) @@ -123,7 +125,6 @@ obj-$(CONFIG_GART_IOMMU) += pci-gart_64.o aperture_64.o obj-$(CONFIG_CALGARY_IOMMU) += pci-calgary_64.o tce_64.o obj-$(CONFIG_AMD_IOMMU) += amd_iommu_init.o amd_iommu.o - obj-$(CONFIG_SWIOTLB) += pci-swiotlb_64.o obj-$(CONFIG_PCI_MMCONFIG) += mmconf-fam10h_64.o endif diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c --- a/arch/x86/kernel/pci-dma.c +++ b/arch/x86/kernel/pci-dma.c @@ -101,11 +101,15 @@ dma32_bootmem_ptr = NULL; dma32_bootmem_size = 0; } +#endif void __init pci_iommu_alloc(void) { +#ifdef CONFIG_X86_64 /* free the range so iommu could get some range less than 4G */ dma32_free_bootmem(); +#endif + /* * The order of these functions is important for * fall-back/fail-over reasons @@ -121,8 +125,6 @@ pci_swiotlb_init(); } -#endif - void *dma_generic_alloc_coherent(struct device *dev, size_t size, dma_addr_t *dma_addr, gfp_t flag) { diff --git a/arch/x86/kernel/pci-swiotlb_64.c b/arch/x86/kernel/pci-swiotlb_64.c --- a/arch/x86/kernel/pci-swiotlb_64.c +++ b/arch/x86/kernel/pci-swiotlb_64.c @@ -62,8 +62,10 @@ void __init pci_swiotlb_init(void) { /* don''t initialize swiotlb if iommu=off (no_iommu=1) */ +#ifdef CONFIG_X86_64 if (!iommu_detected && !no_iommu && max_pfn > MAX_DMA32_PFN) swiotlb = 1; +#endif if (swiotlb_force) swiotlb = 1; if (swiotlb) { diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c --- a/arch/x86/mm/init_32.c +++ b/arch/x86/mm/init_32.c @@ -21,6 +21,7 @@ #include <linux/init.h> #include <linux/highmem.h> #include <linux/pagemap.h> +#include <linux/pci.h> #include <linux/pfn.h> #include <linux/poison.h> #include <linux/bootmem.h> @@ -971,6 +972,8 @@ int codesize, reservedpages, datasize, initsize; int tmp; + pci_iommu_alloc(); + #ifdef CONFIG_FLATMEM BUG_ON(!mem_map); #endif _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 13 of 14] x86/swiotlb: add default phys<->bus conversion
From: Ian Campbell <ian.campbell@citrix.com> Xen will override these later on. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- arch/x86/kernel/pci-swiotlb_64.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/x86/kernel/pci-swiotlb_64.c b/arch/x86/kernel/pci-swiotlb_64.c --- a/arch/x86/kernel/pci-swiotlb_64.c +++ b/arch/x86/kernel/pci-swiotlb_64.c @@ -23,6 +23,16 @@ return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order); } +dma_addr_t swiotlb_phys_to_bus(phys_addr_t paddr) +{ + return paddr; +} + +phys_addr_t swiotlb_bus_to_phys(dma_addr_t baddr) +{ + return baddr; +} + static dma_addr_t swiotlb_map_single_phys(struct device *hwdev, phys_addr_t paddr, size_t size, int direction) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-16 20:17 UTC
[Xen-devel] [PATCH 14 of 14] x86/swiotlb: add default swiotlb_arch_range_needs_mapping
From: Ian Campbell <ian.campbell@citrix.com> Xen will override these later on. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> --- arch/x86/kernel/pci-swiotlb_64.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kernel/pci-swiotlb_64.c b/arch/x86/kernel/pci-swiotlb_64.c --- a/arch/x86/kernel/pci-swiotlb_64.c +++ b/arch/x86/kernel/pci-swiotlb_64.c @@ -33,6 +33,11 @@ return baddr; } +int __weak swiotlb_arch_range_needs_mapping(void *ptr, size_t size) +{ + return 0; +} + static dma_addr_t swiotlb_map_single_phys(struct device *hwdev, phys_addr_t paddr, size_t size, int direction) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ingo Molnar
2008-Dec-16 20:35 UTC
[Xen-devel] Re: [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 use of swiotlb
* Jeremy Fitzhardinge <jeremy@goop.org> wrote:> Hi Ingo, > > Here''s some patches to clean up swiotlb to prepare for some Xen dom0 > patches. These have been posted before, but undergone a round of > cleanups to address various comments.applied to tip/core/iommu, thanks Jeremy. the only patch that seems to have the potential to break drivers is: be4ac7b: swiotlb: consistently use address_needs_mapping everywhere looks fine to me, but the gents on the Cc: might have a dissenting opinion? Ingo ------------->>From be4ac7b87b27380bc43fa4f686e39b46eca2c866 Mon Sep 17 00:00:00 2001From: Jeremy Fitzhardinge <jeremy@goop.org> Date: Tue, 16 Dec 2008 12:17:28 -0800 Subject: [PATCH] swiotlb: consistently use address_needs_mapping everywhere Impact: remove open-coded DMA mask assumptions Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> --- lib/swiotlb.c | 10 +++------- 1 files changed, 3 insertions(+), 7 deletions(-) diff --git a/lib/swiotlb.c b/lib/swiotlb.c index db724ba..76821f0 100644 --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -465,13 +465,9 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size, dma_addr_t dev_addr; void *ret; int order = get_order(size); - u64 dma_mask = DMA_32BIT_MASK; - - if (hwdev && hwdev->coherent_dma_mask) - dma_mask = hwdev->coherent_dma_mask; ret = (void *)__get_free_pages(flags, order); - if (ret && !is_buffer_dma_capable(dma_mask, virt_to_bus(ret), size)) { + if (ret && !address_needs_mapping(hwdev, virt_to_bus(ret), size)) { /* * The allocated memory isn''t reachable by the device. * Fall back on swiotlb_map_single(). @@ -495,9 +491,9 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size, dev_addr = virt_to_bus(ret); /* Confirm address can be DMA''d by device */ - if (!is_buffer_dma_capable(dma_mask, dev_addr, size)) { + if (!address_needs_mapping(hwdev, dev_addr, size)) { printk("hwdev DMA mask = 0x%016Lx, dev_addr = 0x%016Lx\n", - (unsigned long long)dma_mask, + (unsigned long long)dma_get_mask(hwdev), (unsigned long long)dev_addr); /* DMA_TO_DEVICE to avoid memcpy in unmap_single */ _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2008-Dec-17 08:47 UTC
[Xen-devel] Re: [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 useof swiotlb
>I think that the whole patchset is against the swiotlb design. swiotlb >is designed to be used as a library. Each architecture implements the >own swiotlb by using swiotlb library >(e.g. arch/x86/kernel/pci-swiotlb_64.c).If it is a library, then it should be prepared to serve all possible users.>For example, adding the following code (9/14) for just Xen that the >majority of swiotbl users (x86_64 and IA64) don''t need to the library >is against the design."Don''t" in terms of "currently don''t": Once x86-64 wants to support more than 46 physical address bits, it''s not impossible that this would lead to CONFIG_HIGHMEM getting introduced there, and then it''ll be helpful that the code is already prepared to deal with that case. After all, the code portion in question ought to compile to nothing if !CONFIG_HIGHMEM. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-17 16:31 UTC
[Xen-devel] Re: [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 use of swiotlb
FUJITA Tomonori wrote:> On Tue, 16 Dec 2008 21:35:13 +0100 > Ingo Molnar <mingo@elte.hu> wrote: > > >> * Jeremy Fitzhardinge <jeremy@goop.org> wrote: >> >> >>> Hi Ingo, >>> >>> Here''s some patches to clean up swiotlb to prepare for some Xen dom0 >>> patches. These have been posted before, but undergone a round of >>> cleanups to address various comments. >>> >> applied to tip/core/iommu, thanks Jeremy. >> >> the only patch that seems to have the potential to break drivers is: >> >> be4ac7b: swiotlb: consistently use address_needs_mapping everywhere >> > > Yeah, as I already wrote, this patch is wrong. >I''ll have a look.> I think that the whole patchset is against the swiotlb design. swiotlb > is designed to be used as a library. Each architecture implements the > own swiotlb by using swiotlb library > (e.g. arch/x86/kernel/pci-swiotlb_64.c). >The whole patchset? The bulk of the changes to lib/swiotlb.c are relatively minor to remove the unwarranted assumptions it is making in the face of a new user. They will have no effect on other existing users, including non-Xen x86 builds. If you have specific objections we can discuss those, but I don''t think there''s anything fundamentally wrong with making lib/swiotlb.c a bit more generically useful.> For example, adding the following code (9/14) for just Xen that the > majority of swiotbl users (x86_64 and IA64) don''t need to the library > is against the design. >If the architecture doesn''t support highmem then this code will compile to nothing - PageHighMem() will always evaluate to 0. It will therefore have zero effect on the code generated for IA64 or x86-64. This is not really a Xen-specific change, but a result of adding swiotlb support for i386. Other architectures which support a notion of highmem would also need this code if they wanted to use swiotlb. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-17 16:40 UTC
[Xen-devel] Re: [PATCH 04 of 14] swiotlb: consistently use address_needs_mapping everywhere
FUJITA Tomonori wrote:> On Tue, 16 Dec 2008 12:17:28 -0800 > Jeremy Fitzhardinge <jeremy@goop.org> wrote: > > >> Signed-off-by: Ian Campbell <ian.campbell@citrix.com> >> --- >> lib/swiotlb.c | 10 +++------- >> 1 file changed, 3 insertions(+), 7 deletions(-) >> > > This patch is wrong. > > dma_mask and coherent_dma_mask represent different restrictions. > > You can''t use coherent_dma_mask for alloc_coherent. >OK. It looks like this patch is just an (erroneous) cleanup, so we can drop it. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2008-Dec-17 16:43 UTC
[Xen-devel] Re: [PATCH 04 of 14] swiotlb: consistently use address_needs_mapping everywhere
On Wed, 2008-12-17 at 11:51 +0900, FUJITA Tomonori wrote:> On Wed, 17 Dec 2008 11:48:41 +0900 > FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp> wrote: > > > On Tue, 16 Dec 2008 12:17:28 -0800 > > Jeremy Fitzhardinge <jeremy@goop.org> wrote: > > > > > Signed-off-by: Ian Campbell <ian.campbell@citrix.com> > > > --- > > > lib/swiotlb.c | 10 +++------- > > > 1 file changed, 3 insertions(+), 7 deletions(-) > > > > This patch is wrong. > > > > dma_mask and coherent_dma_mask represent different restrictions. > > > > You can''t use coherent_dma_mask for alloc_coherent. > > Oops, "you can''t use dma_mask for alloc_coherent."You are right, I completely overlooked that it was coherent_dma_mask in one place and just dma_mask in the other, sorry for the noise. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-17 18:58 UTC
[Xen-devel] Re: [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 use of swiotlb
FUJITA Tomonori wrote:> On Wed, 17 Dec 2008 08:31:43 -0800 > Jeremy Fitzhardinge <jeremy@goop.org> wrote: > > >>> I think that the whole patchset is against the swiotlb design. swiotlb >>> is designed to be used as a library. Each architecture implements the >>> own swiotlb by using swiotlb library >>> (e.g. arch/x86/kernel/pci-swiotlb_64.c). >>> >>> >> The whole patchset? The bulk of the changes to lib/swiotlb.c are >> relatively minor to remove the unwarranted assumptions it is making in >> the face of a new user. They will have no effect on other existing >> users, including non-Xen x86 builds. >> >> If you have specific objections we can discuss those, but I don''t think >> there''s anything fundamentally wrong with making lib/swiotlb.c a bit >> more generically useful. >> > > Sorry, but the highmem support is not generically useful. >That''s a circular argument. lib/swiotlb currently used by 1 1/2 of the 23 architectures, neither of which happens to use highmem. If you consider swiotlb to be a general purpose mechanism, then presumably the other 21 1/2 architectures are at least potential users (and 6 1/2 of those have highmem configurations). If you base your judgement of what''s a "generically useful" change based on what the current users need, then you''ll naturally exclude the requirements of all the other (potential) users. And the matter arises now because we''re trying to unify the use of swiotlb in x86, bringing the number of users up to 2.> I''m especially against the highmem support. As you said, the rest > looks fine but if you go with pci-swiotlb_32.c, I think that you don''t > need the most of them. >I really don''t want to have to duplicate a lot of code just to incorporate a few small changes. In fact the original Xen patch set included its own swiotlb implementation, and that was rejected on the grounds that we should use the common swiotlb.c. This patch set abstracts out a few more architecture-specific details, to allow the common swiotlb logic to be in one place. By not putting highmem support into lib/swiotlb.c, you''re effectively saying that we should have separate lib/swiotlb-nohighmem.c and lib/swiotlb-highmem.c files, with almost identical contents aside from the few places where we need to use page+offset rather than a kernel vaddr. Yes, highmem is ugly, and there''ll be cheering in the streets when we can finally kill the last highmem user. But for now we need to deal with it, and unfortunately swiotlb is now one of the places that it affects. Its true that adding highmem support to swiotlb.c increases the maintenance burden to some extent. But I can''t see an alternative that doesn''t increase the burden a lot more. We can look at abstracting things a bit more so that non-highmem architectures can keep using simple addresses rather than page+offset, but that highmem code has got to be *somewhere*.>> > have zero effect on the code generated for IA64 or x86-64. This is not >> > really a Xen-specific change, but a result of adding swiotlb support for >> > i386. Other architectures which support a notion of highmem would also >> > need this code if they wanted to use swiotlb. >> > > Can you be more specific? What architecture is plan to use highmem > support in swiotlb? >Well, concretely, x86-32. I don''t know about the others, but I don''t see why there''s an inherent reason that they won''t have a problem that swiotlb solves. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ingo Molnar
2008-Dec-18 13:23 UTC
[Xen-devel] Re: [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 use of swiotlb
* Jeremy Fitzhardinge <jeremy@goop.org> wrote:> FUJITA Tomonori wrote: >> On Wed, 17 Dec 2008 08:31:43 -0800 >> Jeremy Fitzhardinge <jeremy@goop.org> wrote: >> >> >>>> I think that the whole patchset is against the swiotlb design. swiotlb >>>> is designed to be used as a library. Each architecture implements the >>>> own swiotlb by using swiotlb library >>>> (e.g. arch/x86/kernel/pci-swiotlb_64.c). >>>> >>> The whole patchset? The bulk of the changes to lib/swiotlb.c are >>> relatively minor to remove the unwarranted assumptions it is making >>> in the face of a new user. They will have no effect on other >>> existing users, including non-Xen x86 builds. >>> >>> If you have specific objections we can discuss those, but I don''t >>> think there''s anything fundamentally wrong with making lib/swiotlb.c >>> a bit more generically useful. >>> >> >> Sorry, but the highmem support is not generically useful. >> > > That''s a circular argument. lib/swiotlb currently used by 1 1/2 of the > 23 architectures, neither of which happens to use highmem. If you > consider swiotlb to be a general purpose mechanism, then presumably the > other 21 1/2 architectures are at least potential users (and 6 1/2 of > those have highmem configurations). If you base your judgement of > what''s a "generically useful" change based on what the current users > need, then you''ll naturally exclude the requirements of all the other > (potential) users. > > And the matter arises now because we''re trying to unify the use of > swiotlb in x86, bringing the number of users up to 2. > >> I''m especially against the highmem support. As you said, the rest looks >> fine but if you go with pci-swiotlb_32.c, I think that you don''t need >> the most of them. >> > > I really don''t want to have to duplicate a lot of code just to > incorporate a few small changes. In fact the original Xen patch set > included its own swiotlb implementation, and that was rejected on the > grounds that we should use the common swiotlb.c.duplicating that would not be a very good design - and 32-bit highmem is a reality we have to live with for some time to come. The impact: 10 files changed, 251 insertions(+), 81 deletions(-) looks rather to the point and seems relatively compressed. In fact 32-bit Xen could end up being the largest user (and tester) of swiotlb facilities in general, as modern 64-bit platforms tend to have hw IOMMUs. Having more code sharing and more testers is a plus. Ingo _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-18 20:09 UTC
[Xen-devel] Re: [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 use of swiotlb
Becky Bruce wrote:> 32-bit powerpc needs this support - I was actually about to push a > similar set of patches. We have several processors that support 36 > bits of physical address space and do not have any iommu capability. > The rest of the kernel support for those processors is now in place, > so swiotlb is the last piece of the puzzle for that to be fully > functional. I need to take a closer look at this series to see > exactly what it''s doing and how it differs from what I''ve been testing. > > So there is another immediate use case, and I''d really hate to see > this code duplicated. It should be entirely possible to remove the > assumption that we can save off the VA of the original buffer, which > is the thing that precludes HIGHMEM support, and still have nice > readable, maintainable code.Good, I''m glad I''m not just making things up ;) The gist of the patch is to convert all the kernel virtual addresses into struct page+offset. It''s a fairly generic way to handle highmem systems, so it should work for you too. Of course, if you have a cleaner patch to achieve the same result, then we evaluate that too. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ingo Molnar
2008-Dec-18 21:02 UTC
[Xen-devel] Re: [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 use of swiotlb
* Becky Bruce <beckyb@kernel.crashing.org> wrote:>> Can you be more specific? What architecture is plan to use highmem >> support in swiotlb? > > 32-bit powerpc needs this support - I was actually about to push a > similar set of patches. We have several processors that support 36 bits > of physical address space and do not have any iommu capability. The > rest of the kernel support for those processors is now in place, so > swiotlb is the last piece of the puzzle for that to be fully functional. > I need to take a closer look at this series to see exactly what it''s > doing and how it differs from what I''ve been testing. > > So there is another immediate use case, and I''d really hate to see this > code duplicated. It should be entirely possible to remove the > assumption that we can save off the VA of the original buffer, which is > the thing that precludes HIGHMEM support, and still have nice readable, > maintainable code.you can have a look at Jeremy''s current lineup of patches in tip/core/iotlb: http://people.redhat.com/mingo/tip.git/README (also integrated into tip/master) What would be needed to make it suit your usecase too? Ingo _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-19 07:02 UTC
[Xen-devel] Re: [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 use of swiotlb
Becky Bruce wrote:> I''ve taken a quick look at the series posted to the list, and they''re > actually very similar to what I''ve done. I think there are really > only 3 fairly minor issues that need to be resolved to make this work > on ppc: > > 1) We need to add swiotlb_map/unmap_page calls, as those are part the > ppc dma_mapping_ops (in fact, we don''t have map/unmap_single in the > dma_mapping_ops struct on ppc anymore - those just call > map/unmap_page). This is really just a matter of moving some code > around and making some minor changes, as swiotlb_map/unmap_single can > call swiotlb_map/unmap_page once the args have been converted. > There''s a patch in my series that should make this pretty obvious. > > 2) To convert any address to/from a bus address, we also need the > hwdev pointer passed as an argument since ppc supports a per-device > offset accessed via the device ptr that is used to calculate the bus > address. I''d also given my conversion functions more generic names, > as it seemed highly likely that these would eventually be useful > outside of the swiotlb code. > > 3) powerpc uses enum dma_data_direction for the direction argument to > its dma_ops, which is, from reading the kernel docs, the correct > argument type for the DMA API functions. However, the iotlb/x86/ia64 > code is currently using an int. This causes a build warning when we > initialize the dma_ops struct using the swiotlb funcs. Is there a > reason for the use of "int" in x86/ia64? The Right Thing(TM) here > seems to be to convert those over to using the enum, and I have a big > patch that starts doing that, but I''ve probably missed some places. I > could instead do some hackery on the ppc side, and leave the other > platforms alone, but I''d prefer to do it cleanly. Thoughts? > > Unfortunately, this is horrible timing for me, as starting tomorrow, > I''m going to be offline for a week and a half or so in podunk > Louisiana with essentially no net access. I can integrate my code > into your tree and test on PPC as soon as I return to the real world.Yeah, I think that''s OK. The important thing at this point is to determine whether the two patch sets are aligned or conflicting. It sounds like they''re largely aligned, and so generating a delta from my patches to match your needs will be relatively straightforward. I''m trying to line all this Xen stuff up for this merge window, so I''d prefer to revisit it in the next dev cycle. Did you want to get something into this merge window? J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ingo Molnar
2008-Dec-19 08:18 UTC
[Xen-devel] Re: [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 use of swiotlb
* FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp> wrote:> On Thu, 18 Dec 2008 12:17:54 -0600 > Becky Bruce <beckyb@kernel.crashing.org> wrote: > > > > Can you be more specific? What architecture is plan to use highmem > > > support in swiotlb? > > > > 32-bit powerpc needs this support - I was actually about to push a > > similar set of patches. We have several processors that support 36 > > bits of physical address space and do not have any iommu capability. > > The rest of the kernel support for those processors is now in place, > > so swiotlb is the last piece of the puzzle for that to be fully > > functional. I need to take a closer look at this series to see > > exactly what it''s doing and how it differs from what I''ve been testing. > > Ah, thanks, > > Then I accept that highmem support in lib/swiotbl.c is generically > useful and we have to live with the ugly complication due to it.Great! And we of course want to make the best possible job here - so whenever you notice any area that could be done better and less intrusively, please mention it. Ingo _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-19 17:48 UTC
[Xen-devel] Re: [PATCH 00 of 14] swiotlb/x86: lay groundwork for xen dom0 use of swiotlb
Becky Bruce wrote:> > On Dec 19, 2008, at 1:02 AM, Jeremy Fitzhardinge wrote: > >> Becky Bruce wrote: >>> >>> >>> Unfortunately, this is horrible timing for me, as starting tomorrow, >>> I''m going to be offline for a week and a half or so in podunk >>> Louisiana with essentially no net access. I can integrate my code >>> into your tree and test on PPC as soon as I return to the real world. >> >> Yeah, I think that''s OK. The important thing at this point is to >> determine whether the two patch sets are aligned or conflicting. It >> sounds like they''re largely aligned, and so generating a delta from >> my patches to match your needs will be relatively straightforward. >> >> I''m trying to line all this Xen stuff up for this merge window, so >> I''d prefer to revisit it in the next dev cycle. Did you want to get >> something into this merge window? > > I was originally hoping to get something into this merge window, but > the timing just isn''t working out for me. The rest of the support is > already in the kernel for ppc (actually, most everything is in > 2.6.28), and swiotlb is the last bit I need to get this working. Ah > well.... I''ll hop to work on this from the ppc point of view as soon > as I get back online, and we''ll see where that takes us.Hm, yes, they''re really very similar patch sets. We can definitely turn this into something that will work for both. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel