search for: ttm_pl_flag_uncached

...is doesn't make it into v4.8 (which is due to be released in a week), it should get tagged for stable as well. By the way, Alexandre, I believe Martin was reporting issues like this on his TK1 before this change as well. Probably worth investigating - sounds like there's a wider issue with TTM_PL_FLAG_UNCACHED and/or force_coherent. > >> >> Signed-off-by: Karol Herbst <karolherbst at gmail.com> >> --- >> drm/nouveau/include/nvkm/core/device.h | 1 + >> drm/nouveau/nouveau_bo.c | 3 ++- >> drm/nouveau/nvkm/engine/device/pci.c | 1 + >> d...

..._bo.c index 528bdef..6887d0a 100644 --- a/drm/nouveau/nouveau_bo.c +++ b/drm/nouveau/nouveau_bo.c @@ -209,7 +209,8 @@ nouveau_bo_new(struct drm_device *dev, int size, int align, nvbo->tile_flags = tile_flags; nvbo->bo.bdev = &drm->ttm.bdev; - nvbo->force_coherent = flags & TTM_PL_FLAG_UNCACHED; + if (!nvxx_device(&drm->device)->func->cpu_coherent) + nvbo->force_coherent = flags & TTM_PL_FLAG_UNCACHED; nvbo->page_shift = 12; if (drm->client.vm) { diff --git a/drm/nouveau/nvkm/engine/device/pci.c b/drm/nouveau/nvkm/engine/device/pci.c index b1b6932..62ad030...

...veau/nouveau_bo.c index 8db54a217232..9cfb8e61f5c4 100644 --- a/drivers/gpu/drm/nouveau/nouveau_bo.c +++ b/drivers/gpu/drm/nouveau/nouveau_bo.c @@ -552,7 +552,11 @@ nouveau_bo_init_mem_type(struct ttm_bo_device *bdev, uint32_t type, TTM_MEMTYPE_FLAG_MAPPABLE; man->available_caching = TTM_PL_FLAG_UNCACHED | TTM_PL_FLAG_WC; +#if defined(__arm__) + man->default_caching = TTM_PL_FLAG_UNCACHED; +#else man->default_caching = TTM_PL_FLAG_WC; +#endif break; case TTM_PL_TT: if (nv_device(drm->device)->card_type >= NV_50) -- 1.9.2

It has been a couple of months since v4 - apologies for this. v4 has not received many comments, but this version addresses them and makes a new attempt at pushing the critical bit for GK20A and Nouveau on ARM in general. As a reminder, this series addresses the memory coherency issue that we are seeing on ARM platforms. Contrary to x86 which invalidates the PCI caches whenever a write is made by

...egra > > > > > > > in-SoC GPU, at least the fence logic was adjusted. However perhaps we > > > > > > > didn't do it for notifier memory... worth checking. > > > > > > > > > > > > Hm, looks like we should be passing in TTM_PL_FLAG_UNCACHED for the > > > > > > notifier, which should cause nouveau_bo to do the right thing (by > > > > > > setting force_coherent=true in this environment): > > > > > > > > > > > > ret = nouveau_bo_new(&drm->client, 4096, 0x1000,...

...requests an uncached mapping, we actually get an "undefined caching policy" one and this has strange side-effects described on Freedesktop bug 86690. To prevent this from happening, allow user-space to explicitly specify which objects should be coherent, and create such objects with the TTM_PL_FLAG_UNCACHED flag. This will make TTM allocate memory using the DMA API, which will fix the identify mapping and allow us to safely map the objects to user-space uncached. Signed-off-by: Alexandre Courbot <acourbot at nvidia.com> --- Patches that take advantage of this in Mesa will follow up shortly. I...

...domain = nvxx_fb(&drm->device)->ram ? TTM_PL_FLAG_VRAM : TTM_PL_FLAG_TT; > ret = nouveau_bo_new(drm->dev, 16 * priv->base.contexts, 0, > - TTM_PL_FLAG_VRAM, 0, 0, NULL, NULL, &priv->bo); > + domain | TTM_PL_FLAG_UNCACHED, And the TTM_PL_FLAG_UNCACHED here. Are those intentional, I don't see any hints about them in the commit summary ? Thanks Emil

...> synchronization as it will get flushed on pushbuf kickoff anyways. I tried to go that way, and something interesting happened. What I did: remove this patch and instead set the following caching parameters for the TTM_PL_TT case in nouveau_bo_init_mem_type(): man->available_caching = TTM_PL_FLAG_UNCACHED | TTM_PL_FLAG_WC; man->default_caching = TTM_PL_FLAG_WC; What happened: no runtime errors as what happened when caching is enabled. However, many of the vertex and texture buffers seem to be partially corrupted. In glmark2 the 3d models had many vertices (but not all) at the wrong position....

...are used by the kernel, typically fences and GPFIFO buffers, are accessed rarely and thus should not trigger a costly flush or cache invalidation. For these, we want to guarantee coherent access and use the DMA API if necessary. This series attempts to implement this behavior by allowing the TTM_PL_FLAG_UNCACHED flag to be passed to nouveau_bo_new(). On coherent architectures this flag is a no-op ; on non-coherent architectures, it will force the creation of a coherent buffer using the DMA-API. Several fixes and changes were necessary to enable this behavior: - CPU addresses of DMA-allocated BOs must be...

...db54a217232..9cfb8e61f5c4 100644 > --- a/drivers/gpu/drm/nouveau/nouveau_bo.c > +++ b/drivers/gpu/drm/nouveau/nouveau_bo.c > @@ -552,7 +552,11 @@ nouveau_bo_init_mem_type(struct ttm_bo_device *bdev, uint32_t type, > TTM_MEMTYPE_FLAG_MAPPABLE; > man->available_caching = TTM_PL_FLAG_UNCACHED | > TTM_PL_FLAG_WC; > +#if defined(__arm__) > + man->default_caching = TTM_PL_FLAG_UNCACHED; > +#else > man->default_caching = TTM_PL_FLAG_WC; > +#endif > break; > case TTM_PL_TT: > if (nv_device(drm->device)->card_type >= NV_50) -- Pen...

...nouveau/nouveau_bo.c > +++ b/drm/nouveau/nouveau_bo.c > @@ -209,7 +209,8 @@ nouveau_bo_new(struct drm_device *dev, int size, int align, > nvbo->tile_flags = tile_flags; > nvbo->bo.bdev = &drm->ttm.bdev; > > - nvbo->force_coherent = flags & TTM_PL_FLAG_UNCACHED; > + if (!nvxx_device(&drm->device)->func->cpu_coherent) > + nvbo->force_coherent = flags & TTM_PL_FLAG_UNCACHED; > > nvbo->page_shift = 12; > if (drm->client.vm) { > diff --git a/drm/nouveau/nvkm/engine/device/pci.c b/...

...ouveau/nouveau_bo.c @@ -500,18 +500,25 @@ nouveau_bo_init_mem_type(struct ttm_bo_device *bdev, uint32_t type, man->default_caching = TTM_PL_FLAG_CACHED; break; case TTM_PL_VRAM: + man->flags = TTM_MEMTYPE_FLAG_FIXED | + TTM_MEMTYPE_FLAG_MAPPABLE; + man->available_caching = TTM_PL_FLAG_UNCACHED | + TTM_PL_FLAG_WC; + man->default_caching = TTM_PL_FLAG_WC; + if (nv_device(drm->device)->card_type >= NV_50) { + /* Some BARs do not support being ioremapped WC */ + if (nouveau_bar(drm->device)->iomap_uncached) { + man->available_caching = TTM_PL_FLAG_UNCACHE...

...get flushed on pushbuf kickoff anyways. > > I tried to go that way, and something interesting happened. > > What I did: remove this patch and instead set the following caching > parameters for the TTM_PL_TT case in nouveau_bo_init_mem_type(): > > man->available_caching = TTM_PL_FLAG_UNCACHED | TTM_PL_FLAG_WC; > man->default_caching = TTM_PL_FLAG_WC; > > What happened: no runtime errors as what happened when caching is > enabled. However, many of the vertex and texture buffers seem to be > partially corrupted. In glmark2 the 3d models had many vertices (but > no...

TTM-allocated coherent objects were populated using the DMA API and accessed using the mapping it returned to workaround coherency issues. These issues seem to have been solved, thus remove this extra case to handle and use the regular kernel mapping functions. Signed-off-by: Alexandre Courbot <acourbot at nvidia.com> --- drm/nouveau/nouveau_bo.c | 61

...struct drm_device *dev, int size, int align, > > nouveau_bo_fixup_align(nvbo, flags, &align, &size); > nvbo->bo.mem.num_pages = size >> PAGE_SHIFT; > + > + if (tile_flags & NOUVEAU_GEM_TILE_WCUS) > + nvbo->valid_caching = TTM_PL_FLAG_UNCACHED | TTM_PL_FLAG_WC; > + else > + nvbo->valid_caching = TTM_PL_MASK_CACHING; > + > nouveau_bo_placement_set(nvbo, flags, 0); > > acc_size = ttm_bo_dma_acc_size(&drm->ttm.bdev, size, > @@ -292,7 +298,7 @@ void > nouveau_bo_placement...

...om [<c00ea5ec>] (do_vfs_ioctl+0x3f0/0x5bc) [ 78.440277] [<c00ea5ec>] (do_vfs_ioctl) from [<c00ea7ec>] (SyS_ioctl+0x34/0x5c) [ 78.453918] [<c00ea7ec>] (SyS_ioctl) from [<c000e5a0>] (ret_fast_syscall+0x0/0x30) To avoid these I need to set the VRAM default_caching to TTM_PL_FLAG_UNCACHED. It is not clear to me why this is needed. The BO being accessed through the BAR, they are correctly considered as IO memory and mapped using ttm_bo_ioremap(), so it really seems to be unhappy with the WC mapping itself. Note that if I go ahead and force the use of pgprot_writecombine() in ttm_io_...

Am Mittwoch, den 06.05.2020, 10:26 -0400 schrieb Ilia Mirkin: > [please keep list cc'd in your replies] > > On Wed, May 6, 2020 at 10:15 AM Milan Bu?ka <milan.buska at gmail.com> wrote: > > [ 0.000000] Linux version 5.6.10-zotac (root at saux) (gcc version 9.3.0 (SAUX Aarch64)) #1 SMP PREEMPT Tue May 5 22:16:40 CEST 2020 > > [ 0.000000] Machine model: NVIDIA

..._t type, man->available_caching = man->default_caching = 0; break; + case TTM_PL_PRIV1: /* Mappable but unusable as scanout. */ + { + man->flags = TTM_MEMTYPE_FLAG_FIXED | + TTM_MEMTYPE_FLAG_MAPPABLE | + TTM_MEMTYPE_FLAG_NEEDS_IOREMAP; + man->available_caching = TTM_PL_FLAG_UNCACHED | + TTM_PL_FLAG_WC; + man->default_caching = TTM_PL_FLAG_WC; + + man->io_addr = NULL; + man->io_offset = drm_get_resource_start(dev, 1); + man->io_size = drm_get_resource_len(dev, 1); + man->gpu_offset = dev_priv->vm_vram_base; + break; + } case TTM_PL_TT: switch (...

...nto v4.8 (which is due to be released in a > week), it should get tagged for stable as well. > > By the way, Alexandre, I believe Martin was reporting issues like this > on his TK1 before this change as well. Probably worth investigating - > sounds like there's a wider issue with TTM_PL_FLAG_UNCACHED and/or > force_coherent. Yes, it may be indeed related. I need to sync with Martin to understand his exact repro case.

...m/nouveau/nouveau_bo.c @@ -231,6 +231,12 @@ nouveau_bo_new(struct drm_device *dev, int size, int align, nouveau_bo_fixup_align(nvbo, flags, &align, &size); nvbo->bo.mem.num_pages = size >> PAGE_SHIFT; + + if (tile_flags & NOUVEAU_GEM_TILE_WCUS) + nvbo->valid_caching = TTM_PL_FLAG_UNCACHED | TTM_PL_FLAG_WC; + else + nvbo->valid_caching = TTM_PL_MASK_CACHING; + nouveau_bo_placement_set(nvbo, flags, 0); acc_size = ttm_bo_dma_acc_size(&drm->ttm.bdev, size, @@ -292,7 +298,7 @@ void nouveau_bo_placement_set(struct nouveau_bo *nvbo, uint32_t type, uint32_t busy) { stru...