thr3ads.net - Nouveau - [Nouveau] [Intel-gfx] [PATCH v6 08/17] drm/ttm: use gem vma

If this information is useful, please help other people find it:
Share via:

Gerd Hoffmann

2019-Aug-14 05:58 UTC

[Nouveau] [Intel-gfx] [PATCH v6 08/17] drm/ttm: use gem vma_node

> Hi Gerd,
> 
> I've been seeing a regression on Nouveau with recent linux-next
releases
> and git bisect points at this commit as the first bad one. If I revert
> it (there's a tiny conflict with a patch that was merged subsequently),
> things are back to normal.
> 
> I think the reason for this issue is that Nouveau doesn't use GEM
> objects for all buffer objects,
That shouldn't be a problem ...
> and even when it uses GEM objects, the
> code will not initialize the GEM object until after the buffer objects
> and the backing TTM objects have been created.
... but the initialization order is.

ttm_bo_uses_embedded_gem_object() assumes gem gets initialized first.

drm_gem_object_init() init calling drm_vma_node_reset() again is
probably the root cause for the breakage.
> I tried to fix that by making sure drm_gem_object_init() gets called by
> Nouveau before ttm_bo_init(), but the changes are fairly involved and I
> was unable to get the GEM reference counting right. I can look into the
> proper fix some more, but it might be worth reverting this patch for
> now to get Nouveau working again.
Changing the order doesn't look hard.  Patch attached (untested, have no
test hardware).  But maybe I missed some detail ...

The other patch attached works around the issue with a flag, to avoid
drm_vma_node_reset() being called twice.

cheers,
  Gerd

-------------- next part -------------->From af43f933533140e2df58176a68df0c60ba082273 Mon Sep 17 00:00:00 2001From: Gerd Hoffmann <kraxel at redhat.com>
Date: Wed, 14 Aug 2019 07:23:31 +0200
Subject: [PATCH 1/2] try unbreak nouveau #1

---
 include/drm/drm_gem.h        | 11 +++++++++++
 drivers/gpu/drm/drm_gem.c    |  6 ++++--
 drivers/gpu/drm/ttm/ttm_bo.c |  3 ++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index ae693c0666cd..24e8fc58a3e1 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -297,6 +297,17 @@ struct drm_gem_object {
 	 *
 	 */
 	const struct drm_gem_object_funcs *funcs;
+
+	/**
+	 * @ttm_init: indicate ttm has initialized _resv and vma_node fields.
+	 *
+	 * ttm_bo_uses_embedded_gem_object() assumes gem is
+	 * initialized before ttm, nouveau does it the other way
+	 * around though.
+	 *
+	 * This is a temporary stopgap to handle that case.
+	 */
+	bool ttm_init;
 };
 
 /**
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index afc38cece3f5..0a75d8cf7ac7 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -159,11 +159,13 @@ void drm_gem_private_object_init(struct drm_device *dev,
 	kref_init(&obj->refcount);
 	obj->handle_count = 0;
 	obj->size = size;
-	reservation_object_init(&obj->_resv);
 	if (!obj->resv)
 		obj->resv = &obj->_resv;
 
-	drm_vma_node_reset(&obj->vma_node);
+	if (!obj->ttm_init) {
+		reservation_object_init(&obj->_resv);
+		drm_vma_node_reset(&obj->vma_node);
+	}
 }
 EXPORT_SYMBOL(drm_gem_private_object_init);
 
diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c
index 10a861a1690c..83b389fc117e 100644
--- a/drivers/gpu/drm/ttm/ttm_bo.c
+++ b/drivers/gpu/drm/ttm/ttm_bo.c
@@ -160,7 +160,7 @@ static void ttm_bo_release_list(struct kref *list_kref)
 	ttm_tt_destroy(bo->ttm);
 	atomic_dec(&bo->bdev->glob->bo_count);
 	dma_fence_put(bo->moving);
-	if (!ttm_bo_uses_embedded_gem_object(bo))
+	if (bo->base.ttm_init)
 		reservation_object_fini(&bo->base._resv);
 	mutex_destroy(&bo->wu_mutex);
 	bo->destroy(bo);
@@ -1344,6 +1344,7 @@ int ttm_bo_init_reserved(struct ttm_bo_device *bdev,
 		 */
 		reservation_object_init(&bo->base._resv);
 		drm_vma_node_reset(&bo->base.vma_node);
+		bo->base.ttm_init = true;
 	}
 	atomic_inc(&bo->bdev->glob->bo_count);
 
-- 
2.18.1

-------------- next part -------------->From 3e36d5819ed5330068340e78c7a1bf35451b1dad Mon Sep 17 00:00:00 2001From: Gerd Hoffmann <kraxel at redhat.com>
Date: Wed, 14 Aug 2019 07:41:05 +0200
Subject: [PATCH 2/2] try unbreak nouveau #2

---
 drivers/gpu/drm/nouveau/nouveau_bo.h    |  4 ++--
 drivers/gpu/drm/nouveau/dispnv04/crtc.c |  2 +-
 drivers/gpu/drm/nouveau/dispnv50/disp.c |  2 +-
 drivers/gpu/drm/nouveau/nouveau_bo.c    | 11 ++++++++++-
 drivers/gpu/drm/nouveau/nouveau_chan.c  |  2 +-
 drivers/gpu/drm/nouveau/nouveau_dmem.c  |  2 +-
 drivers/gpu/drm/nouveau/nouveau_gem.c   | 10 +---------
 drivers/gpu/drm/nouveau/nouveau_prime.c | 10 +---------
 drivers/gpu/drm/nouveau/nv17_fence.c    |  2 +-
 drivers/gpu/drm/nouveau/nv50_fence.c    |  2 +-
 drivers/gpu/drm/nouveau/nv84_fence.c    |  2 +-
 11 files changed, 21 insertions(+), 28 deletions(-)

diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.h
b/drivers/gpu/drm/nouveau/nouveau_bo.h
index d675efe8e7f9..4c268f299226 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo.h
+++ b/drivers/gpu/drm/nouveau/nouveau_bo.h
@@ -73,7 +73,7 @@ extern struct ttm_bo_driver nouveau_bo_driver;
 void nouveau_bo_move_init(struct nouveau_drm *);
 int  nouveau_bo_new(struct nouveau_cli *, u64 size, int align, u32 flags,
 		    u32 tile_mode, u32 tile_flags, struct sg_table *sg,
-		    struct reservation_object *robj,
+		    struct reservation_object *robj, bool gem_init,
 		    struct nouveau_bo **);
 int  nouveau_bo_pin(struct nouveau_bo *, u32 flags, bool contig);
 int  nouveau_bo_unpin(struct nouveau_bo *);
@@ -115,7 +115,7 @@ nouveau_bo_new_pin_map(struct nouveau_cli *cli, u64 size,
int align, u32 flags,
 		       struct nouveau_bo **pnvbo)
 {
 	int ret = nouveau_bo_new(cli, size, align, flags,
-				 0, 0, NULL, NULL, pnvbo);
+				 0, 0, NULL, NULL, false, pnvbo);
 	if (ret == 0) {
 		ret = nouveau_bo_pin(*pnvbo, flags, true);
 		if (ret == 0) {
diff --git a/drivers/gpu/drm/nouveau/dispnv04/crtc.c
b/drivers/gpu/drm/nouveau/dispnv04/crtc.c
index f22f01020625..5640f5e45c11 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/crtc.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/crtc.c
@@ -1366,7 +1366,7 @@ nv04_crtc_create(struct drm_device *dev, int crtc_num)
 
 	ret = nouveau_bo_new(&nouveau_drm(dev)->client, 64*64*4, 0x100,
 			     TTM_PL_FLAG_VRAM, 0, 0x0000, NULL, NULL,
-			     &nv_crtc->cursor.nvbo);
+			     false, &nv_crtc->cursor.nvbo);
 	if (!ret) {
 		ret = nouveau_bo_pin(nv_crtc->cursor.nvbo, TTM_PL_FLAG_VRAM, false);
 		if (!ret) {
diff --git a/drivers/gpu/drm/nouveau/dispnv50/disp.c
b/drivers/gpu/drm/nouveau/dispnv50/disp.c
index 8497768f1b41..047c0166f823 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/disp.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/disp.c
@@ -2319,7 +2319,7 @@ nv50_display_create(struct drm_device *dev)
 
 	/* small shared memory area we use for notifiers and semaphores */
 	ret = nouveau_bo_new(&drm->client, 4096, 0x1000, TTM_PL_FLAG_VRAM,
-			     0, 0x0000, NULL, NULL, &disp->sync);
+			     0, 0x0000, NULL, NULL, false, &disp->sync);
 	if (!ret) {
 		ret = nouveau_bo_pin(disp->sync, TTM_PL_FLAG_VRAM, true);
 		if (!ret) {
diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c
b/drivers/gpu/drm/nouveau/nouveau_bo.c
index 99e391be9370..b1b67a427a98 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bo.c
@@ -189,7 +189,7 @@ int
 nouveau_bo_new(struct nouveau_cli *cli, u64 size, int align,
 	       uint32_t flags, uint32_t tile_mode, uint32_t tile_flags,
 	       struct sg_table *sg, struct reservation_object *robj,
-	       struct nouveau_bo **pnvbo)
+	       bool gem_init, struct nouveau_bo **pnvbo)
 {
 	struct nouveau_drm *drm = cli->drm;
 	struct nouveau_bo *nvbo;
@@ -295,6 +295,15 @@ nouveau_bo_new(struct nouveau_cli *cli, u64 size, int
align,
 	acc_size = ttm_bo_dma_acc_size(&drm->ttm.bdev, size,
 				       sizeof(struct nouveau_bo));
 
+	if (gem_init) {
+		/* Initialize the embedded gem-object. */
+		ret = drm_gem_object_init(drm->dev, &nvbo->bo.base, size);
+		if (ret) {
+			kfree(nvbo);
+			return -ENOMEM;
+		}
+	}
+
 	ret = ttm_bo_init(&drm->ttm.bdev, &nvbo->bo, size,
 			  type, &nvbo->placement,
 			  align >> PAGE_SHIFT, false, acc_size, sg,
diff --git a/drivers/gpu/drm/nouveau/nouveau_chan.c
b/drivers/gpu/drm/nouveau/nouveau_chan.c
index 282fd90b65e1..fedf3239b2e4 100644
--- a/drivers/gpu/drm/nouveau/nouveau_chan.c
+++ b/drivers/gpu/drm/nouveau/nouveau_chan.c
@@ -144,7 +144,7 @@ nouveau_channel_prep(struct nouveau_drm *drm, struct
nvif_device *device,
 		target = TTM_PL_FLAG_VRAM;
 
 	ret = nouveau_bo_new(cli, size, 0, target, 0, 0, NULL, NULL,
-			    &chan->push.buffer);
+			     false, &chan->push.buffer);
 	if (ret == 0) {
 		ret = nouveau_bo_pin(chan->push.buffer, target, false);
 		if (ret == 0)
diff --git a/drivers/gpu/drm/nouveau/nouveau_dmem.c
b/drivers/gpu/drm/nouveau/nouveau_dmem.c
index 1333220787a1..4090ba54bbdd 100644
--- a/drivers/gpu/drm/nouveau/nouveau_dmem.c
+++ b/drivers/gpu/drm/nouveau/nouveau_dmem.c
@@ -314,7 +314,7 @@ nouveau_dmem_chunk_alloc(struct nouveau_drm *drm)
 
 	ret = nouveau_bo_new(&drm->client, DMEM_CHUNK_SIZE, 0,
 			     TTM_PL_FLAG_VRAM, 0, 0, NULL, NULL,
-			     &chunk->bo);
+			     false, &chunk->bo);
 	if (ret)
 		goto out;
 
diff --git a/drivers/gpu/drm/nouveau/nouveau_gem.c
b/drivers/gpu/drm/nouveau/nouveau_gem.c
index c7368aa0bdec..1993f2c90e4c 100644
--- a/drivers/gpu/drm/nouveau/nouveau_gem.c
+++ b/drivers/gpu/drm/nouveau/nouveau_gem.c
@@ -189,7 +189,7 @@ nouveau_gem_new(struct nouveau_cli *cli, u64 size, int
align, uint32_t domain,
 		flags |= TTM_PL_FLAG_UNCACHED;
 
 	ret = nouveau_bo_new(cli, size, align, flags, tile_mode,
-			     tile_flags, NULL, NULL, pnvbo);
+			     tile_flags, NULL, NULL, true, pnvbo);
 	if (ret)
 		return ret;
 	nvbo = *pnvbo;
@@ -203,14 +203,6 @@ nouveau_gem_new(struct nouveau_cli *cli, u64 size, int
align, uint32_t domain,
 	if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_TESLA)
 		nvbo->valid_domains &= domain;
 
-	/* Initialize the embedded gem-object. We return a single gem-reference
-	 * to the caller, instead of a normal nouveau_bo ttm reference. */
-	ret = drm_gem_object_init(drm->dev, &nvbo->bo.base,
nvbo->bo.mem.size);
-	if (ret) {
-		nouveau_bo_ref(NULL, pnvbo);
-		return -ENOMEM;
-	}
-
 	nvbo->bo.persistent_swap_storage = nvbo->bo.base.filp;
 	return 0;
 }
diff --git a/drivers/gpu/drm/nouveau/nouveau_prime.c
b/drivers/gpu/drm/nouveau/nouveau_prime.c
index e86ad7ae622b..db1ec2d0d4c9 100644
--- a/drivers/gpu/drm/nouveau/nouveau_prime.c
+++ b/drivers/gpu/drm/nouveau/nouveau_prime.c
@@ -70,21 +70,13 @@ struct drm_gem_object
*nouveau_gem_prime_import_sg_table(struct drm_device *dev,
 
 	reservation_object_lock(robj, NULL);
 	ret = nouveau_bo_new(&drm->client, attach->dmabuf->size, 0,
flags, 0, 0,
-			     sg, robj, &nvbo);
+			     sg, robj, true, &nvbo);
 	reservation_object_unlock(robj);
 	if (ret)
 		return ERR_PTR(ret);
 
 	nvbo->valid_domains = NOUVEAU_GEM_DOMAIN_GART;
 
-	/* Initialize the embedded gem-object. We return a single gem-reference
-	 * to the caller, instead of a normal nouveau_bo ttm reference. */
-	ret = drm_gem_object_init(dev, &nvbo->bo.base, nvbo->bo.mem.size);
-	if (ret) {
-		nouveau_bo_ref(NULL, &nvbo);
-		return ERR_PTR(-ENOMEM);
-	}
-
 	return &nvbo->bo.base;
 }
 
diff --git a/drivers/gpu/drm/nouveau/nv17_fence.c
b/drivers/gpu/drm/nouveau/nv17_fence.c
index 5d613d43b84d..6c589d0b847c 100644
--- a/drivers/gpu/drm/nouveau/nv17_fence.c
+++ b/drivers/gpu/drm/nouveau/nv17_fence.c
@@ -128,7 +128,7 @@ nv17_fence_create(struct nouveau_drm *drm)
 	spin_lock_init(&priv->lock);
 
 	ret = nouveau_bo_new(&drm->client, 4096, 0x1000, TTM_PL_FLAG_VRAM,
-			     0, 0x0000, NULL, NULL, &priv->bo);
+			     0, 0x0000, NULL, NULL, false, &priv->bo);
 	if (!ret) {
 		ret = nouveau_bo_pin(priv->bo, TTM_PL_FLAG_VRAM, false);
 		if (!ret) {
diff --git a/drivers/gpu/drm/nouveau/nv50_fence.c
b/drivers/gpu/drm/nouveau/nv50_fence.c
index a00ecc3de053..17910bfca099 100644
--- a/drivers/gpu/drm/nouveau/nv50_fence.c
+++ b/drivers/gpu/drm/nouveau/nv50_fence.c
@@ -81,7 +81,7 @@ nv50_fence_create(struct nouveau_drm *drm)
 	spin_lock_init(&priv->lock);
 
 	ret = nouveau_bo_new(&drm->client, 4096, 0x1000, TTM_PL_FLAG_VRAM,
-			     0, 0x0000, NULL, NULL, &priv->bo);
+			     0, 0x0000, NULL, NULL, false, &priv->bo);
 	if (!ret) {
 		ret = nouveau_bo_pin(priv->bo, TTM_PL_FLAG_VRAM, false);
 		if (!ret) {
diff --git a/drivers/gpu/drm/nouveau/nv84_fence.c
b/drivers/gpu/drm/nouveau/nv84_fence.c
index f07da00f285f..fecdb036521e 100644
--- a/drivers/gpu/drm/nouveau/nv84_fence.c
+++ b/drivers/gpu/drm/nouveau/nv84_fence.c
@@ -204,7 +204,7 @@ nv84_fence_create(struct nouveau_drm *drm)
 			  */
 			 TTM_PL_FLAG_TT | TTM_PL_FLAG_UNCACHED;
 	ret = nouveau_bo_new(&drm->client, 16 * drm->chan.nr, 0,
-			     domain, 0, 0, NULL, NULL, &priv->bo);
+			     domain, 0, 0, NULL, NULL, false, &priv->bo);
 	if (ret == 0) {
 		ret = nouveau_bo_pin(priv->bo, domain, false);
 		if (ret == 0) {
-- 
2.18.1

Thierry Reding

2019-Aug-14 09:35 UTC

head link

[Nouveau] [Intel-gfx] [PATCH v6 08/17] drm/ttm: use gem vma_node

On Wed, Aug 14, 2019 at 07:58:27AM +0200, Gerd Hoffmann
wrote:> > Hi Gerd,
> > 
> > I've been seeing a regression on Nouveau with recent linux-next
releases
> > and git bisect points at this commit as the first bad one. If I revert
> > it (there's a tiny conflict with a patch that was merged
subsequently),
> > things are back to normal.
> > 
> > I think the reason for this issue is that Nouveau doesn't use GEM
> > objects for all buffer objects,
> 
> That shouldn't be a problem ...
> 
> > and even when it uses GEM objects, the
> > code will not initialize the GEM object until after the buffer objects
> > and the backing TTM objects have been created.
> 
> ... but the initialization order is.
> 
> ttm_bo_uses_embedded_gem_object() assumes gem gets initialized first.
> 
> drm_gem_object_init() init calling drm_vma_node_reset() again is
> probably the root cause for the breakage.
> 
> > I tried to fix that by making sure drm_gem_object_init() gets called
by
> > Nouveau before ttm_bo_init(), but the changes are fairly involved and
I
> > was unable to get the GEM reference counting right. I can look into
the
> > proper fix some more, but it might be worth reverting this patch for
> > now to get Nouveau working again.
> 
> Changing the order doesn't look hard.  Patch attached (untested, have
no
> test hardware).  But maybe I missed some detail ...
> 
> The other patch attached works around the issue with a flag, to avoid
> drm_vma_node_reset() being called twice.
I came up with something very similar by splitting up nouveau_bo_new()
into allocation and initialization steps, so that when necessary the GEM
object can be initialized in between. I think that's slightly more
flexible and easier to understand than a boolean flag.

Thierry
-------------- next part --------------

Gerd Hoffmann

2019-Aug-14 10:14 UTC

head link

[Nouveau] [Intel-gfx] [PATCH v6 08/17] drm/ttm: use gem vma_node

Hi,
> > Changing the order doesn't look hard.  Patch attached (untested,
have no
> > test hardware).  But maybe I missed some detail ...
> 
> I came up with something very similar by splitting up nouveau_bo_new()
> into allocation and initialization steps, so that when necessary the GEM
> object can be initialized in between. I think that's slightly more
> flexible and easier to understand than a boolean flag.
Yes, that should work too.

Acked-by: Gerd Hoffmann <kraxel at redhat.com>

cheers,
  Gerd

Possibly Parallel Threads

Search for more reasonably related threads

Nouveau - Aug 2019 - [Intel-gfx] [PATCH v6 08/17] drm/ttm: use gem vma_node

[Nouveau] [Intel-gfx] [PATCH v6 08/17] drm/ttm: use gem vma_node

[Nouveau] [Intel-gfx] [PATCH v6 08/17] drm/ttm: use gem vma_node

[Nouveau] [Intel-gfx] [PATCH v6 08/17] drm/ttm: use gem vma_node

Possibly Parallel Threads