Linux Ethernet Bridging - Dec 2010 - [Bridge] [PATCH] bridge: Fix return value of br_multicast_add_group()

If br_multicast_new_group returns NULL, we would return 0 (no error) to
the caller, which is not what we want. Instead we should return -ENOMEM
in this case.

Also replace IS_ERR(x) || !x by IS_ERR_OR_NULL(x)

Signed-off-by: Tobias Klauser <tklauser at distanz.ch>
---
 net/bridge/br_multicast.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/net/bridge/br_multicast.c b/net/bridge/br_multicast.c
index 326e599..d4e1e81 100644
--- a/net/bridge/br_multicast.c
+++ b/net/bridge/br_multicast.c
@@ -713,8 +713,11 @@ static int br_multicast_add_group(struct net_bridge *br,
 
 	mp = br_multicast_new_group(br, port, group);
 	err = PTR_ERR(mp);
-	if (unlikely(IS_ERR(mp) || !mp))
+	if (IS_ERR_OR_NULL(mp)) {
+		if (!mp)
+			err = -ENOMEM;
 		goto err;
+	}
 
 	if (!port) {
 		hlist_add_head(&mp->mglist, &br->mglist);
-- 
1.7.0.4

On Thu,  9 Dec 2010 15:02:36 +0100
Tobias Klauser <tklauser at distanz.ch> wrote:
> If br_multicast_new_group returns NULL, we would return 0 (no error) to
> the caller, which is not what we want. Instead we should return -ENOMEM
> in this case.
> 
> Also replace IS_ERR(x) || !x by IS_ERR_OR_NULL(x)
> 
> Signed-off-by: Tobias Klauser <tklauser at distanz.ch>
> ---
>  net/bridge/br_multicast.c |    5 ++++-
>  1 files changed, 4 insertions(+), 1 deletions(-)
> 
> diff --git a/net/bridge/br_multicast.c b/net/bridge/br_multicast.c
> index 326e599..d4e1e81 100644
> --- a/net/bridge/br_multicast.c
> +++ b/net/bridge/br_multicast.c
> @@ -713,8 +713,11 @@ static int br_multicast_add_group(struct net_bridge
*br,
>  
>  	mp = br_multicast_new_group(br, port, group);
>  	err = PTR_ERR(mp);
> -	if (unlikely(IS_ERR(mp) || !mp))
> +	if (IS_ERR_OR_NULL(mp)) {
> +		if (!mp)
> +			err = -ENOMEM;
>  		goto err;
> +	}
>  
>  	if (!port) {
>  		hlist_add_head(&mp->mglist, &br->mglist);
I would rather fix br_multicast_new_group so it never returns
NULL. Instead return PTR_ERR(-ENOMEM) on out of memory.

--

If br_multicast_new_group returns NULL, we would return 0 (no error) to
the caller of br_multicast_add_group, which is not what we want. Instead
br_multicast_new_group should return ERR_PTR(-ENOMEM) in this case.
Also propagate the error number returned by br_mdb_rehash properly.

Signed-off-by: Tobias Klauser <tklauser at distanz.ch>
---
 net/bridge/br_multicast.c |   10 ++++++----
 1 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/net/bridge/br_multicast.c b/net/bridge/br_multicast.c
index 326e599..85a0398 100644
--- a/net/bridge/br_multicast.c
+++ b/net/bridge/br_multicast.c
@@ -654,11 +654,13 @@ static struct net_bridge_mdb_entry
*br_multicast_new_group(
 	struct net_bridge_mdb_htable *mdb;
 	struct net_bridge_mdb_entry *mp;
 	int hash;
+	int err;
 
 	mdb = rcu_dereference_protected(br->mdb, 1);
 	if (!mdb) {
-		if (br_mdb_rehash(&br->mdb, BR_HASH_SIZE, 0))
-			return NULL;
+		err = br_mdb_rehash(&br->mdb, BR_HASH_SIZE, 0);
+		if (err)
+			return ERR_PTR(err);
 		goto rehash;
 	}
 
@@ -680,7 +682,7 @@ rehash:
 
 	mp = kzalloc(sizeof(*mp), GFP_ATOMIC);
 	if (unlikely(!mp))
-		goto out;
+		return ERR_PTR(-ENOMEM);
 
 	mp->br = br;
 	mp->addr = *group;
@@ -713,7 +715,7 @@ static int br_multicast_add_group(struct net_bridge *br,
 
 	mp = br_multicast_new_group(br, port, group);
 	err = PTR_ERR(mp);
-	if (unlikely(IS_ERR(mp) || !mp))
+	if (IS_ERR(mp))
 		goto err;
 
 	if (!port) {
-- 
1.7.0.4

From: Tobias Klauser <tklauser at distanz.ch>
Date: Fri, 10 Dec 2010 14:18:04 +0100
> If br_multicast_new_group returns NULL, we would return 0 (no error) to
> the caller of br_multicast_add_group, which is not what we want. Instead
> br_multicast_new_group should return ERR_PTR(-ENOMEM) in this case.
> Also propagate the error number returned by br_mdb_rehash properly.
> 
> Signed-off-by: Tobias Klauser <tklauser at distanz.ch>
Looks good, applied to net-next-2.6

Please in the future make is clear, in your subject line, which
tree this patch is meant for.

I had to figure it out by trial and error, because this patch
does not apply properly to net-2.6