thr3ads.net - Gluster users - [Gluster-users] GlusterFS Mounts at startup consuming needed local ports for host services [Apr 2016]

If this information is useful, please help other people find it:
Share via:

tom at encoding.com

2016-Apr-09 18:20 UTC

[Gluster-users] GlusterFS Mounts at startup consuming needed local ports for host services

Ryan,

What's /proc/sys/net/ipv4/ip_local_port_range set to on the client?

-t
> On Apr 9, 2016, at 8:00 AM, Ryan.J.Wyler at wellsfargo.com wrote:
> 
> Is there any way to configure glusterfs to avoid using specific local ports
when mounting filesystems?
>  
> Ryan Wyler
>  
> From: gluster-users-bounces at gluster.org <mailto:gluster-users-bounces
at gluster.org> [mailto:gluster-users-bounces at gluster.org
<mailto:gluster-users-bounces at gluster.org>] On Behalf Of Ryan.J.Wyler
at wellsfargo.com <mailto:Ryan.J.Wyler at wellsfargo.com>
> Sent: Friday, April 08, 2016 9:31 AM
> To: gluster-users at gluster.org <mailto:gluster-users at
gluster.org>
> Subject: [Gluster-users] GlusterFS Mounts at startup consuming needed local
ports for host services
>  
> Glusterfs mounts are squashing ports I need available for services,
specifically on port 1002.
>  
> How can I configure glusterfs mount points to avoid specific local ports
for the connections?
>  
> A potential workaround I thought of is removing the mounts from starting
automatically at startup and create an rc.d startup script to mount the
glusterfs filesystems after all the host services have started to avoid those
ports.
>  
> Is there a different option?
>  
> The local port range is using 1000-1023 as seen in the information below
for client.domain.com <http://client.domain.com/>.
>  
> # lsof | grep glusterfs | grep IPv4
> glusterfs  2893    root    9u     IPv4              15373       0t0        
TCP client.domain.com
<http://client.domain.com/>:1023->server1.domain.com
<http://server1.domain.com/>:24007 (ESTABLISHED)
> glusterfs  2893    root   10u     IPv4              15447       0t0        
TCP client.domain.com
<http://client.domain.com/>:1015->server2.domain.com
<http://server2.domain.com/>:49171 (ESTABLISHED)
> glusterfs  2893    root   11u     IPv4              15446       0t0        
TCP client.domain.com
<http://client.domain.com/>:1016->server3.domain.com
<http://server3.domain.com/>:49171 (ESTABLISHED)
> glusterfs  2893    root   13u     IPv4              15422       0t0        
TCP client.domain.com
<http://client.domain.com/>:1020->server1.domain.com
<http://server1.domain.com/>:49171 (ESTABLISHED)
> glusterfs  2893    root   15u     IPv4              15439       0t0        
TCP client.domain.com
<http://client.domain.com/>:1018->server4.domain.com
<http://server4.domain.com/>:49171 (ESTABLISHED)
> glusterfs  2998    root    9u     IPv4              15523       0t0        
TCP client.domain.com
<http://client.domain.com/>:1014->server1.domain.com
<http://server1.domain.com/>:24007 (ESTABLISHED)
> glusterfs  2998    root   10u     IPv4              15588       0t0        
TCP client.domain.com
<http://client.domain.com/>:1006->server2.domain.com
<http://server2.domain.com/>:49157 (ESTABLISHED)
> glusterfs  2998    root   11u     IPv4              15576       0t0        
TCP client.domain.com
<http://client.domain.com/>:1007->server3.domain.com
<http://server3.domain.com/>:49157 (ESTABLISHED)
> glusterfs  2998    root   14u     IPv4              15549       0t0        
TCP client.domain.com
<http://client.domain.com/>:1012->server1.domain.com
<http://server1.domain.com/>:49157 (ESTABLISHED)
> glusterfs  2998    root   16u     IPv4              15569       0t0        
TCP client.domain.com
<http://client.domain.com/>:1009->server4.domain.com
<http://server4.domain.com/>:49157 (ESTABLISHED)
> glusterfs 46813    root   10u     IPv4             342418       0t0        
TCP client.domain.com
<http://client.domain.com/>:1019->server1.domain.com
<http://server1.domain.com/>:24007 (ESTABLISHED)
> glusterfs 46813    root   11u     IPv4             342445       0t0        
TCP client.domain.com:busboy->server2.domain.com
<http://server2.domain.com/>:49152 (ESTABLISHED)
> glusterfs 46813    root   15u     IPv4             342428       0t0        
TCP client.domain.com:surf->server1.domain.com
<http://server1.domain.com/>:49152 (ESTABLISHED)
> glusterfs 46813    root   16u     IPv4             342434       0t0        
TCP client.domain.com
<http://client.domain.com/>:1002->server4.domain.com
<http://server4.domain.com/>:49152 (ESTABLISHED)
> glusterfs 46813    root   17u     IPv4             342440       0t0        
TCP client.domain.com:cadlock2->server3.domain.com
<http://server3.domain.com/>:49152 (ESTABLISHED)
>  
>  
> Glusterfs RPMs installed:
>  
> glusterfs-libs-3.6.0.54-1
> glusterfs-api-3.6.0.54-1
> glusterfs-fuse-3.6.0.54-1
> glusterfs-3.6.0.54-1
>  
>  
>  
> Ryan Wyler
> Ryan.j.wyler at wellsfargo.com <mailto:Ryan.j.wyler at
wellsfargo.com>
>  
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
> http://www.gluster.org/mailman/listinfo/gluster-users
<http://www.gluster.org/mailman/listinfo/gluster-users>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160409/92d2d40b/attachment.html>

Ryan.J.Wyler at wellsfargo.com

2016-Apr-09 18:25 UTC

head link

[Gluster-users] GlusterFS Mounts at startup consuming needed local ports for host services

Tom,

# cat /proc/sys/net/ipv4/ip_local_port_range
32768   61000

Ryan

From: tom at encoding.com [mailto:tom at encoding.com]
Sent: Saturday, April 09, 2016 11:20 AM
To: Wyler, Ryan J.
Cc: gluster-users at gluster.org
Subject: Re: [Gluster-users] GlusterFS Mounts at startup consuming needed local
ports for host services

Ryan,

What's /proc/sys/net/ipv4/ip_local_port_range set to on the client?

-t

On Apr 9, 2016, at 8:00 AM, Ryan.J.Wyler at
wellsfargo.com<mailto:Ryan.J.Wyler at wellsfargo.com> wrote:

Is there any way to configure glusterfs to avoid using specific local ports when
mounting filesystems?

Ryan Wyler

From: gluster-users-bounces at gluster.org<mailto:gluster-users-bounces at
gluster.org> [mailto:gluster-users-bounces at gluster.org] On Behalf Of
Ryan.J.Wyler at wellsfargo.com<mailto:Ryan.J.Wyler at wellsfargo.com>
Sent: Friday, April 08, 2016 9:31 AM
To: gluster-users at gluster.org<mailto:gluster-users at gluster.org>
Subject: [Gluster-users] GlusterFS Mounts at startup consuming needed local
ports for host services

Glusterfs mounts are squashing ports I need available for services, specifically
on port 1002.

How can I configure glusterfs mount points to avoid specific local ports for the
connections?

A potential workaround I thought of is removing the mounts from starting
automatically at startup and create an rc.d startup script to mount the
glusterfs filesystems after all the host services have started to avoid those
ports.

Is there a different option?

The local port range is using 1000-1023 as seen in the information below for
client.domain.com<http://client.domain.com/>.

# lsof | grep glusterfs | grep IPv4
glusterfs  2893    root    9u     IPv4              15373       0t0             
TCP
client.domain.com<http://client.domain.com/>:1023->server1.domain.com<http://server1.domain.com/>:24007
(ESTABLISHED)
glusterfs  2893    root   10u     IPv4              15447       0t0             
TCP
client.domain.com<http://client.domain.com/>:1015->server2.domain.com<http://server2.domain.com/>:49171
(ESTABLISHED)
glusterfs  2893    root   11u     IPv4              15446       0t0             
TCP
client.domain.com<http://client.domain.com/>:1016->server3.domain.com<http://server3.domain.com/>:49171
(ESTABLISHED)
glusterfs  2893    root   13u     IPv4              15422       0t0             
TCP
client.domain.com<http://client.domain.com/>:1020->server1.domain.com<http://server1.domain.com/>:49171
(ESTABLISHED)
glusterfs  2893    root   15u     IPv4              15439       0t0             
TCP
client.domain.com<http://client.domain.com/>:1018->server4.domain.com<http://server4.domain.com/>:49171
(ESTABLISHED)
glusterfs  2998    root    9u     IPv4              15523       0t0             
TCP
client.domain.com<http://client.domain.com/>:1014->server1.domain.com<http://server1.domain.com/>:24007
(ESTABLISHED)
glusterfs  2998    root   10u     IPv4              15588       0t0             
TCP
client.domain.com<http://client.domain.com/>:1006->server2.domain.com<http://server2.domain.com/>:49157
(ESTABLISHED)
glusterfs  2998    root   11u     IPv4              15576       0t0             
TCP
client.domain.com<http://client.domain.com/>:1007->server3.domain.com<http://server3.domain.com/>:49157
(ESTABLISHED)
glusterfs  2998    root   14u     IPv4              15549       0t0             
TCP
client.domain.com<http://client.domain.com/>:1012->server1.domain.com<http://server1.domain.com/>:49157
(ESTABLISHED)
glusterfs  2998    root   16u     IPv4              15569       0t0             
TCP
client.domain.com<http://client.domain.com/>:1009->server4.domain.com<http://server4.domain.com/>:49157
(ESTABLISHED)
glusterfs 46813    root   10u     IPv4             342418       0t0             
TCP
client.domain.com<http://client.domain.com/>:1019->server1.domain.com<http://server1.domain.com/>:24007
(ESTABLISHED)
glusterfs 46813    root   11u     IPv4             342445       0t0             
TCP
client.domain.com:busboy->server2.domain.com<http://server2.domain.com/>:49152
(ESTABLISHED)
glusterfs 46813    root   15u     IPv4             342428       0t0             
TCP
client.domain.com:surf->server1.domain.com<http://server1.domain.com/>:49152
(ESTABLISHED)
glusterfs 46813    root   16u     IPv4             342434       0t0             
TCP
client.domain.com<http://client.domain.com/>:1002->server4.domain.com<http://server4.domain.com/>:49152
(ESTABLISHED)
glusterfs 46813    root   17u     IPv4             342440       0t0             
TCP
client.domain.com:cadlock2->server3.domain.com<http://server3.domain.com/>:49152
(ESTABLISHED)


Glusterfs RPMs installed:

glusterfs-libs-3.6.0.54-1
glusterfs-api-3.6.0.54-1
glusterfs-fuse-3.6.0.54-1
glusterfs-3.6.0.54-1



Ryan Wyler
Ryan.j.wyler at wellsfargo.com<mailto:Ryan.j.wyler at wellsfargo.com>

_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org<mailto:Gluster-users at gluster.org>
http://www.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160409/4a161d6a/attachment.html>

Ryan.J.Wyler at wellsfargo.com

2016-Apr-09 18:43 UTC

head link

[Gluster-users] GlusterFS Mounts at startup consuming needed local ports for host services

Tom, this appears to be the same issue.  I can't seem to find any reported
bugs that relate directly to this issue, but others are having the same problem.

http://www.gluster.org/pipermail/gluster-users/2012-June/010380.html

GlusterFS Clients does not appear to be properly using the
/proc/sys/net/ipv4/ip_local_port_range


This issue has been incorrectly identified this issue with glusterfs not
properly using /proc/sys/net/ipv4/ip_local_reserved_ports

http://www.gluster.org/pipermail/gluster-users/2012-June/010383.html

Referring to this bug: https://bugzilla.redhat.com/show_bug.cgi?id=762989

But this is not the same.  Glusterfs should leverage the ip_local_port_range,
and it's not.

I don't want to have to register every port I'm using as reserved, I
want to just have glusterfs use ip_local_port_range properly.

Ryan

From: tom at encoding.com [mailto:tom at encoding.com]
Sent: Saturday, April 09, 2016 11:20 AM
To: Wyler, Ryan J.
Cc: gluster-users at gluster.org
Subject: Re: [Gluster-users] GlusterFS Mounts at startup consuming needed local
ports for host services

Ryan,

What's /proc/sys/net/ipv4/ip_local_port_range set to on the client?

-t

On Apr 9, 2016, at 8:00 AM, Ryan.J.Wyler at
wellsfargo.com<mailto:Ryan.J.Wyler at wellsfargo.com> wrote:

Is there any way to configure glusterfs to avoid using specific local ports when
mounting filesystems?

Ryan Wyler

From: gluster-users-bounces at gluster.org<mailto:gluster-users-bounces at
gluster.org> [mailto:gluster-users-bounces at gluster.org] On Behalf Of
Ryan.J.Wyler at wellsfargo.com<mailto:Ryan.J.Wyler at wellsfargo.com>
Sent: Friday, April 08, 2016 9:31 AM
To: gluster-users at gluster.org<mailto:gluster-users at gluster.org>
Subject: [Gluster-users] GlusterFS Mounts at startup consuming needed local
ports for host services

Glusterfs mounts are squashing ports I need available for services, specifically
on port 1002.

How can I configure glusterfs mount points to avoid specific local ports for the
connections?

A potential workaround I thought of is removing the mounts from starting
automatically at startup and create an rc.d startup script to mount the
glusterfs filesystems after all the host services have started to avoid those
ports.

Is there a different option?

The local port range is using 1000-1023 as seen in the information below for
client.domain.com<http://client.domain.com/>.

# lsof | grep glusterfs | grep IPv4
glusterfs  2893    root    9u     IPv4              15373       0t0             
TCP
client.domain.com<http://client.domain.com/>:1023->server1.domain.com<http://server1.domain.com/>:24007
(ESTABLISHED)
glusterfs  2893    root   10u     IPv4              15447       0t0             
TCP
client.domain.com<http://client.domain.com/>:1015->server2.domain.com<http://server2.domain.com/>:49171
(ESTABLISHED)
glusterfs  2893    root   11u     IPv4              15446       0t0             
TCP
client.domain.com<http://client.domain.com/>:1016->server3.domain.com<http://server3.domain.com/>:49171
(ESTABLISHED)
glusterfs  2893    root   13u     IPv4              15422       0t0             
TCP
client.domain.com<http://client.domain.com/>:1020->server1.domain.com<http://server1.domain.com/>:49171
(ESTABLISHED)
glusterfs  2893    root   15u     IPv4              15439       0t0             
TCP
client.domain.com<http://client.domain.com/>:1018->server4.domain.com<http://server4.domain.com/>:49171
(ESTABLISHED)
glusterfs  2998    root    9u     IPv4              15523       0t0             
TCP
client.domain.com<http://client.domain.com/>:1014->server1.domain.com<http://server1.domain.com/>:24007
(ESTABLISHED)
glusterfs  2998    root   10u     IPv4              15588       0t0             
TCP
client.domain.com<http://client.domain.com/>:1006->server2.domain.com<http://server2.domain.com/>:49157
(ESTABLISHED)
glusterfs  2998    root   11u     IPv4              15576       0t0             
TCP
client.domain.com<http://client.domain.com/>:1007->server3.domain.com<http://server3.domain.com/>:49157
(ESTABLISHED)
glusterfs  2998    root   14u     IPv4              15549       0t0             
TCP
client.domain.com<http://client.domain.com/>:1012->server1.domain.com<http://server1.domain.com/>:49157
(ESTABLISHED)
glusterfs  2998    root   16u     IPv4              15569       0t0             
TCP
client.domain.com<http://client.domain.com/>:1009->server4.domain.com<http://server4.domain.com/>:49157
(ESTABLISHED)
glusterfs 46813    root   10u     IPv4             342418       0t0             
TCP
client.domain.com<http://client.domain.com/>:1019->server1.domain.com<http://server1.domain.com/>:24007
(ESTABLISHED)
glusterfs 46813    root   11u     IPv4             342445       0t0             
TCP
client.domain.com:busboy->server2.domain.com<http://server2.domain.com/>:49152
(ESTABLISHED)
glusterfs 46813    root   15u     IPv4             342428       0t0             
TCP
client.domain.com:surf->server1.domain.com<http://server1.domain.com/>:49152
(ESTABLISHED)
glusterfs 46813    root   16u     IPv4             342434       0t0             
TCP
client.domain.com<http://client.domain.com/>:1002->server4.domain.com<http://server4.domain.com/>:49152
(ESTABLISHED)
glusterfs 46813    root   17u     IPv4             342440       0t0             
TCP
client.domain.com:cadlock2->server3.domain.com<http://server3.domain.com/>:49152
(ESTABLISHED)


Glusterfs RPMs installed:

glusterfs-libs-3.6.0.54-1
glusterfs-api-3.6.0.54-1
glusterfs-fuse-3.6.0.54-1
glusterfs-3.6.0.54-1



Ryan Wyler
Ryan.j.wyler at wellsfargo.com<mailto:Ryan.j.wyler at wellsfargo.com>

_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org<mailto:Gluster-users at gluster.org>
http://www.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160409/e3f4860d/attachment.html>

Gluster users - Apr 2016 - GlusterFS Mounts at startup consuming needed local ports for host services

[Gluster-users] GlusterFS Mounts at startup consuming needed local ports for host services

[Gluster-users] GlusterFS Mounts at startup consuming needed local ports for host services

[Gluster-users] GlusterFS Mounts at startup consuming needed local ports for host services