similar to: rdma or tcp?

Hello I have two peers setup and working with x2 bricks each. They have been working via tcp for the last 4-5 months. I just got two Infiniband cards and put the on the peers. I want to change the transport type to rdma instead of tcp but I don't see an easy way to do this. Can you please help me with proper instructions. Best Regards Ivan Dimitrov

Where can I find documentation about manual configuration of Gluster peers/volumes? All documentation seems to be about the gluster CLI. I would prefer manual configuration to facilitate automation via scripts (e.g. Puppet/Chef). I also read in this list that it is possible to configure Raid10 via text files... I would also like to experiment with this setup. Any related documents on how to do

Hi, we would like to build a gluster storage systems that combines our need for performance with our need for disaster recovery. I saw a couple of posts indicating that this is possible (http://gluster.org/pipermail/gluster-users/2010-February/003862.html) but am not 100% clear if that is possible Let's assume I have a total of 6 storage servers and bricks and want to spread them across 2

Hi, All, I used Infiniband to connect all GlusterFS nodes and the clients. Previously I run IP over IB and everything was OK. Now I used rdma transport mode instead. And then I ran the traffic. After I while, the glusterfs process exited because of segmentation fault. Here were the messages when I saw segmentation fault: pending frames: frame : type(0) op(0) frame : type(0) op(0)

Hi We have been trying back-up a glusterfs (v3.1.4) area using the Tivoli TSM software to an off-site area. The back-up keeps failing with the following typical error messages 06/14/2011 22:22:58 ANS1587W I/O error reading file attributes for: /gdata/projects/philex/OAG/2011/May16/mdor3km10/coast_den2.in. errno = 22, Invalid argument 06/14/2011 22:22:59 ANS4007E Error processing

Data center backends use more and more RDMA or RoCE devices and more and more software runs in virtualized environment. There is a need for a standard to enable RDMA/RoCE on Virtual Machines. Virtio is the optimal solution since is the de-facto para-virtualizaton technology and also because the Virtio specification allows Hardware Vendors to support Virtio protocol natively in order to achieve

Data center backends use more and more RDMA or RoCE devices and more and more software runs in virtualized environment. There is a need for a standard to enable RDMA/RoCE on Virtual Machines. Virtio is the optimal solution since is the de-facto para-virtualizaton technology and also because the Virtio specification allows Hardware Vendors to support Virtio protocol natively in order to achieve

Dear all, I faced a problem with a glusterfs volume (pure distributed, _not_ dispersed) over RDMA transport. One user had a directory with a large number of files (50,000 files) and just doing an "ls" in this directory yields a "Transport endpoint not connected" error. The effect is, that "ls" only shows some files, but not all. The respective log file shows this

This looks like there could be some some problem requesting / leaking / whatever memory but without looking at the core its tought to tell for sure. Note: /usr/lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0x78)[0x7f95bc54e618] Can you open up a bugzilla and get us the core file to review? -b ----- Original Message ----- > From: "???" <21291285 at qq.com> > To:

Hi, I have 2 servers with Mellanox InfiniBand FDR hardware/software installed. A volume with "replica 2 transport rdma" works (create on servers, mount and test on clients) ok. A volume with "stripe 2 transport tcp" works ok, too. A volume with "stripe 2 transport rdma" created ok, and mounted ok on a client, but writing a file caused "endpoint not

Stefan, Sounds like a brick process is not running. I have notice some strangeness in my lab when using RDMA, I often have to forcibly restart the brick process, often as in every single time I do a major operation, add a new volume, remove a volume, stop a volume, etc. gluster volume status <vol> Does any of the self heal daemons show N/A? If that's the case, try forcing a restart on

Hi ,all We found a strange problem. Some clients worked normally while some clients couldn't access sepcial files. For exmaple, Client A couldn't create the directory xxx, but Client B could. However, if Client B created the directory, Client A could acess it and even deleted it. But Client A still couldn't create the same directory later. If I changed the directory name, Client A

It's that time again ? we want to test the GlusterFS 3.4 beta before we unleash it on the world. Like our last test fest, we want you to put the latest GlusterFS beta through real-world usage scenarios that will show you how it compares to previous releases. Unlike the last time, we want to focus this round of testing on Infiniband and RDMA hardware. For a description of how to do this, see

Forgot to mention, sometimes I have to do force start other volumes as well, its hard to determine which brick process is locked up from the logs. Status of volume: rhev_vms_primary Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick spidey.ib.runlevelone.lan:/gluster/brick/rhev_vms_primary

Did any RDMA/Ethernet users see this Gluster error? If so do you know what caused it and how to fix? If you haven't seen it, what RPMs and configuration do you use specific to RDMA/Ethernet? [2011-11-10 10:30:20.595801] C [rdma.c:2417:rdma_connect_qp]0-rpc-transport/rdma: Failed to modify QP to RTR [2011-11-10 10:30:20.595930] E [rdma.c:4159:rdma_handshake_pollin] 0-rpc-transport/rdma:

Dear Dan, thanks for the quick reply! I actually tried restarting all processes (and even rebooting all servers), but the error persists. I can also confirm that all birck processes are running. My volume is a distrubute-only volume (not dispersed, no sharding). I also tried mounting with use_readdirp=no, because the error seems to be connected to readdirp, but this option does not change

Dear Gluster-Users, I am experiencing RDMA problems. I have installed Ubuntu 16.04.4 running with 4.15.0-13-generic kernel, MLNX_OFED_LINUX-4.3-1.0.1.0-ubuntu16.04-x86_64 to 4 different servers. All of them has Mellanox ConnectX-4 LX dual port NICs. These four servers are connected via Mellanox SN2100 Switch. I have installed GlusterFS Server v3.10 (from Ubuntu PPA) to 3 servers. These 3

Forgot to mention that I was using CentOS7.3 and GlusterFS 3.10.3 that is the latest available. From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at gluster.org] On Behalf Of Hatazaki, Takao Sent: Tuesday, August 15, 2017 2:32 AM To: gluster-users at gluster.org Subject: [Gluster-users] Is transport=rdma tested with "stripe"? Hi, I have 2 servers with Mellanox

Is infiniband itself working fine? You can run tools like ibv_rc_pingpong to find out. On Wed, Apr 25, 2018 at 12:23 PM, Necati E. SISECI <siseci at gmail.com> wrote: > Dear Gluster-Users, > > I am experiencing RDMA problems. > > I have installed Ubuntu 16.04.4 running with 4.15.0-13-generic kernel, > MLNX_OFED_LINUX-4.3-1.0.1.0-ubuntu16.04-x86_64 to 4 different servers.

Hi everyone, I hope you can help me with some performance troubles I've been having. I'm doing some tests with gluster 3.2.0, and I can't understand some of the behavior I'm getting with gluster. The test is using a volume with 12 stripped bricks (each brick is an HD), with no replication, via RDMA. I'm doing random readings of 4GByte files with the FIO tool. Gluster