search for: rebalance

We have had a rebalance operation going on for a few days. After a couple days the rebalance status said "failed". We stopped the rebalance operation by doing gluster volume rebalance gv0 stop. Rebalance log indicated gluster did try to stop the rebalance. However, when we try now to stop the volume or try to re...

Dear gluster experts, We're running glusterfs 3.3 and we have met file permission probelems after gluster volume rebalance. Files got stick permissions T--------- after rebalance which break our client normal fops unexpectedly. Any one known this issue? Thank you for your help. -- ??? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-...

Hello everyone, I have problem rebalancing Gluster volume. Gluster version is 3.7.3. My 1x3 replicated volume become full, so I've added three more bricks to make it 2x3 and wanted to rebalance. But every time I start rebalancing, it fails immediately. Rebooting Gluster nodes doesn't help. # gluster volume rebalance gsae_artifactory_cluster_storage start volume rebalance: gsae_artifactory_cluster_storage: success: Rebalance on gsae_artifactory_cluster_storage has been started succes...

Hi guys, we have a rebalance running on eight bricks since July and this is what the status looks like right now: ===Tue Oct 18 13:45:01 CST 2011 ==== rebalance step 1: layout fix in progress: fixed layout 223623 There are roughly 8T photos in the storage,so how long should this rebalance take? What does the number (in th...

Hi Nithya, I see index in context: [2017-07-07 10:07:18.230202] E [MSGID: 106062] [glusterd-utils.c:7997:glusterd_volume_rebalance_use_rsp_dict] 0-glusterd: failed to get index I wonder if there is anything I can do to fix it. I was trying to strace gluster process but still have no clue what exactly is gluster index. Best regards, Szymon Miotk On Thu, Jul 13, 2017 at 10:12 AM, Nithya Balachandran <nbalacha at redhat.com...

...visible files. The files are at the bricks and don't start with a dot, but aren't visible in "ls". Clients still can interact with them by using the explicit path. More information: https://bugzilla.redhat.com/show_bug.cgi?id=1564071 And since this update gluster reported for the rebalance of >16900 PB (Petabyte!) of data for one of our 2 server, when using ?gluster volume rebalance $myvolume status?. The time looks right, but the size of transfered files is absurd. The rebalance was with 3.12.6 in March 2018. The last rebalance log file listed no errors and a realistic size at th...

...'t visible in "ls". Clients still can > interact with them by using the explicit path. > More information: https://bugzilla.redhat.com/show_bug.cgi?id=1564071 I will continue the analysis for this issue in the bug. > > > And since this update gluster reported for the rebalance of >16900 PB > (Petabyte!) of data for one of our 2 server, when using ?gluster volume > rebalance $myvolume status?. The time looks right, but the size of > transfered files is absurd. The rebalance was with 3.12.6 in March 2018. > The last rebalance log file listed no errors and a...

...lt;szymon.miotk at gmail.com> wrote: >> >> Hello everyone, >> >> >> I have problem rebalancing Gluster volume. >> Gluster version is 3.7.3. >> My 1x3 replicated volume become full, so I've added three more bricks >> to make it 2x3 and wanted to rebalance. >> But every time I start rebalancing, it fails immediately. >> Rebooting Gluster nodes doesn't help. >> >> # gluster volume rebalance gsae_artifactory_cluster_storage start >> volume rebalance: gsae_artifactory_cluster_storage: success: Rebalance >> on gsa...

Hi Mahdi, Did you get a chance to verify this fix again? If this fix works for you, is it OK if we move this bug to CLOSED state and revert the rebalance-cli warning patch? -Krutika On Mon, May 29, 2017 at 6:51 PM, Mahdi Adnan <mahdi.adnan at outlook.com> wrote: > Hello, > > > Yes, i forgot to upgrade the client as well. > > I did the upgrade and created a new volume, same options as before, with > one VM running and do...

Hi, This value is an ongoing rough estimate based on the amount of data rebalance has migrated since it started. The values will cange as the rebalance progresses. A few questions: 1. How many files/dirs do you have on this volume? 2. What is the average size of the files? 3. What is the total size of the data on the volume? Can you send us the rebalance log? Thank...

Hello, The past couple of weeks I had some issues with firmware on the OS hard drives in my gluster cluster. I have recently fixed the issue, and am bringing my bricks back into the volume. I am running gluster 3.7.6 and am running into the following issue: When I add the brick and rebalance, the operation fails after a couple minutes. The errors I find in the rebalance log is this: [2017-06-05 13:38:40.441671] E [MSGID: 109010] [dht-rebalance.c:2259:gf_defrag_get_entry] 0-hpcscratch-dht: /LV_Fitting/code/C gfid not present [2017-06-05 13:38:40.450341] E [MSGID: 109010] [dht-rebalance...

...110067712 29199861 80867851 27% /home /dev/md1 131072 363 130709 1% /boot gluster1:/web 2559860992 63694442 2496166550 3% /web tmpfs 8199799 1 8199798 1% /run/user/0 The rebalance log is in the attachment the cluster information gluster volume status web detail Status of volume: web ------------------------------------------------------------ ------------------ Brick : Brick gluster1:/home/export/md3/brick TCP Port : 49154 RDMA Port :...

...7 July 2017 at 15:42, Szymon Miotk <szymon.miotk at gmail.com> wrote: > Hello everyone, > > > I have problem rebalancing Gluster volume. > Gluster version is 3.7.3. > My 1x3 replicated volume become full, so I've added three more bricks > to make it 2x3 and wanted to rebalance. > But every time I start rebalancing, it fails immediately. > Rebooting Gluster nodes doesn't help. > > # gluster volume rebalance gsae_artifactory_cluster_storage start > volume rebalance: gsae_artifactory_cluster_storage: success: Rebalance > on gsae_artifactory_cluster_st...

...i Adnan <mahdi.adnan at outlook.com> > wrote: > >> Hello, >> >> >> Yes, i forgot to upgrade the client as well. >> >> I did the upgrade and created a new volume, same options as before, with >> one VM running and doing lots of IOs. i started the rebalance with force >> and after it completed the process i rebooted the VM, and it did start >> normally without issues. >> >> I repeated the process and did another rebalance while the VM running and >> everything went fine. >> >> But the logs in the client throwi...

Hi, In the past couple of weeks, we've sent the following fixes concerning VM corruption upon doing rebalance - https://review.gluster.org/#/q/status:merged+project:glusterfs+branch:master+topic:bug-1440051 These fixes are very much part of the latest 3.10.2 release. Satheesaran within Red Hat also verified that they work and he's not seeing corruption issues anymore. I'd like to hear feedback f...

Hi, I have a rebalance that has failed on one peer twice now. Rebalance logs below (directories anonomised and some irrelevant log lines cut). It looks like it loses connection to the brick, but immediately stops the rebalance on that peer instead of waiting for reconnection - which happens a second or so later. Is this...

...seemed to work well. And, our initial introduction of GlusterFS into our production environment went well. We had our initial 2 server (80TB) cluster about 50% full and things seemed to be going well. Then we added another pair of servers (for a total of 160TB). This went fine until we did the rebalance. We were running 3.3.1. We ran into the handle leak problem (which unfortunately we didn't know about beforehand). We also found that if any of the bricks went offline while the rebalance was going on, then files were lost or they lost their permissions. We still don't know why some of...

...> >> > >> Hello everyone, > >> > >> > >> I have problem rebalancing Gluster volume. > >> Gluster version is 3.7.3. > >> My 1x3 replicated volume become full, so I've added three more bricks > >> to make it 2x3 and wanted to rebalance. > >> But every time I start rebalancing, it fails immediately. > >> Rebooting Gluster nodes doesn't help. > >> > >> # gluster volume rebalance gsae_artifactory_cluster_storage start > >> volume rebalance: gsae_artifactory_cluster_storage: success:...

...om> >> wrote: >> >>> Hello, >>> >>> >>> Yes, i forgot to upgrade the client as well. >>> >>> I did the upgrade and created a new volume, same options as before, with >>> one VM running and doing lots of IOs. i started the rebalance with force >>> and after it completed the process i rebooted the VM, and it did start >>> normally without issues. >>> >>> I repeated the process and did another rebalance while the VM running >>> and everything went fine. >>> >>> But t...