search for: serkan

Hi, Gluster version is 3.10.5. I am trying to create a 5500 brick volume, but getting an error stating that 4444 bricks is the limit. Is this a known limit? Can I change this with an option? Thanks, Serkan

An upstream bug would be ideal as github issue is mainly used for enhancements. In the mean time, could you point to the exact failure shown at the command line and the log entry from cli.log? On Wed, Aug 23, 2017 at 12:10 AM, Serkan ?oban <cobanserkan at gmail.com> wrote: > Hi, I think this is the line limiting brick count: > https://github.com/gluster/glusterfs/blob/c136024613c697fec87aaff3a07086 > 2b92c57977/cli/src/cli-cmd-parser.c#L84 > > Can gluster-devs increase this limit? Should I open a github is...

...m 80 server cluster gets > rebooted, glusterd process hangs. > > Could you please provide the logs which led glusterd to hang for all the > cases along with gusterd process utilization. > > > Thanks > Gaurav > > > > > > > On Tue, Aug 29, 2017 at 2:44 PM, Serkan ?oban <cobanserkan at gmail.com> wrote: >> >> Here is the requested logs: >> >> https://www.dropbox.com/s/vt187h0gtu5doip/gluster_logs_20_40_80_servers.zip?dl=0 >> >> >> On Tue, Aug 29, 2017 at 7:48 AM, Gaurav Yadav <gyadav at redhat.com> wrote:...

glusterd returned to normal, here is the logs: https://www.dropbox.com/s/41jx2zn3uizvr53/80servers_glusterd_normal_status.zip?dl=0 On Tue, Aug 29, 2017 at 1:47 PM, Serkan ?oban <cobanserkan at gmail.com> wrote: > Here is the logs after stopping all three volumes and restarting > glusterd in all nodes. I waited 70 minutes after glusterd restart but > it is still consuming %100 CPU. > https://www.dropbox.com/s/pzl0f198v03twx3/80servers_after_glusterd...

...found anything significant. > > Can you send me gluster logs along with command-history-logs for these > scenarios: > Scenario1 : 20 servers > Scenario2 : 40 servers > Scenario3: 80 Servers > > > Thanks > Gaurav > > > > On Mon, Aug 28, 2017 at 11:22 AM, Serkan ?oban <cobanserkan at gmail.com> > wrote: >> >> Hi Gaurav, >> Any progress about the problem? >> >> On Thursday, August 24, 2017, Serkan ?oban <cobanserkan at gmail.com> wrote: >>> >>> Thank you Gaurav, >>> Here is more finding...

Hi, I think this is the line limiting brick count: https://github.com/gluster/glusterfs/blob/c136024613c697fec87aaff3a070862b92c57977/cli/src/cli-cmd-parser.c#L84 Can gluster-devs increase this limit? Should I open a github issue? On Mon, Aug 21, 2017 at 7:01 PM, Serkan ?oban <cobanserkan at gmail.com> wrote: > Hi, > Gluster version is 3.10.5. I am trying to create a 5500 brick volume, > but getting an error stating that 4444 bricks is the limit. Is this a > known limit? Can I change this with an option? > > Thanks, > Serkan

----- Original Message ----- > From: "Ben Turner" <bturner at redhat.com> > To: "Serkan ?oban" <cobanserkan at gmail.com> > Cc: "Gluster Users" <gluster-users at gluster.org> > Sent: Sunday, September 3, 2017 2:30:31 PM > Subject: Re: [Gluster-users] Glusterd proccess hangs on reboot > > ----- Original Message ----- > > From: "Mili...

...rting the volume. However, you have mentioned that when a node from 80 server cluster gets rebooted, glusterd process hangs. Could you please provide the logs which led glusterd to hang for all the cases along with gusterd process utilization. Thanks Gaurav On Tue, Aug 29, 2017 at 2:44 PM, Serkan ?oban <cobanserkan at gmail.com> wrote: > Here is the requested logs: > https://www.dropbox.com/s/vt187h0gtu5doip/gluster_logs_ > 20_40_80_servers.zip?dl=0 > > > On Tue, Aug 29, 2017 at 7:48 AM, Gaurav Yadav <gyadav at redhat.com> wrote: > > Till now I haven'...

i usually change event threads to 4. But those logs are from a default installation. On Sun, Sep 3, 2017 at 9:52 PM, Ben Turner <bturner at redhat.com> wrote: > ----- Original Message ----- >> From: "Ben Turner" <bturner at redhat.com> >> To: "Serkan ?oban" <cobanserkan at gmail.com> >> Cc: "Gluster Users" <gluster-users at gluster.org> >> Sent: Sunday, September 3, 2017 2:30:31 PM >> Subject: Re: [Gluster-users] Glusterd proccess hangs on reboot >> >> ----- Original Message ----- >&gt...

No worries Serkan, You can continue to use your 40 node clusters. The backtrace has resolved the function names and it *should* be sufficient to debug the issue. Thanks for letting us know. We'll post on this thread again to notify you about the findings. On Sat, Sep 2, 2017 at 2:42 PM, Serkan ?oban <cob...

Hi Gaurav, Any progress about the problem? On Thursday, August 24, 2017, Serkan ?oban <cobanserkan at gmail.com> wrote: > Thank you Gaurav, > Here is more findings: > Problem does not happen using only 20 servers each has 68 bricks. > (peer probe only 20 servers) > If we use 40 servers with single volume, glusterd cpu %100 state > continues for 5 minut...

----- Original Message ----- > From: "Milind Changire" <mchangir at redhat.com> > To: "Serkan ?oban" <cobanserkan at gmail.com> > Cc: "Gluster Users" <gluster-users at gluster.org> > Sent: Saturday, September 2, 2017 11:44:40 PM > Subject: Re: [Gluster-users] Glusterd proccess hangs on reboot > > No worries Serkan, > You can continue to use your...

Till now I haven't found anything significant. Can you send me gluster logs along with command-history-logs for these scenarios: Scenario1 : 20 servers Scenario2 : 40 servers Scenario3: 80 Servers Thanks Gaurav On Mon, Aug 28, 2017 at 11:22 AM, Serkan ?oban <cobanserkan at gmail.com> wrote: > Hi Gaurav, > Any progress about the problem? > > On Thursday, August 24, 2017, Serkan ?oban <cobanserkan at gmail.com> wrote: > >> Thank you Gaurav, >> Here is more findings: >> Problem does not happen using onl...

On Mon, Sep 4, 2017 at 5:28 PM, Serkan ?oban <cobanserkan at gmail.com> wrote: > >1. On 80 nodes cluster, did you reboot only one node or multiple ones? > Tried both, result is same, but the logs/stacks are from stopping and > starting glusterd only on one server while others are running. > > >2. Are you sure...

...cided against 16+3 as the rebuild time are bottlenecked by matrix operations which scale as the square of the number of data stripes. There are some savings because of larger data chunks but we ended up using 8+3 and heal times are about half compared to 16+3. -Alastair On 30 June 2017 at 02:22, Serkan ?oban <cobanserkan at gmail.com> wrote: > >Thanks for the reply. We will mainly use this for archival - near-cold > storage. > Archival usage is good for EC > > >Anything, from your experience, to keep in mind while planning large > installations? > I am using 3.7....

...eer probe and create the volumes. But after glusterd restart. Also note that this still happens without any volumes. So it is not related with brick count I think... On Mon, Sep 4, 2017 at 5:08 PM, Atin Mukherjee <amukherj at redhat.com> wrote: > > > On Mon, Sep 4, 2017 at 5:28 PM, Serkan ?oban <cobanserkan at gmail.com> wrote: >> >> >1. On 80 nodes cluster, did you reboot only one node or multiple ones? >> Tried both, result is same, but the logs/stacks are from stopping and >> starting glusterd only on one server while others are running. >>...

On Fri, Sep 1, 2017 at 8:47 AM, Milind Changire <mchangir at redhat.com> wrote: > Serkan, > I have gone through other mails in the mail thread as well but responding > to this one specifically. > > Is this a source install or an RPM install ? > If this is an RPM install, could you please install the > glusterfs-debuginfo RPM and retry to capture the gdb backtrace. &gt...

I am working on it and will share my findings as soon as possible. Thanks Gaurav On Thu, Aug 24, 2017 at 3:58 PM, Serkan ?oban <cobanserkan at gmail.com> wrote: > Restarting glusterd causes the same thing. I tried with 3.12.rc0, > 3.10.5. 3.8.15, 3.7.20 all same behavior. > My OS is centos 6.9, I tried with centos 6.8 problem remains... > Only way to a healthy state is destroy gluster config/rpms,...

...> wrote: > Could you be able to provide the pstack dump of the glusterd process? > > On Wed, 23 Aug 2017 at 20:22, Atin Mukherjee <amukherj at redhat.com> wrote: >> >> Not yet. Gaurav will be taking a look at it tomorrow. >> >> On Wed, 23 Aug 2017 at 20:14, Serkan ?oban <cobanserkan at gmail.com> wrote: >>> >>> Hi Atin, >>> >>> Do you have time to check the logs? >>> >>> On Wed, Aug 23, 2017 at 10:02 AM, Serkan ?oban <cobanserkan at gmail.com> >>> wrote: >>> > Same thing h...

...70-80 minutes and I can run cluster commands after that. I will check this again to be sure.. On Mon, Sep 4, 2017 at 1:43 PM, Atin Mukherjee <amukherj at redhat.com> wrote: > > > On Fri, Sep 1, 2017 at 8:47 AM, Milind Changire <mchangir at redhat.com> wrote: >> >> Serkan, >> I have gone through other mails in the mail thread as well but responding >> to this one specifically. >> >> Is this a source install or an RPM install ? >> If this is an RPM install, could you please install the >> glusterfs-debuginfo RPM and retry to captur...