search for: hamb

On 15/09/17 02:45, Sam McLeod wrote: > Out of interest have you tried testing performance > with performance.stat-prefetch enabled? Not yet, because I'm still struggling to understand the current more basic setup's performance behaviour (with it being off), but it's definitely on my list and I'll report the outcome.

On 15/09/17 03:46, Niklas Hamb?chen wrote: >> Out of interest have you tried testing performance >> with performance.stat-prefetch enabled? I have now tested with `performance.stat-prefetch: on` but am not observing a difference. So far the only difference between `ls` and `bup index` I could observe is that `bup i...

On 17/09/17 18:03, Niklas Hamb?chen wrote: > So far the only difference between `ls` and `bup index` I could observe > is that `bup index` chdir()s into the directory to index, ls doesn't. > > But when I `cd` into the dir and run `ls` without directory argument, it > is still much faster than bup index for ea...

Hi Ben, do you know if the smallfile benchmark also does interleaved getdents() and lstat, which is what I found as being the key difference that creates the performance gap (further down this thread)? Also, wouldn't `--threads 8` change the performance numbers by factor 8 versus the plain `ls` and `rsync` that I did? Would you mind running those commands directly/plainly on your cluster to

----- Original Message ----- > From: "Niklas Hamb?chen" <mail at nh2.me> > To: "Ben Turner" <bturner at redhat.com> > Cc: gluster-users at gluster.org > Sent: Sunday, September 17, 2017 9:49:10 PM > Subject: Re: [Gluster-users] Confusing lstat() performance > > Hi Ben, > > do you know if the sma...

I found the reason now, at least for this set of lstat()s I was looking at. bup first does all getdents(), obtaining all file names in the directory, and then stat()s them. Apparently this destroys some of gluster's caching, making stat()s ~100x slower. What caching could this be, and how could I convince gluster to serve these stat()s as fast as if a getdents() had been done just before

Hi Jiffin, On Fri, Jan 19, 2018 at 09:19:45AM +0530, Jiffin Tony Thottan wrote: > It will be very easy to debug if u have core file with u. It looks like > crash is coming from gfapi stack. > > If there is core file can u please share bt of the core file. I'm sorry, there are no core files. :( Is there something I could do to trigger the creation of core files? Or anything else

What's /proc/sys/kernel/core_pattern set to for you? For me it is % cat /proc/sys/kernel/core_pattern core which will drop a core file in the working directory of the process. On 19/01/2018 12.56, Frank Wall wrote: > Is there something I could do to trigger the creation of core files?

Hi, I have a gluster 3.10 volume with a dir with ~1 million small files in them, say mounted at /mnt/dir with FUSE, and I'm observing something weird: When I list and stat them all using rsync, then the lstat() calls that rsync does are incredibly fast (23 microseconds per call on average, definitely faster than a network roundtrip between my 3-machine bricks connected via Ethernet). But

On 18/09/17 17:23, Ben Turner wrote: > Do you want tuned or untuned? If tuned I'd like to try one of my tunings for metadata, but I will use yours if you want. (Re-CC'd list) I would be interested in both, if possible: To confirm that it's not only my machines that exhibit this behaviour given my settings, and to see what can be achieved with your tuned settings. Thank you!

...ure you clear cache on both clients and servers(echo 3 > /proc/sys/vm/drop_caches) before running your tests to eliminate server and client side caching. If you are not dropping cache this could also cause the differences you are seeing. -b ----- Original Message ----- > From: "Niklas Hamb?chen" <mail at nh2.me> > To: gluster-users at gluster.org > Sent: Thursday, September 14, 2017 8:42:44 AM > Subject: [Gluster-users] Confusing lstat() performance > > Hi, > > I have a gluster 3.10 volume with a dir with ~1 million small files in > them, say moun...

Hi Niklas, Out of interest have you tried testing performance with performance.stat-prefetch enabled? -- Sam McLeod @s_mcleod https://smcleod.net > On 14 Sep 2017, at 10:42 pm, Niklas Hamb?chen <mail at nh2.me> wrote: > > Hi, > > I have a gluster 3.10 volume with a dir with ~1 million small files in > them, say mounted at /mnt/dir with FUSE, and I'm observing something weird: > > When I list and stat them all using rsync, then the lstat() calls that...