Gluster users - Dec 2017 - interval or event to evaluate free disk space?

Hi all, 

with the option "cluster.min-free-disk" set, glusterfs avoids placing
files bricks that are "too full".
I'd like to understand when the free space on the bricks  is calculated. It
seems to me that this does not happen for every write call (naturally) but at
some interval or that some other event triggers this.

i.e, if I write two files quickly (that together would fill a brick) I'd get
a error message:

dd if=/dev/zero  of=a    bs=1k count=15000 && sleep 1 && dd
if=/dev/zero  of=aa     bs=1k count=15000
#yiels: dd: error writing ?aa?: No space left on device
#(brick1 is full, but glusterd still tries to place file "aa" on the
same brick

dd if=/dev/zero  of=a    bs=1k count=15000 && sleep 60  && dd
if=/dev/zero  of=aa     bs=1k count=15000
#this works....
#link file on brick1 to "aa" and the real "aa"-file on
brick2

when is statfs (or a simlar call) evaluated?

Thank you!
best wishes,
Stefan

On 19 December 2017 at 00:11, Stefan Solbrig <stefan.solbrig at ur.de>
wrote:
> Hi all,
>
> with the option "cluster.min-free-disk" set, glusterfs avoids
placing
> files bricks that are "too full".
> I'd like to understand when the free space on the bricks  is
calculated.
> It seems to me that this does not happen for every write call (naturally)
> but at some interval or that some other event triggers this.
>
This is checked as part of the file create and mkdir calls.
>
> i.e, if I write two files quickly (that together would fill a brick)
I'd
> get a error message:
>
> dd if=/dev/zero  of=a    bs=1k count=15000 && sleep 1 && dd
if=/dev/zero
> of=aa     bs=1k count=15000
> #yiels: dd: error writing ?aa?: No space left on device
> #(brick1 is full, but glusterd still tries to place file "aa" on
the same
> brick
>
> dd if=/dev/zero  of=a    bs=1k count=15000 && sleep 60  &&
dd
> if=/dev/zero  of=aa     bs=1k count=15000
> #this works....
> #link file on brick1 to "aa" and the real "aa"-file on
brick2
>
Are these being done in parallel? If yes, you can hit this problem as the
value is calculated before the create at which point there is sufficient
disk space.

Regards,
Nithya
>
> when is statfs (or a simlar call) evaluated?
>
> Thank you!
> best wishes,
> Stefan
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20171219/c189bd7e/attachment.html>

Hi, 
> with the option "cluster.min-free-disk" set, glusterfs avoids
placing files bricks that are "too full".
> I'd like to understand when the free space on the bricks  is
calculated. It seems to me that this does not happen for every write call
(naturally) but at some interval or that some other event triggers this.
> 
> This is checked as part of the file create and mkdir calls.  
Ah, Ok. Now I understand all effects I see and I get consistent behavior!

Thank you so mucho for the clarification!
best wishes,
Stefan



-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20171220/1bb2a884/attachment.html>