Gluster users - Mar 2015 - Replica 2 heal blocks virtual machines

Dear List!

I am running a proxmox cluster (3 nodes).
2 of them are acting as glusterfs servers (Version 3.6.2-2) and have a 
16 TB share (XFS) each

I am running virtual machines on the cluster and i have a really big vm 
fileserver running (2 virtual disks a 1 TB).
Now i had to reboot one of the nodes and it came up right after 5 
minutes. At this time, the vm fileserver was running.

After the node came back up, it took a few minutes, then the vm got 
unresponsible. The Disks of the two glusternodes where blinking and
a selfheal was started.

Is there a way to make the selfheal process NOT blocking the vm? Is 
there an option to let gluster do the selfheal in background?

I have no options configured in either /etc/gluster/glusterd.vol or any 
volumes, just plain glusterfs installation.

The nodes are all connected via 10Gbit ethernet and i have only one 
network cable on each server and a 10Gbit switch.

The pure raid performance is good so far, i can write to mounted 16TB 
partition (raid5, 4 6TB HDD) with dd with about 400-500 MB/sec
I can also write to mounted glustervolume /mnt/pve/gluster with about 
200 MB/sec.

Are there any tuning options to make the glusterfs faster and let it do 
the heal nonblocking in background?

We are running about 12 Windows terminalservers on this cluster and 
several linux machines (DB,Mail and so on)

It is really bad if the whole system "hangs" when a heal is running.

Thank you

Alex

https://www.mail-archive.com/gluster-users at gluster.org/msg19152.html maybe?


On 03/20/2015 01:37 PM, Alexander Marx wrote:
> Dear List!
>
> I am running a proxmox cluster (3 nodes).
> 2 of them are acting as glusterfs servers (Version 3.6.2-2) and have a 
> 16 TB share (XFS) each
>
> I am running virtual machines on the cluster and i have a really big 
> vm fileserver running (2 virtual disks a 1 TB).
> Now i had to reboot one of the nodes and it came up right after 5 
> minutes. At this time, the vm fileserver was running.
>
> After the node came back up, it took a few minutes, then the vm got 
> unresponsible. The Disks of the two glusternodes where blinking and
> a selfheal was started.
>
> Is there a way to make the selfheal process NOT blocking the vm? Is 
> there an option to let gluster do the selfheal in background?
>
> I have no options configured in either /etc/gluster/glusterd.vol or 
> any volumes, just plain glusterfs installation.
>
> The nodes are all connected via 10Gbit ethernet and i have only one 
> network cable on each server and a 10Gbit switch.
>
> The pure raid performance is good so far, i can write to mounted 16TB 
> partition (raid5, 4 6TB HDD) with dd with about 400-500 MB/sec
> I can also write to mounted glustervolume /mnt/pve/gluster with about 
> 200 MB/sec.
>
> Are there any tuning options to make the glusterfs faster and let it 
> do the heal nonblocking in background?
>
> We are running about 12 Windows terminalservers on this cluster and 
> several linux machines (DB,Mail and so on)
>
> It is really bad if the whole system "hangs" when a heal is
running.
>
> Thank you
>
> Alex
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users