I am using Gluster for a few years without any significant issue (after I tweaked configuration for v3.5). My configuration is as follows: network.remote-dio: enable cluster.eager-lock: enable performance.stat-prefetch: off performance.io-cache: off performance.read-ahead: off performance.quick-read: off performance.io-thread-count: 6 network.ping-timeout: 2 performance.cache-max-file-size: 0 performance.flush-behind: on features.barrier: disable snap-max-soft-limit: 7 auto-delete: on I use it for running virtual servers on top such a volume. Currently I run this version of Gluster: glusterfs-cli-3.6.5-1.el6.x86_64 glusterfs-3.6.5-1.el6.x86_64 glusterfs-api-3.6.5-1.el6.x86_64 glusterfs-server-3.6.5-1.el6.x86_64 glusterfs-libs-3.6.5-1.el6.x86_64 glusterfs-fuse-3.6.5-1.el6.x86_64 With recent CentOS 6. I have experienced an issue when I move some files from an hdd onto gluster volume such that one node gets overloaded in the middle of file upload. Therefore, I decided to upload it through ssh onto other server than where original images are store. I know that this sounds just weird, but it does not lead to overloading! Along these lines, I decided to upload 10G image onto gluster volume and the upload speed varied, but no overloading at all? Right after upload was done I realized that some virtuals are not running properly. Hence I checked heal status where I discoverd that 4 images are in split-brain state. I had to act quickly, so I resolved the split brain, and let gluster heal. When heal was done everything works? However, I have got a few more VMs to upload, and I am not sure what can happen.. My volume configuration: Volume Name: ph-fs-0 Type: Replicate Volume ID: 71ac6456-03e4-4bb3-a624-937f4605b2cb Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 10.11.100.1:/gfs/s3-sata-10k/fs Brick2: 10.11.100.2:/gfs/s3-sata-10k/fs Options Reconfigured: network.remote-dio: enable cluster.eager-lock: enable performance.stat-prefetch: off performance.io-cache: off performance.read-ahead: off performance.quick-read: off performance.io-thread-count: 6 network.ping-timeout: 2 performance.cache-max-file-size: 0 performance.flush-behind: on features.barrier: disable snap-max-soft-limit: 7 auto-delete: on and logs are attached. Milo? -------------- next part -------------- A non-text attachment was scrubbed... Name: glustershd.log Type: application/octet-stream Size: 9071 bytes Desc: not available URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151130/69e54bf1/attachment.obj> -------------- next part -------------- A non-text attachment was scrubbed... Name: var-lib-one-datastores-103.log Type: application/octet-stream Size: 122560 bytes Desc: not available URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151130/69e54bf1/attachment-0001.obj>
any thoughts? Milo?> 30. 11. 2015 v 23:45, Milo? Koz?k <milos.kozak at lejmr.com>: > > I am using Gluster for a few years without any significant issue (after I tweaked configuration for v3.5). My configuration is as follows: > > network.remote-dio: enable > cluster.eager-lock: enable > performance.stat-prefetch: off > performance.io-cache: off > performance.read-ahead: off > performance.quick-read: off > performance.io-thread-count: 6 > network.ping-timeout: 2 > performance.cache-max-file-size: 0 > performance.flush-behind: on > features.barrier: disable > snap-max-soft-limit: 7 > auto-delete: on > > I use it for running virtual servers on top such a volume. Currently I run this version of Gluster: > > glusterfs-cli-3.6.5-1.el6.x86_64 > glusterfs-3.6.5-1.el6.x86_64 > glusterfs-api-3.6.5-1.el6.x86_64 > glusterfs-server-3.6.5-1.el6.x86_64 > glusterfs-libs-3.6.5-1.el6.x86_64 > glusterfs-fuse-3.6.5-1.el6.x86_64 > > With recent CentOS 6. > > I have experienced an issue when I move some files from an hdd onto gluster volume such that one node gets overloaded in the middle of file upload. Therefore, I decided to upload it through ssh onto other server than where original images are store. I know that this sounds just weird, but it does not lead to overloading! > > Along these lines, I decided to upload 10G image onto gluster volume and the upload speed varied, but no overloading at all? Right after upload was done I realized that some virtuals are not running properly. Hence I checked heal status where I discoverd that 4 images are in split-brain state. I had to act quickly, so I resolved the split brain, and let gluster heal. When heal was done everything works? > > However, I have got a few more VMs to upload, and I am not sure what can happen.. > > My volume configuration: > > Volume Name: ph-fs-0 > Type: Replicate > Volume ID: 71ac6456-03e4-4bb3-a624-937f4605b2cb > Status: Started > Number of Bricks: 1 x 2 = 2 > Transport-type: tcp > Bricks: > Brick1: 10.11.100.1:/gfs/s3-sata-10k/fs > Brick2: 10.11.100.2:/gfs/s3-sata-10k/fs > Options Reconfigured: > network.remote-dio: enable > cluster.eager-lock: enable > performance.stat-prefetch: off > performance.io-cache: off > performance.read-ahead: off > performance.quick-read: off > performance.io-thread-count: 6 > network.ping-timeout: 2 > performance.cache-max-file-size: 0 > performance.flush-behind: on > features.barrier: disable > snap-max-soft-limit: 7 > auto-delete: on > > > and logs are attached. > > Milo? > <glustershd.log><var-lib-one-datastores-103.log>_______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users