On 17 May 2016 at 10:02, WK <wkmail at bneit.com> wrote:> That being said, when we lose a brick, we've traditionally just live > migrated those VMs off onto other clusters because we didn't want to take > the heal hit which at best slowed down our VMs at on the pickier ones cause > them to RO out. > > We have not yet upgraded to 3.7.x yet (still on 3.4 cuz it aint broke) and > are hoping that sharding solves that problem. But it seems everytime it > looks like things are 'safe' for 3.7.x, something comes up. Fortunately, we > like the fuse mount so maybe we are still ok.Unfortunately(?) I get much better performance out of the gfapi - seeing around 30-40% better reads and IOPs over the fuse client in VM's. -- Lindsay
Would you know where the logs of individual vms are with proxmox? In those, do you see any libgfapi/gluster log messages? -Krutika On Tue, May 17, 2016 at 8:38 AM, Lindsay Mathieson < lindsay.mathieson at gmail.com> wrote:> On 17 May 2016 at 10:02, WK <wkmail at bneit.com> wrote: > > That being said, when we lose a brick, we've traditionally just live > > migrated those VMs off onto other clusters because we didn't want to take > > the heal hit which at best slowed down our VMs at on the pickier ones > cause > > them to RO out. > > > > We have not yet upgraded to 3.7.x yet (still on 3.4 cuz it aint broke) > and > > are hoping that sharding solves that problem. But it seems everytime it > > looks like things are 'safe' for 3.7.x, something comes up. Fortunately, > we > > like the fuse mount so maybe we are still ok. > > > Unfortunately(?) I get much better performance out of the gfapi - > seeing around 30-40% better reads and IOPs over the fuse client in > VM's. > > -- > Lindsay > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160517/e13bc7fb/attachment.html>
I believe some of the staff tested gfapi back in the 3.4 days and found that at least that version didn't make a perceptible difference though it tested about 10% faster. We stayed with Fuse because at that time gfapi was still newish and we had used fuse for quite awhile and understood it. Evidently gfapi has improved since then (assuming the corruption issue is resolved). On 5/16/16 8:08 PM, Lindsay Mathieson wrote:> On 17 May 2016 at 10:02, WK <wkmail at bneit.com> wrote: >> That being said, when we lose a brick, we've traditionally just live >> migrated those VMs off onto other clusters because we didn't want to take >> the heal hit which at best slowed down our VMs at on the pickier ones cause >> them to RO out. >> >> We have not yet upgraded to 3.7.x yet (still on 3.4 cuz it aint broke) and >> are hoping that sharding solves that problem. But it seems everytime it >> looks like things are 'safe' for 3.7.x, something comes up. Fortunately, we >> like the fuse mount so maybe we are still ok. > > Unfortunately(?) I get much better performance out of the gfapi - > seeing around 30-40% better reads and IOPs over the fuse client in > VM's. >