Kushnir, Michael (NIH/NLM/LHC) [C]
2012-Oct-23 17:35 UTC
[Gluster-users] Problems with striped-replicated volumes on 3.3.1
Good afternoon, I am playing around with GlusterFS 3.1 in CentOS 6 virtual machines to see if I can get of proof of concept for a bigger project. In my setup, I have 4 GlusterFS servers with two bricks each of 10GB with XFS (per your quick-start guide). So, I have a total of 8 bricks. When bu I have no problem with distributed-replicated volumes. However, when I set up a striped replicated volume and mounted it via the native client, I had problems with file operations. My striped-replicated volume had 8 bricks. I set it up with 4 stripes and 2 replicas. I made sure that no bricks on the same server were in pairs next to each other. The volume created fine and started fine. "gluster volume info" and "gluster volume status" showed no problems. The volume mounted fine. I tried to run "dd if=/dev/zero..." to a new file on the volume, and it would fail every time. However, when I would "touch" the file to create it first, dd would work fine afterwards. The same was true with creating a text file with nano. First, the save of a new file would fail, then when re-tried, it would succeed. The errors I would see in the client log are: [2012-10-21 19:41:18.974559] E [stripe-helpers.c:268:stripe_ctx_handle] 0-dsr-vol-stripe-0: Failed to get stripe-size [2012-10-21 19:41:18.975471] W [fuse-bridge.c:2025:fuse_writev_cbk] 0-glusterfs-fuse: 11545: WRITE => -1 (Invalid argument) Please advise. Thanks, Michael ______________________________________________________________________________________ Michael Kushnir System Architect / Engineer Communications Engineering Branch Lister Hill National Center for Biomedical Communications National Library of Medicine 8600 Rockville Pike, Building 38A, Floor 10 Besthesda, MD 20894 Phone: 301-435-3219 Email: michael.kushnir at nih.gov<mailto:michael.kushnir at nih.gov> [cid:image001.jpg at 01CDB123.54A566B0] -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20121023/18ca47d4/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 8449 bytes Desc: image001.jpg URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20121023/18ca47d4/attachment.jpg>
Ben Turner
2012-Oct-23 21:33 UTC
[Gluster-users] Problems with striped-replicated volumes on 3.3.1
----- Original Message -----> From: "Michael Kushnir (NIH/NLM/LHC) [C]" <michael.kushnir at nih.gov> > To: gluster-users at gluster.org > Sent: Tuesday, October 23, 2012 1:35:56 PM > Subject: [Gluster-users] Problems with striped-replicated volumes on 3.3.1 > > > > > > Good afternoon, > > > > I am playing around with GlusterFS 3.1 in CentOS 6 virtual machines > to see if I can get of proof of concept for a bigger project. In my > setup, I have 4 GlusterFS servers with two bricks each of 10GB with > XFS (per your quick-start guide). So, I have a total of 8 bricks. > When bu > > > > I have no problem with distributed-replicated volumes. However, when > I set up a striped replicated volume and mounted it via the native > client, I had problems with file operations. > > > > My striped-replicated volume had 8 bricks. I set it up with 4 stripes > and 2 replicas. I made sure that no bricks on the same server were > in pairs next to each other. The volume created fine and started > fine. ?gluster volume info? and ?gluster volume status? showed no > problems. The volume mounted fine. > > > > I tried to run ?dd if=/dev/zero?? to a new file on the volume, and it > would fail every time. However, when I would ?touch? the file to > create it first, dd would work fine afterwards. The same was true > with creating a text file with nano. First, the save of a new file > would fail, then when re-tried, it would succeed. > > > > The errors I would see in the client log are: > > > > [2012-10-21 19:41:18.974559] E > [stripe-helpers.c:268:stripe_ctx_handle] 0-dsr-vol-stripe-0: Failed > to get stripe-size > > [2012-10-21 19:41:18.975471] W [fuse-bridge.c:2025:fuse_writev_cbk] > 0-glusterfs-fuse: 11545: WRITE => -1 (Invalid argument) >I opened BZ https://bugzilla.redhat.com/show_bug.cgi?id=861423 a while back. I am also hitting problems with fops on striped replicate volumes on the 3.3.0 branch. I just retested and hit it again, I'll update the BZ with my latest.> > > Please advise. > > > > > > Thanks, > Michael > > > > > > ______________________________________________________________________________________ > > Michael Kushnir > > System Architect / Engineer > > Communications Engineering Branch > > Lister Hill National Center for Biomedical Communications > > National Library of Medicine > > > > 8600 Rockville Pike, Building 38A, Floor 10 > > Besthesda, MD 20894 > > > > Phone: 301-435-3219 > > Email: michael.kushnir at nih.gov > > > > VMW-LGO-CERTIFIED-PRO-5-K > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users