Kevin Lemonnier
2016-Sep-05 22:59 UTC
[Gluster-users] [URGENT] Add-bricks to a volume corrupted the files
> - What was the original (and current) geometry? (status and info)It was a 1x3 that I was trying to bump to 2x3.> - what parameters did you use when adding the bricks? >Just a simple add-brick node1:/path node2:/path node3:/path Then a fix-layout when everything started going wrong. I was able to salvage some VMs by stopping them then starting them again, but most won't start for various reasons (disk corrupted, grub not found ...). For those we are deleting the disks then importing them from backups, that's a huge loss but everything has been down for so long, no choice ..> On 6/09/2016 8:00 AM, Kevin Lemonnier wrote: > > I tried a fix-layout, and since that didn't work I removed the brick (start then commit when it showed > completed). Not better, the volume is now running on the 3 original bricks (replica 3) but the VMs > are still corrupted. I have 880 Mb of shards on the bricks I removed for some reason, thos shards do exist > (and are bigger) on the "live" volume. I don't understand why now that I have removed the new bricks > everything isn't working like before .. > > On Mon, Sep 05, 2016 at 11:06:16PM +0200, Kevin Lemonnier wrote: > > Hi, > > I just added 3 bricks to a volume and all the VMs are doing I/O errors now. > I rebooted a VM to see and it can't start again, am I missing something ? Is the reblance required > to make everything run ? > > That's urgent, thanks. > > -- > Kevin Lemonnier > PGP Fingerprint : 89A5 2283 04A0 E6E9 0111 > > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users > > -- > Lindsay Mathieson> _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users-- Kevin Lemonnier PGP Fingerprint : 89A5 2283 04A0 E6E9 0111 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160906/f9a91216/attachment.sig>
Krutika Dhananjay
2016-Sep-06 04:18 UTC
[Gluster-users] [URGENT] Add-bricks to a volume corrupted the files
Could you please attach the glusterfs client and brick logs? Also provide output of `gluster volume info`. -Krutika On Tue, Sep 6, 2016 at 4:29 AM, Kevin Lemonnier <lemonnierk at ulrar.net> wrote:> > - What was the original (and current) geometry? (status and info) > > It was a 1x3 that I was trying to bump to 2x3. > > > - what parameters did you use when adding the bricks? > > > > Just a simple add-brick node1:/path node2:/path node3:/path > Then a fix-layout when everything started going wrong. > > > I was able to salvage some VMs by stopping them then starting them again, > but most won't start for various reasons (disk corrupted, grub not found > ...). > For those we are deleting the disks then importing them from backups, > that's > a huge loss but everything has been down for so long, no choice .. > > > On 6/09/2016 8:00 AM, Kevin Lemonnier wrote: > > > > I tried a fix-layout, and since that didn't work I removed the brick > (start then commit when it showed > > completed). Not better, the volume is now running on the 3 original > bricks (replica 3) but the VMs > > are still corrupted. I have 880 Mb of shards on the bricks I removed > for some reason, thos shards do exist > > (and are bigger) on the "live" volume. I don't understand why now that > I have removed the new bricks > > everything isn't working like before .. > > > > On Mon, Sep 05, 2016 at 11:06:16PM +0200, Kevin Lemonnier wrote: > > > > Hi, > > > > I just added 3 bricks to a volume and all the VMs are doing I/O errors > now. > > I rebooted a VM to see and it can't start again, am I missing something > ? Is the reblance required > > to make everything run ? > > > > That's urgent, thanks. > > > > -- > > Kevin Lemonnier > > PGP Fingerprint : 89A5 2283 04A0 E6E9 0111 > > > > > > > > > > _______________________________________________ > > Gluster-users mailing list > > Gluster-users at gluster.org > > http://www.gluster.org/mailman/listinfo/gluster-users > > > > > > > > _______________________________________________ > > Gluster-users mailing list > > Gluster-users at gluster.org > > http://www.gluster.org/mailman/listinfo/gluster-users > > > > -- > > Lindsay Mathieson > > > _______________________________________________ > > Gluster-users mailing list > > Gluster-users at gluster.org > > http://www.gluster.org/mailman/listinfo/gluster-users > > > -- > Kevin Lemonnier > PGP Fingerprint : 89A5 2283 04A0 E6E9 0111 > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160906/839c4896/attachment.html>