Gluster users - Dec 2017 - Upgrading from Gluster 3.8 to 3.12

Hi,

I have a cluster of 10 servers all running Fedora 24 along with
Gluster 3.8.  I'm planning on doing rolling upgrades to Fedora 27 with
Gluster 3.12.  I saw the documentation and did some testing but I
would like to run my plan through some (more?) educated minds.

The current setup is:

Volume Name: vol0
Distributed-Replicate
Number of Bricks: 2 x (2 + 1) = 6
Bricks:
Brick1: glt01:/vol/vol0
Brick2: glt02:/vol/vol0
Brick3: glt05:/vol/vol0 (arbiter)
Brick4: glt03:/vol/vol0
Brick5: glt04:/vol/vol0
Brick6: glt06:/vol/vol0 (arbiter)

Volume Name: vol1
Distributed-Replicate
Number of Bricks: 2 x (2 + 1) = 6
Bricks:
Brick1: glt07:/vol/vol1
Brick2: glt08:/vol/vol1
Brick3: glt05:/vol/vol1 (arbiter)
Brick4: glt09:/vol/vol1
Brick5: glt10:/vol/vol1
Brick6: glt06:/vol/vol1 (arbiter)

After performing the upgrade because of differences in checksums, the
upgraded nodes will become:

State: Peer Rejected (Connected)

If I start doing the upgrades one at a time, with nodes glt10 to glt01
except for the arbiters glt05 and glt06, and then upgrading the
arbiters last, everything should remain online at all times through
the process.  Correct?

Thanks.

On Tue, Dec 19, 2017 at 1:10 AM, Ziemowit Pierzycki <ziemowit at
pierzycki.com>
wrote:
> Hi,
>
> I have a cluster of 10 servers all running Fedora 24 along with
> Gluster 3.8.  I'm planning on doing rolling upgrades to Fedora 27 with
> Gluster 3.12.  I saw the documentation and did some testing but I
> would like to run my plan through some (more?) educated minds.
>
> The current setup is:
>
> Volume Name: vol0
> Distributed-Replicate
> Number of Bricks: 2 x (2 + 1) = 6
> Bricks:
> Brick1: glt01:/vol/vol0
> Brick2: glt02:/vol/vol0
> Brick3: glt05:/vol/vol0 (arbiter)
> Brick4: glt03:/vol/vol0
> Brick5: glt04:/vol/vol0
> Brick6: glt06:/vol/vol0 (arbiter)
>
> Volume Name: vol1
> Distributed-Replicate
> Number of Bricks: 2 x (2 + 1) = 6
> Bricks:
> Brick1: glt07:/vol/vol1
> Brick2: glt08:/vol/vol1
> Brick3: glt05:/vol/vol1 (arbiter)
> Brick4: glt09:/vol/vol1
> Brick5: glt10:/vol/vol1
> Brick6: glt06:/vol/vol1 (arbiter)
>
> After performing the upgrade because of differences in checksums, the
> upgraded nodes will become:
>
> State: Peer Rejected (Connected)
>
Have you upgraded all the nodes? If yes, have you bumped up the
cluster.op-version after upgrading all the nodes? Please follow :
http://docs.gluster.org/en/latest/Upgrade-Guide/op_version/ for more
details on how to bump up the cluster.op-version. In case you have done all
of these and you're seeing a checksum issue then I'm afraid you have hit
a
bug. I'd need further details like the checksum mismatch error from
glusterd.log file along with the the exact volume's info file from
/var/lib/glusterd/vols/<volname>/info between both the peers to debug this
further.

> If I start doing the upgrades one at a time, with nodes glt10 to glt01
> except for the arbiters glt05 and glt06, and then upgrading the
> arbiters last, everything should remain online at all times through
> the process.  Correct?
>
> Thanks.
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20171219/6fe43aa5/attachment.html>

I have not done the upgrade yet.  Since this is a production cluster I
need to make sure it stays up or schedule some downtime if it doesn't
doesn't.  Thanks.

On Tue, Dec 19, 2017 at 10:11 AM, Atin Mukherjee <amukherj at redhat.com>
wrote:
>
>
> On Tue, Dec 19, 2017 at 1:10 AM, Ziemowit Pierzycki <ziemowit at
pierzycki.com>
> wrote:
>>
>> Hi,
>>
>> I have a cluster of 10 servers all running Fedora 24 along with
>> Gluster 3.8.  I'm planning on doing rolling upgrades to Fedora 27
with
>> Gluster 3.12.  I saw the documentation and did some testing but I
>> would like to run my plan through some (more?) educated minds.
>>
>> The current setup is:
>>
>> Volume Name: vol0
>> Distributed-Replicate
>> Number of Bricks: 2 x (2 + 1) = 6
>> Bricks:
>> Brick1: glt01:/vol/vol0
>> Brick2: glt02:/vol/vol0
>> Brick3: glt05:/vol/vol0 (arbiter)
>> Brick4: glt03:/vol/vol0
>> Brick5: glt04:/vol/vol0
>> Brick6: glt06:/vol/vol0 (arbiter)
>>
>> Volume Name: vol1
>> Distributed-Replicate
>> Number of Bricks: 2 x (2 + 1) = 6
>> Bricks:
>> Brick1: glt07:/vol/vol1
>> Brick2: glt08:/vol/vol1
>> Brick3: glt05:/vol/vol1 (arbiter)
>> Brick4: glt09:/vol/vol1
>> Brick5: glt10:/vol/vol1
>> Brick6: glt06:/vol/vol1 (arbiter)
>>
>> After performing the upgrade because of differences in checksums, the
>> upgraded nodes will become:
>>
>> State: Peer Rejected (Connected)
>
>
> Have you upgraded all the nodes? If yes, have you bumped up the
> cluster.op-version after upgrading all the nodes? Please follow :
> http://docs.gluster.org/en/latest/Upgrade-Guide/op_version/ for more
details
> on how to bump up the cluster.op-version. In case you have done all of
these
> and you're seeing a checksum issue then I'm afraid you have hit a
bug. I'd
> need further details like the checksum mismatch error from glusterd.log
file
> along with the the exact volume's info file from
> /var/lib/glusterd/vols/<volname>/info between both the peers to debug
this
> further.
>
>>
>> If I start doing the upgrades one at a time, with nodes glt10 to glt01
>> except for the arbiters glt05 and glt06, and then upgrading the
>> arbiters last, everything should remain online at all times through
>> the process.  Correct?
>>
>> Thanks.
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-users
>
>