Dmitry Kopelevich
2014-Feb-18 19:51 UTC
[Gluster-users] upgrading from gluster-3.2.6 to gluster-3.4.2
I am attempting to upgrade my GlusterFS from 3.2.6 to 3.4.2 using the instructions posted at http://vbellur.wordpress.com/2012/05/31/upgrading-to-glusterfs-3-3. These guidelines are for an upgrade to 3.3 but it is stated at http://vbellur.wordpress.com/2013/07/15/upgrading-to-glusterfs-3-4 that they can also be used to upgrade to 3.4.0. So I was hoping that they would also work with an upgrade to 3.4.2. I'm running CentOS 5 and installed the following rpms on the gluster servers: glusterfs-libs-3.4.2-1.el5.x86_64.rpm glusterfs-3.4.2-1.el5.x86_64.rpm glusterfs-fuse-3.4.2-1.el5.x86_64.rpm glusterfs-cli-3.4.2-1.el5.x86_64.rpm glusterfs-server-3.4.2-1.el5.x86_64.rpm glusterfs-rdma-3.4.2-1.el5.x86_64.rpm glusterfs-geo-replication-3.4.2-1.el5.x86_64.rpm According to the installation guidelines, installation from rpms should automatically copy the files from /etc/glusterd to /var/lib/glusterd. This didn't happen for me -- the directory /var/lib/glusterd contained only empty subdirectories. But the content of /etc/glusterd directory has moved to /etc/glusterd/glusterd. So, I decided to manually copy files from /etc/glusterd/glusterd to /var/lib/glusterd and follow step 5 of the installation guidelines (which was supposed to be skipped when installing from rpms): glusterd --xlator-option *.upgrade=on -N This didn't work (error message: glusterd: No match) Then I triedspecifying explicitly the name of my volume: glusterd --xlator-option <volume>.upgrade=on -N This lead to the following messages in file etc-glusterfs-glusterd.vol.log: [2014-02-18 17:22:27.146449] I [glusterd.c:961:init] 0-management: Using /var/lib/glusterd as working directory [2014-02-18 17:22:27.149097] I [socket.c:3480:socket_init] 0-socket.management: SSL support is NOT enabled [2014-02-18 17:22:27.149126] I [socket.c:3495:socket_init] 0-socket.management: using system polling thread [2014-02-18 17:22:29.282665] I [glusterd-store.c:1339:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 1 [2014-02-18 17:22:29.283478] E [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown key: brick-0 [2014-02-18 17:22:29.283513] E [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown key: brick-1 [2014-02-18 17:22:29.283534] E [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown key: brick-2 ... and so on for all other bricks. After that, files nfs.log, glustershd.log, and etc-glusterfs-glusterd.vol.log get filled with a large number of warning messages and nothing else seems to happen. The following messages appear to be relevant: - Files nfs.log, glustershd.log: 2014-02-18 15:58:01.889847] W [rdma.c:1079:gf_rdma_cm_event_handler] 0-data-volume-client-2: cma event RDMA_CM_EVENT_ADDR_ERROR, error -2 (me: peer:) (the name of my volume is data-volume and its transport type is RDMA) - File etc-glusterfs-glusterd.vol.log [2014-02-18 17:22:33.322565] W [socket.c:514:__socket_rwv] 0-management: readv failed (No data available) Also, for some reason the time stamps in the log files are incorrect. Any suggestions for fixing this would be greatly appreciated. Thanks, Dmitry -- Dmitry Kopelevich Associate Professor Chemical Engineering Department University of Florida Gainesville, FL 32611 Phone: (352)-392-4422 Fax: (352)-392-9513 E-mail:dkopelevich at che.ufl.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140218/a7d9f296/attachment.html>
Dmitry Kopelevich
2014-Feb-21 13:22 UTC
[Gluster-users] upgrading from gluster-3.2.6 to gluster-3.4.2
I would like to follow up on my question regarding an upgrade from 3.2.6 to 3.4.2. Can anybody tell me whether I'm doing something completely wrong? Am I trying to skip too many versions of gluster in my upgrade? Is CentOS 5 too old for this? Thanks, Dmitry On 2/18/2014 2:51 PM, Dmitry Kopelevich wrote:> I am attempting to upgrade my GlusterFS from 3.2.6 to 3.4.2 using the > instructions posted at > http://vbellur.wordpress.com/2012/05/31/upgrading-to-glusterfs-3-3. > These guidelines are for an upgrade to 3.3 but it is stated at > http://vbellur.wordpress.com/2013/07/15/upgrading-to-glusterfs-3-4 > that they can also be used to upgrade to 3.4.0. So I was hoping that > they would also work with an upgrade to 3.4.2. > > I'm running CentOS 5 and installed the following rpms on the gluster > servers: > > glusterfs-libs-3.4.2-1.el5.x86_64.rpm > glusterfs-3.4.2-1.el5.x86_64.rpm > glusterfs-fuse-3.4.2-1.el5.x86_64.rpm > glusterfs-cli-3.4.2-1.el5.x86_64.rpm > glusterfs-server-3.4.2-1.el5.x86_64.rpm > glusterfs-rdma-3.4.2-1.el5.x86_64.rpm > glusterfs-geo-replication-3.4.2-1.el5.x86_64.rpm > > According to the installation guidelines, installation from rpms > should automatically copy the files from /etc/glusterd to > /var/lib/glusterd. This didn't happen for me -- the directory > /var/lib/glusterd contained only empty subdirectories. But the content > of /etc/glusterd directory has moved to /etc/glusterd/glusterd. > > So, I decided to manually copy files from /etc/glusterd/glusterd to > /var/lib/glusterd and follow step 5 of the installation guidelines > (which was supposed to be skipped when installing from rpms): > > glusterd --xlator-option *.upgrade=on -N > > This didn't work (error message: glusterd: No match) > > Then I triedspecifying explicitly the name of my volume: > > glusterd --xlator-option <volume>.upgrade=on -N > > This lead to the following messages in file > etc-glusterfs-glusterd.vol.log: > > [2014-02-18 17:22:27.146449] I [glusterd.c:961:init] 0-management: > Using /var/lib/glusterd as working directory > [2014-02-18 17:22:27.149097] I [socket.c:3480:socket_init] > 0-socket.management: SSL support is NOT enabled > [2014-02-18 17:22:27.149126] I [socket.c:3495:socket_init] > 0-socket.management: using system polling thread > [2014-02-18 17:22:29.282665] I > [glusterd-store.c:1339:glusterd_restore_op_version] 0-glusterd: > retrieved op-version: 1 > [2014-02-18 17:22:29.283478] E > [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown > key: brick-0 > [2014-02-18 17:22:29.283513] E > [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown > key: brick-1 > [2014-02-18 17:22:29.283534] E > [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown > key: brick-2 > ... > and so on for all other bricks. > > After that, files nfs.log, glustershd.log, and > etc-glusterfs-glusterd.vol.log get filled with a large number of > warning messages and nothing else seems to happen. The following > messages appear to be relevant: > > - Files nfs.log, glustershd.log: > > 2014-02-18 15:58:01.889847] W [rdma.c:1079:gf_rdma_cm_event_handler] > 0-data-volume-client-2: cma event RDMA_CM_EVENT_ADDR_ERROR, error -2 > (me: peer:) > > (the name of my volume is data-volume and its transport type is RDMA) > > - File etc-glusterfs-glusterd.vol.log > > [2014-02-18 17:22:33.322565] W [socket.c:514:__socket_rwv] > 0-management: readv failed (No data available) > > Also, for some reason the time stamps in the log files are incorrect. > > Any suggestions for fixing this would be greatly appreciated. > > Thanks, > > Dmitry > -- > Dmitry Kopelevich > Associate Professor > Chemical Engineering Department > University of Florida > Gainesville, FL 32611 > > Phone: (352)-392-4422 > Fax: (352)-392-9513 > E-mail:dkopelevich at che.ufl.edu-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140221/d2f90110/attachment.html>
Vijay Bellur
2014-Feb-24 16:40 UTC
[Gluster-users] upgrading from gluster-3.2.6 to gluster-3.4.2
On 02/19/2014 01:21 AM, Dmitry Kopelevich wrote:> > According to the installation guidelines, installation from rpms should > automatically copy the files from /etc/glusterd to /var/lib/glusterd. > This didn't happen for me -- the directory /var/lib/glusterd contained > only empty subdirectories. But the content of /etc/glusterd directory > has moved to /etc/glusterd/glusterd.Did /var/lib/glusterd per chance exist before installing the RPMs? If it does, then contents of /etc/glusterd do not get copied over to /var/lib/glusterd.> > So, I decided to manually copy files from /etc/glusterd/glusterd to > /var/lib/glusterd and follow step 5 of the installation guidelines > (which was supposed to be skipped when installing from rpms): > > glusterd --xlator-option *.upgrade=on -N > > This didn't work (error message: glusterd: No match) > > Then I triedspecifying explicitly the name of my volume: > > glusterd --xlator-option <volume>.upgrade=on -N > > This lead to the following messages in file etc-glusterfs-glusterd.vol.log: > > [2014-02-18 17:22:27.146449] I [glusterd.c:961:init] 0-management: Using > /var/lib/glusterd as working directory > [2014-02-18 17:22:27.149097] I [socket.c:3480:socket_init] > 0-socket.management: SSL support is NOT enabled > [2014-02-18 17:22:27.149126] I [socket.c:3495:socket_init] > 0-socket.management: using system polling thread > [2014-02-18 17:22:29.282665] I > [glusterd-store.c:1339:glusterd_restore_op_version] 0-glusterd: > retrieved op-version: 1 > [2014-02-18 17:22:29.283478] E > [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown key: > brick-0 > [2014-02-18 17:22:29.283513] E > [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown key: > brick-1 > [2014-02-18 17:22:29.283534] E > [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown key: > brick-2 > ... > and so on for all other bricks.These messages related to bricks are benign and can be ignored.> > After that, files nfs.log, glustershd.log, and > etc-glusterfs-glusterd.vol.log get filled with a large number of warning > messages and nothing else seems to happen. The following messages appear > to be relevant: > > - Files nfs.log, glustershd.log: > > 2014-02-18 15:58:01.889847] W [rdma.c:1079:gf_rdma_cm_event_handler] > 0-data-volume-client-2: cma event RDMA_CM_EVENT_ADDR_ERROR, error -2 > (me: peer:)Do you also have IPoIB in your setup? RDMA-CM in 3.4.x releases does need IPoIB to function properly. [1] Raghavendra - can you please help here?> > (the name of my volume is data-volume and its transport type is RDMA) > > - File etc-glusterfs-glusterd.vol.log > > [2014-02-18 17:22:33.322565] W [socket.c:514:__socket_rwv] 0-management: > readv failed (No data available) > > Also, for some reason the time stamps in the log files are incorrect.Starting with 3.4, time stamps in the log files are in UTC by default. Thanks, Vijay [1] https://github.com/gluster/glusterfs/blob/master/doc/features/rdma-cm-in-3.4.0.txt