J. Pablo Fernández
2014-Jun-17 08:25 UTC
[Gluster-users] Mounting localhost at boot time with GlusterFS 3.5 official packages in Ubuntu 12.04
Hello,
I'm running the official GlusterFS 3.5 packages on an Ubuntu 12.04 box that
is acting as both, client and server, and everything seems to be working
fine, except mounting the GlusterFS volumes at boot time. This is what I
see in the log files:
[2014-06-17 08:20:52.969258] I [glusterfsd.c:1959:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.5.0
(/usr/sbin/glusterfs --volfile-server=127.0.0.1
--volfile-id=/public_uploads /var/www/shared/public/uploads)
[2014-06-17 08:20:52.998985] I [socket.c:3561:socket_init] 0-glusterfs:
SSL support is NOT enabled
[2014-06-17 08:20:52.999048] I [socket.c:3576:socket_init] 0-glusterfs:
using system polling thread
[2014-06-17 08:20:53.000373] E [socket.c:2161:socket_connect_finish]
0-glusterfs: connection to 127.0.0.1:24007 failed (Connection refused)
[2014-06-17 08:20:53.000427] E [glusterfsd-mgmt.c:1601:mgmt_rpc_notify]
0-glusterfsd-mgmt: failed to connect with remote-host: 127.0.0.1 (No data
available)
[2014-06-17 08:20:53.000442] I [glusterfsd-mgmt.c:1607:mgmt_rpc_notify]
0-glusterfsd-mgmt: Exhausted all volfile servers
[2014-06-17 08:20:53.013793] W [glusterfsd.c:1095:cleanup_and_exit]
(-->/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_transport_notify+0x27)
[0x7f686e0160f7]
(-->/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x1a4)
[0x7f686e019cc4] (-->/usr/sbin/glusterfs(+0xcada) [0x7f686e6ddada]))) 0-:
received signum (1), shutting down
[2014-06-17 08:20:53.013830] I [fuse-bridge.c:5444:fini] 0-fuse:
Unmounting '/var/www/shared/public/uploads'.
My fstab contains:
proc /proc proc defaults
0 0
/dev/xvda / ext4
noatime,errors=remount-ro 0 1
/dev/xvdb none swap sw
0 0
/dev/xvdc /var/lib/glusterfs/brick01 ext4 defaults
1 2
127.0.0.1:/private_uploads /var/www/shared/private/uploads glusterfs
defaults,_netdev 0 0
I know this used to be a bug in GlusterFS 3.2 for Ubuntu, but I understand
it was solved in the PPA packages for GlusterFS 3.4 as shown here:
https://bugs.launchpad.net/ubuntu/+source/glusterfs/+bug/876648
I also remember this working in an experiment I run with some virtual
machines (but since it just working, I didn't look too deep into it). I see
that the gluster-client packages provides an upstart job called
mounting-glusterfs.conf which contains:
author "Louis Zuckerman <me at louiszuckerman.com>"
description "Block the mounting event for glusterfs filesystems until
the network interfaces are running"
instance $MOUNTPOINT
start on mounting TYPE=glusterfs
task
exec start wait-for-state WAIT_FOR=static-network-up
WAITER=mounting-glusterfs-$MOUNTPOINT
But I'm not so sure how it should work. It doesn't seem to be working
out
of the box. Even though mounting of glusterfs volumes happens after the
network starts, it happens before GlusterFS starts:
* Starting RPC portmapper replacement
[ OK ]
* Stopping rpcsec_gss daemon
[ OK ]
* Starting Start this job to wait until rpcbind is started or fails to
s[ OK ]
* Starting configure network device
[ OK ]
* Stopping Start this job to wait until rpcbind is started or fails to
s[ OK ]
* Starting Bridge socket events into upstart
[ OK ]
* Starting NSM status monitor
[ OK ]
* Stopping cold plug devices
[ OK ]
* Stopping log initial device creation
[ OK ]
* Starting load fallback graphics devices
[ OK ]
* Starting configure network device security
[ OK ]
* Starting load fallback graphics devices
[fail]
* Starting configure virtual network devices
[ OK ]
* Starting Send an event to indicate plymouth is up
[ OK ]
* Stopping Send an event to indicate plymouth is up
[ OK ]
* Starting Mount network filesystems
[ OK ]
* Stopping configure virtual network devices
[ OK ]
* Stopping Mount network filesystems
[ OK ]
* Starting Mount network filesystems
[ OK ]
* Stopping Mount network filesystems
[ OK ]
* Starting configure network device
[ OK ]
* Starting set sysctls from /etc/sysctl.conf
[ OK ]
* Stopping set sysctls from /etc/sysctl.conf
[ OK ]
The disk drive for /var/www/shared/public/uploads is not ready yet or
not present.
Continue to wait, or Press S to skip mounting or M for manual recovery
* Starting Waiting for state
[fail]
* Starting Block the mounting event for glusterfs filesystems until
the [fail]k interfaces are running
mountall: Event failed
Mount failed. Please check the log file for more details.
* Starting GNU Screen Cleanup
[ OK ]
* Starting flush early job output to logs
[ OK ]
* Starting base
[ OK ]
* Starting save udev log and update rules
[ OK ]
* Starting OpenSSH server
[ OK ]
* Stopping Failsafe Boot Delay
[ OK ]
* Starting System V initialisation compatibility
[ OK ]
* Stopping save udev log and update rules
[ OK ]
* Stopping Mount filesystems on boot
[ OK ]
* Stopping GNU Screen Cleanup
[ OK ]
* Stopping flush early job output to logs
[ OK ]
* Starting system logging daemon
[ OK ]
* Stopping System V initialisation compatibility
[ OK ]
* Starting System V runlevel compatibility
[ OK ]
* Starting save kernel messages
[ OK ]
* Starting deferred execution scheduler
[ OK ]
* Starting CPU interrupts balancing daemon
[ OK ]
* Starting regular background program processing daemon
[ OK ]
* Starting automatic crash report generation
[ OK ]
* Starting GlusterFS Management Daemon
[ OK ]
Any ideas what's going on and/or how to fix it?
--
J. Pablo Fern?ndez <pupeno at pupeno.com> (http://pupeno.com)
PS: I did post this in serverfault some time ago, but I didn't receive any
reply other than a custom script to mount things after GlusterFS started:
http://serverfault.com/questions/604860/glusterfs-is-failing-to-mount-on-boot
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140617/ef175199/attachment.html>
J. Pablo Fernández
2014-Jun-19 11:59 UTC
[Gluster-users] Mounting localhost at boot time with GlusterFS 3.5 official packages in Ubuntu 12.04
Hello, I also tried a custom upstart job to mount the volumes after glusterfs-server starts, like this: description "Mount public uploads" start on started glusterfs-server exec mount /var/www/shared/public/uploads but it didn't work. All I could find was:> Mount failed. Please check the log file for more details.in the upstart logs, but no other log. Starting that upstart job manually after the computer finished booting up seemed to work, so I would guess that glusterfs didn't finish starting or something. What's the proper way of doing this? Is there a proper way of mounting a localhost volume at boot time? On Tue, Jun 17, 2014 at 9:25 AM, J. Pablo Fern?ndez <pupeno at pupeno.com> wrote:> Hello, > I'm running the official GlusterFS 3.5 packages on an Ubuntu 12.04 box > that is acting as both, client and server, and everything seems to be > working fine, except mounting the GlusterFS volumes at boot time. This is > what I see in the log files: > > [2014-06-17 08:20:52.969258] I [glusterfsd.c:1959:main] > 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.5.0 > (/usr/sbin/glusterfs --volfile-server=127.0.0.1 > --volfile-id=/public_uploads /var/www/shared/public/uploads) > [2014-06-17 08:20:52.998985] I [socket.c:3561:socket_init] > 0-glusterfs: SSL support is NOT enabled > [2014-06-17 08:20:52.999048] I [socket.c:3576:socket_init] > 0-glusterfs: using system polling thread > [2014-06-17 08:20:53.000373] E [socket.c:2161:socket_connect_finish] > 0-glusterfs: connection to 127.0.0.1:24007 failed (Connection refused) > [2014-06-17 08:20:53.000427] E > [glusterfsd-mgmt.c:1601:mgmt_rpc_notify] 0-glusterfsd-mgmt: failed to > connect with remote-host: 127.0.0.1 (No data available) > [2014-06-17 08:20:53.000442] I > [glusterfsd-mgmt.c:1607:mgmt_rpc_notify] 0-glusterfsd-mgmt: Exhausted all > volfile servers > [2014-06-17 08:20:53.013793] W [glusterfsd.c:1095:cleanup_and_exit] > (-->/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_transport_notify+0x27) > [0x7f686e0160f7] > (-->/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x1a4) > [0x7f686e019cc4] (-->/usr/sbin/glusterfs(+0xcada) [0x7f686e6ddada]))) 0-: > received signum (1), shutting down > [2014-06-17 08:20:53.013830] I [fuse-bridge.c:5444:fini] 0-fuse: > Unmounting '/var/www/shared/public/uploads'. > > My fstab contains: > > proc /proc proc defaults > 0 0 > /dev/xvda / ext4 > noatime,errors=remount-ro 0 1 > /dev/xvdb none swap sw > 0 0 > /dev/xvdc /var/lib/glusterfs/brick01 ext4 defaults > 1 2 > 127.0.0.1:/private_uploads /var/www/shared/private/uploads glusterfs > defaults,_netdev 0 0 > > I know this used to be a bug in GlusterFS 3.2 for Ubuntu, but I understand > it was solved in the PPA packages for GlusterFS 3.4 as shown here: > https://bugs.launchpad.net/ubuntu/+source/glusterfs/+bug/876648 > > I also remember this working in an experiment I run with some virtual > machines (but since it just working, I didn't look too deep into it). I see > that the gluster-client packages provides an upstart job called > mounting-glusterfs.conf which contains: > > author "Louis Zuckerman <me at louiszuckerman.com>" > description "Block the mounting event for glusterfs filesystems until > the network interfaces are running" > > instance $MOUNTPOINT > > start on mounting TYPE=glusterfs > task > exec start wait-for-state WAIT_FOR=static-network-up > WAITER=mounting-glusterfs-$MOUNTPOINT > > But I'm not so sure how it should work. It doesn't seem to be working out > of the box. Even though mounting of glusterfs volumes happens after the > network starts, it happens before GlusterFS starts: > > * Starting RPC portmapper replacement > [ OK ] > * Stopping rpcsec_gss daemon > [ OK ] > * Starting Start this job to wait until rpcbind is started or fails > to s[ OK ] > * Starting configure network device > [ OK ] > * Stopping Start this job to wait until rpcbind is started or fails > to s[ OK ] > * Starting Bridge socket events into upstart > [ OK ] > * Starting NSM status monitor > [ OK ] > * Stopping cold plug devices > [ OK ] > * Stopping log initial device creation > [ OK ] > * Starting load fallback graphics devices > [ OK ] > * Starting configure network device security > [ OK ] > * Starting load fallback graphics devices > [fail] > * Starting configure virtual network devices > [ OK ] > * Starting Send an event to indicate plymouth is up > [ OK ] > * Stopping Send an event to indicate plymouth is up > [ OK ] > * Starting Mount network filesystems > [ OK ] > * Stopping configure virtual network devices > [ OK ] > * Stopping Mount network filesystems > [ OK ] > * Starting Mount network filesystems > [ OK ] > * Stopping Mount network filesystems > [ OK ] > * Starting configure network device > [ OK ] > * Starting set sysctls from /etc/sysctl.conf > [ OK ] > * Stopping set sysctls from /etc/sysctl.conf > [ OK ] > The disk drive for /var/www/shared/public/uploads is not ready yet or > not present. > Continue to wait, or Press S to skip mounting or M for manual recovery > * Starting Waiting for state > [fail] > * Starting Block the mounting event for glusterfs filesystems until > the [fail]k interfaces are running > mountall: Event failed > > > Mount failed. Please check the log file for more details. > * Starting GNU Screen Cleanup > [ OK ] > * Starting flush early job output to logs > [ OK ] > * Starting base > [ OK ] > * Starting save udev log and update rules > [ OK ] > * Starting OpenSSH server > [ OK ] > * Stopping Failsafe Boot Delay > [ OK ] > * Starting System V initialisation compatibility > [ OK ] > * Stopping save udev log and update rules > [ OK ] > * Stopping Mount filesystems on boot > [ OK ] > * Stopping GNU Screen Cleanup > [ OK ] > * Stopping flush early job output to logs > [ OK ] > * Starting system logging daemon > [ OK ] > * Stopping System V initialisation compatibility > [ OK ] > * Starting System V runlevel compatibility > [ OK ] > * Starting save kernel messages > [ OK ] > * Starting deferred execution scheduler > [ OK ] > * Starting CPU interrupts balancing daemon > [ OK ] > * Starting regular background program processing daemon > [ OK ] > * Starting automatic crash report generation > [ OK ] > * Starting GlusterFS Management Daemon > [ OK ] > > Any ideas what's going on and/or how to fix it? > > > -- > J. Pablo Fern?ndez <pupeno at pupeno.com> (http://pupeno.com) > > PS: I did post this in serverfault some time ago, but I didn't receive any > reply other than a custom script to mount things after GlusterFS started: > http://serverfault.com/questions/604860/glusterfs-is-failing-to-mount-on-boot >-- J. Pablo Fern?ndez <pupeno at pupeno.com> (http://pupeno.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140619/38eaea80/attachment.html>