Hi,
I'm rather new to this project, having stumbled across it earlier this
afternoon, so forgive me if I'm still trying to find my way around. I
was in the need of an alternative to NFS that would let me spread the
task of sharing my downloaded source code files across a couple of
boxes, and GlusterFS looked like a great candidate, having had no luck
with Coda or OpenAFS.
I also want to share some files on the web server here with various
network clients (some x86, some MIPS) to make maintenance of web pages
easier, and to allow backups to my file server (a MIPS-based Cobalt Qube2).
All the boxes are running Gentoo Linux... Gentoo Hardened on the x86
systems (mostly with vanilla kernels, SELinux disabled, but with stack
protection enabled in the compiler), and plain Gentoo on MIPS.
For the record, the following setup is in use:
Web Server:
Intel PIII 550MHz running Gentoo Linux with Kernel 2.6.23.1 (vanilla
sources). fuse 2.7.0, glusterfs-1.3.11 (client+server).
Laptop:
Intel PII 300MHz running Gentoo Linux with Kernel 2.6.25 (vanilla
sources). fuse 2.7.0, glusterfs-1.3.11 (client+server)
Cobalt Qube2:
QED RM5231 250MHz (MIPS4 Little Endian) running Gentoo Linux with
kernel 2.6.27-rc4 (latest linux-mips.org git sources). glusterfs-1.3.11
(server only for now, but will be installing fuse and the client soon).
All boxes are synced to the local NTP server which runs at Stratum 2.
Other machines to be added into the mix:
o Two Lemote Fulong computers (Loongson 2E 660MHz CPU; MIPS3 Little Endian)
o SGI O2 (RM527x 300MHz; MIPS4 Big Endian) -- should be good for
endianness checks
o A few more x86 boxes, and one or two SGI boxes when I get around to
it. ;-)
At this time, I have the server, glusterfsd, running nicely on the
Qube2, performing the job of sharing out a repository stored there. The
web server (x86) currently connects to this without issues as does my
x86-based laptop. The problem arose this afternoon when I tried to set
up glusterfsd on the web server to share out Apache's vhost roots (in
/home/httpd) so I could mount them elsewhere.
On the server, I have the following configuration file:> www ~ # cat /etc/glusterfs/glusterfs-server.vol
> volume brick
> type storage/posix
> option directory /home/httpd/
> end-volume
>
> volume server
> type protocol/server
> subvolumes brick
> option transport-type tcp/server # For TCP/IP transport
> option auth.ip.brick.allow *
> end-volume
Save a couple of slight differences, it's more or less a direct copy
from the 10min setup guide. For debugging purposes, I started the
server with the following command (as root) under strace:
# strace -o /tmp/glusterfsd-strace.txt glusterfsd -f \> /etc/glusterfs/glusterfs-server.vol -L DEBUG -l \
> /tmp/glusterfsd-log.txt -N
On my laptop, I have a similar config file:> stuartl at portege ~ $ cat /etc/glusterfs/glusterfs-client.vol
> volume client
> type protocol/client
> option transport-type tcp/client
> option remote-host 10.0.0.254
> option remote-subvolume brick
> end-volume
I attempt to mount the files using the following command:
# strace -o /tmp/glusterfs-strace.txt glusterfs -f \> /etc/glusterfs/glusterfs-client.vol -L DEBUG -l \
> /tmp/glusterfs-log.txt -N
On the client side, the following is generated in the
logfiles:> portege ~ # cat /tmp/glusterfs-log.txt
> 2008-08-24 18:20:33 D [glusterfs.c:167:get_spec_fp] glusterfs: loading spec
from /etc/glusterfs/glusterfs-client.vol
> 2008-08-24 18:20:33 D [spec.y:107:new_section] parser: New node for
'client'
> 2008-08-24 18:20:33 D [xlator.c:115:xlator_set_type] xlator: attempt to
load file /usr/lib/glusterfs/1.3.11/xlator/protocol/client.so
> 2008-08-24 18:20:33 D [spec.y:127:section_type] parser:
Type:client:protocol/client
> 2008-08-24 18:20:33 D [spec.y:141:section_option] parser:
Option:client:transport-type:tcp/client
> 2008-08-24 18:20:33 D [spec.y:141:section_option] parser:
Option:client:remote-host:10.0.0.254
> 2008-08-24 18:20:33 D [spec.y:141:section_option] parser:
Option:client:remote-subvolume:brick
> 2008-08-24 18:20:33 D [spec.y:209:section_end] parser: end:client
> 2008-08-24 18:20:33 D [glusterfs.c:128:fuse_graph] glusterfs: setting
option mount-point to /mnt/net/
> 2008-08-24 18:20:33 D [xlator.c:115:xlator_set_type] xlator: attempt to
load file /usr/lib/glusterfs/1.3.11/xlator/mount/fuse.so
> 2008-08-24 18:20:34 D [client-protocol.c:5013:init] client: defaulting
transport-timeout to 42
> 2008-08-24 18:20:34 D [transport.c:80:transport_load] transport: attempt to
load file /usr/lib/glusterfs/1.3.11/transport/tcp/client.so
> 2008-08-24 18:20:34 D [client-protocol.c:5040:init] client: defaulting
limits.transaction-size to 268435456
> 2008-08-24 18:20:34 D [client-protocol.c:5340:notify] client: got
GF_EVENT_PARENT_UP, attempting connect on transport
> 2008-08-24 18:20:34 D [client-protocol.c:4757:client_protocol_reconnect]
client: attempting reconnect
> 2008-08-24 18:20:34 D [tcp-client.c:77:tcp_connect] client: socket fd = 6
> 2008-08-24 18:20:34 D [tcp-client.c:107:tcp_connect] client: finalized on
port `1023'
> 2008-08-24 18:20:34 D [tcp-client.c:128:tcp_connect] client: defaulting
remote-port to 6996
> 2008-08-24 18:20:34 D [common-utils.c:179:gf_resolve_ip] resolver: DNS
cache not present, freshly probing hostname: 10.0.0.254
> 2008-08-24 18:20:34 D [common-utils.c:204:gf_resolve_ip] resolver:
returning IP:10.0.0.254[0] for hostname: 10.0.0.254
> 2008-08-24 18:20:34 D [common-utils.c:212:gf_resolve_ip] resolver: flushing
DNS cache
> 2008-08-24 18:20:34 D [tcp-client.c:161:tcp_connect] client: connect on 6
in progress (non-blocking)
> 2008-08-24 18:20:34 D [tcp-client.c:205:tcp_connect] client: connection on
6 success
> 2008-08-24 18:20:34 D [client-protocol.c:5362:notify] client: got
GF_EVENT_CHILD_UP
> 2008-08-24 18:20:34 W [client-protocol.c:4784:client_protocol_cleanup]
client: cleaning up state in transport object 0x8b32da8
> 2008-08-24 18:20:34 D [tcp.c:87:tcp_disconnect] client: connection
disconnected
> 2008-08-24 18:20:35 D [client-protocol.c:4757:client_protocol_reconnect]
client: attempting reconnect
> 2008-08-24 18:20:35 D [tcp-client.c:77:tcp_connect] client: socket fd = 6
> 2008-08-24 18:20:35 D [tcp-client.c:107:tcp_connect] client: finalized on
port `1023'
> 2008-08-24 18:20:35 D [tcp-client.c:128:tcp_connect] client: defaulting
remote-port to 6996
> 2008-08-24 18:20:35 D [common-utils.c:179:gf_resolve_ip] resolver: DNS
cache not present, freshly probing hostname: 10.0.0.254
> 2008-08-24 18:20:35 D [common-utils.c:204:gf_resolve_ip] resolver:
returning IP:10.0.0.254[0] for hostname: 10.0.0.254
> 2008-08-24 18:20:35 D [common-utils.c:212:gf_resolve_ip] resolver: flushing
DNS cache
> 2008-08-24 18:20:35 D [tcp-client.c:161:tcp_connect] client: connect on 6
in progress (non-blocking)
> 2008-08-24 18:20:35 D [tcp-client.c:198:tcp_connect] client: connection on
6 still in progress - try later
> 2008-08-24 18:20:37 D [client-protocol.c:4757:client_protocol_reconnect]
client: attempting reconnect
> 2008-08-24 18:20:37 D [tcp-client.c:198:tcp_connect] client: connection on
6 still in progress - try later
> 2008-08-24 18:20:40 D [client-protocol.c:4757:client_protocol_reconnect]
client: attempting reconnect
> 2008-08-24 18:20:40 D [tcp-client.c:205:tcp_connect] client: connection on
6 success
> 2008-08-24 18:20:40 D [client-protocol.c:5362:notify] client: got
GF_EVENT_CHILD_UP
> 2008-08-24 18:20:40 W [client-protocol.c:4784:client_protocol_cleanup]
client: cleaning up state in transport object 0x8b32da8
> 2008-08-24 18:20:40 D [tcp.c:87:tcp_disconnect] client: connection
disconnected
> 2008-08-24 18:20:45 D [client-protocol.c:4757:client_protocol_reconnect]
client: attempting reconnect
> 2008-08-24 18:20:45 D [tcp-client.c:77:tcp_connect] client: socket fd = 6
> 2008-08-24 18:20:45 D [tcp-client.c:107:tcp_connect] client: finalized on
port `1023'
> 2008-08-24 18:20:45 D [tcp-client.c:128:tcp_connect] client: defaulting
remote-port to 6996
> 2008-08-24 18:20:45 D [common-utils.c:179:gf_resolve_ip] resolver: DNS
cache not present, freshly probing hostname: 10.0.0.254
> 2008-08-24 18:20:45 D [common-utils.c:204:gf_resolve_ip] resolver:
returning IP:10.0.0.254[0] for hostname: 10.0.0.254
> 2008-08-24 18:20:45 D [common-utils.c:212:gf_resolve_ip] resolver: flushing
DNS cache
> 2008-08-24 18:20:45 D [tcp-client.c:161:tcp_connect] client: connect on 6
in progress (non-blocking)
> 2008-08-24 18:20:46 D [tcp-client.c:198:tcp_connect] client: connection on
6 still in progress - try later
> 2008-08-24 18:20:54 D [client-protocol.c:4757:client_protocol_reconnect]
client: attempting reconnect
> 2008-08-24 18:20:54 D [tcp-client.c:205:tcp_connect] client: connection on
6 success
> 2008-08-24 18:20:54 D [client-protocol.c:5362:notify] client: got
GF_EVENT_CHILD_UP
> 2008-08-24 18:20:54 W [client-protocol.c:4784:client_protocol_cleanup]
client: cleaning up state in transport object 0x8b32da8
> 2008-08-24 18:20:54 D [tcp.c:87:tcp_disconnect] client: connection
disconnected
> 2008-08-24 18:21:07 D [client-protocol.c:4757:client_protocol_reconnect]
client: attempting reconnect
> 2008-08-24 18:21:07 D [tcp-client.c:77:tcp_connect] client: socket fd = 6
> 2008-08-24 18:21:07 D [tcp-client.c:107:tcp_connect] client: finalized on
port `1023'
> 2008-08-24 18:21:07 D [tcp-client.c:128:tcp_connect] client: defaulting
remote-port to 6996
> 2008-08-24 18:21:07 D [common-utils.c:179:gf_resolve_ip] resolver: DNS
cache not present, freshly probing hostname: 10.0.0.254
> 2008-08-24 18:21:07 D [common-utils.c:204:gf_resolve_ip] resolver:
returning IP:10.0.0.254[0] for hostname: 10.0.0.254
> 2008-08-24 18:21:07 D [common-utils.c:212:gf_resolve_ip] resolver: flushing
DNS cache
> 2008-08-24 18:21:07 D [tcp-client.c:161:tcp_connect] client: connect on 6
in progress (non-blocking)
> 2008-08-24 18:21:07 D [tcp-client.c:205:tcp_connect] client: connection on
6 success
> 2008-08-24 18:21:07 D [client-protocol.c:5362:notify] client: got
GF_EVENT_CHILD_UP
> 2008-08-24 18:21:07 W [client-protocol.c:4784:client_protocol_cleanup]
client: cleaning up state in transport object 0x8b32da8
> 2008-08-24 18:21:07 D [tcp.c:87:tcp_disconnect] client: connection
disconnected
> 2008-08-24 18:21:28 D [client-protocol.c:4757:client_protocol_reconnect]
client: attempting reconnect
> 2008-08-24 18:21:28 D [tcp-client.c:77:tcp_connect] client: socket fd = 6
> 2008-08-24 18:21:28 D [tcp-client.c:107:tcp_connect] client: finalized on
port `1023'
> 2008-08-24 18:21:28 D [tcp-client.c:128:tcp_connect] client: defaulting
remote-port to 6996
> 2008-08-24 18:21:28 D [common-utils.c:179:gf_resolve_ip] resolver: DNS
cache not present, freshly probing hostname: 10.0.0.254
> 2008-08-24 18:21:28 D [common-utils.c:204:gf_resolve_ip] resolver:
returning IP:10.0.0.254[0] for hostname: 10.0.0.254
> 2008-08-24 18:21:28 D [common-utils.c:212:gf_resolve_ip] resolver: flushing
DNS cache
> 2008-08-24 18:21:28 D [tcp-client.c:161:tcp_connect] client: connect on 6
in progress (non-blocking)
> 2008-08-24 18:21:28 D [tcp-client.c:205:tcp_connect] client: connection on
6 success
> 2008-08-24 18:21:28 D [client-protocol.c:5362:notify] client: got
GF_EVENT_CHILD_UP
> 2008-08-24 18:21:28 W [client-protocol.c:4784:client_protocol_cleanup]
client: cleaning up state in transport object 0x8b32da8
> 2008-08-24 18:21:28 D [tcp.c:87:tcp_disconnect] client: connection
disconnected
> 2008-08-24 18:21:32 W [glusterfs.c:416:glusterfs_cleanup_and_exit]
glusterfs: shutting down server
> portege ~ # cat /tmp/glusterfs-
> glusterfs-1.3.11/ glusterfs-CURRENT.tar.gz glusterfs-log.txt
glusterfs-strace.txt
> portege ~ # cat /tmp/glusterfs-strace.txt
> execve("/usr/sbin/glusterfs", ["glusterfs",
"-f", "/etc/glusterfs/glusterfs-client."...,
"-L"..., "DEBUG"..., "-l"...,
"/tmp/glusterfs-log.txt"..., "-N"...,
"/mnt/net/"...], [/* 67 vars */]) = 0
> brk(0) = 0x8b2d000
> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file
or directory)
> open("/etc/ld.so.cache", O_RDONLY) = 3
> fstat64(3, {st_mode=S_IFREG|0644, st_size=168554, ...}) = 0
> mmap2(NULL, 168554, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f2b000
> close(3) = 0
> open("/usr/lib/libglusterfs.so.0", O_RDONLY) = 3
> read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0pW\0\0004\0\0\0"...,
512) = 512
> fstat64(3, {st_mode=S_IFREG|0755, st_size=387428, ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7f2a000
> mmap2(NULL, 115488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb7f0d000
> mmap2(0xb7f28000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b) = 0xb7f28000
> close(3) = 0
> open("/lib/libdl.so.2", O_RDONLY) = 3
> read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\20\f\0\0004\0\0\0"...,
512) = 512
> fstat64(3, {st_mode=S_IFREG|0755, st_size=10652, ...}) = 0
> mmap2(NULL, 12344, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb7f09000
> mmap2(0xb7f0b000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb7f0b000
> close(3) = 0
> open("/lib/tls/libpthread.so.0", O_RDONLY) = 3
> read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340K\0\0004\0\0\0"...,
512) = 512
> fstat64(3, {st_mode=S_IFREG|0755, st_size=129332, ...}) = 0
> mmap2(NULL, 90336, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb7ef2000
> mmap2(0xb7f05000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x12) = 0xb7f05000
> mmap2(0xb7f07000, 4320, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7f07000
> close(3) = 0
> open("/lib/tls/libc.so.6", O_RDONLY) = 3
> read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220Y\1\0004\0\0\0"...,
512) = 512
> fstat64(3, {st_mode=S_IFREG|0755, st_size=1299536, ...}) = 0
> mmap2(NULL, 1226876, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0)
= 0xb7dc6000
> mmap2(0xb7eec000, 12288, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x125) = 0xb7eec000
> mmap2(0xb7eef000, 10364, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7eef000
> close(3) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7dc5000
> set_thread_area({entry_number:-1 -> 6, base_addr:0xb7dc5ac0,
limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1,
seg_not_present:0, useable:1}) = 0
> open("/dev/urandom", O_RDONLY) = 3
> read(3, "\31\223Uu", 4) = 4
> close(3) = 0
> mprotect(0xb7eec000, 4096, PROT_READ) = 0
> mprotect(0xb7f05000, 4096, PROT_READ) = 0
> mprotect(0xb7f0b000, 4096, PROT_READ) = 0
> mprotect(0xb7f28000, 4096, PROT_READ) = 0
> mprotect(0x804c000, 4096, PROT_READ) = 0
> mprotect(0xb7f70000, 4096, PROT_READ) = 0
> munmap(0xb7f2b000, 168554) = 0
> set_tid_address(0xb7dc5b08) = 11503
> set_robust_list(0xb7dc5b10, 0xc) = 0
> rt_sigaction(SIGRTMIN, {0xb7ef66d0, [], SA_SIGINFO}, NULL, 8) = 0
> rt_sigaction(SIGRT_1, {0xb7ef6750, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0
> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
> getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
> uname({sys="Linux", node="portege", ...}) = 0
> brk(0) = 0x8b2d000
> brk(0x8b4e000) = 0x8b4e000
> setrlimit(RLIMIT_CORE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) =
0
> open("/tmp/glusterfs-log.txt",
O_WRONLY|O_CREAT|O_APPEND|O_LARGEFILE, 0666) = 3
> fstat64(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7f54000
> fstat64(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
> _llseek(3, 0, [0], SEEK_SET) = 0
> open("/etc/glusterfs/glusterfs-client.vol", O_RDONLY|O_LARGEFILE)
= 4
> time(NULL) = 1219566033
> open("/etc/localtime", O_RDONLY) = 5
> fstat64(5, {st_mode=S_IFREG|0644, st_size=413, ...}) = 0
> fstat64(5, {st_mode=S_IFREG|0644, st_size=413, ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7f53000
> read(5,
"TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0\0"...,
4096) = 413
> close(5) = 0
> munmap(0xb7f53000, 4096) = 0
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:33 D [glusterfs"..., 117) = 117
> rt_sigaction(SIGUSR1, {0xb7e2d6d0, [USR1], SA_RESTART}, {SIG_DFL}, 8) = 0
> rt_sigaction(SIGUSR2, {0xb7f1bfa0, [USR2], SA_RESTART}, {SIG_DFL}, 8) = 0
> rt_sigaction(SIGSEGV, {0xb7f1c8e0, [SEGV], SA_RESTART}, {SIG_DFL}, 8) = 0
> rt_sigaction(SIGABRT, {0xb7f1c8e0, [ABRT], SA_RESTART}, {SIG_DFL}, 8) = 0
> rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
> rt_sigaction(SIGHUP, {0xb7f18350, [HUP], SA_RESTART}, {SIG_IGN}, 8) = 0
> rt_sigaction(SIGTERM, {0x8049b30, [TERM], SA_RESTART}, {SIG_IGN}, 8) = 0
> _llseek(-1, 0, 0xbfd6f090, SEEK_SET) = -1 EBADF (Bad file descriptor)
> ftruncate64(-1, 0) = -1 EBADF (Bad file descriptor)
> write(-1, "11503\n", 6) = -1 EBADF (Bad file
descriptor)
> close(-1) = -1 EBADF (Bad file descriptor)
> mmap2(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb75c4000
> mprotect(0xb75c4000, 4096, PROT_NONE) = 0
> clone(child_stack=0xb7dc44c4,
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
parent_tidptr=0xb7dc4bd8, {entry_number:6, base_addr:0xb7dc4b90, limit:1048575,
seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0,
useable:1}, child_tidptr=0xb7dc4bd8) = 11515
> ioctl(4, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfd6e9d8) = -1 ENOTTY
(Inappropriate ioctl for device)
> fstat64(4, {st_mode=S_IFREG|0644, st_size=143, ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7f53000
> read(4, "volume client\n\ttype protocol/cli"..., 8192) = 143
> read(4, "", 4096) = 0
> time(NULL) = 1219566033
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:33 D [spec.y:10"..., 77) = 77
> time(NULL) = 1219566033
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:33 D [xlator.c:"..., 134) = 134
> futex(0xb7f0c02c, FUTEX_WAKE, 2147483647) = 0
> open("/usr/lib/glusterfs/1.3.11/xlator/protocol/client.so",
O_RDONLY) = 5
> read(5, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 at
+\0\0004\0\0\0"..., 512) = 512
> fstat64(5, {st_mode=S_IFREG|0755, st_size=141051, ...}) = 0
> mmap2(NULL, 74660, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) =
0xb7f40000
> mmap2(0xb7f51000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x10) = 0xb7f51000
> close(5) = 0
> mprotect(0xb7f51000, 4096, PROT_READ) = 0
> time(NULL) = 1219566033
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:33 D [spec.y:12"..., 84) = 84
> time(NULL) = 1219566033
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:33 D [spec.y:14"..., 98) = 98
> time(NULL) = 1219566033
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:33 D [spec.y:14"..., 95) = 95
> time(NULL) = 1219566033
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:33 D [spec.y:14"..., 95) = 95
> time(NULL) = 1219566033
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:33 D [spec.y:20"..., 66) = 66
> read(4, "", 8192) = 0
> ioctl(4, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfd6e9d8) = -1 ENOTTY
(Inappropriate ioctl for device)
> close(4) = 0
> munmap(0xb7f53000, 4096) = 0
> time(NULL) = 1219566033
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:33 D [glusterfs"..., 102) = 102
> time(NULL) = 1219566033
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:33 D [xlator.c:"..., 129) = 129
> open("/usr/lib/glusterfs/1.3.11/xlator/mount/fuse.so", O_RDONLY)
= 4
> read(4,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\30\0\0004\0\0\0"...,
512) = 512
> fstat64(4, {st_mode=S_IFREG|0755, st_size=101283, ...}) = 0
> mmap2(NULL, 50028, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) =
0xb7f33000
> mmap2(0xb7f3e000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0xa) = 0xb7f3e000
> close(4) = 0
> open("/etc/ld.so.cache", O_RDONLY) = 4
> fstat64(4, {st_mode=S_IFREG|0644, st_size=168554, ...}) = 0
> mmap2(NULL, 168554, PROT_READ, MAP_PRIVATE, 4, 0) = 0xb759a000
> close(4) = 0
> open("/usr/lib/libfuse.so.2", O_RDONLY) = 4
> read(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0
N\0\0004\0\0\0"..., 512) = 512
> fstat64(4, {st_mode=S_IFREG|0755, st_size=85344, ...}) = 0
> mmap2(NULL, 88248, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) =
0xb7584000
> mmap2(0xb7598000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x13) = 0xb7598000
> close(4) = 0
> open("/lib/tls/librt.so.1", O_RDONLY) = 4
> read(4,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0P\35\0\0004\0\0\0"...,
512) = 512
> fstat64(4, {st_mode=S_IFREG|0755, st_size=33492, ...}) = 0
> mmap2(NULL, 33100, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) =
0xb757b000
> mmap2(0xb7582000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x6) = 0xb7582000
> close(4) = 0
> mprotect(0xb7582000, 4096, PROT_READ) = 0
> mprotect(0xb7598000, 4096, PROT_READ) = 0
> mprotect(0xb7f3e000, 4096, PROT_READ) = 0
> munmap(0xb759a000, 168554) = 0
> lstat64("/mnt/net/", {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
> open("/dev/fuse", O_RDWR|O_LARGEFILE) = 4
> getgid32() = 0
> getuid32() = 0
> mount("glusterfs", "/mnt/net/", "fuse"...,
MS_NOSUID|MS_NODEV, "allow_other,default_permissions,"...) = 0
> geteuid32() = 0
> lstat64("/mnt", {st_mode=S_IFDIR|0755, st_size=102, ...}) = 0
> clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0xb7dc5b08) = 11565
> waitpid(11565, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0) = 11565
> --- SIGCHLD (Child exited) @ 0 (0) ---
> getuid32() = 0
> rt_sigaction(SIGHUP, NULL, {0xb7f18350, [HUP], SA_RESTART}, 8) = 0
> rt_sigaction(SIGINT, NULL, {SIG_IGN}, 8) = 0
> rt_sigaction(SIGTERM, NULL, {0x8049b30, [TERM], SA_RESTART}, 8) = 0
> rt_sigaction(SIGPIPE, NULL, {SIG_IGN}, 8) = 0
> time(NULL) = 1219566034
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:34 D [client-pr"..., 95) = 95
> time(NULL) = 1219566034
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:34 D [transport"..., 136) = 136
> open("/usr/lib/glusterfs/1.3.11/transport/tcp/client.so",
O_RDONLY) = 5
> read(5,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\f\0\0004\0\0\0"...,
512) = 512
> fstat64(5, {st_mode=S_IFREG|0755, st_size=46004, ...}) = 0
> mmap2(NULL, 16580, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) =
0xb75bf000
> mmap2(0xb75c2000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2) = 0xb75c2000
> close(5) = 0
> mprotect(0xb75c2000, 4096, PROT_READ) = 0
> time(NULL) = 1219566034
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:34 D [client-pr"..., 108) = 108
> setrlimit(RLIMIT_NOFILE, {rlim_cur=1024*1024, rlim_max=1024*1024}) = 0
> time(NULL) = 1219566034
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:34 D [client-pr"..., 118) = 118
> gettimeofday({1219566034, 561531}, NULL) = 0
> epoll_create(1024) = 5
> futex(0x8b342cc, FUTEX_WAIT, 1, NULL) = 0
> futex(0x8b342b0, FUTEX_WAKE, 1) = 0
> epoll_wait(5, {{EPOLLIN|EPOLLERR|EPOLLHUP, {u32=145960360,
u64=145960360}}}, 1, -1) = 1
> time(NULL) = 1219566034
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:34 W [client-pr"..., 127) = 127
> time(NULL) = 1219566034
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:34 D [tcp.c:87:"..., 80) = 80
> epoll_ctl(5, EPOLL_CTL_DEL, 6,
{EPOLLOUT|EPOLLMSG|EPOLLERR|EPOLLET|0x37f0e820, {u32=3086127092,
u64=626792034022363124}}) = 0
> close(6) = 0
> futex(0x8b342cc, FUTEX_WAIT, 3, NULL) = 0
> futex(0x8b342b0, FUTEX_WAKE, 1) = 0
> epoll_wait(5, {{EPOLLIN|EPOLLERR|EPOLLHUP, {u32=145960360,
u64=145960360}}}, 1, -1) = 1
> time(NULL) = 1219566040
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:40 W [client-pr"..., 127) = 127
> time(NULL) = 1219566040
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:40 D [tcp.c:87:"..., 80) = 80
> epoll_ctl(5, EPOLL_CTL_DEL, 6,
{EPOLLRDNORM|EPOLLRDBAND|EPOLLWRBAND|EPOLLMSG|0x19000, {u32=3218534156,
u64=13250337929468243724}}) = 0
> close(6) = 0
> futex(0x8b342cc, FUTEX_WAIT, 5, NULL) = 0
> futex(0x8b342b0, FUTEX_WAKE, 1) = 0
> epoll_wait(5, {{EPOLLIN|EPOLLERR|EPOLLHUP, {u32=145960360,
u64=145960360}}}, 1, -1) = 1
> time(NULL) = 1219566054
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:54 W [client-pr"..., 127) = 127
> time(NULL) = 1219566054
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:20:54 D [tcp.c:87:"..., 80) = 80
> epoll_ctl(5, EPOLL_CTL_DEL, 6,
{EPOLLRDNORM|EPOLLRDBAND|EPOLLWRBAND|EPOLLMSG|0x19000, {u32=3218534156,
u64=13250337929468243724}}) = 0
> close(6) = 0
> futex(0x8b342cc, FUTEX_WAIT, 7, NULL) = 0
> futex(0x8b342b0, FUTEX_WAIT, 2, NULL) = 0
> futex(0x8b342b0, FUTEX_WAKE, 1) = 0
> epoll_wait(5, {{EPOLLIN|EPOLLERR|EPOLLHUP, {u32=145960360,
u64=145960360}}}, 1, -1) = 1
> time(NULL) = 1219566067
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:21:07 W [client-pr"..., 127) = 127
> time(NULL) = 1219566067
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:21:07 D [tcp.c:87:"..., 80) = 80
> epoll_ctl(5, EPOLL_CTL_DEL, 6,
{EPOLLRDNORM|EPOLLRDBAND|EPOLLWRBAND|EPOLLMSG|0x19000, {u32=3218534156,
u64=13250337929468243724}}) = 0
> close(6) = 0
> futex(0x8b342cc, FUTEX_WAIT, 9, NULL) = ? ERESTARTSYS (To be restarted)
> --- SIGINT (Interrupt) @ 0 (0) ---
> futex(0x8b342cc, FUTEX_WAIT, 9, NULL) = ? ERESTARTSYS (To be restarted)
> --- SIGINT (Interrupt) @ 0 (0) ---
> futex(0x8b342cc, FUTEX_WAIT, 9, NULL) = 0
> futex(0x8b342b0, FUTEX_WAKE, 1) = 0
> epoll_wait(5, {{EPOLLIN|EPOLLERR|EPOLLHUP, {u32=145960360,
u64=145960360}}}, 1, -1) = 1
> time(NULL) = 1219566088
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:21:28 W [client-pr"..., 127) = 127
> time(NULL) = 1219566088
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:21:28 D [tcp.c:87:"..., 80) = 80
> epoll_ctl(5, EPOLL_CTL_DEL, 6,
{EPOLLRDNORM|EPOLLRDBAND|EPOLLWRBAND|EPOLLMSG|0x19000, {u32=3218534156,
u64=13250337929468243724}}) = 0
> close(6) = 0
> futex(0x8b342cc, FUTEX_WAIT, 11, NULL) = ? ERESTARTSYS (To be restarted)
> --- SIGTERM (Terminated) @ 0 (0) ---
> time(NULL) = 1219566092
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:21:32 W [glusterfs"..., 99) = 99
> exit_group(0) = ?
> portege ~ #
On the server, the following messages are logged (and yes, at the end I
do manually try to shut it down with SIGINT before resorting to
SIGTERM).> www ~ # cat /tmp/glusterfsd-log.txt
> 2008-08-24 18:10:56 D [glusterfs.c:167:get_spec_fp] glusterfs: loading spec
from /etc/glusterfs/glusterfs-server.vol
> 2008-08-24 18:10:56 D [spec.y:107:new_section] parser: New node for
'brick'
> 2008-08-24 18:10:56 D [xlator.c:115:xlator_set_type] xlator: attempt to
load file /usr/lib/glusterfs/1.3.11/xlator/storage/posix.so
> 2008-08-24 18:10:56 D [spec.y:127:section_type] parser:
Type:brick:storage/posix
> 2008-08-24 18:10:56 D [spec.y:141:section_option] parser:
Option:brick:directory:/home/httpd/
> 2008-08-24 18:10:56 D [spec.y:209:section_end] parser: end:brick
> 2008-08-24 18:10:56 D [spec.y:107:new_section] parser: New node for
'server'
> 2008-08-24 18:10:56 D [xlator.c:115:xlator_set_type] xlator: attempt to
load file /usr/lib/glusterfs/1.3.11/xlator/protocol/server.so
> 2008-08-24 18:10:56 D [spec.y:127:section_type] parser:
Type:server:protocol/server
> 2008-08-24 18:10:56 D [spec.y:196:section_sub] parser:
child:server->brick
> 2008-08-24 18:10:56 D [spec.y:141:section_option] parser:
Option:server:transport-type:tcp/server
> 2008-08-24 18:10:56 D [spec.y:141:section_option] parser:
Option:server:auth.ip.brick.allow:*
> 2008-08-24 18:10:56 D [spec.y:209:section_end] parser: end:server
> 2008-08-24 18:10:56 D [server-protocol.c:6309:init] server: protocol/server
xlator loaded
> 2008-08-24 18:10:56 D [transport.c:80:transport_load] transport: attempt to
load file /usr/lib/glusterfs/1.3.11/transport/tcp/server.so
> 2008-08-24 18:10:56 D [server-protocol.c:6350:init] server: defaulting
limits.transaction-size to 4194304
> 2008-08-24 18:11:45 D [tcp-server.c:145:tcp_server_notify] server:
Registering socket (6) for new transport object of 10.0.0.87
> 2008-08-24 18:11:45 E [protocol.c:342:gf_block_unserialize_transport]
server: frame size (168) > max (-2147458000)
> 2008-08-24 18:11:45 D [tcp.c:87:tcp_disconnect] server: connection
disconnected
> 2008-08-24 18:11:45 D [server-protocol.c:6269:server_protocol_cleanup]
server: cleaned up transport state for client 10.0.0.87:1022
> 2008-08-24 18:11:45 D [tcp-server.c:257:gf_transport_fini] server:
destroying transport object for 10.0.0.87:1022 (fd=6)
> 2008-08-24 18:11:49 D [tcp-server.c:145:tcp_server_notify] server:
Registering socket (6) for new transport object of 10.0.0.87
> 2008-08-24 18:11:51 E [protocol.c:342:gf_block_unserialize_transport]
server: frame size (168) > max (-2147458000)
> 2008-08-24 18:11:51 D [tcp.c:87:tcp_disconnect] server: connection
disconnected
> 2008-08-24 18:11:51 D [server-protocol.c:6269:server_protocol_cleanup]
server: cleaned up transport state for client 10.0.0.87:1022
> 2008-08-24 18:11:51 D [tcp-server.c:257:gf_transport_fini] server:
destroying transport object for 10.0.0.87:1022 (fd=6)
> 2008-08-24 18:11:59 D [tcp-server.c:145:tcp_server_notify] server:
Registering socket (6) for new transport object of 10.0.0.87
> 2008-08-24 18:12:05 E [protocol.c:342:gf_block_unserialize_transport]
server: frame size (168) > max (-2147458000)
> 2008-08-24 18:12:05 D [tcp.c:87:tcp_disconnect] server: connection
disconnected
> 2008-08-24 18:12:05 D [server-protocol.c:6269:server_protocol_cleanup]
server: cleaned up transport state for client 10.0.0.87:1022
> 2008-08-24 18:12:05 D [tcp-server.c:257:gf_transport_fini] server:
destroying transport object for 10.0.0.87:1022 (fd=6)
> 2008-08-24 18:13:13 W [glusterfs.c:416:glusterfs_cleanup_and_exit]
glusterfs: shutting down server
> www ~ # cat /tmp/glusterfsd-strace.txt
> execve("/usr/sbin/glusterfsd", ["glusterfsd",
"-f", "/etc/glusterfs/glusterfs-server."...,
"-L"..., "DEBUG"..., "-l"...,
"/tmp/glusterfsd-log.txt"..., "-N"...], [/* 45 vars */]) = 0
> brk(0) = 0x80006000
> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file
or directory)
> open("/etc/ld.so.cache", O_RDONLY) = 3
> fstat64(3, {st_mode=S_IFREG|0644, st_size=62235, ...}) = 0
> mmap2(NULL, 62235, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f08000
> close(3) = 0
> open("/usr/lib/libglusterfs.so.0", O_RDONLY) = 3
> read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320W\0\0004\0\0\0"...,
512) = 512
> fstat64(3, {st_mode=S_IFREG|0755, st_size=416299, ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7f07000
> mmap2(NULL, 143628, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb7ee3000
> mmap2(0xb7f05000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7f05000
> close(3) = 0
> open("/lib/libdl.so.2", O_RDONLY) = 3
> read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\20\f\0\0004\0\0\0"...,
512) = 512
> fstat64(3, {st_mode=S_IFREG|0755, st_size=10652, ...}) = 0
> mmap2(NULL, 12344, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb7edf000
> mmap2(0xb7ee1000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb7ee1000
> close(3) = 0
> open("/lib/libpthread.so.0", O_RDONLY) = 3
> read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340K\0\0004\0\0\0"...,
512) = 512
> fstat64(3, {st_mode=S_IFREG|0755, st_size=129424, ...}) = 0
> mmap2(NULL, 90336, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb7ec8000
> mmap2(0xb7edb000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x12) = 0xb7edb000
> mmap2(0xb7edd000, 4320, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7edd000
> close(3) = 0
> open("/lib/libc.so.6", O_RDONLY) = 3
> read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220Y\1\0004\0\0\0"...,
512) = 512
> fstat64(3, {st_mode=S_IFREG|0755, st_size=1299536, ...}) = 0
> mmap2(NULL, 1226876, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0)
= 0xb7d9c000
> mmap2(0xb7ec2000, 12288, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x125) = 0xb7ec2000
> mmap2(0xb7ec5000, 10364, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7ec5000
> close(3) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7d9b000
> set_thread_area({entry_number:-1 -> 6, base_addr:0xb7d9bac0,
limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1,
seg_not_present:0, useable:1}) = 0
> open("/dev/urandom", O_RDONLY) = 3
> read(3, ",\367\25\214", 4) = 4
> close(3) = 0
> mprotect(0xb7ec2000, 4096, PROT_READ) = 0
> mprotect(0xb7edb000, 4096, PROT_READ) = 0
> mprotect(0xb7ee1000, 4096, PROT_READ) = 0
> mprotect(0xb7f05000, 4096, PROT_READ) = 0
> mprotect(0x80004000, 4096, PROT_READ) = 0
> mprotect(0xb7f33000, 4096, PROT_READ) = 0
> munmap(0xb7f08000, 62235) = 0
> set_tid_address(0xb7d9bb08) = 9299
> set_robust_list(0xb7d9bb10, 0xc) = 0
> rt_sigaction(SIGRTMIN, {0xb7ecc6d0, [], SA_SIGINFO}, NULL, 8) = 0
> rt_sigaction(SIGRT_1, {0xb7ecc750, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0
> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
> getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
> uname({sys="Linux", node="www", ...}) = 0
> brk(0) = 0x80006000
> brk(0x80027000) = 0x80027000
> setrlimit(RLIMIT_CORE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) =
0
> open("/tmp/glusterfsd-log.txt",
O_WRONLY|O_CREAT|O_APPEND|O_LARGEFILE, 0666) = 3
> fstat64(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7f17000
> fstat64(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
> _llseek(3, 0, [0], SEEK_SET) = 0
> open("/etc/glusterfs/glusterfs-server.vol", O_RDONLY|O_LARGEFILE)
= 4
> time(NULL) = 1219565456
> open("/etc/localtime", O_RDONLY) = 5
> fstat64(5, {st_mode=S_IFREG|0644, st_size=413, ...}) = 0
> fstat64(5, {st_mode=S_IFREG|0644, st_size=413, ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7f16000
> read(5,
"TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0\0"...,
4096) = 413
> close(5) = 0
> munmap(0xb7f16000, 4096) = 0
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [glusterfs"..., 117) = 117
> rt_sigaction(SIGUSR1, {0xb7e03560, [USR1], SA_RESTART}, {SIG_DFL}, 8) = 0
> rt_sigaction(SIGUSR2, {0xb7ef4d20, [USR2], SA_RESTART}, {SIG_DFL}, 8) = 0
> rt_sigaction(SIGSEGV, {0xb7ef5680, [SEGV], SA_RESTART}, {SIG_DFL}, 8) = 0
> rt_sigaction(SIGABRT, {0xb7ef5680, [ABRT], SA_RESTART}, {SIG_DFL}, 8) = 0
> rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
> rt_sigaction(SIGHUP, {0xb7eef6d0, [HUP], SA_RESTART}, {SIG_IGN}, 8) = 0
> rt_sigaction(SIGTERM, {0x80001e60, [TERM], SA_RESTART}, {SIG_IGN}, 8) = 0
> _llseek(-1, 0, 0xbfd2a560, SEEK_SET) = -1 EBADF (Bad file descriptor)
> ftruncate64(-1, 0) = -1 EBADF (Bad file descriptor)
> write(-1, "9299\n", 5) = -1 EBADF (Bad file
descriptor)
> close(-1) = -1 EBADF (Bad file descriptor)
> mmap2(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb759a000
> mprotect(0xb759a000, 4096, PROT_NONE) = 0
> clone(child_stack=0xb7d9a4c4,
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
parent_tidptr=0xb7d9abd8, {entry_number:6, base_addr:0xb7d9ab90, limit:1048575,
seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0,
useable:1}, child_tidptr=0xb7d9abd8) = 9300
> ioctl(4, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfd29de8) = -1 ENOTTY
(Inappropriate ioctl for device)
> fstat64(4, {st_mode=S_IFREG|0644, st_size=228, ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7f16000
> read(4, "volume brick\n\ttype storage/posix"..., 8192) = 228
> read(4, "", 4096) = 0
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [spec.y:10"..., 76) = 76
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [xlator.c:"..., 132) = 132
> futex(0xb7ee202c, FUTEX_WAKE, 2147483647) = 0
> open("/usr/lib/glusterfs/1.3.11/xlator/storage/posix.so",
O_RDONLY) = 5
> read(5,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\37\0\0004\0\0\0"...,
512) = 512
> fstat64(5, {st_mode=S_IFREG|0755, st_size=90384, ...}) = 0
> mmap2(NULL, 49552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) =
0xb7f09000
> mmap2(0xb7f14000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0xa) = 0xb7f14000
> close(5) = 0
> mprotect(0xb7f14000, 4096, PROT_READ) = 0
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [spec.y:12"..., 81) = 81
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [spec.y:14"..., 94) = 94
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [spec.y:20"..., 65) = 65
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [spec.y:10"..., 77) = 77
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [xlator.c:"..., 134) = 134
> open("/usr/lib/glusterfs/1.3.11/xlator/protocol/server.so",
O_RDONLY) = 5
> read(5,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0P=\0\0004\0\0\0"...,
512) = 512
> fstat64(5, {st_mode=S_IFREG|0755, st_size=165201, ...}) = 0
> mmap2(NULL, 86664, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) =
0xb7584000
> mmap2(0xb7598000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x14) = 0xb7598000
> close(5) = 0
> mprotect(0xb7598000, 4096, PROT_READ) = 0
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [spec.y:12"..., 84) = 84
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [spec.y:19"..., 75) = 75
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [spec.y:14"..., 98) = 98
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [spec.y:14"..., 94) = 94
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [spec.y:20"..., 66) = 66
> read(4, "", 8192) = 0
> ioctl(4, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfd29e08) = -1 ENOTTY
(Inappropriate ioctl for device)
> close(4) = 0
> munmap(0xb7f16000, 4096) = 0
> umask(0) = 022
> mkdir("/home/httpd/", 0777) = -1 EEXIST (File exists)
> stat64("/home/httpd/", {st_mode=S_IFDIR|0775, st_size=4096, ...})
= 0
> gettimeofday({1219565456, 60809}, NULL) = 0
> gettimeofday({1219565456, 60930}, NULL) = 0
> setrlimit(RLIMIT_NOFILE, {rlim_cur=1024*1024, rlim_max=1024*1024}) = 0
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [server-pr"..., 90) = 90
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [transport"..., 136) = 136
> open("/usr/lib/glusterfs/1.3.11/transport/tcp/server.so",
O_RDONLY) = 4
> read(4,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\20\16\0\0004\0\0\0"...,
512) = 512
> fstat64(4, {st_mode=S_IFREG|0755, st_size=46330, ...}) = 0
> mmap2(NULL, 16420, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) =
0xb757f000
> mmap2(0xb7582000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x2) = 0xb7582000
> close(4) = 0
> mprotect(0xb7582000, 4096, PROT_READ) = 0
> socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 4
> setsockopt(4, SOL_SOCKET, SO_SNDTIMEO, "*\0\0\0\0\0\0\0", 8) = 0
> setsockopt(4, SOL_SOCKET, SO_RCVTIMEO, "*\0\0\0\0\0\0\0", 8) = 0
> setsockopt(4, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
> bind(4, {sa_family=AF_INET, sin_port=htons(6996),
sin_addr=inet_addr("0.0.0.0")}, 16) = 0
> listen(4, 10) = 0
> epoll_create(1024) = 5
> epoll_ctl(5, EPOLL_CTL_ADD, 4, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP,
{u32=2147509296, u64=2147509296}}) = 0
> open("/usr/lib/glusterfs/1.3.11/auth/ip.so", O_RDONLY) = 6
> read(6,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0"...,
512) = 512
> fstat64(6, {st_mode=S_IFREG|0755, st_size=10122, ...}) = 0
> mmap2(NULL, 8192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 6, 0) =
0xb757d000
> mmap2(0xb757e000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 6, 0) = 0xb757e000
> close(6) = 0
> mprotect(0xb757e000, 4096, PROT_READ) = 0
> time(NULL) = 1219565456
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:10:56 D [server-pr"..., 106) = 106
> setrlimit(RLIMIT_NOFILE, {rlim_cur=1024*1024, rlim_max=1024*1024}) = 0
> mmap2(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb6d7c000
> mprotect(0xb6d7c000, 4096, PROT_NONE) = 0
> clone(child_stack=0xb757c4c4,
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
parent_tidptr=0xb757cbd8, {entry_number:6, base_addr:0xb757cb90, limit:1048575,
seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0,
useable:1}, child_tidptr=0xb757cbd8) = 9301
> epoll_wait(5, {{EPOLLIN, {u32=2147509296, u64=2147509296}}}, 1, -1) = 1
> accept(4, {sa_family=AF_INET, sin_port=htons(1022),
sin_addr=inet_addr("10.0.0.87")}, [16]) = 6
> futex(0xb7ec6cf0, FUTEX_WAKE, 2147483647) = 0
> getpeername(6, {sa_family=AF_INET, sin_port=htons(1022),
sin_addr=inet_addr("10.0.0.87")}, [16]) = 0
> time(NULL) = 1219565505
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:45 D [tcp-serve"..., 128) = 128
> epoll_ctl(5, EPOLL_CTL_ADD, 6, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP,
{u32=2147531600, u64=2147531600}}) = 0
> epoll_wait(5, {{EPOLLIN, {u32=2147531600, u64=2147531600}}}, 2, -1) = 1
> read(6, "Block Start\n0000000000067932\n000"..., 113) = 113
> time(NULL) = 1219565505
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:45 E [protocol."..., 115) = 115
> time(NULL) = 1219565505
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:45 D [tcp.c:87:"..., 80) = 80
> epoll_ctl(5, EPOLL_CTL_DEL, 6,
{EPOLLRDNORM|EPOLLWRNORM|EPOLLHUP|EPOLLET|0xc020, {u32=3218252780,
u64=13822290701277373420}}) = 0
> close(6) = 0
> time(NULL) = 1219565505
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:45 D [server-pr"..., 132) = 132
> time(NULL) = 1219565505
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:45 D [tcp-serve"..., 121) = 121
> epoll_wait(5, {{EPOLLIN, {u32=2147509296, u64=2147509296}}}, 2, -1) = 1
> accept(4, {sa_family=AF_INET, sin_port=htons(1022),
sin_addr=inet_addr("10.0.0.87")}, [16]) = 6
> getpeername(6, {sa_family=AF_INET, sin_port=htons(1022),
sin_addr=inet_addr("10.0.0.87")}, [16]) = 0
> time(NULL) = 1219565509
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:49 D [tcp-serve"..., 128) = 128
> epoll_ctl(5, EPOLL_CTL_ADD, 6, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP,
{u32=2147531584, u64=2147531584}}) = 0
> epoll_wait(5, {{EPOLLIN, {u32=2147531584, u64=2147531584}}}, 2, -1) = 1
> read(6, "Block Start\n0000000000067932\n000"..., 113) = 113
> time(NULL) = 1219565511
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:51 E [protocol."..., 115) = 115
> time(NULL) = 1219565511
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:51 D [tcp.c:87:"..., 80) = 80
> epoll_ctl(5, EPOLL_CTL_DEL, 6,
{EPOLLWRNORM|EPOLLERR|EPOLLHUP|EPOLLET|0xc020, {u32=3218252780,
u64=13822290701277373420}}) = 0
> close(6) = 0
> time(NULL) = 1219565511
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:51 D [server-pr"..., 132) = 132
> time(NULL) = 1219565511
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:51 D [tcp-serve"..., 121) = 121
> epoll_wait(5, {{EPOLLIN, {u32=2147509296, u64=2147509296}}}, 2, -1) = 1
> accept(4, {sa_family=AF_INET, sin_port=htons(1022),
sin_addr=inet_addr("10.0.0.87")}, [16]) = 6
> getpeername(6, {sa_family=AF_INET, sin_port=htons(1022),
sin_addr=inet_addr("10.0.0.87")}, [16]) = 0
> time(NULL) = 1219565519
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:11:59 D [tcp-serve"..., 128) = 128
> epoll_ctl(5, EPOLL_CTL_ADD, 6, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP,
{u32=2147532080, u64=2147532080}}) = 0
> epoll_wait(5, {{EPOLLIN, {u32=2147532080, u64=2147532080}}}, 2, -1) = 1
> read(6, "Block Start\n0000000000067932\n000"..., 113) = 113
> time(NULL) = 1219565525
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:12:05 E [protocol."..., 115) = 115
> time(NULL) = 1219565525
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:12:05 D [tcp.c:87:"..., 80) = 80
> epoll_ctl(5, EPOLL_CTL_DEL, 6,
{EPOLLRDNORM|EPOLLWRNORM|EPOLLWRBAND|EPOLLERR|EPOLLET|0xb820, {u32=3218252780,
u64=13822290701277373420}}) = 0
> close(6) = 0
> time(NULL) = 1219565525
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:12:05 D [server-pr"..., 132) = 132
> time(NULL) = 1219565525
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:12:05 D [tcp-serve"..., 121) = 121
> epoll_wait(5, 8000bcc8, 2, -1) = -1 EINTR (Interrupted system
call)
> --- SIGINT (Interrupt) @ 0 (0) ---
> epoll_wait(5, 8000bcc8, 2, -1) = -1 EINTR (Interrupted system
call)
> --- SIGINT (Interrupt) @ 0 (0) ---
> epoll_wait(5, 8000bcc8, 2, -1) = -1 EINTR (Interrupted system
call)
> --- SIGINT (Interrupt) @ 0 (0) ---
> epoll_wait(5, 8000bcc8, 2, -1) = -1 EINTR (Interrupted system
call)
> --- SIGINT (Interrupt) @ 0 (0) ---
> epoll_wait(5, 8000bcc8, 2, -1) = -1 EINTR (Interrupted system
call)
> --- SIGINT (Interrupt) @ 0 (0) ---
> epoll_wait(5, 8000bcc8, 2, -1) = -1 EINTR (Interrupted system
call)
> --- SIGINT (Interrupt) @ 0 (0) ---
> epoll_wait(5, 8000bcc8, 2, -1) = -1 EINTR (Interrupted system
call)
> --- SIGQUIT (Quit) @ 0 (0) ---
> epoll_wait(5, 8000bcc8, 2, -1) = -1 EINTR (Interrupted system
call)
> --- SIGINT (Interrupt) @ 0 (0) ---
> epoll_wait(5, 8000bcc8, 2, -1) = -1 EINTR (Interrupted system
call)
> --- SIGTERM (Terminated) @ 0 (0) ---
> time(NULL) = 1219565593
> stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=413,
...}) = 0
> write(3, "2008-08-24 18:13:13 W [glusterfs"..., 99) = 99
> exit_group(0) = ?
The key error message seems to be this one (note the high negative
maximum number):> 2008-08-24 18:11:45 E [protocol.c:342:gf_block_unserialize_transport]
server: frame size (168) > max (-2147458000)
Speaking in the IRC channel, Anand Babu (aka rooty) had reason to
suspect the following segment of code:> [2008-08-24 18:28] <rooty> if (max_block_size &&
(blk->size > max_block_size)) {
> [2008-08-24 18:28] <rooty> gf_log (trans->xl->name,
GF_LOG_ERROR,
> [2008-08-24 18:28] <rooty> "frame size
(%"PRId32") > max (%"PRId32")",
> [2008-08-24 18:28] <rooty> blk->size, max_block_size);
> [2008-08-24 18:28] <rooty> /* block size exceeds the maximum
block size permitted by the protocol controlling
> [2008-08-24 18:28] <rooty> * this transport */
> [2008-08-24 18:28] <rooty> goto herr;
> [2008-08-24 18:28] <rooty> }
I've recompiled it with debugging symbols, tried running it in gdb, but
so far (quite likely because I'm not very experienced with gdb) haven't
had any luck tracking the problem down.
The problem arises regardless of whether the client exists on a remote
computer, or on the server itself connecting via the loopback device.
The web server is successfully able to mount GlusterFS shares on other
computers, but isn't able to share its own.
It's an unusual bug, and seems specific to just this machine, so
exactly what is going on is anyone's guess at this point. Clearly
there's some edge case that I'm triggering here. If there's any
further
information I can provide to aid tracking down this bug, let me know.
Thanks in advance.
Regards,
--
Stuart Longland (aka Redhatter) .'''.
Gentoo Linux/MIPS Cobalt and Docs Developer '.'` :
. . . . . . . . . . . . . . . . . . . . . . .'.'
http://dev.gentoo.org/~redhatter :.'
I haven't lost my mind...
...it's backed up on a tape somewhere.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20080824/0ffdcbd5/attachment.sig>