Bill Mair
2013-Dec-05 19:05 UTC
[Gluster-users] Errors from PHP stat() on files and directories in a glusterfs mount
Hi,
I'm trying to use glusterfs to mirror the ownCloud "data" area
between 2
servers.
They are using debian jessie due to some dependancies that I have for
other components.
This is where my issue rears it's ugly head. This is failing because I
can't stat the files and directories on my glusterfs mount.
/var/www/owncloud/data is where I am mounting the volume and I can
reproduce the error using a simple php test application, so I don't
think that it is apache or owncloud related.
I'd be grateful for any pointers on how to resolve this problem.
Thanks,
Bill
Attached is "simple.php" test and the results of executing
"strace php5
simple.php" twice, once with the glusterfs mounted
(simple.php.strace-glusterfs) and once against the file system when
unmounted (simple.php.strace-unmounted).
------------------------------------------------------------------------
Here is what I get in the gluster log when I run the test (as root):
/var/log/glusterfs/var-www-owncloud-data.log
[2013-12-05 18:33:50.802250] D
[client-handshake.c:185:client_start_ping] 0-gv-ocdata-client-0:
returning as transport is already disconnected OR there are no frames (0
|| 0)
[2013-12-05 18:33:50.825132] D
[afr-self-heal-common.c:138:afr_sh_print_pending_matrix]
0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ]
[2013-12-05 18:33:50.825322] D
[afr-self-heal-common.c:138:afr_sh_print_pending_matrix]
0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ]
[2013-12-05 18:33:50.825393] D
[afr-self-heal-common.c:887:afr_mark_sources] 0-gv-ocdata-replicate-0:
Number of sources: 0
[2013-12-05 18:33:50.825456] D
[afr-self-heal-data.c:825:afr_lookup_select_read_child_by_txn_type]
0-gv-ocdata-replicate-0: returning read_child: 0
[2013-12-05 18:33:50.825511] D
[afr-common.c:1380:afr_lookup_select_read_child]
0-gv-ocdata-replicate-0: Source selected as 0 for /
[2013-12-05 18:33:50.825579] D
[afr-common.c:1117:afr_lookup_build_response_params]
0-gv-ocdata-replicate-0: Building lookup response from 0
[2013-12-05 18:33:50.827069] D
[afr-common.c:131:afr_lookup_xattr_req_prepare] 0-gv-ocdata-replicate-0:
/check.txt: failed to get the gfid from dict
[2013-12-05 18:33:50.829409] D
[client-handshake.c:185:client_start_ping] 0-gv-ocdata-client-0:
returning as transport is already disconnected OR there are no frames (0
|| 0)
[2013-12-05 18:33:50.836719] D
[afr-self-heal-common.c:138:afr_sh_print_pending_matrix]
0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ]
[2013-12-05 18:33:50.836870] D
[afr-self-heal-common.c:138:afr_sh_print_pending_matrix]
0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ]
[2013-12-05 18:33:50.836941] D
[afr-self-heal-common.c:887:afr_mark_sources] 0-gv-ocdata-replicate-0:
Number of sources: 0
[2013-12-05 18:33:50.837002] D
[afr-self-heal-data.c:825:afr_lookup_select_read_child_by_txn_type]
0-gv-ocdata-replicate-0: returning read_child: 0
[2013-12-05 18:33:50.837058] D
[afr-common.c:1380:afr_lookup_select_read_child]
0-gv-ocdata-replicate-0: Source selected as 0 for /check.txt
[2013-12-05 18:33:50.837129] D
[afr-common.c:1117:afr_lookup_build_response_params]
0-gv-ocdata-replicate-0: Building lookup response from 0
Other bits of information
root at bbb-1:/var/www/owncloud# uname -a
Linux bbb-1 3.8.13-bone30 #1 SMP Thu Nov 14 02:59:07 UTC 2013 armv7l
GNU/Linux
root at bbb-1:/var/www/owncloud# dpkg -l glusterfs-*
Desired=Unknown/Install/Remove/Purge/Hold
|
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name
Version Architecture Description
+++-============================================-===========================-===========================-=============================================================================================ii
glusterfs-client
3.4.1-1 armhf clustered
file-system (client package)
ii glusterfs-common
3.4.1-1 armhf GlusterFS common
libraries and translator modules
ii glusterfs-server
3.4.1-1 armhf clustered
file-system (server package)
mount
bbb-1:gv-ocdata on /var/www/owncloud/data type fuse.glusterfs
(rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
/etc/fstab
UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /sdhc ext4 defaults 0 0
bbb-1:gv-ocdata /var/www/owncloud/data glusterfs
defaults,_netdev,log-level=DEBUG 0 0
ls -al on the various paths
root at bbb-1:/var/log/glusterfs# ll -d /sdhc/
drwxrwxr-x 7 root root 4096 Nov 28 19:15 /sdhc/
root at bbb-1:/var/log/glusterfs# ll -d /sdhc/gv-ocdata/
drwxrwx--- 5 www-data www-data 4096 Dec 5 00:50 /sdhc/gv-ocdata/
root at bbb-1:/var/log/glusterfs# ll -d /sdhc/gv-ocdata/check.txt
-rw-r--r-- 2 root root 10 Dec 5 00:50 /sdhc/gv-ocdata/check.txt
root at bbb-1:/var/www/owncloud# ll -d /var/www/owncloud/data/
drwxrwx--- 5 www-data www-data 4096 Dec 5 00:50 /var/www/owncloud/data/
root at bbb-1:/var/www/owncloud# ll -d /var/www/owncloud/data/check.txt
-rw-r--r-- 1 root root 10 Dec 5 00:50 /var/www/owncloud/data/check.txt
file & dir attr information:
root at bbb-1:/var/www/owncloud# attr -l /var/www/owncloud/data
Attribute "glusterfs.volume-id" has a 16 byte value for
/var/www/owncloud/data
root at bbb-1:/var/www/owncloud# attr -l /var/www/owncloud/data/check.txt
root at bbb-1:/var/www/owncloud#
root at bbb-1:/var/www/owncloud# attr -l /sdhc/gv-ocdata/
Attribute "glusterfs.volume-id" has a 16 byte value for
/sdhc/gv-ocdata/
Attribute "gfid" has a 16 byte value for /sdhc/gv-ocdata/
Attribute "glusterfs.dht" has a 16 byte value for /sdhc/gv-ocdata/
Attribute "afr.gv-ocdata-client-0" has a 12 byte value for
/sdhc/gv-ocdata/
Attribute "afr.gv-ocdata-client-1" has a 12 byte value for
/sdhc/gv-ocdata/
root at bbb-1:/var/www/owncloud# attr -l /sdhc/gv-ocdata/check.txt
Attribute "gfid" has a 16 byte value for /sdhc/gv-ocdata/check.txt
Attribute "afr.gv-ocdata-client-0" has a 12 byte value for
/sdhc/gv-ocdata/check.txt
Attribute "afr.gv-ocdata-client-1" has a 12 byte value for
/sdhc/gv-ocdata/check.txt
root at bbb-1:/var/www/owncloud#
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131205/a11c93d9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: simple-php-example.zip
Type: application/zip
Size: 21249 bytes
Desc: not available
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131205/a11c93d9/attachment.zip>
Johan Huysmans
2013-Dec-10 09:42 UTC
[Gluster-users] Errors from PHP stat() on files and directories in a glusterfs mount
Hi, It seems I have a related problem (just posted this on the mailing list). Do you already have a solution for this problem? gr. Johan Huysmans On 05-12-13 20:05, Bill Mair wrote:> Hi, > > I'm trying to use glusterfs to mirror the ownCloud "data" area between > 2 servers. > > They are using debian jessie due to some dependancies that I have for > other components. > > This is where my issue rears it's ugly head. This is failing because I > can't stat the files and directories on my glusterfs mount. > > /var/www/owncloud/data is where I am mounting the volume and I can > reproduce the error using a simple php test application, so I don't > think that it is apache or owncloud related. > > I'd be grateful for any pointers on how to resolve this problem. > > Thanks, > > Bill > > Attached is "simple.php" test and the results of executing "strace > php5 simple.php" twice, once with the glusterfs mounted > (simple.php.strace-glusterfs) and once against the file system when > unmounted (simple.php.strace-unmounted). > > ------------------------------------------------------------------------ > > Here is what I get in the gluster log when I run the test (as root): > > /var/log/glusterfs/var-www-owncloud-data.log > > [2013-12-05 18:33:50.802250] D > [client-handshake.c:185:client_start_ping] 0-gv-ocdata-client-0: > returning as transport is already disconnected OR there are no frames > (0 || 0) > [2013-12-05 18:33:50.825132] D > [afr-self-heal-common.c:138:afr_sh_print_pending_matrix] > 0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ] > [2013-12-05 18:33:50.825322] D > [afr-self-heal-common.c:138:afr_sh_print_pending_matrix] > 0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ] > [2013-12-05 18:33:50.825393] D > [afr-self-heal-common.c:887:afr_mark_sources] 0-gv-ocdata-replicate-0: > Number of sources: 0 > [2013-12-05 18:33:50.825456] D > [afr-self-heal-data.c:825:afr_lookup_select_read_child_by_txn_type] > 0-gv-ocdata-replicate-0: returning read_child: 0 > [2013-12-05 18:33:50.825511] D > [afr-common.c:1380:afr_lookup_select_read_child] > 0-gv-ocdata-replicate-0: Source selected as 0 for / > [2013-12-05 18:33:50.825579] D > [afr-common.c:1117:afr_lookup_build_response_params] > 0-gv-ocdata-replicate-0: Building lookup response from 0 > [2013-12-05 18:33:50.827069] D > [afr-common.c:131:afr_lookup_xattr_req_prepare] > 0-gv-ocdata-replicate-0: /check.txt: failed to get the gfid from dict > [2013-12-05 18:33:50.829409] D > [client-handshake.c:185:client_start_ping] 0-gv-ocdata-client-0: > returning as transport is already disconnected OR there are no frames > (0 || 0) > [2013-12-05 18:33:50.836719] D > [afr-self-heal-common.c:138:afr_sh_print_pending_matrix] > 0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ] > [2013-12-05 18:33:50.836870] D > [afr-self-heal-common.c:138:afr_sh_print_pending_matrix] > 0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ] > [2013-12-05 18:33:50.836941] D > [afr-self-heal-common.c:887:afr_mark_sources] 0-gv-ocdata-replicate-0: > Number of sources: 0 > [2013-12-05 18:33:50.837002] D > [afr-self-heal-data.c:825:afr_lookup_select_read_child_by_txn_type] > 0-gv-ocdata-replicate-0: returning read_child: 0 > [2013-12-05 18:33:50.837058] D > [afr-common.c:1380:afr_lookup_select_read_child] > 0-gv-ocdata-replicate-0: Source selected as 0 for /check.txt > [2013-12-05 18:33:50.837129] D > [afr-common.c:1117:afr_lookup_build_response_params] > 0-gv-ocdata-replicate-0: Building lookup response from 0 > > Other bits of information > > root at bbb-1:/var/www/owncloud# uname -a > Linux bbb-1 3.8.13-bone30 #1 SMP Thu Nov 14 02:59:07 UTC 2013 armv7l > GNU/Linux > > root at bbb-1:/var/www/owncloud# dpkg -l glusterfs-* > Desired=Unknown/Install/Remove/Purge/Hold > | > Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend > |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) > ||/ Name Version Architecture Description > +++-============================================-===========================-===========================-=============================================================================================> ii glusterfs-client 3.4.1-1 armhf clustered > file-system (client package) > ii glusterfs-common 3.4.1-1 armhf GlusterFS > common libraries and translator modules > ii glusterfs-server 3.4.1-1 armhf clustered > file-system (server package) > > mount > > bbb-1:gv-ocdata on /var/www/owncloud/data type fuse.glusterfs > (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) > > /etc/fstab > > UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /sdhc ext4 defaults 0 0 > bbb-1:gv-ocdata /var/www/owncloud/data glusterfs > defaults,_netdev,log-level=DEBUG 0 0 > > ls -al on the various paths > > root at bbb-1:/var/log/glusterfs# ll -d /sdhc/ > drwxrwxr-x 7 root root 4096 Nov 28 19:15 /sdhc/ > > root at bbb-1:/var/log/glusterfs# ll -d /sdhc/gv-ocdata/ > drwxrwx--- 5 www-data www-data 4096 Dec 5 00:50 /sdhc/gv-ocdata/ > > root at bbb-1:/var/log/glusterfs# ll -d /sdhc/gv-ocdata/check.txt > -rw-r--r-- 2 root root 10 Dec 5 00:50 /sdhc/gv-ocdata/check.txt > > root at bbb-1:/var/www/owncloud# ll -d /var/www/owncloud/data/ > drwxrwx--- 5 www-data www-data 4096 Dec 5 00:50 /var/www/owncloud/data/ > > root at bbb-1:/var/www/owncloud# ll -d /var/www/owncloud/data/check.txt > -rw-r--r-- 1 root root 10 Dec 5 00:50 /var/www/owncloud/data/check.txt > > file & dir attr information: > > root at bbb-1:/var/www/owncloud# attr -l /var/www/owncloud/data > Attribute "glusterfs.volume-id" has a 16 byte value for > /var/www/owncloud/data > > root at bbb-1:/var/www/owncloud# attr -l /var/www/owncloud/data/check.txt > root at bbb-1:/var/www/owncloud# > > root at bbb-1:/var/www/owncloud# attr -l /sdhc/gv-ocdata/ > Attribute "glusterfs.volume-id" has a 16 byte value for /sdhc/gv-ocdata/ > Attribute "gfid" has a 16 byte value for /sdhc/gv-ocdata/ > Attribute "glusterfs.dht" has a 16 byte value for /sdhc/gv-ocdata/ > Attribute "afr.gv-ocdata-client-0" has a 12 byte value for > /sdhc/gv-ocdata/ > Attribute "afr.gv-ocdata-client-1" has a 12 byte value for > /sdhc/gv-ocdata/ > > root at bbb-1:/var/www/owncloud# attr -l /sdhc/gv-ocdata/check.txt > Attribute "gfid" has a 16 byte value for /sdhc/gv-ocdata/check.txt > Attribute "afr.gv-ocdata-client-0" has a 12 byte value for > /sdhc/gv-ocdata/check.txt > Attribute "afr.gv-ocdata-client-1" has a 12 byte value for > /sdhc/gv-ocdata/check.txt > root at bbb-1:/var/www/owncloud# > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131210/d77e25bb/attachment.html>