I'm having an interesting/odd problem with nfs (I think). We recently
(Monday/Tuesday) upgraded our file server from an ancient redhat 7.3
system to a shiny new centos 6.4 system. We don't see any issues between
the other centos boxes, but things get a bit weird when we start
mounting on the old solaris clients.
The initial symptom was that the 'tab complete' wasn't working, and
then
we noticed that typing 'ls *' in the mounted directory was bombing. I
tried forcing the mounting back to nfs3 but it's not consistent. I've
set up two boxes as servers and one of the solaris boxes is my client.
Each server has two shares that are mounted on the client. Of those
four, one of them works properly and the other three do not. I've spent
most of the day trying to debug this and I cannot for the life of me
tell why one share works and the rest don't. Nothing seems to be special
about that share versus the rest.
Here are notes on how things are set up.
on duke: (nis server)
vi /etc/ypfiles/automap
scrs1_bolt -soft,intr,retrans=1 boltzmann:/scrs1_bolt
summit_bolt -soft,intr,retrans=1 boltzmann:/summit_bolt
scrs1.mirror -soft,intr,retrans=1 goblin:/scrs1.mirror
summit.mirror -soft,intr,retrans=1 goblin:/summit.mirror
( cd /var/yp ; make )
on boltzmann: (nfs server)
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sdb2 50G 13G 37G 26% /
tmpfs 3.9G 1.2M 3.9G 1% /dev/shm
/dev/sdb3 177G 188M 175G 1% /aux
/dev/sda3 208G 44G 164G 21% /aux2
mkdir /aux/scrs1_bolt
mkdir /aux2/summit_bolt
ln -s /aux/scrs1_bolt /scrs1_bolt
ln -s /aux2/summit_bolt /summit_bolt
chmod 777 /aux/scrs1_bolt /aux2/summit_bolt
service nfs restart
vi /etc/exports
/scrs1_bolt xxx.xxx.xxx.0/24(rw,no_root_squash,sync,insecure)
/summit_bolt xxx.xxx.xxx.0/24(rw,no_root_squash,sync,insecure)
exportfs -rv
on bigdog: (client)
mkdir /tmp/test/b1 /tmp/test/b2 /tmp/test/g1 /tmp/test/g2
touch /tmp/test/b1/nothing_is_mounted
/tmp/test/b2/nothing_is_mounted /tmp/test/g1/nothing_is_mounted
/tmp/test/g2/nothing_is_mounted
mount -F nfs -o nfsvers=3 boltzmann:/summit_bolt /tmp/test/b1
mount -F nfs -o nfsvers=3 boltzmann:/scrs1_bolt /tmp/test/b2
mount -F nfs -o nfsvers=3 goblin:/summit.mirror /tmp/test/g1
mount -F nfs -o nfsvers=3 goblin:/scrs1.mirror /tmp/test/g2
ls -l /tmp/test/*
-rw-r--r-- 1 root other 0 Jul 3 14:39
/tmp/test/nothing_is_mounted
/tmp/test/b1:
total 24
-rw-r--r-- 1 root other 0 Jul 3 12:32 SUMMIT_BOLT
-rw-r--r-- 1 root other 0 Jul 3 09:26
boltzmann_test_summit
/tmp/test/b2:
total 32
-rw-r--r-- 1 root other 0 Jul 3 12:31 SCRS1_BOLT
-rw-r--r-- 1 root other 0 Jul 3 09:26
boltzmann_test_scrs1
/tmp/test/g1:
total 280
-rw-r--r-- 1 root other 0 Jul 3 15:40
.00_summit_nas_volume
-rw-rw-r-- 1 root other 0 Jul 3 15:03
SUMMIT_MIRROR
/tmp/test/g2:
total 120
-rw-r--r-- 1 root other 0 Jul 3 15:40
.00_scrs1_nas_volume
-rw-rw-r-- 1 root other 0 Jul 3 15:02 SCRS1_MIRROR
ls -la /tmp/test/b1/*
-rw-r--r-- 1 root other 0 Jul 3 12:32
/tmp/test/b1/SUMMIT_BOLT
-rw-r--r-- 1 root other 0 Jul 3 09:26
/tmp/test/b1/boltzmann_test_summit
ls -l /tmp/test/b2/*
ls: No match.
ls -l /tmp/test/g1/*
ls: No match.
ls -l /tmp/test/g2/*
ls: No match.
mount
/tmp/test/b1 on boltzmann:/summit_bolt read/write/remote on
Wed Jul 3 15:41:11 2013
/tmp/test/b2 on boltzmann:/scrs1_bolt read/write/remote on Wed
Jul 3 15:41:11 2013
/tmp/test/g1 on goblin:/summit.mirror read/write/remote on Wed
Jul 3 15:41:11 2013
/tmp/test/g2 on goblin:/scrs1.mirror read/write/remote on Wed
Jul 3 15:41:11 2013
umount -a /tmp/test/b1 /tmp/test/b2 /tmp/test/g1 /tmp/test/g2
If I use automount to access the shares on the client, the mounts are
made by default with nfs4 and we see this same 'no match' behaviour.
It's probably something really stupid but I'm just not seeing it...
If anyone has ideas and/or needs more info, please let me know.
--
Thanks!
Miranda
> > If anyone has ideas and/or needs more info, please let me know. >Step 1 in debugging and troubleshooting... Use the KISS principle. Right now in that you have NIS, NFS, CentOS server and Solaris client (version? Given the red hat 7.3 instance you had would a safe assumption be not 11 or even 10?) Cut this down to work out which cog in the wheel is broken as there are just too many variables... Can you mount the NFS share locally to another directory on the NFS server? If that works can you mount it with a CentOS 6.4 client on another system? It's been a while since I had to deal with NFS but I see you have symlinks in exports... I thought at least under 3 that was definitely not supported and you should use bind mounts... http://mail-index.netbsd.org/tech-kern/1995/05/28/0000.html Start there and see how you go...
----- Original Message ----- | I'm having an interesting/odd problem with nfs (I think). We recently | (Monday/Tuesday) upgraded our file server from an ancient redhat 7.3 | system to a shiny new centos 6.4 system. We don't see any issues | between | the other centos boxes, but things get a bit weird when we start | mounting on the old solaris clients. | | The initial symptom was that the 'tab complete' wasn't working, and | then | we noticed that typing 'ls *' in the mounted directory was bombing. I | tried forcing the mounting back to nfs3 but it's not consistent. I've | set up two boxes as servers and one of the solaris boxes is my | client. | Each server has two shares that are mounted on the client. Of those | four, one of them works properly and the other three do not. I've | spent | most of the day trying to debug this and I cannot for the life of me | tell why one share works and the rest don't. Nothing seems to be | special | about that share versus the rest. | | Here are notes on how things are set up. | | on duke: (nis server) | vi /etc/ypfiles/automap | scrs1_bolt -soft,intr,retrans=1 boltzmann:/scrs1_bolt | summit_bolt -soft,intr,retrans=1 boltzmann:/summit_bolt | scrs1.mirror -soft,intr,retrans=1 goblin:/scrs1.mirror | summit.mirror -soft,intr,retrans=1 goblin:/summit.mirror | ( cd /var/yp ; make ) | | on boltzmann: (nfs server) | df -h | Filesystem Size Used Avail Use% Mounted on | /dev/sdb2 50G 13G 37G 26% / | tmpfs 3.9G 1.2M 3.9G 1% /dev/shm | /dev/sdb3 177G 188M 175G 1% /aux | /dev/sda3 208G 44G 164G 21% /aux2 | mkdir /aux/scrs1_bolt | mkdir /aux2/summit_bolt | ln -s /aux/scrs1_bolt /scrs1_bolt | ln -s /aux2/summit_bolt /summit_bolt | chmod 777 /aux/scrs1_bolt /aux2/summit_bolt | service nfs restart | vi /etc/exports | /scrs1_bolt | xxx.xxx.xxx.0/24(rw,no_root_squash,sync,insecure) | /summit_bolt | xxx.xxx.xxx.0/24(rw,no_root_squash,sync,insecure) | exportfs -rv | | on bigdog: (client) | mkdir /tmp/test/b1 /tmp/test/b2 /tmp/test/g1 /tmp/test/g2 | touch /tmp/test/b1/nothing_is_mounted | /tmp/test/b2/nothing_is_mounted /tmp/test/g1/nothing_is_mounted | /tmp/test/g2/nothing_is_mounted | mount -F nfs -o nfsvers=3 boltzmann:/summit_bolt /tmp/test/b1 | mount -F nfs -o nfsvers=3 boltzmann:/scrs1_bolt /tmp/test/b2 | mount -F nfs -o nfsvers=3 goblin:/summit.mirror /tmp/test/g1 | mount -F nfs -o nfsvers=3 goblin:/scrs1.mirror /tmp/test/g2 | ls -l /tmp/test/* | -rw-r--r-- 1 root other 0 Jul 3 14:39 | /tmp/test/nothing_is_mounted | | /tmp/test/b1: | total 24 | -rw-r--r-- 1 root other 0 Jul 3 12:32 | SUMMIT_BOLT | -rw-r--r-- 1 root other 0 Jul 3 09:26 | boltzmann_test_summit | | /tmp/test/b2: | total 32 | -rw-r--r-- 1 root other 0 Jul 3 12:31 | SCRS1_BOLT | -rw-r--r-- 1 root other 0 Jul 3 09:26 | boltzmann_test_scrs1 | | /tmp/test/g1: | total 280 | -rw-r--r-- 1 root other 0 Jul 3 15:40 | .00_summit_nas_volume | -rw-rw-r-- 1 root other 0 Jul 3 15:03 | SUMMIT_MIRROR | | /tmp/test/g2: | total 120 | -rw-r--r-- 1 root other 0 Jul 3 15:40 | .00_scrs1_nas_volume | -rw-rw-r-- 1 root other 0 Jul 3 15:02 | SCRS1_MIRROR | | ls -la /tmp/test/b1/* | -rw-r--r-- 1 root other 0 Jul 3 12:32 | /tmp/test/b1/SUMMIT_BOLT | -rw-r--r-- 1 root other 0 Jul 3 09:26 | /tmp/test/b1/boltzmann_test_summit | | ls -l /tmp/test/b2/* | ls: No match. | | ls -l /tmp/test/g1/* | ls: No match. | | ls -l /tmp/test/g2/* | ls: No match. | | mount | /tmp/test/b1 on boltzmann:/summit_bolt read/write/remote on | Wed Jul 3 15:41:11 2013 | /tmp/test/b2 on boltzmann:/scrs1_bolt read/write/remote on | Wed | Jul 3 15:41:11 2013 | /tmp/test/g1 on goblin:/summit.mirror read/write/remote on | Wed | Jul 3 15:41:11 2013 | /tmp/test/g2 on goblin:/scrs1.mirror read/write/remote on | Wed | Jul 3 15:41:11 2013 | | umount -a /tmp/test/b1 /tmp/test/b2 /tmp/test/g1 /tmp/test/g2 | | If I use automount to access the shares on the client, the mounts are | made by default with nfs4 and we see this same 'no match' behaviour. | It's probably something really stupid but I'm just not seeing it... | | If anyone has ideas and/or needs more info, please let me know. | | -- | Thanks! | Miranda in our automounts we specified vers=3 as an option which consistently mounted with NFSv3. You may also wish to check out /etc/sysconfig/nfs and see if there are any inconsistencies there between the working and non-working servers. Note, you can specify the NFS version there too ;) -- James A. Peltier Manager, IT Services - Research Computing Group Simon Fraser University - Burnaby Campus Phone : 778-782-6573 Fax : 778-782-3045 E-Mail : jpeltier at sfu.ca Website : http://www.sfu.ca/itservices ?A successful person is one who can lay a solid foundation from the bricks others have thrown at them.? -David Brinkley via Luke Shaw
Am 04.07.2013 um 04:22 schrieb Miranda Hawarden-Ogata <hawarden at ifa.hawaii.edu>:> I'm having an interesting/odd problem with nfs (I think). We recently > (Monday/Tuesday) upgraded our file server from an ancient redhat 7.3 > system to a shiny new centos 6.4 system. We don't see any issues between > the other centos boxes, but things get a bit weird when we start > mounting on the old solaris clients.Just some hints (even in case you know them all :-)): https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/ch-nfs.html cat /etc/sysconfig/nfs tcp-wrappers (/etc/hosts.{allow.deny})? iptables (iptables -L -n)? rpcinfo -p rpcinfo -p nfs-server -- LF