Gluster users - Jan 2012 - gluster no longer serving all of its files

Hi,

We recently upgraded our version of gluster from 3.1.4 to
3.2.5. Shortly after I noticed that gluster was not serving
up all of its files, that is if I log onto one of the individual
bricks and do an ls of the underlying nfs directories, I can
see files that I do not see if do an ls from a client node
in the equivalent gluster directory.

We may have caused this by also running a script that went
through and removed pointer files (this had been how we had
been dealing with bad pointers under gluster 3.1.4).

So far, we have tried running a gluster self-heal under
version 3.2.5 which did not solve the problem.  We have
also tried rolling back to version 3.1.4 and running the
self-heal there, but that does not solve the problem either.

Any advice you can give us would be greatly appreciated.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley                          Email:  phaley at mit.edu
Center for Ocean Engineering       Phone:  (617) 253-6824
Dept. of Mechanical Engineering    Fax:    (617) 253-8125
MIT, Room 5-222B                   http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA  02139-4301

Hi,

We recently upgraded our version of gluster from 3.1.4 to
3.2.5. Shortly after I noticed that gluster was not serving
up all of its files, that is if I log onto one of the individual
bricks and do an ls of the underlying nfs directories, I can
see files that I do not see if do an ls from a client node
in the equivalent gluster directory.

We may have caused this by also running a script that went
through and removed pointer files (this had been how we had
been dealing with bad pointers under gluster 3.1.4).

So far, we have tried
    - running a gluster self-heal under version 3.2.5
    - rolling back to version 3.1.4
    - running a gluster self-heal under version 3.1.4
    - running a rebalance under version 3.1.4
None of these have solved the problem.  The one other
piece of data we have is that all the files which are
not being served appear to reside on a single brick
(at least every file we missed so far has been on
  that server and we have not found one missing from
  the other servers).

Any advice you can give us would be greatly appreciated.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley                          Email:  phaley at mit.edu
Center for Ocean Engineering       Phone:  (617) 253-6824
Dept. of Mechanical Engineering    Fax:    (617) 253-8125
MIT, Room 5-222B                   http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA  02139-4301