Gluster users - Jul 2011 - 3.0.5 RDMA clients seem broken

Just ran through a long exercise with a customer on trying to find out 
why only 1.6M files out of 4+M files appeared when using RDMA glusterfs 
clients.

The tools we put together previously were all about trying to work 
around other issues (not gfid, though it looks like they might have 
utility there).

It looks like the issue was/is in the RDMA mounting.  I don't know why, 
but the same file system mounted from the gluster client over tcp saw 
all the files, but when mounted over IB, we did not see all the files. 
More disconcerting was that several clients saw radically different 
numbers of files.

Just a note ... if you use 3.0.5, it looks like rdma is broken.  Caching 
is (badly) broken there as well, but thats a whole other story.

n.b. we are updating them to 3.2.2 tomorrow.

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615

> n.b. we are updating them to 3.2.2 tomorrow.
anyone know if upgrading from 3.0.x to 3.2.x works without data loss
for gluster storage nodes where clients are accessing the storage
nodes in a stripe (or distribute for that matter)?