lxcypp
2009-Jun-26 22:22 UTC
[Gluster-users] list files very slow when 400K+ files in Glusterfs 1.3.12
I've bind 6 harddisk to one big volume with unify mode and mount it to /disk . After copied 400,000+ files into it /disk/files/ directory ( each file size is about 500K~1M ), when I execute then command "ls /disk", I get no response for serval minitues. Has anyone ever encountered with this problem? I want to know how to solve this problem (at least reduce the time of waiting)? Has the newest version of glusterfs (v2.0) solved this problem? Thanks for your help! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20090627/f78010e9/attachment.html>
Martin Reissner
2009-Jun-27 09:11 UTC
[Gluster-users] list files very slow when 400K+ files in Glusterfs 1.3.12
Hello, I don't have experience with your exact setup especially not with GlusterFS 1.3 but what you could try to increase performance is - Update to GlusterFS 2.0.2 - Use newest FUSE (and try GlusterFS patched FUSE) - Replace the Unify Translator by DHT - If possible put the files in some subfolders, that should increase performance when dirlisting I tried with 4 Servers unified with the DHT Translator and ~20000 files that were a bit smaller than yours and the performance was reasonable compared to a single NFS share. Martin lxcypp wrote:> I've bind 6 harddisk to one big volume with unify mode and mount it to /disk > . > After copied 400,000+ files into it /disk/files/ directory ( each file size > is about 500K~1M ), > when I execute then command "ls /disk", > I get no response for serval minitues. > > Has anyone ever encountered with this problem? > I want to know how to solve this problem (at least reduce the time of > waiting)? > Has the newest version of glusterfs (v2.0) solved this problem? > > Thanks for your help! > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users