[Gluster-users] cache before threads or visa versa

Ian Rogers
Sun, 21 Mar 2010 01:07:47 -0000

Can someone familiar with performance/io-threads explain how it actually
works (or write in up in the wiki)?...

I've been doing some experiments with performance/io-threads and
performance/io-cache. I'd expect that if io-threads was the root of the
vol tree then the io-cache would be duplicated along with the memory
usage. But that doesn't seem to be the case...

E.g. on one test client I have

...

volume iothreads_web_upload_ct
   type performance/io-threads
   option thread-count 64
   subvolumes mirror_web_upload_ct
end-volume

volume io_cache_web_upload_ct
   type performance/io-cache
   option cache-size 256MB
   option cache-timeout 60
   subvolumes iothreads_web_upload_ct
end-volume


and on the other

...

volume io_cache_web_upload_tc
   type performance/io-cache
   option cache-size 256MB
   option cache-timeout 60
   subvolumes mirror_web_upload_tc
end-volume

volume iothreads_web_upload_tc
   type performance/io-threads
   option thread-count 64  # default is 1
   subvolumes io_cache_web_upload_tc
end-volume

I then run a bunch of concurrent greps on each client on a large set of
files.  On both clients VSZ of glusterfs for that mount never get above
357492k which is actually great but not what I expected :-)

If one puts io-threads in front of protocol/client will the client be
making multiple concurrent TCP connections to the server or not?

Ian