[Gluster-users] Readdirp (ls -l) Performance Improvement

RAFI KC
Wed, 27 May 2020 05:56:40 -0000

Hi All,

I have been working on POC to improve readdirp performance improvement.
At the end of the experiment, The results are showing promising result
in performance, overall there is a 104% improvement for full filesystem
crawl compared to the existing solution. Here is the short test numbers.
The tests were carried out in 16*3 setup with 1.5 Million dentries (Both
files and dir). The system also contains some empty directories. *In the
result the proposed solution is 287% faster than the plane volume and
104% faster than the parallel-readdir based solution.*

*
*

Configuration



Plain volume



Parallel-readdir



Proposed Solution

FS Crawl Time in Seconds



16497.523



8717.872



4261.401

In short, the basic idea behind the proposal is the efficient managing
of readdir buffer in gluster along with prefetching the dentries for
intelligent switch-over to the next buffer. The detailed problem
description, deign description and results are available in the
doc.https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit



If anybody can help with the testing on a different kind of workloads, I
would be very happy to assist. If wanted to test the patch and run a
performance test on your setup, I could help with back-porting the patch
to the version of your choice.


https://review.gluster.org/24469

https://review.gluster.org/24470


Regards

Rafi KC

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20200527/596c6ffd/attachment.html>