I made the following changes for small file performance as suggested by http://blog.gluster.org/gluster-tiering-and-small-file-performance/ I am still seeing du -sh /data/shared taking 39 minutes. Any other tuning I can do. Most of my files are 15K. Here is sample of small files with size and number of occurrences FileSize. # of occurrence ==== =========== 1.1K 1122 1.1M 1040 1.2K 1281 1.2M 1357 1.3K 1149 1.3M 1098 1.4K 1119 1.5K 1189 1.6K 1036 1.7K 1169 11K 2157 12K 2398 13K 2402 14K 2406*15K 2426* 16K 2386 17K 1986 18K 2037 19K 1829 2.0K 1027 2.1K 1048 2.4K 1013 20K 1585 21K 1713 22K 1590 23K 1371 24K 1428 25K 1444 26K 1391 27K 1217 28K 1485 29K 1282 30K 1303 31K 1275 32K 1296 33K 1058 36K 1023 37K 1107 39K 1092 41K 1034 42K 1187 46K 1030 On Mon, May 25, 2020 at 5:30 PM <vadud3 at gmail.com> wrote:> time du -sh /data/shared > > 431G /data/shared > > real 45m49.992s > user 0m20.043s > sys 2m32.456s > > > gluster fs is extremely slow > > Any suggestions on what settings to change to improve it? > > > -- > Asif Iqbal > PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu > A: Because it messes up the order in which people normally read text. > Q: Why is top-posting such a bad thing? > >-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200526/a27697fd/attachment.html>
I ran the smallfile utility like below for create and read and LOOKUP
number looks too high
# smallfile_cli.py --top /data/shared/tmp/ --threads 8 --files 2000
--file-size 15 --record-size 15 --fsync N --operation create
# smallfile_cli.py --top /data/shared/tmp/ --threads 8 --files 2000
--file-size 15 --record-size 15 --fsync N --operation read
total threads = 8
total files = 16000
total IOPS = 125
total data = 0.229 GiB
100.00% of requested files processed, warning threshold is 70.00
elapsed time = 127.516
files/sec = 125.931253
IOPS = 125.931253
MiB/sec = 1.844696
# gluster volume profile atlassian info cumulative | egrep
'Brick|LOOKUP'
Brick: node1:/data/foo/gluster
37.43 1424.67 us 13.78 us 2244438.29 us 2024515
LOOKUP
Brick: note2:/data/foo/gluster
53.01 413.87 us 14.03 us 262160.52 us 3122542
LOOKUP
On Tue, May 26, 2020 at 2:59 PM <vadud3 at gmail.com> wrote:
> I made the following changes for small file performance as suggested by
> http://blog.gluster.org/gluster-tiering-and-small-file-performance/
>
> I am still seeing du -sh /data/shared taking 39 minutes.
>
> Any other tuning I can do. Most of my files are 15K. Here is sample of
> small files with size and number of occurrences
>
> FileSize. # of occurrence
> ==== ===========>
> 1.1K 1122
> 1.1M 1040
> 1.2K 1281
> 1.2M 1357
> 1.3K 1149
> 1.3M 1098
> 1.4K 1119
> 1.5K 1189
> 1.6K 1036
> 1.7K 1169
> 11K 2157
> 12K 2398
> 13K 2402
> 14K 2406*15K 2426*
> 16K 2386
> 17K 1986
> 18K 2037
> 19K 1829
> 2.0K 1027
> 2.1K 1048
> 2.4K 1013
> 20K 1585
> 21K 1713
> 22K 1590
> 23K 1371
> 24K 1428
> 25K 1444
> 26K 1391
> 27K 1217
> 28K 1485
> 29K 1282
> 30K 1303
> 31K 1275
> 32K 1296
> 33K 1058
> 36K 1023
> 37K 1107
> 39K 1092
> 41K 1034
> 42K 1187
> 46K 1030
>
>
>
>
>
> On Mon, May 25, 2020 at 5:30 PM <vadud3 at gmail.com> wrote:
>
>> time du -sh /data/shared
>>
>> 431G /data/shared
>>
>> real 45m49.992s
>> user 0m20.043s
>> sys 2m32.456s
>>
>>
>> gluster fs is extremely slow
>>
>> Any suggestions on what settings to change to improve it?
>>
>>
>> --
>> Asif Iqbal
>> PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
>> A: Because it messes up the order in which people normally read text.
>> Q: Why is top-posting such a bad thing?
>>
>>
>
> --
> Asif Iqbal
> PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
> A: Because it messes up the order in which people normally read text.
> Q: Why is top-posting such a bad thing?
>
>
--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20200526/38fda5f4/attachment.html>
Sunil Kumar Heggodu Gopala Acharya
2020-May-26 19:06 UTC
[Gluster-users] File system very slow
Hi, Please share the gluster volume information. # gluster vol info Regards, Sunil kumar Acharya On Wed, May 27, 2020 at 12:30 AM <vadud3 at gmail.com> wrote:> I made the following changes for small file performance as suggested by > http://blog.gluster.org/gluster-tiering-and-small-file-performance/ > > I am still seeing du -sh /data/shared taking 39 minutes. > > Any other tuning I can do. Most of my files are 15K. Here is sample of > small files with size and number of occurrences > > FileSize. # of occurrence > ==== ===========> > 1.1K 1122 > 1.1M 1040 > 1.2K 1281 > 1.2M 1357 > 1.3K 1149 > 1.3M 1098 > 1.4K 1119 > 1.5K 1189 > 1.6K 1036 > 1.7K 1169 > 11K 2157 > 12K 2398 > 13K 2402 > 14K 2406*15K 2426* > 16K 2386 > 17K 1986 > 18K 2037 > 19K 1829 > 2.0K 1027 > 2.1K 1048 > 2.4K 1013 > 20K 1585 > 21K 1713 > 22K 1590 > 23K 1371 > 24K 1428 > 25K 1444 > 26K 1391 > 27K 1217 > 28K 1485 > 29K 1282 > 30K 1303 > 31K 1275 > 32K 1296 > 33K 1058 > 36K 1023 > 37K 1107 > 39K 1092 > 41K 1034 > 42K 1187 > 46K 1030 > > > > > > On Mon, May 25, 2020 at 5:30 PM <vadud3 at gmail.com> wrote: > >> time du -sh /data/shared >> >> 431G /data/shared >> >> real 45m49.992s >> user 0m20.043s >> sys 2m32.456s >> >> >> gluster fs is extremely slow >> >> Any suggestions on what settings to change to improve it? >> >> >> -- >> Asif Iqbal >> PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu >> A: Because it messes up the order in which people normally read text. >> Q: Why is top-posting such a bad thing? >> >> > > -- > Asif Iqbal > PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu > A: Because it messes up the order in which people normally read text. > Q: Why is top-posting such a bad thing? > > ________ > > > > Community Meeting Calendar: > > Schedule - > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > Gluster-users at gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200527/f3ce2cdb/attachment.html>