Gluster users - Dec 2015 - Disabling gluster cache

Hi,
One of my gluster deployment requires to completely disable Gluster caching
and put in proprietary caching logic. Setup is as follows:

- PCI flash will be used as flash
- Two node connected using 40GB Melanox cards
- Gluster version 3.7.6

With fresh installation of Gluster, using fio with libgapi with bs=1M/4M,
seq_write i am able to achieve speeds up to 2.4GB/s from user point of
view. When i check the raw flash speed, seeing bandwidth of 1.1GB/s but
flash is capable of getting speeds of 1.5GB/s.
Before putting in our own cache, i disabled Gluster  caching by disabling
below options and measured fio speed

gluster vol set $vol_name performance.io-cache off
gluster vol set $vol_name performance.write-behind off
gluster vol set $vol_name performance.flush-behind off
service glusterd restart

[ $vol_name was my name name passed in shell script ]

With the above parameters disabled, i was drop in fio speed ( as expected )
to around 1.4GB/s but raw flash speed measured was around 250-350MB/s using
iostat. This shows that Gluster caching might not be completely disabled or
i am missing some piece of information.
Please provide your valuable inputs if any other settings need to be
disabled.

-Prashanth
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20151217/2b1af442/attachment.html>

Hi,
Can anyone please provide your inputs on the issue?

-Prashanth


On Thu, Dec 17, 2015 at 11:33 AM, Prashanth kumar N <
prashanth.kumar.n at gmail.com> wrote:
> Hi,
> One of my gluster deployment requires to completely disable Gluster
> caching and put in proprietary caching logic. Setup is as follows:
>
> - PCI flash will be used as flash
> - Two node connected using 40GB Melanox cards
> - Gluster version 3.7.6
>
> With fresh installation of Gluster, using fio with libgapi with bs=1M/4M,
> seq_write i am able to achieve speeds up to 2.4GB/s from user point of
> view. When i check the raw flash speed, seeing bandwidth of 1.1GB/s but
> flash is capable of getting speeds of 1.5GB/s.
> Before putting in our own cache, i disabled Gluster  caching by disabling
> below options and measured fio speed
>
> gluster vol set $vol_name performance.io-cache off
> gluster vol set $vol_name performance.write-behind off
> gluster vol set $vol_name performance.flush-behind off
> service glusterd restart
>
> [ $vol_name was my name name passed in shell script ]
>
> With the above parameters disabled, i was drop in fio speed ( as expected
> ) to around 1.4GB/s but raw flash speed measured was around 250-350MB/s
> using iostat. This shows that Gluster caching might not be completely
> disabled or i am missing some piece of information.
> Please provide your valuable inputs if any other settings need to be
> disabled.
>
> -Prashanth
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20151222/f3a11924/attachment.html>