On Thu, May 25, 2017 at 8:39 PM, Joe Julian <joe at julianfamily.org>
wrote:> Maybe hooks?
Yes, we were thinking of the same :)
Christopher,
Gluster has hook-scripts facility that admins can write and set those
to be run on certain events in Gluster. We have a event for volume
creation.
Here are the steps for using hook scripts.
1. deploy the gluster pods and create a cluster as you have already done.
2. on the kubernetes nodes that are running gluster pods(make sure
they are running now, because we want to write into the bind mount),
create a new file in location /var/lib/glusterd/hooks/1/create/post/
3. name of the fie could be S29disable-perf.sh , important part being
that the number should have capital S as first letter.
4. I tried out a sample script with content as below
```
#!/bin/bash
PROGNAME="Sdisable-perf"
OPTSPEC="volname:,gd-workdir:"
VOLCONFIGFILELOGFILEBASEPIDDIRGLUSTERD_WORKDIR
function parse_args () {
ARGS=$(getopt -l $OPTSPEC -name $PROGNAME $@)
eval set -- "$ARGS"
while true; do
case $1 in
--volname)
shift
VOL=$1
;;
--gd-workdir)
shift
GLUSTERD_WORKDIR=$1
;;
*)
shift
break
;;
esac
shift
done
}
function disable_perf_xlators () {
volname=$1
gluster volume set $volname performance.write-behind off
echo "executed and return is $?" >>
/var/lib/glusterd/hooks/1/create/post/log
}
echo "starting" >> /var/lib/glusterd/hooks/1/create/post/log
parse_args $@
disable_perf_xlators $VOL
```
5. set execute permissions on the file
I tried this out and it worked for me. Let us know if that helps!
Thanks,
Raghavendra Talur
>
>
> On May 25, 2017 6:48:04 AM PDT, Christopher Schmidt <fakod666 at
gmail.com>
> wrote:
>>
>> Hi Humble,
>>
>> thanks for that, it is really appreciated.
>>
>> In the meanwhile, using K8s 1.5, what can I do to disable the
performance
>> translator that doesn't work with Kafka? Maybe something while
generating
>> the Glusterfs container for Kubernetes?
>>
>> Best Christopher
>>
>> Humble Chirammal <hchiramm at redhat.com> schrieb am Do., 25. Mai
2017,
>> 09:36:
>>>
>>> On Thu, May 25, 2017 at 12:57 PM, Raghavendra Talur <rtalur at
redhat.com>
>>> wrote:
>>>>
>>>> On Thu, May 25, 2017 at 11:21 AM, Christopher Schmidt
>>>> <fakod666 at gmail.com> wrote:
>>>> > So this change of the Gluster Volume Plugin will make it
into K8s 1.7
>>>> > or
>>>> > 1.8. Unfortunately too late for me.
>>>> >
>>>> > Does anyone know how to disable performance translators by
default?
>>>>
>>>> Humble,
>>>>
>>>> Do you know of any way Christopher can proceed here?
>>>
>>>
>>> I am trying to get it in 1.7 branch, will provide an update here as
soon
>>> as its available.
>>>>
>>>>
>>>> >
>>>> >
>>>> > Raghavendra Talur <rtalur at redhat.com> schrieb am
Mi., 24. Mai 2017,
>>>> > 19:30:
>>>> >>
>>>> >> On Wed, May 24, 2017 at 4:08 PM, Christopher Schmidt
>>>> >> <fakod666 at gmail.com>
>>>> >> wrote:
>>>> >> >
>>>> >> >
>>>> >> > Vijay Bellur <vbellur at redhat.com>
schrieb am Mi., 24. Mai 2017 um
>>>> >> > 05:53
>>>> >> > Uhr:
>>>> >> >>
>>>> >> >> On Tue, May 23, 2017 at 1:39 AM, Christopher
Schmidt
>>>> >> >> <fakod666 at gmail.com>
>>>> >> >> wrote:
>>>> >> >>>
>>>> >> >>> OK, seems that this works now.
>>>> >> >>>
>>>> >> >>> A couple of questions:
>>>> >> >>> - What do you think, are all these
options necessary for Kafka?
>>>> >> >>
>>>> >> >>
>>>> >> >> I am not entirely certain what subset of
options will make it work
>>>> >> >> as I
>>>> >> >> do
>>>> >> >> not understand the nature of failure with
Kafka and the default
>>>> >> >> gluster
>>>> >> >> configuration. It certainly needs further
analysis to identify the
>>>> >> >> list
>>>> >> >> of
>>>> >> >> options necessary. Would it be possible for
you to enable one
>>>> >> >> option
>>>> >> >> after
>>>> >> >> the other and determine the configuration
that ?
>>>> >> >>
>>>> >> >>
>>>> >> >>>
>>>> >> >>> - You wrote that there have to be kind of
application profiles.
>>>> >> >>> So to
>>>> >> >>> find out, which set of options work is
currently a matter of
>>>> >> >>> testing
>>>> >> >>> (and
>>>> >> >>> hope)? Or are there any experiences for
MongoDB / ProstgreSQL /
>>>> >> >>> Zookeeper
>>>> >> >>> etc.?
>>>> >> >>
>>>> >> >>
>>>> >> >> Application profiles are work in progress. We
have a few that are
>>>> >> >> focused
>>>> >> >> on use cases like VM storage, block storage
etc. at the moment.
>>>> >> >>
>>>> >> >>>
>>>> >> >>> - I am using Heketi and Dynamik Storage
Provisioning together
>>>> >> >>> with
>>>> >> >>> Kubernetes. Can I set this volume options
somehow by default or
>>>> >> >>> by
>>>> >> >>> volume
>>>> >> >>> plugin?
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >> Adding Raghavendra and Michael to help
address this query.
>>>> >> >
>>>> >> >
>>>> >> > For me it would be sufficient to disable some (or
all) translators,
>>>> >> > for
>>>> >> > all
>>>> >> > volumes that'll be created, somewhere here:
>>>> >> >
https://github.com/gluster/gluster-containers/tree/master/CentOS
>>>> >> > This is the container used by the GlusterFS
DaemonSet for
>>>> >> > Kubernetes.
>>>> >>
>>>> >> Work is in progress to give such option at volume
plugin level. We
>>>> >> currently have a patch[1] in review for Heketi that
allows users to
>>>> >> set Gluster options using heketi-cli instead of going
into a Gluster
>>>> >> pod. Once this is in, we can add options in
storage-class of
>>>> >> Kubernetes that pass down Gluster options for every
volume created in
>>>> >> that storage-class.
>>>> >>
>>>> >> [1] https://github.com/heketi/heketi/pull/751
>>>> >>
>>>> >> Thanks,
>>>> >> Raghavendra Talur
>>>> >>
>>>> >> >
>>>> >> >>
>>>> >> >>
>>>> >> >> -Vijay
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >>>
>>>> >> >>>
>>>> >> >>> Thanks for you help... really
appreciated.. Christopher
>>>> >> >>>
>>>> >> >>> Vijay Bellur <vbellur at
redhat.com> schrieb am Mo., 22. Mai 2017 um
>>>> >> >>> 16:41
>>>> >> >>> Uhr:
>>>> >> >>>>
>>>> >> >>>> Looks like a problem with caching.
Can you please try by
>>>> >> >>>> disabling
>>>> >> >>>> all
>>>> >> >>>> performance translators? The
following configuration commands
>>>> >> >>>> would
>>>> >> >>>> disable
>>>> >> >>>> performance translators in the
gluster client stack:
>>>> >> >>>>
>>>> >> >>>> gluster volume set <volname>
performance.quick-read off
>>>> >> >>>> gluster volume set <volname>
performance.io-cache off
>>>> >> >>>> gluster volume set <volname>
performance.write-behind off
>>>> >> >>>> gluster volume set <volname>
performance.stat-prefetch off
>>>> >> >>>> gluster volume set <volname>
performance.read-ahead off
>>>> >> >>>> gluster volume set <volname>
performance.readdir-ahead off
>>>> >> >>>> gluster volume set <volname>
performance.open-behind off
>>>> >> >>>> gluster volume set <volname>
performance.client-io-threads off
>>>> >> >>>>
>>>> >> >>>> Thanks,
>>>> >> >>>> Vijay
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>> On Mon, May 22, 2017 at 9:46 AM,
Christopher Schmidt
>>>> >> >>>> <fakod666 at gmail.com> wrote:
>>>> >> >>>>>
>>>> >> >>>>> Hi all,
>>>> >> >>>>>
>>>> >> >>>>> has anyone ever successfully
deployed a Kafka (Cluster) on
>>>> >> >>>>> GlusterFS
>>>> >> >>>>> volumes?
>>>> >> >>>>>
>>>> >> >>>>> I my case it's a Kafka
Kubernetes-StatefulSet and a Heketi
>>>> >> >>>>> GlusterFS.
>>>> >> >>>>> Needless to say that I am getting
a lot of filesystem related
>>>> >> >>>>> exceptions like this one:
>>>> >> >>>>>
>>>> >> >>>>> Failed to read `log header` from
file channel
>>>> >> >>>>> `sun.nio.ch.FileChannelImpl at
67afa54a`. Expected to read 12
>>>> >> >>>>> bytes,
>>>> >> >>>>> but
>>>> >> >>>>> reached end of file after reading
0 bytes. Started read from
>>>> >> >>>>> position
>>>> >> >>>>> 123065680.
>>>> >> >>>>>
>>>> >> >>>>> I limited the amount of
exceptions with the
>>>> >> >>>>> log.flush.interval.messages=1
option, but not all...
>>>> >> >>>>>
>>>> >> >>>>> best Christopher
>>>> >> >>>>>
>>>> >> >>>>>
>>>> >> >>>>>
_______________________________________________
>>>> >> >>>>> Gluster-users mailing list
>>>> >> >>>>> Gluster-users at gluster.org
>>>> >> >>>>>
http://lists.gluster.org/mailman/listinfo/gluster-users
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >
>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>> Humble
>>>
>>> Sr.Software Engineer - Red Hat Storage Engineering
>>> website: http://humblec.com
>
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users