Gluster users - Feb 2010 - gluster and system options for logging

Hi all,

I'm currently in the process of testing a Gluster 3.0.0 setup (4 nodes, 
2 servers, 2 clients, client-side replication), and have run into the 
occasional hiccup (and at least one big crash).

Unfortunately, the logged remnants of these incidents leave something to 
be desired, and as anyone who's tried to track down a crash knows, the 
more log data you have, the better.  Thus, i'd like to know from the 
Gluster community and devs what options, functionalities, and features i 
should be using in order to ensure that i'm getting all the log data i 
can when something goes wrong.

Beyond increasing the Gluster log level, what other sorts of things 
should i be doing ?  What sorts of logging have you done during your tests ?

Thanks !


-- 
Daniel Maher <dma+gluster AT witbe DOT net>

* Apart from logging, it would be better to build glusterfs with CFLAGS set
to '-g -O0', so that it is built with debugging information and no code
optimization. And when crash happens, back trace got by using gdb with
command 'bt full' will be helpful for us to debug the issue.

Following may not be useful to debug a crash, but they are helpful to debug
other issues like memory leak etc.
* debug/trace translator when used judisciously can help debug issues, since
we can know that what each translator does to the results of calls.

* process state dump of glusterfs got by sending SIGUSR1 (kill -SIGUSR1
<glusterfs pid>) is useful to debug memory leaks.

* while debugging issues involved with libglusterfsclient or booster, a
loglevel of TRACE will be useful.

regards,
On Fri, Feb 5, 2010 at 1:50 PM, Daniel Maher
<dma+gluster at witbe.net<dma%2Bgluster at
witbe.net>
> wrote:
> Hi all,
>
> I'm currently in the process of testing a Gluster 3.0.0 setup (4 nodes,
2
> servers, 2 clients, client-side replication), and have run into the
> occasional hiccup (and at least one big crash).
>
> Unfortunately, the logged remnants of these incidents leave something to be
> desired, and as anyone who's tried to track down a crash knows, the
more log
> data you have, the better.  Thus, i'd like to know from the Gluster
> community and devs what options, functionalities, and features i should be
> using in order to ensure that i'm getting all the log data i can when
> something goes wrong.
>
> Beyond increasing the Gluster log level, what other sorts of things should
> i be doing ?  What sorts of logging have you done during your tests ?
>
> Thanks !
>
>
> --
> Daniel Maher <dma+gluster AT witbe DOT net>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>


-- 
Raghavendra G