samba - Jul 2007 - What are my smbd's doing ? (was Re: secrets.tdb locking fun!)

James R Grinter wrote:-
>
>On Fri, Jul 13, 2007 at 09:39:37AM +0100, Mac wrote:
>> On one previous occasion, the whole thing seemed to grind to a virtual
>> halt, and we suspected (but couldn't prove) that a locking battle
over
>> (something like) secrets.tdb was to blame.
>
>(I recognise that symptom, see below)
> 
>> Something that stands out is a huge ammount of time (1.9 - 2.0 seconds)
>> spent reading a file in /var/tmp  (/var/tmp/firesun1_044_0). This seems
>> to happen quite often (maybe every few seconds). This file isn't
terribly
>> (human) readable, and I don't recall Samba ever having needed
>> a temp file outside of 'var/locks'.
>
>that's not one I recognise.
Closer inspection of the 'truss' output reveals that each process,
having 'read' its way through the entire file, then writes 81 (ish)
bytes out to the end of the file. These bytes look a bit like a machine
trust account name.

Here's an extract from the 'truss' output.

write(29, "\0\0\018 g k m t x 2 j $".., 81)     = 81
fdsync(29, O_RDONLY|O_SYNC)                     = 0

Repeated 'ls -l' of the file in /var/tmp reveals that it both shrinks
and
grows over time.

>Try running 'smbd -b' to look at the build paths in your binary
(I'm
>assuming it's one you've built in-house, but it may be worthwhile
>checking anyway, to see if anything with /var/tmp shows up, or even to
>make sure that you're running the build of the binary that you think
>you should be running. There's nothing like that path on my
>3.0.25-running or earlier systems.)

Me either.  No sign of 'var' or 'var/tmp' anywhere (except
'/var/adm/utmp') in any of the binaries we compiled.

I'm guessing it is picking up an Environment variable?  Or there's some
kernel system call that made this file (a bit like 'mktemp').

>We had problems with large amounts of fcntl locking on Solaris 8, which
>create exactly the sort of 'virtual halt' that you describe and
>undoubtedly that bug is in earlier Solaris 9 kernels too (bug 4735093
>and 2057706. A quick check on Sunsolve suggests it was fixed in
>112233-04 for SunOS 5.9/sparc.)
We're currently running 112233-12, so I don't think that's likely to
be
our issue.


                               Mac
          Assistant Systems Administrator @nibsc.ac.uk
                           mac@nibsc.ac.uk
   Work: +44 1707 641565          Everything else: +44 7956 237670 (anytime)

On Fri, Jul 13, 2007 at 12:48:14PM +0100, Mac wrote:
> 
> James R Grinter wrote:-
> >
> >On Fri, Jul 13, 2007 at 09:39:37AM +0100, Mac wrote:
> >> On one previous occasion, the whole thing seemed to grind to a
virtual
> >> halt, and we suspected (but couldn't prove) that a locking
battle over
> >> (something like) secrets.tdb was to blame.
> >
> >(I recognise that symptom, see below)
> > 
> >> Something that stands out is a huge ammount of time (1.9 - 2.0
seconds)
> >> spent reading a file in /var/tmp  (/var/tmp/firesun1_044_0). This
seems
> >> to happen quite often (maybe every few seconds). This file
isn't terribly
> >> (human) readable, and I don't recall Samba ever having needed
> >> a temp file outside of 'var/locks'.
> >
> >that's not one I recognise.
> 
> Closer inspection of the 'truss' output reveals that each process,
> having 'read' its way through the entire file, then writes 81 (ish)
> bytes out to the end of the file. These bytes look a bit like a machine
> trust account name.
> 
> Here's an extract from the 'truss' output.
> 
> write(29, "\0\0\018 g k m t x 2 j $".., 81)     = 81
> fdsync(29, O_RDONLY|O_SYNC)                     = 0
> 
> Repeated 'ls -l' of the file in /var/tmp reveals that it both
shrinks and
> grows over time.
Looks like this might be a utmp record to me (maybe libc trying to
create one) ? Have you tried compiling with utmp turned off ?

Jeremy.