samba - Apr 2015 - Samba process pegging CPU

On Thu, Apr 16, 2015 at 02:33:20PM +0000, Donaldson Jeff
wrote:
> My apologies, went to paste and accidentally sent the message...see below
for rest of detail.
> 
> 
> Greetings,
> 
> 
> I'm currently experiencing an issue with two of my three Samba servers
where one Samba process is pegging the CPU usage and eating up a lot of virtual
and resident memory. While this is happening I get a "ERROR(runtime):
DsReplicaGetInfo of type 0 failed - (-1073610723,
'NT_STATUS_RPC_PROTOCOL_ERROR')" error when I try to run the
samba-tool drs showrepl command. When that singular process drops (albeit
temporarily) I can get the samba-tool command to complete. It shows everything
is replicating successfully. This is happening on both of my Samba 4.1.9
servers. One is Ubuntu Server 12.04 and the other is Ubuntu Server 14.04. It
seems to be pegged more on the 12.04 box though. I'm also seeing these
errors in 12.04 log.samba...
> 
> 
> [2015/04/16 09:54:16.470468,  1]
../source4/dsdb/kcc/kcc_deleted.c:134(kccsrv_check_deleted)
>   ../source4/dsdb/kcc/kcc_deleted.c:134: Failed to remove deleted object
DC=NCS-HPLAP28\0ADEL:ac84ad1f-a645-47db-873f-ccfccd9c621c,CN=Deleted
Objects,DC=DomainDnsZones,DC=ncs,DC=k12,DC=de,DC=us
> [2015/04/16 09:54:16.911186,  1]
../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>   ldb: Invalid data for index  DN=@INDEX:OBJECTCLASS:DNSNODE
> 
> 
> Could this just be an issue with re-indexing?
What does an strace log say ?

I did, but I don't know how to decypher it. Here's a 1.5 second snippet
of the the strace -p PID command.


Jeff Donaldson
Technology Director
Newark Charter School
jeff.donaldson at ncs.k12.de.us
(302) 369-2001 ext: 425

________________________________________
From: Jeremy Allison <jra at samba.org>
Sent: Thursday, April 16, 2015 6:49 PM
To: Donaldson Jeff
Cc: samba at lists.samba.org
Subject: Re: [Samba] Samba process pegging CPU

On Thu, Apr 16, 2015 at 02:33:20PM +0000, Donaldson Jeff
wrote:
> My apologies, went to paste and accidentally sent the message...see below
for rest of detail.
>
>
> Greetings,
>
>
> I'm currently experiencing an issue with two of my three Samba servers
where one Samba process is pegging the CPU usage and eating up a lot of virtual
and resident memory. While this is happening I get a "ERROR(runtime):
DsReplicaGetInfo of type 0 failed - (-1073610723,
'NT_STATUS_RPC_PROTOCOL_ERROR')" error when I try to run the
samba-tool drs showrepl command. When that singular process drops (albeit
temporarily) I can get the samba-tool command to complete. It shows everything
is replicating successfully. This is happening on both of my Samba 4.1.9
servers. One is Ubuntu Server 12.04 and the other is Ubuntu Server 14.04. It
seems to be pegged more on the 12.04 box though. I'm also seeing these
errors in 12.04 log.samba...
>
>
> [2015/04/16 09:54:16.470468,  1]
../source4/dsdb/kcc/kcc_deleted.c:134(kccsrv_check_deleted)
>   ../source4/dsdb/kcc/kcc_deleted.c:134: Failed to remove deleted object
DC=NCS-HPLAP28\0ADEL:ac84ad1f-a645-47db-873f-ccfccd9c621c,CN=Deleted
Objects,DC=DomainDnsZones,DC=ncs,DC=k12,DC=de,DC=us
> [2015/04/16 09:54:16.911186,  1]
../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>   ldb: Invalid data for index  DN=@INDEX:OBJECTCLASS:DNSNODE
>
>
> Could this just be an issue with re-indexing?
What does an strace log say ?
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: strace output.txt
URL:
<http://lists.samba.org/pipermail/samba/attachments/20150417/a3656d87/attachment-0001.txt>

On Fri, Apr 17, 2015 at 03:08:12PM +0000, Donaldson Jeff
wrote:
> I did, but I don't know how to decypher it. Here's a 1.5 second
snippet of the the strace -p PID command.
All these:

brk(0x7f4d856de000)                     = 0x7f4d856de000
brk(0x7f4d856ff000)                     = 0x7f4d856ff000
brk(0x7f4d85720000)                     = 0x7f4d85720000

mean it's in a tight loop allocating memory.

Rebuild with -g (debug), reproduce the problem
then attach to the errent process using
gdb -p, and the type "bt" to get a backtrace.