Fedora directory users - Oct 2009 - [389-users] server crash (389 server 1.2.1 on FC10 domu / centos 5.3 dom0)

Hello,

We''ve recently migrated our old FDS 1.0.2/FC4 setup to a new one
hosted on a xen virtualised FC10. The dom0 is centos 5.3 and 389
server version is 1.2.1.

It has been working about a week, but we have recently faced some crashes.

First ones were the day after a schema modification on the master
which had not been weel propagated on the slaves. This could be the
explanation but I''m not really sure as it worked well several hours.

The last one was today when using the console.

Problem is we have no logs. Server suddenly freezes and is no more
available via ssh , vnc or xen console.

Would somebody have an idea about which steps to debug this situation ?
Could it be that virtualizing FC1 on Centos 5.3 was a bad idea ?

Regards,

--
Mikael Kermorgant

Mikael Kermorgant wrote:
> Hello,
>
> We''ve recently migrated our old FDS 1.0.2/FC4 setup to a new one
> hosted on a xen virtualised FC10. The dom0 is centos 5.3 and 389
> server version is 1.2.1.
>
> It has been working about a week, but we have recently faced some crashes.
>
> First ones were the day after a schema modification on the master
> which had not been weel propagated on the slaves.
How did you modify the schema on the master?
> This could be the
> explanation but I''m not really sure as it worked well several
hours.
>
> The last one was today when using the console.
>   
Try console -D 9 -f console.log if you can reproduce the
> Problem is we have no logs. Server suddenly freezes and is no more
> available via ssh , vnc or xen console.
>   
So the entire VM freezes, not just the directory server?
So you see the server crashing and the VM freezing?
> Would somebody have an idea about which steps to debug this situation ?
>   
Anything in the error log?  /var/log/messages ?  any other log files in 
/var/log?
> Could it be that virtualizing FC1 on Centos 5.3 was a bad idea ?
>
> Regards,
>
> --
> Mikael Kermorgant
>
> --
> 389 users mailing list
> 389-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

On Fri, Oct 2, 2009 at 5:24 PM, Rich Megginson <rmeggins@redhat.com>
wrote:
> Mikael Kermorgant wrote:
>>
>> Hello,
>>
>> First ones were the day after a schema modification on the master
>> which had not been weel propagated on the slaves.
>
> How did you modify the schema on the master?
Via the console, by adding an attribute.
Had to copy the schema files to the slaves later.
>>
>> This could be the
>> explanation but I''m not really sure as it worked well several
hours.
>>
>> The last one was today when using the console.
>>
>
> Try console -D 9 -f console.log if you can reproduce the
Ok, I''ll try that next time.
>> Problem is we have no logs. Server suddenly freezes and is no more
>> available via ssh , vnc or xen console.
>>
>
> So the entire VM freezes, not just the directory server?
> So you see the server crashing and the VM freezing?
Yes, sort of. I still had control of the cursor but could not ctrl+c
the console on the xterm

>> Would somebody have an idea about which steps to debug this situation ?
>>
>
> Anything in the error log?  /var/log/messages ?  any other log files in
> /var/log?
Only in /var/log/xen/xend.log which indicates that this is quite offtopic.

I''ll see what I can find on the xen side,

Thanks,

Mikael

Mikael Kermorgant wrote:
> On Fri, Oct 2, 2009 at 5:24 PM, Rich Megginson <rmeggins@redhat.com>
wrote:
>   
>> Mikael Kermorgant wrote:
>>     
>>> Hello,
>>>
>>> First ones were the day after a schema modification on the master
>>> which had not been weel propagated on the slaves.
>>>       
>> How did you modify the schema on the master?
>>     
>
> Via the console, by adding an attribute.
> Had to copy the schema files to the slaves later.
>   
Were there any messages in the errors logs on the supplier or consumers 
about not being able to replicate the schema?
>   
>>> This could be the
>>> explanation but I''m not really sure as it worked well
several hours.
>>>
>>> The last one was today when using the console.
>>>
>>>       
>> Try console -D 9 -f console.log if you can reproduce the
>>     
>
> Ok, I''ll try that next time.
>
>   
>>> Problem is we have no logs. Server suddenly freezes and is no more
>>> available via ssh , vnc or xen console.
>>>
>>>       
>> So the entire VM freezes, not just the directory server?
>> So you see the server crashing and the VM freezing?
>>     
>
> Yes, sort of. I still had control of the cursor but could not ctrl+c
> the console on the xterm
>
>
>   
>>> Would somebody have an idea about which steps to debug this
situation ?
>>>
>>>       
>> Anything in the error log?  /var/log/messages ?  any other log files in
>> /var/log?
>>     
>
> Only in /var/log/xen/xend.log which indicates that this is quite offtopic.
>
> I''ll see what I can find on the xen side,
>
> Thanks,
>
> Mikael
>
> --
> 389 users mailing list
> 389-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

Hello,

On Fri, Oct 2, 2009 at 10:40 PM, Rich Megginson <rmeggins@redhat.com>
wrote:
>>>> First ones were the day after a schema modification on the
master
>>>> which had not been weel propagated on the slaves.
>>>>
>>>
>>> How did you modify the schema on the master?
>>>
>>
>> Via the console, by adding an attribute.
>> Had to copy the schema files to the slaves later.
>>
>
> Were there any messages in the errors logs on the supplier or consumers
> about not being able to replicate the schema?
Yep, there were some :

[23/Sep/2009:10:59:58 +0200] NSMMReplicationPlugin -
agmtlist_add_callback: Can''t start agreement "cn=supann0& -
bsupann01,cn=replica,cn="dc=dede, dc=deded, dc=fr",cn=mapping
tree,cn=config"

And many of these :

[23/Sep/2009:14:37:13 +0200] NSMMReplicationPlugin -
agmt="cn=bsupann01" (192:389): Replica has a different generation ID
than the local data.
[23/Sep/2009:14:37:17 +0200] NSMMReplicationPlugin -
agmt="cn=bsupann01" (192:389): Replica has a different generation ID
than the local data.


However, I now really want to migrate to some safer virtualisation
environment. Maybe a bit over pessimistic but I''m affraid of the FC11
domU on centos dom0...


Regards,

Mikael

Mikael Kermorgant wrote:
> Hello,
>
> On Fri, Oct 2, 2009 at 10:40 PM, Rich Megginson <rmeggins@redhat.com>
wrote:
>
>   
>>>>> First ones were the day after a schema modification on the
master
>>>>> which had not been weel propagated on the slaves.
>>>>>
>>>>>           
>>>> How did you modify the schema on the master?
>>>>
>>>>         
>>> Via the console, by adding an attribute.
>>> Had to copy the schema files to the slaves later.
>>>
>>>       
>> Were there any messages in the errors logs on the supplier or consumers
>> about not being able to replicate the schema?
>>     
>
> Yep, there were some :
>
> [23/Sep/2009:10:59:58 +0200] NSMMReplicationPlugin -
> agmtlist_add_callback: Can''t start agreement "cn=supann0&
-
> bsupann01,cn=replica,cn="dc=dede, dc=deded, dc=fr",cn=mapping
> tree,cn=config"
>   
That''s really odd.  Do you get that at startup?  That indicates some 
sort of configuration problem.  You could try starting the server with 
/usr/lib/dirsrv/slapd-instance/start-slapd -d 8192 to see if there is 
more detailed information.
> And many of these :
>
> [23/Sep/2009:14:37:13 +0200] NSMMReplicationPlugin -
> agmt="cn=bsupann01" (192:389): Replica has a different generation
ID
> than the local data.
> [23/Sep/2009:14:37:17 +0200] NSMMReplicationPlugin -
> agmt="cn=bsupann01" (192:389): Replica has a different generation
ID
> than the local data.
>   
This usually means the consumer has not been
initialized.
>
> However, I now really want to migrate to some safer virtualisation
> environment. Maybe a bit over pessimistic but I''m affraid of the
FC11
> domU on centos dom0...
>
>
> Regards,
>
> Mikael
>
> --
> 389 users mailing list
> 389-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>