I have a CentOS 7 system that I run a home grown python daemon on. I run this same daemon on many other systems without any incident. On this one system the daemon seems to die or be killed every day around 3:30am. There is nothing it its log or any system logs that tell me why it dies. However in /var/log/messages every day I see something like this: May 10 03:35:58 localhost pure-ftpd: (tool at xx.xx.xx.xx) [NOTICE] /usrMay 10 03:57:57 localhost pure-ftpd: (tool at xx.xx.xx.xx) [NOTICE] /usr/local/motor/data//B31/today/Images/CP0982436.00C_T6PH8_M0-R_1/T6PH8_M0-RN_TT6PH8_M0-R_P4_M1_FX-1_FY4_RR1_TR1_Ver3.jpg uploaded (90666 bytes, 8322.34KB/sec) Notice how the message that was being printed at 03:35:58 is truncated mid-message and the next message is at 03:57:57 ? there are always messages like around the time when the daemon dies. What could be going on here?
Am 10.05.2017 um 20:40 schrieb Larry Martell:> I have a CentOS 7 system that I run a home grown python daemon on. I > run this same daemon on many other systems without any incident. On > this one system the daemon seems to die or be killed every day around > 3:30am. There is nothing it its log or any system logs that tell me > why it dies. However in /var/log/messages every day I see something > like this: > > May 10 03:35:58 localhost pure-ftpd: (tool at xx.xx.xx.xx) [NOTICE] > /usrMay 10 03:57:57 localhost pure-ftpd: (tool at xx.xx.xx.xx) [NOTICE] > /usr/local/motor/data//B31/today/Images/CP0982436.00C_T6PH8_M0-R_1/T6PH8_M0-RN_TT6PH8_M0-R_P4_M1_FX-1_FY4_RR1_TR1_Ver3.jpg > uploaded (90666 bytes, 8322.34KB/sec) > > > Notice how the message that was being printed at 03:35:58 is truncated > mid-message and the next message is at 03:57:57 ? there are always > messages like around the time when the daemon dies. What could be > going on here?Have you checked /etc/cron.daily/ for a cron job to restart the pure-ftpd service? /etc/cron.daily is called from /etc/anacrontab with some randomness so that the execution time varies. Alexander
On Wed, May 10, 2017 at 02:40:04PM -0400, Larry Martell wrote:> I have a CentOS 7 system that I run a home grown python daemon on. I > run this same daemon on many other systems without any incident. On > this one system the daemon seems to die or be killed every day around > 3:30am. There is nothing it its log or any system logs that tell me > why it dies. However in /var/log/messages every day I see something > like this:How are you starting this daemon? Can you check the journal? Perhaps you'll see more useful information than what you see in the syslogs? -- Jonathan Billings <billings at negate.org>
On Wed, May 10, 2017 at 3:05 PM, Alexander Dalloz <ad+lists at uni-x.org> wrote:> Am 10.05.2017 um 20:40 schrieb Larry Martell: >> >> I have a CentOS 7 system that I run a home grown python daemon on. I >> run this same daemon on many other systems without any incident. On >> this one system the daemon seems to die or be killed every day around >> 3:30am. There is nothing it its log or any system logs that tell me >> why it dies. However in /var/log/messages every day I see something >> like this: >> >> May 10 03:35:58 localhost pure-ftpd: (tool at xx.xx.xx.xx) [NOTICE] >> /usrMay 10 03:57:57 localhost pure-ftpd: (tool at xx.xx.xx.xx) [NOTICE] >> >> /usr/local/motor/data//B31/today/Images/CP0982436.00C_T6PH8_M0-R_1/T6PH8_M0-RN_TT6PH8_M0-R_P4_M1_FX-1_FY4_RR1_TR1_Ver3.jpg >> uploaded (90666 bytes, 8322.34KB/sec) >> >> >> Notice how the message that was being printed at 03:35:58 is truncated >> mid-message and the next message is at 03:57:57 ? there are always >> messages like around the time when the daemon dies. What could be >> going on here? > > > Have you checked /etc/cron.daily/ for a cron job to restart the pure-ftpd > service? /etc/cron.daily is called from /etc/anacrontab with some randomness > so that the execution time varies.I will check that, but the the pure-ftpd service is not my daemon that is getting killed - I was thinking the sudden stopping of the logging and subsequent time gap would be some sort of clue as to what is happing at that time. This is a system I do not have remote access to and it's in Japan so I have to ask someone there to do things for me. Such a pain.
On Wed, May 10, 2017 at 3:07 PM, Jonathan Billings <billings at negate.org> wrote:> On Wed, May 10, 2017 at 02:40:04PM -0400, Larry Martell wrote: >> I have a CentOS 7 system that I run a home grown python daemon on. I >> run this same daemon on many other systems without any incident. On >> this one system the daemon seems to die or be killed every day around >> 3:30am. There is nothing it its log or any system logs that tell me >> why it dies. However in /var/log/messages every day I see something >> like this: > > How are you starting this daemon?I am using code something like this: https://gist.github.com/slor/5946334.> Can you check the journal? Perhaps > you'll see more useful information than what you see in the syslogs?Thanks, I will do that.