On 27 Feb 2020, at 14:42, Jonathan Billings wrote:> On Feb 27, 2020, at 08:01, Tobias Kirchhofer <collect at shift.agency> > wrote: >> >> ?Hi, >> >> we experience difficulties with crond behaviour sending mail since >> CentOS 8.1. The cron job is the same like we used in CentOS 7. >> >> crontab -l >> /usr/bin/python3 -c 'import random; import time; >> time.sleep(random.random() * 3600)' && /usr/local/bin/backup.sh > > Is this literally what your crontab looks like? Because that?s not > valid crontab syntax.This is what it is literally: 0 5 * * * /usr/bin/python3 -c 'import random; import time; time.sleep(random.random() * 3600)' && /usr/local/bin/backup.sh>> backup.sh writes the backup to the remote backup server. There are >> around 30 machines with the same cron job. The python part spreads >> the jobs over an hour so that the backup server is not struck at once >> from about 30 machines. >> >> The change in behaviour is that crond sends no mail anymore. It is >> because of the logical and (&&). Without this crond sends mails. >> >> What has changed in CentOS 8 and does anyone has an idea how we could >> fix it? > > There aren?t any significant changes in ?cronie? in 8.1, looking > at the spec file. > > Assuming the crontab you wrote above included the time spec too, I?d > check to make sure the first command isn?t exiting with a non-zero > exit code.The command chain is running propery, STDOUT output is visible in system log (excerpt from logwatch email): ``` --------------------- Cron Begin ------------------------ **Unmatched Entries** CMDOUT (### Starting backup. Host: host.example.com Backupserver: 10.9.1.5 Path: /borgbackup/vm/host-example-com Date: Fri Mar 20 05:13:46 CET 2020 ###) CMDOUT (Creating archive at "borg at IP:/borgbackup/vm/host-example-com::{now:%Y-%m-%d_%H:%M}") ? ``` Its just not sending an email. What is additionally irritating is that it is only on a few machines. All machines are identically provisioned with ansible. The situation is not mission critical but it should work. Strange. Tobias> -- > Jonathan Billings <billings at negate.org> > > > _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos-- collect at shift.agency
>>> we experience difficulties with crond behaviour sending mail since >>> CentOS 8.1. The cron job is the same like we used in CentOS 7.Meanwhile we found the reason for the bug - actually we do not know if it is related to a specific version of CentOS or a specific kind of command as cron job. Let me explain what we have: - sssd for ssh login of ldap user - crond for cron jobs :) If we stop sssd and restart crond cron starts to send mails again! We started with sssd on newly provisioned machines with CentOS 8. We do not know if this is the same on CentOS 7. We send mails only to root. So no remote user is involved in cron. From our perspective it is a bug. How could we dive deeper to find the specific reason? What do you think about this? Tobias On 21 Mar 2020, at 11:56, Tobias Kirchhofer wrote:> On 27 Feb 2020, at 14:42, Jonathan Billings wrote: > >> On Feb 27, 2020, at 08:01, Tobias Kirchhofer <collect at shift.agency> >> wrote: >>> >>> ?Hi, >>> >>> we experience difficulties with crond behaviour sending mail since >>> CentOS 8.1. The cron job is the same like we used in CentOS 7. >>> >>> crontab -l >>> /usr/bin/python3 -c 'import random; import time; >>> time.sleep(random.random() * 3600)' && /usr/local/bin/backup.sh >> >> Is this literally what your crontab looks like? Because that?s >> not valid crontab syntax. > > This is what it is literally: > > 0 5 * * * /usr/bin/python3 -c 'import random; import time; > time.sleep(random.random() * 3600)' && /usr/local/bin/backup.sh > >>> backup.sh writes the backup to the remote backup server. There are >>> around 30 machines with the same cron job. The python part spreads >>> the jobs over an hour so that the backup server is not struck at >>> once from about 30 machines. >>> >>> The change in behaviour is that crond sends no mail anymore. It is >>> because of the logical and (&&). Without this crond sends mails. >>> >>> What has changed in CentOS 8 and does anyone has an idea how we >>> could fix it? >> >> There aren?t any significant changes in ?cronie? in 8.1, >> looking at the spec file. >> >> Assuming the crontab you wrote above included the time spec too, >> I?d check to make sure the first command isn?t exiting with a >> non-zero exit code. > > The command chain is running propery, STDOUT output is visible in > system log (excerpt from logwatch email): > > > ``` > --------------------- Cron Begin ------------------------ > > **Unmatched Entries** > CMDOUT (### Starting backup. Host: host.example.com Backupserver: > 10.9.1.5 Path: /borgbackup/vm/host-example-com Date: Fri Mar 20 > 05:13:46 CET 2020 ###) > CMDOUT (Creating archive at > "borg at IP:/borgbackup/vm/host-example-com::{now:%Y-%m-%d_%H:%M}") > ? > ``` > > Its just not sending an email. > > What is additionally irritating is that it is only on a few machines. > All machines are identically provisioned with ansible. > > The situation is not mission critical but it should work. Strange. > > Tobias-- collect at shift.agency
On 5 Apr 2020, at 21:20, Tobias Kirchhofer wrote:>>>> we experience difficulties with crond behaviour sending mail since >>>> CentOS 8.1. The cron job is the same like we used in CentOS 7. > > Meanwhile we found the reason for the bug - actually we do not know if > it is related to a specific version of CentOS or a specific kind of > command as cron job. > > Let me explain what we have: > > - sssd for ssh login of ldap user > - crond for cron jobs :) > > If we stop sssd and restart crond cron starts to send mails again! > > We started with sssd on newly provisioned machines with CentOS 8. We > do not know if this is the same on CentOS 7. > > We send mails only to root. So no remote user is involved in cron. > > From our perspective it is a bug. How could we dive deeper to find the > specific reason?To sum it up: - Install CentOS 8 - Enabled and started crond - crond sends emails properly - Enable and start sssd - crond stops sending emails and starts journal logging - Restart crond (or reboot) - crond sends emails and stops journal logging It is a matter of order. At boot time crond starts after sssd. This situation is bearable if you know it but has cost us some hours. Thanks for reading and sorry for this public clarification process ;) Tobias> What do you think about this? > > Tobias > > > On 21 Mar 2020, at 11:56, Tobias Kirchhofer wrote: > >> On 27 Feb 2020, at 14:42, Jonathan Billings wrote: >> >>> On Feb 27, 2020, at 08:01, Tobias Kirchhofer <collect at shift.agency> >>> wrote: >>>> >>>> ?Hi, >>>> >>>> we experience difficulties with crond behaviour sending mail since >>>> CentOS 8.1. The cron job is the same like we used in CentOS 7. >>>> >>>> crontab -l >>>> /usr/bin/python3 -c 'import random; import time; >>>> time.sleep(random.random() * 3600)' && /usr/local/bin/backup.sh >>> >>> Is this literally what your crontab looks like? Because that?s >>> not valid crontab syntax. >> >> This is what it is literally: >> >> 0 5 * * * /usr/bin/python3 -c 'import random; import time; >> time.sleep(random.random() * 3600)' && /usr/local/bin/backup.sh >> >>>> backup.sh writes the backup to the remote backup server. There are >>>> around 30 machines with the same cron job. The python part spreads >>>> the jobs over an hour so that the backup server is not struck at >>>> once from about 30 machines. >>>> >>>> The change in behaviour is that crond sends no mail anymore. It is >>>> because of the logical and (&&). Without this crond sends mails. >>>> >>>> What has changed in CentOS 8 and does anyone has an idea how we >>>> could fix it? >>> >>> There aren?t any significant changes in ?cronie? in 8.1, >>> looking at the spec file. >>> >>> Assuming the crontab you wrote above included the time spec too, >>> I?d check to make sure the first command isn?t exiting with a >>> non-zero exit code. >> >> The command chain is running propery, STDOUT output is visible in >> system log (excerpt from logwatch email): >> >> >> ``` >> --------------------- Cron Begin ------------------------ >> >> **Unmatched Entries** >> CMDOUT (### Starting backup. Host: host.example.com Backupserver: >> 10.9.1.5 Path: /borgbackup/vm/host-example-com Date: Fri Mar 20 >> 05:13:46 CET 2020 ###) >> CMDOUT (Creating archive at >> "borg at IP:/borgbackup/vm/host-example-com::{now:%Y-%m-%d_%H:%M}") >> ? >> ``` >> >> Its just not sending an email. >> >> What is additionally irritating is that it is only on a few machines. >> All machines are identically provisioned with ansible. >> >> The situation is not mission critical but it should work. Strange. >> >> Tobias-- collect at shift.agency