CentOS - Nov 2019 - Issue with "ExecStartPost" attribute in systemd daemon faile

Hello,

I'm trying to configure a daemon (I'm doing tests with "crond"
daemon) to send me an email after daemon restart. My "crond.service"
file is:
# /etc/systemd/system/crond.service
[Unit]
Description=Command Scheduler
After=auditd.service systemd-user-sessions.service time-sync.target
#OnFailure=crond-notify-email@%i.service

[Service]
EnvironmentFile=/etc/sysconfig/crond
ExecStart=/usr/sbin/crond -n $CRONDARGS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartSec=10s
ExecStartPost=/bin/sh -c "/etc/systemd/system/test.sh"

[Install]
WantedBy=multi-user.target
My "test.sh" is very simple:
#!/bin/sh
echo "CROND is restarting" | /usr/bin/mailx -s "crond failure
notification" myemailaddress at mydomain

As you can see, I have added lines "Restart=on-failure",
"RestartSec=10s" and " ExecStartPost=/bin/sh -c
"/etc/systemd/system/test.sh"" to the original crond.service
daemon file. Then, I run "systemctl daemon-reconfigure" and, from one
console, I run "kill -9 `pidof cron`" for restarting crond daemon.
After it, I receive an email... But now, if I run again ""kill -9
`pidof cron`", I don't receive any mail... I have notice that if I run
"systemctl daemon-reload" and then kill crond process, mail is sent
perfectly... but if I don't run "systemctl daemon-reload", mail is
sent ONLY first time...

Why?

Thanks.

Am 21.11.19 um 14:28 schrieb Gesti? Servidors:
> Hello,
> 
> I'm trying to configure a daemon (I'm doing tests with
"crond" daemon) to send me an email after daemon restart. My
"crond.service" file is:
> # /etc/systemd/system/crond.service
> [Unit]
> Description=Command Scheduler
> After=auditd.service systemd-user-sessions.service time-sync.target
> #OnFailure=crond-notify-email@%i.service
> 
> [Service]
> EnvironmentFile=/etc/sysconfig/crond
> ExecStart=/usr/sbin/crond -n $CRONDARGS
> ExecReload=/bin/kill -HUP $MAINPID
> KillMode=process
> Restart=on-failure
> RestartSec=10s
> ExecStartPost=/bin/sh -c "/etc/systemd/system/test.sh"
> 
> [Install]
> WantedBy=multi-user.target
> My "test.sh" is very simple:
> #!/bin/sh
> echo "CROND is restarting" | /usr/bin/mailx -s "crond
failure notification" myemailaddress at mydomain
> 
> As you can see, I have added lines "Restart=on-failure",
"RestartSec=10s" and " ExecStartPost=/bin/sh -c
"/etc/systemd/system/test.sh"" to the original crond.service
daemon file. Then, I run "systemctl daemon-reconfigure" and, from one
console, I run "kill -9 `pidof cron`" for restarting crond daemon.
After it, I receive an email... But now, if I run again ""kill -9
`pidof cron`", I don't receive any mail... I have notice that if I run
"systemctl daemon-reload" and then kill crond process, mail is sent
perfectly... but if I don't run "systemctl daemon-reload", mail is
sent ONLY first time...
> 
> Why?
What is "status" showing (systemctl status crond)? For both cases 
(first/second kill)? Any  differences?

--
Leon

Gesti? Servidors wrote:
> 
> Hello,
> 
> I'm trying to configure a daemon (I'm doing tests with
"crond" daemon) to send me an email after daemon restart. My
"crond.service" file is:
> # /etc/systemd/system/crond.service
> [Unit]
> Description=Command Scheduler
> After=auditd.service systemd-user-sessions.service time-sync.target
> #OnFailure=crond-notify-email@%i.service
> 
> [Service]
> EnvironmentFile=/etc/sysconfig/crond
> ExecStart=/usr/sbin/crond -n $CRONDARGS
> ExecReload=/bin/kill -HUP $MAINPID
> KillMode=process
> Restart=on-failure
> RestartSec=10s
> ExecStartPost=/bin/sh -c "/etc/systemd/system/test.sh"
> 
> [Install]
> WantedBy=multi-user.target
> My "test.sh" is very simple:
> #!/bin/sh
> echo "CROND is restarting" | /usr/bin/mailx -s "crond
failure notification" myemailaddress at mydomain
> 
> As you can see, I have added lines "Restart=on-failure",
"RestartSec=10s" and " ExecStartPost=/bin/sh -c
"/etc/systemd/system/test.sh"" to the original crond.service
daemon file. Then, I run "systemctl daemon-reconfigure" and, from one
console, I run "kill -9 `pidof cron`" for restarting crond daemon.
After it, I receive an email... But now, if I run again ""kill -9
`pidof cron`", I don't receive any mail... I have notice that if I run
"systemctl daemon-reload" and then kill crond process, mail is sent
perfectly... but if I don't run "systemctl daemon-reload", mail is
sent ONLY first time...
> 
> Why?
I guess it is probably the way systemd works - subsequent 'crashes' (or 
kills in you case) probably only restart the main process (ExecStart) 
and don't re-run the ExecStartPost bits ...

I notice you have a commented out an 'OnFailure' line in the unit file -
so I guess you've been experimenting with that - which I believe is 
actually the way to do what you need ?

James Pearson

Hellos,

Yes, I did some tests with "OnFailure" attribute, but the result was
the same: system didn't sent any mail.
Daemon and script files were:

  *   crond.service:
     *   [Unit]
Description=Command Scheduler
After=auditd.service systemd-user-sessions.service time-sync.target
OnFailure=crond-notify-email.service
[Service]
EnvironmentFile=/etc/sysconfig/crond
ExecStart=/usr/sbin/crond -n $CRONDARGS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target


  *   crond-notify-email.service:
     *   [Unit]
Description=crond failure email notification
After=crond.service
Requires=crond.service
[Service]
Type=oneshot
ExecStart=/etc/systemd/system/mail-notification.sh


  *   /etc/systemd/system/mail-notification.sh
     *   #!/bin/bash
echo "Crond restarted" | /usr/bin/mailx -s "Crond failure
notification" myemail at mydomain

After "systemctl daemon-reload" and "kill -9 `pidof crond`",
message log are:
Nov 22 11:04:36 mysystem systemd: crond.service: main process exited,
code=killed, status=9/KILL
Nov 22 11:04:36 mysystem systemd: Unit crond.service entered failed state.
Nov 22 11:04:36 mysystem systemd: Triggering OnFailure= dependencies of
crond.service.
Nov 22 11:04:36 mysystem systemd: crond.service failed.
Nov 22 11:04:41 mysystem systemd: crond.service holdoff time over, scheduling
restart.
Nov 22 11:04:41 mysystem systemd: Stopped Command Scheduler.
Nov 22 11:04:41 mysystem systemd: Started Command Scheduler.
Nov 22 11:04:41 mysystem systemd: Starting crond failure email notification...
Nov 22 11:04:41 mysystem systemd: Started crond failure email notification.

... but I don't receive any email...

Why?

El 21/11/2019 a las 18:34, James Pearson escribi?:
Gesti? Servidors wrote:

Hello,

I'm trying to configure a daemon (I'm doing tests with "crond"
daemon) to send me an email after daemon restart. My "crond.service"
file is:
# /etc/systemd/system/crond.service
[Unit]
Description=Command Scheduler
After=auditd.service systemd-user-sessions.service time-sync.target
#OnFailure=crond-notify-email@%i.service

[Service]
EnvironmentFile=/etc/sysconfig/crond
ExecStart=/usr/sbin/crond -n $CRONDARGS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartSec=10s
ExecStartPost=/bin/sh -c "/etc/systemd/system/test.sh"

[Install]
WantedBy=multi-user.target
My "test.sh" is very simple:
#!/bin/sh
echo "CROND is restarting" | /usr/bin/mailx -s "crond failure
notification" myemailaddress at mydomain

As you can see, I have added lines "Restart=on-failure",
"RestartSec=10s" and " ExecStartPost=/bin/sh -c
"/etc/systemd/system/test.sh"" to the original crond.service
daemon file. Then, I run "systemctl daemon-reconfigure" and, from one
console, I run "kill -9 `pidof cron`" for restarting crond daemon.
After it, I receive an email... But now, if I run again ""kill -9
`pidof cron`", I don't receive any mail... I have notice that if I run
"systemctl daemon-reload" and then kill crond process, mail is sent
perfectly... but if I don't run "systemctl daemon-reload", mail is
sent ONLY first time...

Why?

I guess it is probably the way systemd works - subsequent 'crashes' (or
kills in you case) probably only restart the main process (ExecStart) and
don't re-run the ExecStartPost bits ...

I notice you have a commented out an 'OnFailure' line in the unit file -
so I guess you've been experimenting with that - which I believe is actually
the way to do what you need ?

James Pearson