Nut upsuser - Aug 2020 - Synology NAS is shutting down Ubuntu servers after very brief power outage (fwd)

On Thu, 6 Aug 2020, Todd Benivegna wrote:
> ... I grep’d the syslog and here’s the results:
Could you also grep for upsd and upsmon in the NAS log?  Is this possible?
>       proton at proton:~$ sudo grep upsmon /var/log/syslog
>       Aug  6 19:19:09 proton upsmon[1552]: UPS ups at 192.168.1.70 on
battery
>       Aug  6 19:19:14 proton upsmon[1552]: UPS ups at 192.168.1.70 on line
power
>       Aug  6 19:19:44 proton upsmon[1552]: UPS ups at 192.168.1.70: forced
shutdown in progress
>       Aug  6 19:19:44 proton upsmon[1552]: Executing automatic power-fail
shutdown
So proton is a slave, and is being made to shut down.
>       Aug  6 19:19:45 proton upsmon[1552]: Auto logout and shutdown
proceeding
>       Aug  6 19:19:50 proton upsmon.conf: UPS status is 
The variable USPstatus set by getUPSstatus was not in the environment for the 
echo command.  To get this to work, you'll have to put the upsc commands 
directly in SHUTDOWNCMD, or create a short script and call the script in 
SHUTDOWNCMD.
>       Aug  6 19:19:59 proton upsmon[1545]: UPS: ups at 192.168.1.70 (slave)
(power value 1)
Could you tell us your topology?  Which system is master and which is slave? 
Do you have multiple slaves?  The decision to shutdown is taken by the master, 
not the slave.

Is it possible to run script http://rogerprice.org/NUT/nut-report on the NAS? 
Could you run the script on one of the slaves and post the result here?

Roger

On Fri, 7 Aug 2020, Roger Price wrote:
> Is it possible to run script http://rogerprice.org/NUT/nut-report on the
NAS?
No need, it's sufficient to tell us the contents of files upsd.conf,
upsd.users
and ups.conf probably in DS416 directory /usr/syno/etc/ups . Remove your 
passwords, and please remove comments and blank lines. A command such as

  grep ^[^#] upsd.conf

will do the job.

It's implicit in the Synology documentation I've read that upsmon never
runs on
a Synology NAS.  Is this true?

It would still be nice to see the output of the nut-report script for your 
master Ubuntu box.

Roger

Hi Roger,

I am not home, but when I do get home I will check out everything you've
mentioned.  I do have some time where I can give you a breakdown of my
topology though.  I have a feeling all of this is probably due to a
configuration error somewhere on my part.   Here's what I have done so far.

*APC Back-UPS NS 650M1 UPS* ---USB---> *Synology NAS (DS416 - Master?)*
---Ethernet---> *Netgear Managed Switch* w/ uplink to router
<---Ethernet--- *Servers (Ubuntu 20.04 - Plex, Pulsar, Proton - All three
set as slaves)*

I have all three servers set as slaves, so is the Synology considered the
master?  Or do I need to set one of the servers as the Master?  I've been
under the impression that the Synology is the master, but have been unable
to confirm this.

So what I have done so far is enable the "Network UPS Server" on the
Synology, entered the three IPs of the servers in there, set it to shutdown
when battery is low and enabled "Shutdown UPS when the system enters safe
mode".  I then installed NUT on all three servers. In nut.conf I changed
MODE to "MODE=netclient".  I then added my MONITOR line in
upsmon.conf. on
all three.  Looks something like this:

MONITOR ups at 192.168.1.70 1 monuser secret slave

My SHUTDOWNCMD looks like this:

SHUTDOWNCMD "getUPSstatus ups at 192.168.1.70 ; logger -t upsmon.conf
\"UPS
status is $UPSstatus\" ; /sbin/shutdown -h +0"

Would that not work?  I believe that's what you told me to put there.

I think I've definitely made a mistake though, in that I have not
set RUN_AS_USER in upsmon.conf and set up the appropriate permissions.  I
read somewhere that it would then run as user NOBODY and not be able read
upsmon.conf.  Is that true?  Could that be the whole problem?

I'm thinking I need to put in..

RUN_AS_USER nut

in upsmon.conf and then do:

$ sudo chown -R root:nut /etc/nut
$ sudo chmod 0770 /etc/nut
$ sudo chmod 0640 /etc/nut/*

Does that last part look correct to you?  I think what I will do is put the
contents of the files in pastebin and send the links to you.  May be easier
that way.

Thanks,

Todd

On Fri, Aug 7, 2020 at 5:47 AM Roger Price <roger at rogerprice.org>
wrote:
> On Thu, 6 Aug 2020, Todd Benivegna wrote:
>
> > ... I grep’d the syslog and here’s the results:
>
> Could you also grep for upsd and upsmon in the NAS log?  Is this possible?
>
> >       proton at proton:~$ sudo grep upsmon /var/log/syslog
> >       Aug  6 19:19:09 proton upsmon[1552]: UPS ups at 192.168.1.70 on
> battery
> >       Aug  6 19:19:14 proton upsmon[1552]: UPS ups at 192.168.1.70 on
line
> power
> >       Aug  6 19:19:44 proton upsmon[1552]: UPS ups at 192.168.1.70:
forced
> shutdown in progress
> >       Aug  6 19:19:44 proton upsmon[1552]: Executing automatic
> power-fail shutdown
>
> So proton is a slave, and is being made to shut down.
>
> >       Aug  6 19:19:45 proton upsmon[1552]: Auto logout and shutdown
> proceeding
> >       Aug  6 19:19:50 proton upsmon.conf: UPS status is
>
> The variable USPstatus set by getUPSstatus was not in the environment for
> the
> echo command.  To get this to work, you'll have to put the upsc
commands
> directly in SHUTDOWNCMD, or create a short script and call the script in
> SHUTDOWNCMD.
>
> >       Aug  6 19:19:59 proton upsmon[1545]: UPS: ups at 192.168.1.70
> (slave) (power value 1)
>
> Could you tell us your topology?  Which system is master and which is
> slave?
> Do you have multiple slaves?  The decision to shutdown is taken by the
> master,
> not the slave.
>
> Is it possible to run script http://rogerprice.org/NUT/nut-report on the
> NAS?
> Could you run the script on one of the slaves and post the result here?
>
> Roger_______________________________________________
> Nut-upsuser mailing list
> Nut-upsuser at alioth-lists.debian.net
> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/nut-upsuser


-- 
*Todd Benivegna* // todd at benivegna.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://alioth-lists.debian.net/pipermail/nut-upsuser/attachments/20200807/c7d6b3ef/attachment.html>

upsmon.conf on server:  https://pastebin.com/z4CrUTxb
nut.conf on server:  https://pastebin.com/540ShZH7
Permissions for /etc/nut:  https://hastebin.com/qecolodapi.diff

On the Synology (I didn’t edit any of these files):
ups.conf:  https://hastebin.com/dedereqizi.shell
upsd.conf:  https://hastebin.com/pupeseweda.css
upsd.users:  https://hastebin.com/ocenamecex.cs

I don’t think I am able to run that script.  If you can, I wouldn’t know how.

Please let me know what you think.

Thanks for all the help!

Todd

--
Todd Benivegna // todd at benivegna.com
On Aug 7, 2020, 12:14 PM -0400, Todd Benivegna <todd at benivegna.com>,
wrote:
> Hi Roger,
>
> I am not home, but when I do get home I will check out everything
you've mentioned.  I do have some time where I can give you a breakdown of
my topology though.  I have a feeling all of this is probably due to a
configuration error somewhere on my part.   Here's what I have done so far.
>
> APC Back-UPS NS 650M1 UPS ---USB---> Synology NAS (DS416 - Master?)
---Ethernet---> Netgear Managed Switch w/ uplink to router <---Ethernet---
Servers (Ubuntu 20.04 - Plex, Pulsar, Proton - All three set as slaves)
>
> I have all three servers set as slaves, so is the Synology considered the
master?  Or do I need to set one of the servers as the Master?  I've been
under the impression that the Synology is the master, but have been unable to
confirm this.
>
> So what I have done so far is enable the "Network UPS Server" on
the Synology, entered the three IPs of the servers in there, set it to shutdown
when battery is low and enabled "Shutdown UPS when the system enters safe
mode".  I then installed NUT on all three servers. In nut.conf I changed
MODE to "MODE=netclient".  I then added my MONITOR line in
upsmon.conf. on all three.  Looks something like this:
>
> MONITOR ups at 192.168.1.70 1 monuser secret slave
>
> My SHUTDOWNCMD looks like this:
>
> SHUTDOWNCMD "getUPSstatus ups at 192.168.1.70 ; logger -t upsmon.conf
\"UPS status is $UPSstatus\" ; /sbin/shutdown -h +0"
>
> Would that not work?  I believe that's what you told me to put there.
>
> I think I've definitely made a mistake though, in that I have not
set RUN_AS_USER in upsmon.conf and set up the appropriate permissions.  I read
somewhere that it would then run as user NOBODY and not be able read
upsmon.conf.  Is that true?  Could that be the whole problem?
>
> I'm thinking I need to put in..
>
> RUN_AS_USER nut
>
> in upsmon.conf and then do:
>
> $ sudo chown -R root:nut /etc/nut
> $ sudo chmod 0770 /etc/nut
> $ sudo chmod 0640 /etc/nut/*
>
> Does that last part look correct to you?  I think what I will do is put the
contents of the files in pastebin and send the links to you.  May be easier that
way.
>
> Thanks,
>
> Todd
>
> > On Fri, Aug 7, 2020 at 5:47 AM Roger Price <roger at
rogerprice.org> wrote:
> > > On Thu, 6 Aug 2020, Todd Benivegna wrote:
> > >
> > > > ... I grep’d the syslog and here’s the results:
> > >
> > > Could you also grep for upsd and upsmon in the NAS log?  Is this
possible?
> > >
> > > >       proton at proton:~$ sudo grep upsmon /var/log/syslog
> > > >       Aug  6 19:19:09 proton upsmon[1552]: UPS ups at
192.168.1.70 on battery
> > > >       Aug  6 19:19:14 proton upsmon[1552]: UPS ups at
192.168.1.70 on line power
> > > >       Aug  6 19:19:44 proton upsmon[1552]: UPS ups at
192.168.1.70: forced shutdown in progress
> > > >       Aug  6 19:19:44 proton upsmon[1552]: Executing
automatic power-fail shutdown
> > >
> > > So proton is a slave, and is being made to shut down.
> > >
> > > >       Aug  6 19:19:45 proton upsmon[1552]: Auto logout and
shutdown proceeding
> > > >       Aug  6 19:19:50 proton upsmon.conf: UPS status is
> > >
> > > The variable USPstatus set by getUPSstatus was not in the
environment for the
> > > echo command.  To get this to work, you'll have to put the
upsc commands
> > > directly in SHUTDOWNCMD, or create a short script and call the
script in
> > > SHUTDOWNCMD.
> > >
> > > >       Aug  6 19:19:59 proton upsmon[1545]: UPS: ups at
192.168.1.70 (slave) (power value 1)
> > >
> > > Could you tell us your topology?  Which system is master and
which is slave?
> > > Do you have multiple slaves?  The decision to shutdown is taken
by the master,
> > > not the slave.
> > >
> > > Is it possible to run script http://rogerprice.org/NUT/nut-report
on the NAS?
> > > Could you run the script on one of the slaves and post the result
here?
> > >
> > > Roger_______________________________________________
> > > Nut-upsuser mailing list
> > > Nut-upsuser at alioth-lists.debian.net
> > >
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/nut-upsuser
>
>
> --
> Todd Benivegna // todd at benivegna.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://alioth-lists.debian.net/pipermail/nut-upsuser/attachments/20200807/664644b8/attachment-0001.html>

On Fri, 7 Aug 2020, Todd Benivegna wrote:
> APC Back-UPS NS 650M1 UPS ---USB---> Synology NAS (DS416 - Master?) 
> ---Ethernet---> Netgear Managed Switch w/ uplink to router
<---Ethernet---
> Servers (Ubuntu 20.04 - Plex, Pulsar, Proton - All three set as slaves)
I'm guessing that the UPS supplies only the NAS, not the 3 Ubuntu machines. 
Do
they have their own UPS's?
> I have all three servers set as slaves, so is the Synology considered the 
> master?  Or do I need to set one of the servers as the Master?  I've
been
> under the impression that the Synology is the master, but have been unable
to
> confirm this.
I've been looking at the Synology documentation and their NUT setup is not
at
all clear.  Some of their site is nonsense.  I gather from other sites that the 
NAS is indeed the master and that upsmon runs in the NAS. This makes sense if 
the UPS is for the NAS and nothing else.  It also simplifies shutdown if NAS 
users mount NFS supplied directories in the NAS.
> So what I have done so far is enable the "Network UPS Server" on
the Synology,
> entered the three IPs of the servers in there, set it to shutdown when
battery
> is low and enabled "Shutdown UPS when the system enters safe
mode".  I then
> installed NUT on all three servers. In nut.conf I changed MODE to 
> "MODE=netclient".  I then added my MONITOR line in upsmon.conf.
on all three. 
> Looks something like this:  
> 
> MONITOR ups at 192.168.1.70 1 monuser secret slave
> 
> My SHUTDOWNCMD looks like this:
> SHUTDOWNCMD "getUPSstatus ups at 192.168.1.70 ; logger -t upsmon.conf
\"UPS status is $UPSstatus\" ; /sbin/shutdown -h +0"  
>From your previous reports it looks as if getUPSstatus does not work in a 
SHUTDOWNCMD declaration since the shell variable it creates gets lost. It 
probably better to declare something like

  SHUTDOWNCMD "logger -t upsmon.conf \"UPS status [$( upsc ups at
192.168.1.70 ups.status )]:$( upsc ups at 192.168.1.70 )\" ; /sbin/shutdown
-h +0"  
> I think I've definitely made a mistake though, in that I have not
set RUN_AS_USER in upsmon.conf
The default user is usually set when NUT is built for a specific Linux 
distribution.  I don't know what user Ubuntu have chosen, but I will guess
that
they have followed Debian and use "nut".  I suggest you do not change
this.
> and set up the appropriate permissions.
Again, I assume Ubuntu build NUT with the correct file permissions for their 
default user.

Roger