Nut upsuser - Nov 2023 - A proper way to modify battery.charge.low persistently

Dear fellow NUT-UPS users,

I'm writing this message mostly "for the record" - for others who 
will follow in my footsteps.

Apparently, I'm not the only one, trying to find a way to "make my 
UPS to initiate the graceful shutdown process earlier".

The topic is kind of documented in various different places, and 
there are a number of howto's and forum posts in the interwebs that 
don't seem to carry all the nuance in one place :-)

My scenario: I have a relatively cheap and low-end UPS, called the 
Ellipse ECO 650, nowadays under the Eaton brand (MGE no more).
I did some research ahead of my purchase and picked a UPS with 
support in "nut" (and not having a noisy transformer or a fan).
In Debian 11, I have installed the nut-client and the nut-server and 
pretty much went headlong into the config files in /etc/nut, where 
the config isn't very complicated.

- you need to load the driver for your UPS 
   (config in ups.conf, mine uses the usbhid-ups)

- in my case nut.conf only says mode=standalone

- upsd.conf can in fact stay empty

- I have two user entries in upsd.users, one of which is for upsmon, 
the other for a human admin (me)

- and upsmon.conf contains several relevant rows, first and foremost 
a pointer to the UPS driver instance on the local host.

Everything went pretty much smoothly, except for one thing: 
in its default config, my UPS waits until the charge percentage is 
depleted down to 20%, before finally initiating a shutdown.
I prefer the UPS to initiate the shutdown earlier, because my major 
concern is that the small server must shut down gracefully. 
It is NOT my point to keep the server alive for as long as possible, 
during a power outage.

If you follow the docs to NUT, you will soon stumble over upsctl / 
upsc / upsrw.
And, upsc will tell you about a UPS variable called

 battery.charge.low 

This is an internal, "device-side" variable kept by the UPS firmware, 
and in my case it comes at 20% ex works.
It took a few tries (and checking a manpage) to get upsrw to change 
this to 90%, and run a practical test.
That worked well.

I then restarted the server and tried again... hmm. No response to 
pulling the mains cord, the battery was already several % below 90% 
and the system kept running! What ho?
Try upsc... ahh yes, battery.charge.low is back at 20%.
So... the variable is not stored in my UPS, and the configured 
threshold is ephemeral... not sure if to the UPS or to the driver 
running on the host PC. Makes no difference. The point is that, to 
put this UPS variable to practical use, we need to configure it 
permanently / persistently.
And, this is where the documentation feels a little sketchy.

The setting command in the shell goes like

upsrw -s battery.charge.low=90 -u my_user -p my_passwd eaton1

But... is there perhaps some systemic way where to configure a UPS 
variable? So that it gets configured first thing after a reboot?

Turns out that there isn't.
You need to enter this into a script somewhere.

I was wondering about Systemd's ExecStartPost, but as the nut-server 
service unit specifies "type=forking", there is a risk that the 
ExecStartPost will run earlier than the nut-server has managed to 
start listening... which can be solved by some minimal further 
scripting, ...but I ended up just tucking that single command into 
/etc/rc.local :-) And I had to prepend a few seconds of delay even 
there, otherwise the variable sometimes would not get configured on 
startup.
_Now_ the variable is persistent.

I said that there wasn't a systemic place to put the variable alone, 
in order to have NUT do the setting on startup for you.
Which may sound perplexing to you, if you have noticed config options 
such as

override.battery.charge.low

in ups.conf.

Note that this is a bit of a false friend.
This only applies, if at the same time you configure

ignorelb

Which is an option telling NUT, *not* to act upon the Low Battery 
status flag, as reported by the UPS. 
In that case, NUT *itself* follows the instantaneous battery.charge 
(as reported by the UPS) and NUT *itself* evaluates that the 
override.battery.charge.low has been trespassed.

Without ignorelb, NUT merely watches for the LB status flag from the 
UPS, and it is the UPS (or its driver?) watching the charge level and 
flagging LB when the conditions are met. And, the 
override.battery.charge.low    option in ups.conf does not apply = 
does not get posted to the UPS driver+firmware.

See also the ups.status variable in the output of "upsc eaton1":
OL = on line
OB = on battery
LB = low battery

So, effectively you have a choice:

A) put upsrw in /etc/rc.local or some such, to tell the UPS when to 
flag "low battery"

B) tell NUT to ignore LB signaling from the UPS, and instead evaluate 
the threshold in software, based on an override option

Another thing that took me a while was to realize, how the whole 
graceful shutdown process is going. The process isn't nearly as 
straightforward as it may seem at a first glance: "Battery low, SNAP,
 lights out". Nonono. 
The sequence is described in quite some detail in official 
documentation (link to "configuration notes" follows below).

In rough outlines, in a standalone setup, the process goes like:

1)  the power goes out, which the UPS signals to the server. 
You get a message on the console and in the log, but so far nothing 
else happens.

2) the UPS reaches the "battery low threshold", and raises the LB 
flag

3) NUTS on the host machine (master monitor instance) notices this 
and initiates a shutdown = tells the "init" daemon to orchestrate a 
graceful shutdown.

4) at the very end of the "init" daemon's shutdown dance, there is
a
stage where all meaningful services have been shut down, all volumes 
have been unmounted, and there is a "space" where pieces of software 
or config can have a last wish, before the power switch gets flipped.
In systemd parlance, you are free to place your scripts in 

/lib/systemd/system-shutdown/

Or you could probably write some unit files to achieve a similar 
effect including some dependencies... but just dropping a script in 
this directory does work.

And, this is where the nut-monitor package in Debian places its 
script, called "nutshutdown". 
This script conditionally calls "upsdrvctl shutdown", if a killpower 
file is found. You don't want to shut down your UPS upon any regular 
shutdown, right? So that's the purpose of
POWERDOWNFLAG /etc/killpower
in /etc/upsmon.conf .

Within the systemd "system-shutdown" section, on my Debian system, 
the only other script is mdadm.shutdown = disassemble any MD RAID 
arrays for a good measure. (Not really needed IMO, as by now, all 
data has been synced to spinning rust as part of umount.)

So in item 4), the UPS gets commanded to disconnect power on its 
output.

5) but wait, that is not all yet. The UPS waits a few more seconds, 
before actually cutting power to the load. See the UPS variable
ups.delay.shutdown
in my case == 20 seconds.
This means that anything within the systemd system-shutdown directory 
should have a chance to finish.
Actually my server manages to perform an ATX powerdown (transition 
into ACPI S5) - and then it waits in this state for about a dozen 
seconds, before the external UPS actually flips the switch.

If it comes to this end, the server is ready to auto-start upon the 
mains coming back up.

Now... what happens if mains power at the wall is restored within 
that last ~20s grace period? (I mean ups.delay.shutdown.)
That's an interesting question, and to me the jury is out.
May have to do with the configuration/arrangement of "wakeup by USB".
I've tried poking around in 
cat /proc/acpi/wakeup
grep . /sys/bus/usb/devices/*/power/wakeup
cat /sys/bus/usb/devices/1-4/product
echo enabled > /sys/bus/usb/devices/1-4/power/wakeup
...but that didn't seem to have any effect.
During the 20s grace period just before "UPS load off", I don't
get
the PC woken up by the UPS.

Upon closer inspection, it actually seems that:
ups.delay.shutdown
ups.delay.start
are not respected by the UPS.
It waits about 15 seconds with the shutdown, and about 7 seconds with 
load startup.
Also, if I plug the cord back within its hardwired 15second "grace 
period upon the shutdown command received via USB", and I am patient, 
the UPS still turns the load off. For those roughly 7 seconds. So the 
wakeup via USB is not actually needed. Except that, within those 
approx 7 seconds, the StandBy consumption of my PC doesn't manage to 
deplete the PSU's primary capacitor, i.e. the server doesn't register 
a cold power outage, therefore does not even try to start up, when 
the UPS turns the mains back on.

The UPS is relatively dumb. You get what you pay for. The UPS 
Companion software from Eaton, when used with this UPS model, has 
relatively little to offer in the way of configuration. There are 
some options for host-based operation if the "UPS Companion software" 
itself, and probably no configurable knobs for the UPS.

I hope this helps someone...

Frank


References:

https://networkupstools.org/docs/user-manual.chunked/Configuration_not
es.html

https://networkupstools.org/docs/user-manual.chunked/ar01s06.html#UPS_
shutdown

https://www.mail-archive.com/nut-upsuser at alioth-lists.debian.net/msg01
334.html

https://forum.openwrt.org/t/nut-ups-does-not-trigger-lowbattery-shutdo
wn-please-help/91252

https://forums.debian.net/viewtopic.php?t=156046
https://devicetests.com/wakeup-linux-wireless-usb-keyboard-mouse