Nut upsdev - Jul 2014 - Documenting the NUT driver-qualification process

I think the time for me to get involved in NUT documentation has come
again.

Late last week I had to buy a UPS under time pressure.  The Eaton unit 
that thus project gifted me with in 2006(?) died during a severe
thunderstorm watch, so it was off to MicroCenter to get a replacement
pronto.

I wound up buying an APC BN700MC.  It was what they had in the
performance range I needed.  The removable battery door was pleasing.

Based on the experience, I have updated the UPS HOWTO:

http://www.catb.org/esr/ldp/UPS-HOWTO.html

The bad news, however, is that (a) this is not a NUT-supported device,
and (b) the (poorly documented) NUT process for discovering and
customizing a driver failed at the first step.  Running upsstart
gave a driver fail message containing no clues as to how to recover.

This is definitely USB and probably a fairly generic hidups device. 
There is no good reason for customizing a driver profile to be
so difficult.

What I'd like to do is this: confer in real-time, perhaps via IRC,
with someone who knows this process.  Ask about every step (thought
processes and diagnostics).  *Write them down* and turn this into a 
document on how to qualify and support a new device.

Volunteer, please?
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Of all tyrannies, a tyranny exercised for the good of its victims may
be the most oppressive. It may be better to live under robber barons
than under omnipotent moral busybodies. The robber baron's cruelty may
sometimes sleep, his cupidity may at some point be satiated; but those
who torment us for our own good will torment us without end, for they
do so with the approval of their consciences.
	-- C. S. Lewis

On Jul 9, 2014, at 6:31 AM, Eric S. Raymond <esr at thyrsus.com> wrote:
> What I'd like to do is this: confer in real-time, perhaps via IRC,
> with someone who knows this process.  Ask about every step (thought
> processes and diagnostics).  *Write them down* and turn this into a 
> document on how to qualify and support a new device.
Happy to help, but scheduling something in real-time might not happen before
this weekend. Feel free to email me off-list to schedule.

I would also recommend that someone who is less involved with the development of
NUT volunteer as well. I fully admit that I have some preferences about how
things "should" be done, and they don't seem to match what people
are asking for. (A prime example: I think an UPS should shut down the system
when the UPS says the battery has gone below a certain level, as determined by
periodic battery tests, rather than shutting down after a certain time period on
battery.)

A few miscellaneous points about the UPS HOWTO:

* While RS-232C interfaces are being phased out on servers and desktops, UPS
vendors are just wrapping the serial protocol (or contact-closure lines) in a
cheap USB-to-serial converter that may or may not show up as such in the OS. So
you get the intersection of the flakiness of serial, plus the flakiness of USB.
(The USB devices tend not to reconnect with the same device name, if the OS has
a driver for the USB-to-serial converter.)

* The suggestions also seem to be subtly Linux-centric: a lot of the advantages
of USB are lost on systems with less robust USB stacks, such as the BSD family.

* Several years ago, while developing drivers for early Tripp Lite models, I
would have definitely recommended another vendor such as APC or MGE instead.
Today, the tables are turned: MGE/Eaton have decreased their involvement in NUT,
and APC has switched to a proprietary MODBUS-over-USB protocol that they have
not publicly documented [1]. On the other hand, Tripp Lite approached the NUT
project unsolicited with several documents worth of NUT integration testing
results with their newer USB UPS models.[2] There is still some work to be done
on correcting some of the values, but it's a move in the right direction.

[1] http://forums.apc.com/thread/5374

[2]
http://lists.alioth.debian.org/pipermail/nut-upsuser/2013-October/008675.html

* I am not sure that "bigger is better" when it comes to replacement
batteries. I have been buying slightly larger-capacity batteries for several
different UPS brands and models, and the effect ranges from confusion about
state-of-charge (~2002 350W APC; with battery door, incidentally) to fried
battery charging circuitry (350W MGE Evolution).

* For testing power failure scenarios, I would use a power strip switch or a
circuit breaker before tripping a GFCI. I have no scientific basis for that,
just feels better for some reason.

* Although you certainly have a point about not letting UPS batteries wither on
the shelf, if someone is planning to set up many independent UPS-computer pairs
with inexpensive UPSes, it might be worth having a "hot spare" UPS
that is rotated in occasionally during service windows. The battery charger
failure I mentioned above did not allow me to continue using the UPS even
without a battery, so I had to swap in another unit.

-- 
Charles Lepple
clepple at gmail

On 7/9/2014 6:51 AM, Charles Lepple wrote:
> On Jul 9, 2014, at 6:31 AM, Eric S. Raymond<esr at thyrsus.com> 
wrote:
>
>> What I'd like to do is this: confer in real-time, perhaps via IRC,
>> with someone who knows this process.  Ask about every step
>> (thought processes and diagnostics).  *Write them down* and turn
>> this into a document on how to qualify and support a new device.
>
> Happy to help, but scheduling something in real-time might not happen
> before this weekend. Feel free to email me off-list to schedule.
>
> I would also recommend that someone who is less involved with the
> development of NUT volunteer as well. I fully admit that I have some
> preferences about how things "should" be done, and they don't
seem to
> match what people are asking for. (A prime example: I think an UPS
> should shut down the system when the UPS says the battery has gone
> below a certain level, as determined by periodic battery tests,
> rather than shutting down after a certain time period on battery.)
>
> A few miscellaneous points about the UPS HOWTO:
>
> * While RS-232C interfaces are being phased out on servers and
> desktops, UPS vendors are just wrapping the serial protocol (or
> contact-closure lines) in a cheap USB-to-serial converter that may or
> may not show up as such in the OS. So you get the intersection of the
> flakiness of serial, plus the flakiness of USB. (The USB devices tend
> not to reconnect with the same device name, if the OS has a driver
> for the USB-to-serial converter.)
>
> * The suggestions also seem to be subtly Linux-centric: a lot of the
> advantages of USB are lost on systems with less robust USB stacks,
> such as the BSD family.
>
> * Several years ago, while developing drivers for early Tripp Lite
> models, I would have definitely recommended another vendor such as
> APC or MGE instead. Today, the tables are turned: MGE/Eaton have
> decreased their involvement in NUT, and APC has switched to a
> proprietary MODBUS-over-USB protocol that they have not publicly
> documented [1].
That is not true.  The MODBUS protocol has been documented by APC and 
the open source apcupsd program supports it.  The proprietary protocol
APC uses is yet another protocol they call microlink.  And the UPSLink 
protocol that NUT supports is still available via use of an add-on card. 
  Plus, many of the APC UPSes (the backups line) are simple UPSes
and don't use either the smartlink or the modbus protocol they use a
variant of the old upslink over usb.

Someone could write a driver for MODBUS for NUT, all of the information
and materials are available to do it.  The simple fact of it is that the
reason no one has done so is that the apcupsd program is available.

I guess the story about the big evil impersonal corporation beating up 
on the small OSS software project is more compelling than the reality 
that the big corporation has made development info available and the
small OSS software project isn't interested, even though the story is
fabrication.
> On the other hand, Tripp Lite approached the NUT
> project unsolicited with several documents worth of NUT integration
> testing results with their newer USB UPS models.[2] There is still
> some work to be done on correcting some of the values, but it's a
> move in the right direction.
>
> [1] http://forums.apc.com/thread/5374
>
> [2]
>
http://lists.alioth.debian.org/pipermail/nut-upsuser/2013-October/008675.html
>
>  * I am not sure that "bigger is better" when it comes to
replacement
> batteries. I have been buying slightly larger-capacity batteries for
> several different UPS brands and models, and the effect ranges from
> confusion about state-of-charge (~2002 350W APC; with battery door,
> incidentally) to fried battery charging circuitry (350W MGE
> Evolution).
>
I agree with that.
> * For testing power failure scenarios, I would use a power strip
> switch or a circuit breaker before tripping a GFCI. I have no
> scientific basis for that, just feels better for some reason.
>
The contacts in a GFCI  (if your talking about the GFCI's that are
inside an electrical outlet) are much smaller and not as robust as those
in a power strip switch or circuit breaker.  They are mainly intended as
emergency use, to save the life of the dumb bunny who accidentally drops 
their hairdryer into a sink full of water and reaches for it,
or a similar situation.
> * Although you certainly have a point about not letting UPS batteries
> wither on the shelf, if someone is planning to set up many
> independent UPS-computer pairs with inexpensive UPSes, it might be
> worth having a "hot spare" UPS that is rotated in occasionally
during
> service windows. The battery charger failure I mentioned above did
> not allow me to continue using the UPS even without a battery, so I
> had to swap in another unit.
>
It would be far better to refrain from using many independent 
inexpensive UPSes and use a fewer larger expensive UPSes, then you don't
have to bother with battery charger failures and the like.

There's a reason inexpensive UPSes exist - cheap parts.  Cheap parts 
fail a lot.

Ted

On 7/9/2014 3:31 AM, Eric S. Raymond wrote:
> I think the time for me to get involved in NUT documentation has come
> again.
>
> Late last week I had to buy a UPS under time pressure.  The Eaton unit
> that thus project gifted me with in 2006(?) died during a severe
> thunderstorm watch, so it was off to MicroCenter to get a replacement
> pronto.
>
> I wound up buying an APC BN700MC.  It was what they had in the
> performance range I needed.  The removable battery door was pleasing.
>
> Based on the experience, I have updated the UPS HOWTO:
>
> http://www.catb.org/esr/ldp/UPS-HOWTO.html
>
There are some other things in that FAQ that are wrong, they are as follows:

"...The aging power grid in the U.S. has made this a more urgent issue 
than it used to be even for American hackers..."

Pure rubbish.  The following study says the grid has been unchanged
for events below 2000MW and only slightly increased for events above that:

https://www.ece.cmu.edu/cascadingfailures/blackout_frequencies.htm

Of course, there are those who have flat-out lied about this study:

http://www.leonardo-energy.org/increasing-frequency-black-outs-us

"...increase during the period from 1984 to 2006. That is the main 
conclusion of a working paper, published last January, by the Carnegie 
Mellon Electricity Industry Centre..."

The paper they cite DOES NOT SAY it is the "main conclusion"

The fact is that multiple public agencies (like NERC) are attempting to 
gain more funding in the United States - this is what bureaucracies do 
after all, is try to get more money.

So they do these studies and find a slight change then bang the gong
running around saying the sky is falling.

Eric, consider that global warming has increased the power and frequency
of storms in the US over the past 20 years - we have had more tornados
and hurricanes than ever which have destroyed more stuff - and there is
a far, far greater correlation between that and outages, then any
"insufficient spending" arguments that people like Massoud Amin who go
on NPR and make noise would have you believe.  Dr. Amin completely
ignores storms in his analysis even though those are the largest cause
of blackouts.

If you feel compelled to Chicken Little your FAQ than change it to:

"...Global Warming and the increased storms it causes have made this a 
more urgent issue than it used to be even for American hackers..."

Then at least your Chicken Litteling it for a REAL reason not acting
as an unwitting participant in some political scheme by a government 
bureaucracy to make a money-grab.



"An important fact about surge suppressors is that they need to be 
replaced if they absorb a large surge. Besides fuses, most suppressors 
rely on on components called Metal-Oxide Varistors (or MOVs) for spike 
suppression, which degrade when they take a voltage hit. The problem 
with cheap suppressors is that they don't tell you when the MOV is 
cooked, so you can end up with no spike protection and a false sense of 
security. Better ones have an indicator"


The indicator is a power indicator there's no way to build an indicator
that tells if the MOV is cooked or not.  Any indicators you see are
"feel good" indicators.

I personally think MOV's degrade over time due to the plethora of SMALL
spikes they absorb, but I have seen NO data on this one way or another.
This would be a great thesis or graduate study for an engineering 
student to do.

"The basic principle is this: ZS units slow down the surge with a 
network of passive elements and then sends it back out the neutral line"

Too much watching of Star Trek there.  The transformer in a conditioner 
protects against a surge because a surge is so brief it is far higher 
frequency than the transformer is able to resonate at - in short the
transformer simply does not work for the high frequency spikes and
surges it only works for the low frequency AC power.


" in these days of journalling filesystems like Linux's EXT3 or JFS
from "


Who still uses ext3?  I think you mean EXT4.

"UPSes are nowadays very inexpensive. In the U.S. in 2006, quite"

change the date there

"by things like the size of monitor you use (big ones can be quite 
power-hungry"

big ones with tubes in them - we are all using flat panels now...


"My advice is to forget the numbers game. Just go online or to your 
local computer store and buy one of the higher-end consumer or SOHO 
models from APC, CyberPower, Eaton, Tripp-Lite, Belkin, or some other 
reputable manufacturer. Go ahead and grab the model with the longest 
dwell time, highest watt rating, or biggest VA number you can find; the 
premium for it is not likely to be more than US$75 over the 
bargain-basement model"

APC BackUPS models under the 700 level NO LONGER contain a monitoring 
port. You need to be careful when advising people what UPS to buy to 
tell them to look for this port.


"Do not buy a serial line UPS (one that communicates via an RS-232C 
cable). These are passing out of use in favor of UPS designs that use 
USB or Ethernet, for the very excellent reason that RS-232C interfaces 
are flaky"


Serial is passing out of use because Microsoft and Intel said to stop
using it in the PC 2001 standard:

http://en.wikipedia.org/wiki/PC_System_Design_Guide

where (for cost reasons) they strongly emphasized Legacy Free PCs'.

http://en.wikipedia.org/wiki/Legacy-free_PC

It has absolutely nothing to do with flakiness of RC232C.  The reality 
is that RS232C when properly implemented per the standard is NOT flaky.

You could say that 100foot USB cable is flaky - because it is - because 
it's a gross standard violation of USB - are you then going to label USB
flaky because some bozo isn't implementing the USB standard properly?


Ted
> The bad news, however, is that (a) this is not a NUT-supported device,
> and (b) the (poorly documented) NUT process for discovering and
> customizing a driver failed at the first step.  Running upsstart
> gave a driver fail message containing no clues as to how to recover.
>
> This is definitely USB and probably a fairly generic hidups device.
> There is no good reason for customizing a driver profile to be
> so difficult.
>
> What I'd like to do is this: confer in real-time, perhaps via IRC,
> with someone who knows this process.  Ask about every step (thought
> processes and diagnostics).  *Write them down* and turn this into a
> document on how to qualify and support a new device.
>
> Volunteer, please?

Hi Eric,

sorry for the lag, summer time...

I'm first seconding Charles comments

2014-07-09 12:31 GMT+02:00 Eric S. Raymond <esr at thyrsus.com>:
> I think the time for me to get involved in NUT documentation has come
> again.
>
welcome back

> Late last week I had to buy a UPS under time pressure.  The Eaton unit
> that thus project gifted me with in 2006(?) died during a severe
> thunderstorm watch, so it was off to MicroCenter to get a replacement
> pronto.
>
> I wound up buying an APC BN700MC.  It was what they had in the
> performance range I needed.  The removable battery door was pleasing.
>
> Based on the experience, I have updated the UPS HOWTO:
>
> http://www.catb.org/esr/ldp/UPS-HOWTO.html
>
> The bad news, however, is that (a) this is not a NUT-supported device,
>
I've logged an entry for the APC Modbus:
https://github.com/networkupstools/nut/issues/139

> and (b) the (poorly documented) NUT process for discovering and
> customizing a driver failed at the first step.  Running upsstart
> gave a driver fail message containing no clues as to how to recover.
>
yup, definitely room for enhancement!
I've got some ideas, beside from the obvious need to improve the
documentation, that I'll like to discuss.

> This is definitely USB and probably a fairly generic hidups device.
> There is no good reason for customizing a driver profile to be
> so difficult.
>
well, I'm unsure since I'm just opening this "APC Modbus Pandora
box"
but if I understand correctly, your new unit is a microlink one.
though the serial version should be Modbus only, I can't say at first if
the USB ones implement if full Modbus over HID or any other mean.
I would need traces to give more info.

anyway, this may not be as generic USB as you think.

What I'd like to do is this: confer in real-time, perhaps via
IRC,
> with someone who knows this process.  Ask about every step (thought
> processes and diagnostics).  *Write them down* and turn this into a
> document on how to qualify and support a new device.
>
> Volunteer, please?
>
I am

I would also be more than happy to have you looking more closely at NUT
documentation in general [1].
Charles and myself have worked a lot on the Asciidoc conversion, with some
improvements.
But I know it needs an overall revamp, with usability in mind.
Would you be willing to work on that one?

cheers,
Arno
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.alioth.debian.org/pipermail/nut-upsdev/attachments/20140726/b367941c/attachment.html>

On 7/26/2014 12:18 PM, Arnaud Quette wrote:
> Hi Eric,
>
> sorry for the lag, summer time...
>
> I'm first seconding Charles comments
>
> 2014-07-09 12:31 GMT+02:00 Eric S. Raymond <esr at thyrsus.com
> <mailto:esr at thyrsus.com>>:
>
>     I think the time for me to get involved in NUT documentation has come
>     again.
>
>
> welcome back
>
>     Late last week I had to buy a UPS under time pressure.  The Eaton unit
>     that thus project gifted me with in 2006(?) died during a severe
>     thunderstorm watch, so it was off to MicroCenter to get a replacement
>     pronto.
>
>     I wound up buying an APC BN700MC.  It was what they had in the
>     performance range I needed.  The removable battery door was pleasing.
>
>     Based on the experience, I have updated the UPS HOWTO:
>
>     http://www.catb.org/esr/ldp/UPS-HOWTO.html
>
>     The bad news, however, is that (a) this is not a NUT-supported device,
>
>
> I've logged an entry for the APC Modbus:
> https://github.com/networkupstools/nut/issues/139
>
>     and (b) the (poorly documented) NUT process for discovering and
>     customizing a driver failed at the first step.  Running upsstart
>     gave a driver fail message containing no clues as to how to recover.
>
>
> yup, definitely room for enhancement!
> I've got some ideas, beside from the obvious need to improve the
> documentation, that I'll like to discuss.
>
>     This is definitely USB and probably a fairly generic hidups device.
>     There is no good reason for customizing a driver profile to be
>     so difficult.
>
>
> well, I'm unsure since I'm just opening this "APC Modbus
Pandora box"
> but if I understand correctly, your new unit is a microlink one.
MAYBE.

HIS UPS is a BackUPS not a SmartUPS.  Originally the APC BackUPS DID NOT
SUPPORT any Smart protocol (the original UPSLink protocol)

But over the years it's been observed that APC has somewhat
"fudged" on
the BackUPS UPSes.  Newer ones did appear to implement the UPSlink 
protocol over USB.

Microlink protocol is officially only for APC SmartUPSes.  Officially,
what your supposed to get with a BackUPS is dumb signalling only.  But,
with the newer BackUPSes that have USB output, some seemed to act like
SmartUPSes even though they were not.

APC has very much blurred the line on the model BN UPSes.  These are the
APC UPSes that look like an oversized power strip.  That's the one that 
he has.  They seem to have dropped monitoring ports completely on these 
UPSes that are under 700VA in size.  They used to make monitoring ports
on these UPSes all the way down to 350VA.  But the current models, only 
the 700VA unit like the one he bought, still support the monitoring 
port.  That port is the special wide RJ45 that has 10 conductors not 8.
It is selectable between serial and USB depending on what cable is used
with it.
> though the serial version should be Modbus only,
There is no "serial only" APC UPS that supports Modbus, in fact no APC
smart UPS since the Microlink protocol was implemented has been "serial 
only", all have been dual-use  Serial and USB.
> I can't say at first if
> the USB ones implement if full Modbus over HID or any other mean.
> I would need traces to give more info.
>
The BackUPSes with USB ports might implement full UPSlink over HID.

Or they might implement Microlink over HID

Or they might implement nothing like either of that.

But they definitely don't implement Modbus.

It is best I think to refer to BackUPS and SmartUPS in the 
documentation.  Please do not refer to APC UPSes in a generic fashion. 
And please don't assume that any APC BackUPS implements any particular
protocol over USB or Serial unless you have directly tested it.  APC
BackUPS upses are the "cheap, poor relations" and are for consumers
and
penny-pinching businesses that are too cheap to spend a couple hundred 
bucks on decent gear.  Purchasers who want full functionality should be
encouraged to buy an APC SmartUPS.


Ted
> anyway, this may not be as generic USB as you think.
>
>     What I'd like to do is this: confer in real-time, perhaps via IRC,
>     with someone who knows this process.  Ask about every step (thought
>     processes and diagnostics).  *Write them down* and turn this into a
>     document on how to qualify and support a new device.
>
>     Volunteer, please?
>
>
> I am
>
> I would also be more than happy to have you looking more closely at NUT
> documentation in general [1].
> Charles and myself have worked a lot on the Asciidoc conversion, with
> some improvements.
> But I know it needs an overall revamp, with usability in mind.
> Would you be willing to work on that one?
>
> cheers,
> Arno
>
>
> _______________________________________________
> Nut-upsdev mailing list
> Nut-upsdev at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/nut-upsdev