gnucash announce - May 2024 - ANNOUNCE: [ERR-MAINT] GnuCash-Devel list config (and sub list) reverted to OLD config

Hi GnuCashers-

TL;DR:

As some of you may have noticed, email to GnuCash-Devel has been, well,
wonky.  The short version is that due to a cascade of issues and failures,
the list configuration was reverted to an old config, including the
subscription list.  This caused several hundred old subscriptions to
re-appear, and several hundred newer subscriptions to disappear. 
Unfortunately the only remedy is to re-subscribe to the list.

LONG VERSION

The GnuCash server is set up for a daily backup.  This backup creates an
LVM snapshot so the backup has a stable dataset during the ~4-hour backup
process.  When the backup finishes, the snapshots are removed.  All well
and good.

Unfortunately, at some point (unknown) there was a system failure during
that backup window which caused the snapshots to remain, and not get
cleaned up.  System failures like this happen frequently (mostly due to
the VM host spontaneously rebooting, which happens a couple times a year).
 However usually the system cleans up after the failure.  This time, for
some reason, that cleanup did not occur.

Simultaneously, the backup server email system wedged and stopped sending
out status emails, so I didn't get notified of the backup failures.

The leftover snapshot eventually filled up, which caused the gnucash-devel
list to fail to write out its config file in a catastrophic way.  This
resulted in a corrupted config.pck.  Mailman, "helpfully", reverted
back
to its config.db, which was its configuration back in 2002.  This caused
the list to disappear, due to old old changes.

So why not revert to another backup?  Two reasons:

1. It's unclear exactly when this failure occurred, so there's not a
good
way to know which backup to revert to.
2. Due to the snapshot failure, the backups were not running for way too
long.  The previous backup was from October 2023!

I apologize for all the inconveniences if you unsubscribed in the past and
somehow found yourself receiving gnucash-devel mail again.  I also
apologize for those who will have to re-subscribe because your
subscription was lost.

Going forward, I've made additional changes to recover if there is a
leftover snapshot to attempt to prevent this kind of failure.  I've also
been paying much closer attention to the daily log emails and backup logs
to keep this from happening again.

Thanks,

-derek

-- 
       Derek Atkins                 617-623-3745
       derek at ihtfp.com             www.ihtfp.com
       Computer and Internet Security Consultant