CentOS - Feb 2021 - Transition test report going from CentOS8 to Debian 10.

On Feb 4, 2021, at 8:39 AM, Lamar Owen <lowen at pari.edu>
wrote:
> 
> I posted a pretty complete rundown on the scientific linux users mailing
list, so I won't recap it all here.
Link?
> the transition was not any more difficult, really, than moving from CentOS
7 to CentOS 8.
That?s not my experience.

I keep several of my packages running on CentOS and Debian (and more) and I keep
running into several common problems:

1. The package names are often different, and not always differing by an obvious
translation rule.  For instance, it?s ?openldap-devel? on CentOS but
?libldap2-dev? on Debian, where the normal rule would make it ?libopenldap-dev?.
Why the difference?  Dunno, but I have to track such things down when setting up
scripts that do cross-distro builds.  If I automate that translation, now I?m
setting myself up for a future breakage when the package names change again. 
(libldap3-dev?)

2. Some packages simply won?t be available.  Most often this happens in the
Debian ? CentOS direction, but I?ve run into cases going the other way.  Just
for one, I currently have to install NPM from source on Debian because the
platform version won?t work properly with the platform version of Node, last
time I tested it.  Why?  Same answer as above.

3. Debian adopted systemd, but it didn?t adopt the rest of the Red Hat userland
tooling.  For instance, it?s firewalld on CentOS, UFW on Ubuntu, and raw kernel
firewall manipulation on Debian unless you install one of those two.  And then,
which?

4. Network configuration is almost entirely different unless you turn off all
the automation on all platforms, in which case you might as well switch to macOS
or FreeBSD for all the good your muscle memory and training will do you.

I?m not saying ?don?t do it,? but to say it?s as smooth as from CentOS 7 to 8? 
Hard sell.

I?ll give you one mulligan: the changes to the security rules in CentOS 8 caused
a huge upheaval for one of my applications, since it basically stopped it from
running, being naughty in Red Hat?s omnisciently beneficent eyes.  We spent
about a year fixing breakages due to 25 years of built-up assumptions about what
was correct and sensible, which don?t affect us on other Linuxes because they
didn?t implement the same SELinux rules.

The details aren?t super-important, because the real take-away is this: it?s
always *something.*

(For those that must know, the biggie was that our systemd-based service used to
run from /home/$APPNAME but that?s a no-no on C8 now.  Moving it all under
/opt/$APPNAME and rearranging it all according to LFS rules, then finding and
fixing all the places we depended on such paths was *painful*.)

On Thu, Feb 4, 2021 at 10:23 AM Warren Young <warren at etr-usa.com>
wrote:
>
> On Feb 4, 2021, at 8:39 AM, Lamar Owen <lowen at pari.edu> wrote:
> >
> > I posted a pretty complete rundown on the scientific linux users
mailing list, so I won't recap it all here.
>
> Link?
https://listserv.fnal.gov/scripts/wa.exe?A2=ind2102&L=SCIENTIFIC-LINUX-USERS&P=5275

Akemi

> On Feb 4, 2021, at 8:39 AM, Lamar Owen <lowen at pari.edu> wrote:
>>
>> I posted a pretty complete rundown on the scientific linux users
mailing
>> list, so I won't recap it all here.
>
> Link?
>
>> the transition was not any more difficult, really, than moving from
>> CentOS 7 to CentOS 8.
>
> That?s not my experience.
>
> I keep several of my packages running on CentOS and Debian (and more) and
> I keep running into several common problems:
>
> 1. The package names are often different, and not always differing by an
> obvious translation rule.  For instance, it?s ?openldap-devel? on CentOS
> but ?libldap2-dev? on Debian, where the normal rule would make it
> ?libopenldap-dev?.  Why the difference?  Dunno, but I have to track such
> things down when setting up scripts that do cross-distro builds.  If I
> automate that translation, now I?m setting myself up for a future breakage
> when the package names change again.  (libldap3-dev?)
>
> 2. Some packages simply won?t be available.  Most often this happens in
> the Debian ? CentOS direction, but I?ve run into cases going the other
> way.  Just for one, I currently have to install NPM from source on Debian
> because the platform version won?t work properly with the platform version
> of Node, last time I tested it.  Why?  Same answer as above.
>
> 3. Debian adopted systemd, but it didn?t adopt the rest of the Red Hat
> userland tooling.  For instance, it?s firewalld on CentOS, UFW on Ubuntu,
> and raw kernel firewall manipulation on Debian unless you install one of
> those two.  And then, which?
>
> 4. Network configuration is almost entirely different unless you turn off
> all the automation on all platforms, in which case you might as well
> switch to macOS or FreeBSD for all the good your muscle memory and
> training will do you.
>
> I?m not saying ?don?t do it,? but to say it?s as smooth as from CentOS 7
> to 8?  Hard sell.
>
> I?ll give you one mulligan: the changes to the security rules in CentOS 8
> caused a huge upheaval for one of my applications, since it basically
> stopped it from running, being naughty in Red Hat?s omnisciently
> beneficent eyes.  We spent about a year fixing breakages due to 25 years
> of built-up assumptions about what was correct and sensible, which don?t
> affect us on other Linuxes because they didn?t implement the same SELinux
> rules.
>
> The details aren?t super-important, because the real take-away is this:
> it?s always *something.*
>
> (For those that must know, the biggie was that our systemd-based service
> used to run from /home/$APPNAME but that?s a no-no on C8 now.  Moving it
> all under /opt/$APPNAME and rearranging it all according to LFS rules,
> then finding and fixing all the places we depended on such paths was
> *painful*.)
Thanks for the big fat warning, now I know exactly what I'll have to do
sooner or later :)

Part of the problem is this: there was FHS 2.x and all was good. Then some
devs started with new tools like systemd and didn't care about FHS and
reinvented some new wheels. Later, after the new facts were established,
the FHS was adjusted to the new taste of things and people like you and me
were left with the mess :)

Simon

On 2/4/21 1:23 PM, Warren Young wrote:
> On Feb 4, 2021, at 8:39 AM, Lamar Owen <lowen at pari.edu> wrote:
>> I posted a pretty complete rundown on the scientific linux users
mailing list, so I won't recap it all here.
> Link?
Yeah, I forgot to post the link... sorry about that.? Akemi beat me to 
it, so I'll not repost.? Thanks, Akemi!
>> the transition was not any more difficult, really, than moving from
CentOS 7 to CentOS 8.
> That?s not my experience.

I thought long and hard about it before posting to the CentOS lists, 
because I don't want to be perceived as advocating for a particular 
transition path.? I am exploring different paths, and I started with my 
own workstation.? Of course workstation needs are going to be some 
different than server needs; and thus I posted here to sincerely see 
others' experiences.? Thanks for the great information!
> I keep several of my packages running on CentOS and Debian (and more) and I
keep running into several common problems:
>
> 1. The package names are often different, and not always differing by an
obvious translation rule.  ...

This one I've run across, and yes the package name differences are 
annoying.? I did some cross-packaging myself back in the day, all on RPM 
systems, and I had to do special cases for SuSE since the package names 
are quite different there, too.? Why are they different?? Different 
naming standards developed at different times by different people with 
different ideas; I would consider it very odd if different distributions 
had identical package naming after going through such different paths.?? 
I consider this to be very minor in comparison to other items.
> 2. Some packages simply won?t be available.  Most often this happens in the
Debian ? CentOS direction, but I?ve run into cases going the other way. ...

Yes, true.? True going from C7 to C8, too, especially if you rely on 
third-party repositories for packages.
> 3. Debian adopted systemd, but it didn?t adopt the rest of the Red Hat
userland tooling.  For instance, it?s firewalld on CentOS, UFW on Ubuntu, and
raw kernel firewall manipulation on Debian unless you install one of those two. 
And then, which?
That one is more serious for the server than the other two, for sure.? 
If migrating from CentOS I would probably go with firewalld.? I haven't 
decided yet in my evaluations.? But I put an ACL on the Cisco 7609's 
here on all server-bound traffic, so not yet an important consideration 
for me.? But it will be soon enough.
> 4. Network configuration is almost entirely different unless you turn off
all the automation on all platforms, in which case you might as well switch to
macOS or FreeBSD for all the good your muscle memory and training will do you.

Again I started with my workstation, and since I chose a GNOME install I 
got all the NetworkManager tooling; once NM is in control there aren't 
too many differences.? Red Hat and Debian chose different paths for 
system configuration information, that much is for sure.? But I try to 
not EVER use strict muscle memory in a sysadmin role, since that is 
guaranteed to break on CentOS version upgrades too, and I have too many 
different systems to rely on muscle memory, although I do find myself 
using that out of habit.? Constantly doing things differently keeps me 
sharp, hopefully, at least.? YMMV.
> I?m not saying ?don?t do it,? but to say it?s as smooth as from CentOS 7 to
8?  Hard sell.
On my workstation with my workload it hasn't so far been that big of a 
deal and was less of a hassle than what C7 to C8 was; that is my 
specific workstation and my specific workloads, of course.? I am getting 
ready to take an older internal server that is on CentOS 6 (yes, I know, 
EOL etc etc) to Debian 10 as a test; I had planned to take to CentOS 8 
in early December, at least until the news hit, causing me to hold it up 
until I had a better idea of the issues.? I'll know a whole lot more 
after that is done.

> ... it?s always *something.*
True that.


I will say that thus far my experience is mostly on the workstation, 
although I did set up a couple of test servers.? One is an old Dell 
PE1950; had no issues with the Red Hat-unsupported Dell PERC RAID 
controller and the C8-removed PCI IDs from the megaraid_sas driver.? The 
other one is....... well, INTERESTING: for grins and giggles I installed 
the 32-bit version on an ancient IBM eSeries x330 rocking dual Pentium 
III-S? 1.4GHz Tualatins with a full 4GB of RAM and dual 146GB U320 SCSI 
drives - maximum performance you'll get from a Pentium III (and almost 
as good as a much newer 64-bit Netburst-based Xeon at 3.2GHz; Byte 
Unixbench on the Tualatins at 324.6; on the Noconas 491.4, at over twice 
the clock speed, too). And it's fun to just be able to say I have one of 
the most maxxed-out Pentium III-S systems you'll find, at least without 
overclocking or going past a dually.? YMMV, and you might not think it 
so much fun, but you'll see my idea of computer fun in my SL-users list 
post when you get down to the paragraph about the TRS-80 emulators.....

Hi,


On Thu, 4 Feb 2021, Warren Young wrote:
> On Feb 4, 2021, at 8:39 AM, Lamar Owen <lowen at pari.edu> wrote:
>>
>> I posted a pretty complete rundown on the scientific linux users
mailing
>> list, so I won't recap it all here.
>
> Link?
>
>> the transition was not any more difficult, really, than moving from
CentOS 7
>> to CentOS 8.
>
> That's not my experience.
>
> I keep several of my packages running on CentOS and Debian (and more) and I
> keep running into several common problems:
>
> 1. The package names are often different, and not always differing by an
> obvious translation rule.  For instance, it's openldap-devel on
> CentOS but libldap2-dev on Debian, where the normal rule would make
> it libopenldap-dev. Why the difference? Dunno, but I have to track
> such things down when setting up scripts that do cross-distro builds. If I
> automate that translation, now I'm setting myself up for a future
breakage
> when the package names change again.  (libldap3-dev?)
Yep!! It is a pita when trying to get things running for the first time.
I started this journey on a couple samba DC's before the Red Hat
announcement.
Libraries are almost always different names but even common packages like dhcp
and bind have different names, configuration files and commands to do the same
thing. Most of it is not that hard to figure out but it does take time to 
do it and it is a lot more work than going from CentOS 7 to CentOS 8.
> 2. Some packages simply won't??t be available.  Most often this happens
in
> the Debian -> CentOS direction, but I've run into cases going the
other
> way.  Just for one, I currently have to install NPM from source on Debian
> because the platform version won't work properly with the platform
version
> of Node, last time I tested it.  Why?  Same answer as above.
>
> 3. Debian adopted systemd, but it didn't adopt the rest of the Red Hat
> userland tooling.  For instance, it's firewalld on CentOS, UFW on
Ubuntu,
> and raw kernel firewall manipulation on Debian unless you install one of
those
> two.  And then, which?
Their systemd implementation is my biggest problem with Debian based systems.
Debian moved to systemd but only partially. Apparently Debian decided to only
kind of move to systemd. Tab completion on systemd commands does not work.
If you look in /etc/init.d there are a bunch of sysv init scripts. Converting to
systemd is a big enough pita without having a system that is half sysv and half
systemd. It would be nice if they would make up their minds. Either bite the
bullet and convert everything to systemd or stay with sysv :-(

Before somebody brings it up, yes, I know there is Devuan. I do not wish to go
even further back in time.
>
> 4. Network configuration is almost entirely different unless you turn off
all
> the automation on all platforms, in which case you might as well switch to
> macOS or FreeBSD for all the good your muscle memory and training will do
you.
+1 It is a whole different process. It takes me back about 15 years.
>
> I'm not saying don't do it, but to say it's as smooth as
> from CentOS 7 to 8?  Hard sell.
Agreed!!

Regards,

-- 
Tom			me at tdiehl.org