CentOS - Jan 2021 - CentOS Stream suitability as a production webserver

Hello

I've recently discovered the announcement regarding the change in direction
for the CentOS project and I imagine like many others, I'm confused and
concerned about what this means moving forward.

I work for a small web development agency and we offer hosting as part of
our package to clients who need it. We have many CentOS 7 web servers
(DigitalOcean droplets) (LAMP/LEMP) that I look after and today I'm
thankful I have only migrated one of those to CentOS 8, given the recent
announcement about its curtailed EOL. I literally just went to the Wiki
today to confirm the EOL date for EL7 and boy am I glad I spotted it.

Given we are not developing drivers or applications (other than websites
and web applications), is the change a non-issue for my use-case? I've seen
it written that CentOS Stream is the "development version" of RHEL but
also
that we shouldn't have considered RHEL to be the beta for CentOS. Others
have said to think of CentOS more like RHEL RC-1. I just don't know how the
stability will compare and we have historically always chosen CentOS for
its stability (and of course price).

Sure, I could migrate to Ubuntu (I use this locally in WSL), but I've
become somewhat "comfy slippers" with CentOS and have built our setup
around it (including custom ansible scripts etc) and don't want to change
everything unncessarily.

Of course, a lot of this is somewhat dependent on what DigitalOcean will
decide to provide image wise moving forward.

I'm sorry if this has already been answered, I spent a good few hours
reading through the respective threads in the devel list and ended up more
confused than I started.

Cheers,
Jamie

On 05/01/2021 19:32, Jamie Burchell wrote:
> Hello
> 
> I've recently discovered the announcement regarding the change in
direction
> for the CentOS project and I imagine like many others, I'm confused and
> concerned about what this means moving forward.
> 
> I work for a small web development agency and we offer hosting as part of
> our package to clients who need it. We have many CentOS 7 web servers
> (DigitalOcean droplets) (LAMP/LEMP) that I look after and today I'm
> thankful I have only migrated one of those to CentOS 8, given the recent
> announcement about its curtailed EOL. I literally just went to the Wiki
> today to confirm the EOL date for EL7 and boy am I glad I spotted it.
> 
> Given we are not developing drivers or applications (other than websites
> and web applications), is the change a non-issue for my use-case? I've
seen
> it written that CentOS Stream is the "development version" of
RHEL but also
> that we shouldn't have considered RHEL to be the beta for CentOS.
Others
> have said to think of CentOS more like RHEL RC-1. I just don't know how
the
> stability will compare and we have historically always chosen CentOS for
> its stability (and of course price).
> 
> Sure, I could migrate to Ubuntu (I use this locally in WSL), but I've
> become somewhat "comfy slippers" with CentOS and have built our
setup
> around it (including custom ansible scripts etc) and don't want to
change
> everything unncessarily.
> 
> Of course, a lot of this is somewhat dependent on what DigitalOcean will
> decide to provide image wise moving forward.
> 
> I'm sorry if this has already been answered, I spent a good few hours
> reading through the respective threads in the devel list and ended up more
> confused than I started.
> 
> Cheers,
> Jamie
Hi Jamie,

Unfortunately no one can advise you as to what may be a suitable 
operating system for your business needs.

One thing is clear, the operating system you are currently running 
(CentOS Linux) is being brought to end of life, version 7 in 2024 and 
version 8 in 2021.

That gives you at least a year (for 8) if not longer to consider and 
evaluate alternatives. As your current OS will no longer exist, I would 
start with a blank sheet, look at the OSes that do exist and evaluate 
each based on it's merits and suitability for your business needs and 
requirements.

Cheers,
Phil

> Given we are not developing drivers or applications (other than
> websites
> and web applications), is the change a non-issue for my use-case? 
If you decide to go with Stream, you will need to test carefully each
version and use some kind of repository management - as there will be
no older version of the packages.
Thankfully the 'Boom boot manager' is now fully working, so you can
easily roll back an OS update.

If you decide that you don't want to fight with updates and the short
life cycle of Stream, you got plenty of clones that are available:
- Springdale Linux
- Oracle Enterprise Linux

And 2 more expected to come:
- Rocky Linux (founder of the original CentOS -> Gregory Kurtzer)
- Lenix (backed by CloudLinux)

I would prefer the full lifecycle of a RHEL clone instead of Stream.

Best Regards,
Strahil Nikolov

On 1/5/21 11:32 AM, Jamie Burchell wrote:
> is the change a non-issue for my use-case?

Probably.? For a lot of users, Stream is a drop-in replacement that's 
better than CentOS was, because it gets updates consistently and doesn't 
suffer from periods in which no updates are available, including 
security updates.

If security was a priority for you, as it was for me, then CentOS wasn't 
really suitable for public-facing services, but CentOS Stream might be.

If you're building software that you intend to deploy on RHEL, Stream 
might not be a suitable build root for you.? Compiling software in a 
Stream build root may result in a binary that has dependencies which 
aren't yet available in RHEL.? And if you're building kernel modules 
(like Phil @elrepo), then there is the issue that the kernel isn't 
subject to RHEL's ABI policy, but Red Hat developers have expressed 
interest in making the kernel interfaces more stable and using external 
kernel module builds as a test to flag interfaces that have changed.? So 
that situation may improve...

> 
> Given we are not developing drivers or applications (other than websites
> and web applications), is the change a non-issue for my use-case? I've
seen
> it written that CentOS Stream is the "development version" of
RHEL but also
> that we shouldn't have considered RHEL to be the beta for CentOS.
Others
> have said to think of CentOS more like RHEL RC-1. I just don't know how
the
> stability will compare and we have historically always chosen CentOS for
> its stability (and of course price).
There's been a lot of information and mis-information being bandied
around on websites from people who don't really quite understand what's
going on. I hope I don't contribute to the confusion! It wasn't helped
by the, frankly, heavy-handed way it was handled by RH.

One of the problems is that people are trying to put a label on what 8-
stream is - such as development version, or RC, or beta version or
whatever. To be honest all we can do is to try and understand what RH
want. As far as I understand it, 8-stream accumulates new versions of
packages that will collectively go to make up the next point release of
RHEL8. We have been told, and we can only take it at face value, that
the versions that go into 8-stream will be final, QC'd packages: they
are not test, development or beta versions, nor are they "work in
progress". 8-stream will be a complete and functioning, stable distro.
So rather than waiting to get the new versions of things once every 6
months, 8-stream gets them when they are ready.

The confusion about the "development" label is that RH said that 8-
stream will be the distro used for their development process. So
internally things will be developed and compiled in an 8-stream
environment. They have never said that the development packages will
ever be visible or available in 8-stream itself until they are ready to
be set free.

TBH I would have thought that this exactly how RH operate internally at
the moment - they must have, say, a pre-8.3 environment that they put
packages in so that when new packages are developed that can be
compiled and everything is compatible. I really can't imagine that
packages are developed in isolation until there's a big 8.3 compile
time. All they are doing is making that internal system a public thing.

Now it's certainly possible that from RH point of view, releasing the
packages into the wild is a very good way of finding bugs that might
have slipped through QC - there is after all already a steady stream of
updates between point releases. So the benefit for RH is that paying
customers get potentially fewer updates between releases, but the
implication is that 8-stream will be no less stable than CentOS 8
currently is.

The rhetoric from RH is that the tooling of the 8-stream system is not
fully in place yet, but should be soon.  Again, we can only take them
at their word and watch what happens. And I must stress that I am no RH
apologist: I think it was all handled incredibly badly by them and they
desperately need to get some change management experience!!

If you are considering using 8-stream then you need to understand that
there is no specific point-release configuration that you can base
things on - you cann't say that this is "equivalent to RHEL 8.5"
or
whatever; this is important if you need to use 3rd party drivers during
install as they are based on specific configurations (but hey, install
CentOS 8.2 and move to 8-stream from there and upgrade). Also the
lifetime of 8-stream is half what you've been used to - so come 2024,
it will die; but 9-stream will have existed for at least a couple of
years by then, so there is a roadmap. 

As for what you should do, than no one can really tell you. My advice
to others has been to watch, evaluate, test. If you are running bog
standard web servers with nothing exotic, then I have a feeling that 8-
stream will work; if you are running 3rd party apps on a web service
where versions matter, then you need to think carefully and consider
switching to one of the rebuild distros.
> 
> Of course, a lot of this is somewhat dependent on what DigitalOcean will
> decide to provide image wise moving forward.
I suspect that as more and more things become containerised (and boy do
I dislike containers), the actual underlying OS will become
considerably less important. 


P.

On 05/01/2021 20:32, Jamie Burchell wrote:
> Hello
> 
> I've recently discovered the announcement regarding the change in
direction
> for the CentOS project and I imagine like many others, I'm confused and
> concerned about what this means moving forward.
> 
<snip>
> 
> Given we are not developing drivers or applications (other than websites
> and web applications), is the change a non-issue for my use-case? I've
seen
> it written that CentOS Stream is the "development version" of
RHEL but also
> that we shouldn't have considered RHEL to be the beta for CentOS.
Others
> have said to think of CentOS more like RHEL RC-1. I just don't know how
the
> stability will compare and we have historically always chosen CentOS for
> its stability (and of course price).
> 
<snip>

Well, let me just quickly chime in this thread ...
If you have already automated things (or not btw) for CentOS 8, current
Stream (8-stream) will continue to just work.

For CentOS Infra, I started to deploy Stream nodes and it continues to
work fine.

Fun fact : new coming Stream buildsystem infra *is* build exclusively on
top of CentOS Stream ... hopefully that would give people confidence
about platform (dog fooding) :)

Will there be some changes suddenly happening faster than in usual
major.minor releases lifecyles ? yes

Will it differ really ? well, it's what coming in the same major.minor
version that people *are* waiting for ..

Is it perfect *now* ? probably not, but there is a chance to look at it
and it's up to (and not tied to Stream vs Linux effect imho)
sysadmin/devops engineers/$pick-your-title-here in charge of infra to
have validation platform before rolling out versions/updates/etc ...
(nothing *should* change here, except if one still manage single box
like in the 90's) ;-)

With my SysAdmin hat on, I'd say that the only real impacting bit is the
shorter lifetime (5y instead of 10), but with overlap between stream
versions, so one would have time to have a look, reflect in automation,
reinstall/migrate, enjoy

Just my 0.02$ here
-- 
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | twitter: @arrfab