I reboot when I yum update to a new kernel or systemd, which seems to come out about once a month. Should I do it for this week's glibc? Is that "core" enough to justify a reboot or should I wait for the next kernel update? I know the glibc update was mainly to handle the new Japanese calendar, so that shouldn't affect my usage. So my question is more about how shared libraries work and whether anything bad would happen with different forks of running services (mainly the mail suite with dovecot and the various content scanners launched by sendmail) running different versions of the library based on when they were started.
In article <A39602BA6D703571AC020AA0@[192.168.1.16]>, Kenneth Porter <shiva at sewingwitch.com> wrote:> I reboot when I yum update to a new kernel or systemd, which seems to come > out about once a month. Should I do it for this week's glibc? Is that > "core" enough to justify a reboot or should I wait for the next kernel > update? I know the glibc update was mainly to handle the new Japanese > calendar, so that shouldn't affect my usage. So my question is more about > how shared libraries work and whether anything bad would happen with > different forks of running services (mainly the mail suite with dovecot and > the various content scanners launched by sendmail) running different > versions of the library based on when they were started.That shouldn't matter. The running programs will have mapped the original glibc into memory, which will create a reference to the original inode, even though the directory entries pointing to it are gone. See the output of "lsof" for one of those processes, and you will see the libraries tagged as (deleted). Any program started after the glibc update will open and map the new libraries, independently of any open instances of the old ones. Both old and new libraries will occupy their own separate disk space until the last reference to the old library is closed, by terminating all programs using it, at which time the disk space occupied by the old libraries will be released. Cheers Tony -- Tony Mountifield Work: tony at softins.co.uk - http://www.softins.co.uk Play: tony at mountifield.org - http://tony.mountifield.org
On 4/13/19 3:32 AM, Kenneth Porter wrote:> I reboot when I yum update to a new kernel or systemd, which seems to > come out about once a month. Should I do it for this week's glibc? Is > that "core" enough to justify a reboot or should I wait for the next > kernel update?This is basically your decision, and you seem to know what update brings. On an uptime note: in my observation since about the time kernel 2.6 was introduced Linux has to be rebooted on average every 45 days (either kernel or glibc security update). This was mainly what made me move my servers from CentOS Linux to FreeBSD. Valeri> I know the glibc update was mainly to handle the new > Japanese calendar, so that shouldn't affect my usage. So my question is > more about how shared libraries work and whether anything bad would > happen with different forks of running services (mainly the mail suite > with dovecot and the various content scanners launched by sendmail) > running different versions of the library based on when they were > started. _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos-- ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++
On Apr 13, 2019, at 2:32 AM, Kenneth Porter <shiva at sewingwitch.com> wrote:> > I reboot when I yum update to a new kernel or systemd, which seems to come out about once a month.You can use similar logic as in Tony Mountfield?s answer to put off reboots in those cases as well. If the reason for the kernel update is a bug in a Realtek NIC driver but your systems all use Intel NICs, you don?t need to reboot. Let?s get concrete. Just a few days ago, this CVE was filed against the Linux kernel: https://nvd.nist.gov/vuln/detail/CVE-2019-11191 I assume CentOS doesn?t ship any a.out binaries, so this bug is of no consequence to most CentOS systems. For it to matter to your systems, your threat model must either allow: 1. Arbitrary code upload by someone with root privileges so they can setuid a newly uploaded a.out binary. The only way such a situation is not already Game Over would be something like a VPS host where there are multiple ?root? privilege levels. If you?re not running such a hosting service, you probably don?t care about this bug. 2. Local staff to create a.out binaries and setuid them. But why would that happen? That?s two very uncommon conditions back to back. On top of that, the threat model then must include the ability for your attacker to run one of these binaries; if the threat model is network outsiders only and these are not network services, the bug *still* doesn?t affect you. Now let?s take systemd. Systemd isn?t a single binary, and most of those binaries don?t run continuously. (On a near-stock CentOS 7 VM I have here, only 5 of the 41 programs under bin/ in the systemd RPM are running right now.) If the systemd component being updated doesn?t run continuously or can safely be restarted individually, you don?t need to reboot. The component might not be running at upgrade time, or it might be easily restarted if it is running. The glibc updates can also be put off, depending on the bug in question and the system?s threat model. If you deem that the only threats worth responding to are those from the network, with everything internal to the server being deemed ?good,? then the questions become ?What?s listening to the network, can it/they be restarted, and which ones use affected glibc facilities?? Let?s take a recent glibc CVE as an example: https://nvd.nist.gov/vuln/detail/CVE-2019-9169 If your network listening services aren?t doing case-insensitive POSIX regex matches, this bug cannot affect them, so under our stated threat model, the network services don?t need to be restarted, much less the whole system. If you have network-listening services that *are* doing case-insensitive POSIX regex matches, then I assume the bug must only be happening with *particular* regexes, else we?d have learned of this bug decades ago, so your threat model must also allow the attacker to provide the regex. That excludes, for example, regexes in your Apache configuration file, unless you?re running a shared web hosting service and allow arbitrary changes to the Apache config.> I know the glibc update was mainly to handle the new Japanese calendarNot ?mainly,? ?solely.?> So my question is more about how shared libraries work and whether anything bad would happen with different forks of running services (mainly the mail suite with dovecot and the various content scanners launched by sendmail) running different versions of the library based on when they were started.As Tony said, each running binary continues with its prior copy of glibc and newly launched binaries get the new one. Unless you?ve got a set of binaries that can end up with different glibc underpinnings and they are passing around Japanese date strings with the assumption that they agree on their interpretation, I can?t see how this can affect you. Every CVE does not affect everybody, but Red Hat has to respond to most[*] of those affecting the code bases behind the binaries they build, ship, and support, because chances are, not doing so will affect some nontrivial subset of their user base. But whether the bug affects *you* is a wholly different question. That?s why they publish these advisories, and why those advisories often link to further information. You have to be willing and able to absorb and analyze this information if you don?t want to fall back on generic advice like ?Reboot on every kernel, glibc, or systemd update.? On the other hand, maybe you have good reason to reboot once a month or so anyway, and all of this provides a convenient excuse: ?Because security.? You might be subject to an uptime SLA that excludes security reboots, which let you slip other maintenance downtime into those reboot windows. [*] I assume there are conditions that would lead Red Hat to ignore a CVE that does affect code it ships, but I have no ready examples. If it happens, I trust their judgement.
On Sat, Apr 13, 2019 at 01:32:11AM -0700, Kenneth Porter wrote:> I reboot when I yum update to a new kernel or systemd, which seems to come > out about once a month. Should I do it for this week's glibc? Is that "core" > enough to justify a reboot or should I wait for the next kernel update? I > know the glibc update was mainly to handle the new Japanese calendar, so > that shouldn't affect my usage. So my question is more about how shared > libraries work and whether anything bad would happen with different forks of > running services (mainly the mail suite with dovecot and the various content > scanners launched by sendmail) running different versions of the library > based on when they were started.As a general rule we recommend that you reboot after updates for the kernel, glibc, ssh/openssl and systemd due to a lingering bug that breaks connectivity to dbus. Rebooting activates the new kernel and as almost all of our kernel updates are security updates it's quite highly recommended to do so. systemd has a lingering bug where it loses connectivity to dbus during a daemon restart and it doesn't get it back. Rebooting is the only method (at least that I am aware of) that addresses this. ssh/openssl are always security updates, you really want to reboot to ensure everything is running off the new libraries. We have seen odd issues with glibc updates without reboots that have cleared up after reboot; from memory I think they were locale issues but don't quote me on that. Considering how integral glibc is to the entire system you are strongly urged to reboot after said updates. John -- It's like picking a team when you really don't want to root for either team. -- Kristin Johnson, 23, on the two political parties, New York Times, 3 September 2010 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: <http://lists.centos.org/pipermail/centos/attachments/20190413/2497e252/attachment.sig>
Once upon a time, Tony Mountifield <tony at softins.co.uk> said:> That shouldn't matter. The running programs will have mapped the original > glibc into memory, which will create a reference to the original inode, even > though the directory entries pointing to it are gone. See the output of "lsof" > for one of those processes, and you will see the libraries tagged as (deleted).There can be problems when a running process tries to dlopen() a shared library file and gets a new version. For example, if a running process tries to do a host or user lookup (and hadn't used the method before), it could crash. Also, just because a process is still running OK with the old library doesn't mean you want it to; there could be a security change in the update that means old processes are vulnerable. You can use the "needs-restarting" program from the yum-utils package to see a list of processes that appear to need a restart due to library (or binary) changes. It isn't 100% accurate, but it is pretty close. There are some things that can't be restarted (like PID 1); then you should probably reboot. -- Chris Adams <linux at cmadams.net>