Hi Leon,
Thanks a lot for taking a look at my test and the suggestion.
We have tested this suggestion
echo never > /sys/kernel/mm/transparent_hugepage/enabled
Unfortunately, the Ram usage was the same with this option changed as
well.
Just to give some additional context to the original question to all,
We see 10-25 percent more RSS on CentOS 8 ( as compared to CentOS 7) for
the same Java vm process that we are benchmarking . And in that case , the
RSS growth for the executable and the shared library of our application is
considerably higher.
Perhaps, for the big server use cases the rss growth did not matter, but
for us, we are trying to use it in an embedded scenario and this is a major
issue.
We will be glad to hear any further suggestions/ tests that we could check
to further understand the differences between the 2 versions.
Regards,
-Anand
On Wed, Jul 7, 2021, 5:30 PM <centos-request at centos.org> wrote:
> Send CentOS mailing list submissions to
> centos at centos.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.centos.org/mailman/listinfo/centos
> or, via email, send a message with subject or body 'help' to
> centos-request at centos.org
>
> You can reach the person managing the list at
> centos-owner at centos.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of CentOS digest..."
>
>
> Today's Topics:
>
> 1. RSS usage on centos8.4 is higher for user-application
> compared to centos7.9 (Anand Babu)
> 2. Re: RSS usage on centos8.4 is higher for user-application
> compared to centos7.9 (Leon Fauster)
> 3. Re: Centos versions in the future? (Nikolaos Milas)
> 4. Re: Centos versions in the future? (J Martin Rushton)
> 5. Re: Centos versions in the future? (Nikolaos Milas)
> 6. Re: Centos versions in the future? (Leon Fauster)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 6 Jul 2021 15:02:49 +0200
> From: Anand Babu <mail2anand at gmail.com>
> To: centos at centos.org
> Subject: [CentOS] RSS usage on centos8.4 is higher for
> user-application compared to centos7.9
> Message-ID:
> <CAB5++c4-Z> 7FheHD02RW6YLP6d-trwCqqQqvDdAKsoBZF4tbUg at
mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> Hi Centos Community,
>
> This is my first time here and I apologize in advance if I made a mistake
> here and hope you will correct me , if i made any.
>
>
> With that said,
>
> This is a native example that was written to narrow down the higher RAM
> usage that we see when we are using Centos8 as opposed to Centos7.
>
> The c-code is very small and looks like this :
>
> #include<stdio.h>
>
> #include<unistd.h>
>
> int main()
>
> {
>
> pause();
>
> return 0;
>
> }
>
> I compile and run the binary on centos7 and memory backed section of pmap
> output looks like below:
>
> pmap -X $(pidof sleep) | head -n -2 | awk '{ if (NR > 2 &&
$5 > 0 )
> printf "%12s %8s %8s %4s %s\n", $1, $6, $7, $2,$13}'
> 00400000 4 4 r-xp sleep
> 00600000 4 4 r--p sleep
> 00601000 4 4 rw-p sleep*7f54514f9000 1808
> 204 r-xp libc-2.17.so <http://libc-2.17.so>
> *7f54516bd000 2044 0 ---p libc-2.17.so
> 7f54518bc000 16 16 r--p libc-2.17.so
> 7f54518c0000 8 8 rw-p libc-2.17.so
> 7f54518c7000 136 108 r-xp ld-2.17.so
> 7f5451ae8000 4 4 r--p ld-2.17.so
> 7f5451ae9000 4 4 rw-p ld-2.17.so
>
> and ps says the following:
>
> ** ps -o rss= pidof sleep 352**
>
> Running the same binary on centos8 leads to
>
> ** ps -o rss= pidof pause 784 **
>
> and the pmap output looks like below:
>
> 00400000 4 4 r-xp pause
> 00600000 4 4 r--p pause
> 00601000 4 4 rw-p pause* 7f24029a8000 1776
> 788 r-xp libc-2.28.so <http://libc-2.28.so> *
> 7f2402b64000 2044 0 ---p libc-2.28.so
> 7f2402d63000 16 16 r--p libc-2.28.so
> 7f2402d67000 8 8 rw-p libc-2.28.so
> 7f2402d6d000 176 176 r-xp ld-2.28.so
> 7f2402f99000 4 4 r--p ld-2.28.so
> 7f2402f9a000 8 8 rw-p ld-2.28.so
>
> For running the same executable, the libc.so takes 788KB(204KB on centos7).
>
> Note:
>
> 1.
>
> This is not the only library that is showing this behavior, but we see
> the same behavior for other shared-library as well as executables that
> were
> compiled on centos7. Running the same executables/shared objects take a
> higher amount of pages on centos8 than on centos7.
> 2.
>
> Since the glibc version on Centos8 was 2.28 , i have compiled 2.17
> version and then used patchelf to patch the centos7 built binary to make
> use of 2.17 glibc on centos8 host like patchelf --set-interpreter
>
>
>
/home/babu/RSS_measurement/2_17_downloaded/glibc_home/lib/ld-linux-x86-64.so.2
> --set-rpath /home/babu/RSS_measurement/2_17_downloaded/glibc_home/lib
> /tmp/pause
>
> and then run it under LD_DEBUG=libs and verified the 2.17 version of libc
> libraries were used and still the memory usage on centos8 was higher than
> on centos7 (about the same numbers as above).
>
> What could be the reason for the higher RAM usage on centos8 vs centos7? I
> can imagine some defaults have changed on centos8 and that has meant this
> impact and i could change this default and could reproduce the same RSS
> numbers on centos8 as well?
>
>
> I am posting my environment details in the P.S below.
>
> Regards,
>
> -Anand
>
>
> P.S:
>
> cat /etc/centos-release
>
> CentOS Linux release 8.4.2105
>
> [root at localhost RSS_measurement]# cat /etc/centos-release
>
> CentOS Linux release 7.9.2009 (Core)
>
> I am running both the centos versions under Virtualbox.
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 6 Jul 2021 17:59:11 +0200
> From: Leon Fauster <leonfauster at googlemail.com>
> To: centos at centos.org
> Subject: Re: [CentOS] RSS usage on centos8.4 is higher for
> user-application compared to centos7.9
> Message-ID: <4e84ea30-9d8b-2e9d-a898-e469303a537c at gmail.com>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> On 06.07.21 15:02, Anand Babu wrote:
> > Hi Centos Community,
> >
> > This is my first time here and I apologize in advance if I made a
mistake
> > here and hope you will correct me , if i made any.
> >
> >
> > With that said,
> >
> > This is a native example that was written to narrow down the higher
RAM
> > usage that we see when we are using Centos8 as opposed to Centos7.
> >
> > The c-code is very small and looks like this :
> >
> > #include<stdio.h>
> >
> > #include<unistd.h>
> >
> > int main()
> >
> > {
> >
> > pause();
> >
> > return 0;
> >
> > }
> >
> > I compile and run the binary on centos7 and memory backed section of
pmap
> > output looks like below:
> >
> > pmap -X $(pidof sleep) | head -n -2 | awk '{ if (NR > 2
&& $5 > 0 )
> > printf "%12s %8s %8s %4s %s\n", $1, $6, $7, $2,$13}'
> > 00400000 4 4 r-xp sleep
> > 00600000 4 4 r--p sleep
> > 00601000 4 4 rw-p sleep*7f54514f9000 1808
> > 204 r-xp libc-2.17.so <http://libc-2.17.so>
> > *7f54516bd000 2044 0 ---p libc-2.17.so
> > 7f54518bc000 16 16 r--p libc-2.17.so
> > 7f54518c0000 8 8 rw-p libc-2.17.so
> > 7f54518c7000 136 108 r-xp ld-2.17.so
> > 7f5451ae8000 4 4 r--p ld-2.17.so
> > 7f5451ae9000 4 4 rw-p ld-2.17.so
> >
> > and ps says the following:
> >
> > ** ps -o rss= pidof sleep 352**
> >
> > Running the same binary on centos8 leads to
> >
> > ** ps -o rss= pidof pause 784 **
> >
> > and the pmap output looks like below:
> >
> > 00400000 4 4 r-xp pause
> > 00600000 4 4 r--p pause
> > 00601000 4 4 rw-p pause* 7f24029a8000 1776
> > 788 r-xp libc-2.28.so <http://libc-2.28.so> *
> > 7f2402b64000 2044 0 ---p libc-2.28.so
> > 7f2402d63000 16 16 r--p libc-2.28.so
> > 7f2402d67000 8 8 rw-p libc-2.28.so
> > 7f2402d6d000 176 176 r-xp ld-2.28.so
> > 7f2402f99000 4 4 r--p ld-2.28.so
> > 7f2402f9a000 8 8 rw-p ld-2.28.so
> >
> > For running the same executable, the libc.so takes 788KB(204KB on
> centos7).
> >
> > Note:
> >
> > 1.
> >
> > This is not the only library that is showing this behavior, but we
> see
> > the same behavior for other shared-library as well as executables
> that were
> > compiled on centos7. Running the same executables/shared objects
> take a
> > higher amount of pages on centos8 than on centos7.
> > 2.
> >
> > Since the glibc version on Centos8 was 2.28 , i have compiled 2.17
> > version and then used patchelf to patch the centos7 built binary
to
> make
> > use of 2.17 glibc on centos8 host like patchelf --set-interpreter
> >
> >
>
/home/babu/RSS_measurement/2_17_downloaded/glibc_home/lib/ld-linux-x86-64.so.2
> > --set-rpath /home/babu/RSS_measurement/2_17_downloaded/glibc_home/lib
> > /tmp/pause
> >
> > and then run it under LD_DEBUG=libs and verified the 2.17 version of
libc
> > libraries were used and still the memory usage on centos8 was higher
than
> > on centos7 (about the same numbers as above).
> >
> > What could be the reason for the higher RAM usage on centos8 vs
centos7?
> I
> > can imagine some defaults have changed on centos8 and that has meant
this
> > impact and i could change this default and could reproduce the same
RSS
> > numbers on centos8 as well?
> >
> >
>
> How does the results look like after doing:
>
> echo never > /sys/kernel/mm/transparent_hugepage/enabled
>
> --
> Leon
>
>
>
> ------------------------------
>
> Message: 3
> Date: Wed, 7 Jul 2021 12:44:15 +0300
> From: Nikolaos Milas <nmilas at noa.gr>
> To: CentOS mailing list <centos at centos.org>
> Subject: Re: [CentOS] Centos versions in the future?
> Message-ID: <6b3c136f-5aae-1b0a-e65c-868377875f7a at noa.gr>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> On 30/4/2021 7:27 ?.?., Gionatan Danti wrote:
>
> >
> > The correct answer is to buy RH: fine. But do not let Stream touch
> > anything which require a kABI compatible modules. As said above, the
> > Stream move is squarely addresses *cloud* vendor requests and needs.
> > Again, fine. But please leave apart the RH comparison, this is not
> > going to help Stream.
> >
> > Again, don't let me wrong: I wishes the best to Stream, and I will
use
> > it where appropriate. But "where" is much smaller today than
> > yesterday. But this aside, I really thank you all CentOS maintainer
> > for your monumental work, and I really hope Stream will be a success.
>
> I re-visit this thread, since it is crucial for CentOS 8 users.
>
> RESF / Rocky Linux is gaining worldwide recognition and sets itself as
> the primary organization / platform to become the CentOS 8 heir /
> successor in the future.
>
> Google and Microsoft become RESF sponsors/partners:
>
> https://rockylinux.org/news/community-update-june-2021/
>
> And so IBM/RH lose the tremendous advantage they had by owning the
> CentOS project, which - it seems - never evaluated correctly.
>
> From now on, it is clear that hundreds of thousands of CentOS
> installations will be migrating to Rocky Linux.
>
> I also wish the best of success to CentOS Stream, but this is not what
> the CentOS community expected.
>
> My 2c.
> Nick
>
>
>
> ------------------------------
>
> Message: 4
> Date: Wed, 7 Jul 2021 10:47:16 +0100
> From: J Martin Rushton <martinrushton56 at btinternet.com>
> To: centos at centos.org
> Subject: Re: [CentOS] Centos versions in the future?
> Message-ID: <e6298010-bece-df14-98d0-9f8b5e04e8e2 at btinternet.com>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> There's also Alma, which is where I've gone after being with CentOS
> since 5.3
>
> On 07/07/2021 10:44, Nikolaos Milas wrote:
> > On 30/4/2021 7:27 ?.?., Gionatan Danti wrote:
> >
> >>
> >> The correct answer is to buy RH: fine. But do not let Stream touch
> >> anything which require a kABI compatible modules. As said above,
the
> >> Stream move is squarely addresses *cloud* vendor requests and
needs.
> >> Again, fine. But please leave apart the RH comparison, this is not
> >> going to help Stream.
> >>
> >> Again, don't let me wrong: I wishes the best to Stream, and I
will use
> >> it where appropriate. But "where" is much smaller today
than
> >> yesterday. But this aside, I really thank you all CentOS
maintainer
> >> for your monumental work, and I really hope Stream will be a
success.
> >
> > I re-visit this thread, since it is crucial for CentOS 8 users.
> >
> > RESF / Rocky Linux is gaining worldwide recognition and sets itself as
> > the primary organization / platform to become the CentOS 8 heir /
> > successor in the future.
> >
> > Google and Microsoft become RESF sponsors/partners:
> >
> > ?? https://rockylinux.org/news/community-update-june-2021/
> >
> > And so IBM/RH lose the tremendous advantage they had by owning the
> > CentOS project, which - it seems - never evaluated correctly.
> >
> > From now on, it is clear that hundreds of thousands of CentOS
> > installations will be migrating to Rocky Linux.
> >
> > I also wish the best of success to CentOS Stream, but this is not what
> > the CentOS community expected.
> >
> > My 2c.
> > Nick
> >
> > _______________________________________________
> > CentOS mailing list
> > CentOS at centos.org
> > https://lists.centos.org/mailman/listinfo/centos
>
> --
> J Martin Rushton MBCS
>
>
> ------------------------------
>
> Message: 5
> Date: Wed, 7 Jul 2021 13:07:12 +0300
> From: Nikolaos Milas <nmilas at noa.gr>
> To: CentOS mailing list <centos at centos.org>
> Subject: Re: [CentOS] Centos versions in the future?
> Message-ID: <ce7e72f7-7159-78ee-4576-e75e806025f7 at noa.gr>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> On 7/7/2021 12:47 ?.?., J Martin Rushton via CentOS wrote:
>
> > There's also Alma, which is where I've gone after being with
CentOS
> > since 5.3
>
> AlmaLinux is a great project too, IMHO, but things show that the new
> industry standard (replacing CentOS) will probably be Rocky Linux.
>
> (Yes, RHEL **AND** CentOS have indeed been industry standards - the
> point of reference -, IMHO, and this is what IBM/RHEL have failed to
> realize: You don't alter a point of reference.)
>
> It is interesting to see what Service Providers will do with their (huge
> numbers of) CentOS installations, when they migrate...
>
> From the users/admins' perspective it is to their interest to have
> robust and healthy alternatives.
>
> In our org, I am now using Rocky Linux on new installations (without
> issues) and will be migrating several CentOS 8 boxes to Rocky Linux as
> well.
>
> Cheers,
> Nick
>
>
>
> ------------------------------
>
> Message: 6
> Date: Wed, 7 Jul 2021 12:28:58 +0200
> From: Leon Fauster <leonfauster at googlemail.com>
> To: centos at centos.org
> Subject: Re: [CentOS] Centos versions in the future?
> Message-ID: <9d2d8304-6779-a452-1a8b-62c58d7722e1 at gmail.com>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> On 07.07.21 12:07, Nikolaos Milas wrote:
> > On 7/7/2021 12:47 ?.?., J Martin Rushton via CentOS wrote:
> >
> >> There's also Alma, which is where I've gone after being
with CentOS
> >> since 5.3
> >
> > AlmaLinux is a great project too, IMHO, but things show that the new
> > industry standard (replacing CentOS) will probably be Rocky Linux.
> >
> > (Yes, RHEL **AND** CentOS have indeed been industry standards - the
> > point of reference -, IMHO, and this is what IBM/RHEL have failed to
> > realize: You don't alter a point of reference.)
>
>
> It should not be forgotten that Rocky Linux will be a 1:1 rebuild, also
> in the future. So, to shape this future everyone is invited to
> participate at CentOS Stream. This is a great future or not?
>
>
>
> > It is interesting to see what Service Providers will do with their
(huge
> > numbers of) CentOS installations, when they migrate...
> >
> > From the users/admins' perspective it is to their interest to
have
> > robust and healthy alternatives.
> >
> > In our org, I am now using Rocky Linux on new installations (without
> > issues) and will be migrating several CentOS 8 boxes to Rocky Linux as
> > well.
> >
> > Cheers,
> > Nick
> >
> > _______________________________________________
> > CentOS mailing list
> > CentOS at centos.org
> > https://lists.centos.org/mailman/listinfo/centos
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos
>
>
> ------------------------------
>
> End of CentOS Digest, Vol 198, Issue 6
> **************************************
>