freebsd stable - Feb 2018 - 50 percent swap used, but "ps auxww" output shows no processes swapped out

Brandon Allbery allbery.b at gmail.com wrote on
Sat Feb 3 21:18:53 UTC 2018 :
> Swapping whole processes out is not really a thing any more. Individual
> pages are paged to/from memory; if a memory page has no backing file, it
> will be allocated a block in swap space as its backing storage.
> 
> (I'm not sure "W" status even means swap; I thought
whole-process swapping
> wasn't even supported any more.)
From what I've seen on the lists there is a technical distinction
made between "kernel stacks for the process no longer memory resident"
(swapped out) and other pages for the process having paged to disk and
not being resident.

But many tools do not seem to present that point of view and still
reflect an older view in the terminology used, including in
documentation. One has to interpret what one is shown as I understand.

As an example, top can show RES being zero despite the kernel stacks
for the process not having been moved to disk. RES zero might not
mean what one might expect about "swapped out".

I do not know if a W after the first letter in state (STAT) for
"ps auxww" track the kernel-stacks' resident-vs-not status for the
process or not. (Matching your not sure status.)

> On Sat, Feb 3, 2018 at 4:14 PM, Michael Voorhis <mvoorhis at
mcvau.net> wrote:
> 
> > Hi all,
> >
> > I've got an amd64 system running 11.1-STABLE r325027, with
something
> > like 20G of swap. "swapinfo" shows that half the swap is
used.
> >
> > So of course I'm curious to know which processes have been swapped
> > out. I'm not using any "tmpfs" filesystems; no ZFS, no
huge amounts of
> > wired-down memory. The system's got 16 processors and 128G of RAM.
"ps
> > auxww" output shows *no* processes that are swapped out (2nd
character
> > in "STAT" field is "W"). Not a single one. The
only process with a W in
> > the stat field at all is the "[intr]" kernel thread.
> >
> > What is using the swapspace
The so-called swapspace is really the paging/swap-space with
most of the use being paging typically. (As Brandon indicated.)

Once a page is paged out, if the process sticks around but
does not use or free the page, the page likely stays
paged-out. (I'm guessing some at the intended results for
default tuning --and that you probably are using default
tuning.) So the in-use swapspace is likely from one or
more existing processes that did page-outs earlier.

(Expect my descriptions to be over simplified, but hopefully
pointing in the right general direction.)
> > Please educate me.
> >

==Mark Millard
marklmi at yahoo.com
( markmi at dsl-only.net is
going away in 2018-Feb, late)

Also worth noting is that likely candidates for such pageouts include
long-lived daemons that are only needed, or which only need certain pages,
during startup/shutdown. So evicting only those pages to swap allows
optimal use of memory that would otherwise be wasted unnecessarily.

Studying demand paging and unified page management is worth the effort.
Modern OSes, including Windows, make heavy use of this to optimize memory
usage --- but it means that old-style notions of process memory usage will
leave you wondering how the numbers make any sense. (I see this quite a
lot; most people still seem to think the basic unit of memory management is
a process, not a memory page, despite unified page management being over a
decade old and basic demand paging going back to 4BSD days.)

On Sat, Feb 3, 2018 at 5:09 PM, Mark Millard via freebsd-stable <
freebsd-stable at freebsd.org> wrote:
> Brandon Allbery allbery.b at gmail.com wrote on
> Sat Feb 3 21:18:53 UTC 2018 :
>
> > Swapping whole processes out is not really a thing any more.
Individual
> > pages are paged to/from memory; if a memory page has no backing file,
it
> > will be allocated a block in swap space as its backing storage.
> >
> > (I'm not sure "W" status even means swap; I thought
whole-process
> swapping
> > wasn't even supported any more.)
>
> From what I've seen on the lists there is a technical distinction
> made between "kernel stacks for the process no longer memory
resident"
> (swapped out) and other pages for the process having paged to disk and
> not being resident.
>
> But many tools do not seem to present that point of view and still
> reflect an older view in the terminology used, including in
> documentation. One has to interpret what one is shown as I understand.
>
> As an example, top can show RES being zero despite the kernel stacks
> for the process not having been moved to disk. RES zero might not
> mean what one might expect about "swapped out".
>
> I do not know if a W after the first letter in state (STAT) for
> "ps auxww" track the kernel-stacks' resident-vs-not status
for the
> process or not. (Matching your not sure status.)
>
>
> > On Sat, Feb 3, 2018 at 4:14 PM, Michael Voorhis <mvoorhis at
mcvau.net>
> wrote:
> >
> > > Hi all,
> > >
> > > I've got an amd64 system running 11.1-STABLE r325027, with
something
> > > like 20G of swap. "swapinfo" shows that half the swap
is used.
> > >
> > > So of course I'm curious to know which processes have been
swapped
> > > out. I'm not using any "tmpfs" filesystems; no ZFS,
no huge amounts of
> > > wired-down memory. The system's got 16 processors and 128G of
RAM. "ps
> > > auxww" output shows *no* processes that are swapped out (2nd
character
> > > in "STAT" field is "W"). Not a single one.
The only process with a W in
> > > the stat field at all is the "[intr]" kernel thread.
> > >
> > > What is using the swapspace
>
> The so-called swapspace is really the paging/swap-space with
> most of the use being paging typically. (As Brandon indicated.)
>
> Once a page is paged out, if the process sticks around but
> does not use or free the page, the page likely stays
> paged-out. (I'm guessing some at the intended results for
> default tuning --and that you probably are using default
> tuning.) So the in-use swapspace is likely from one or
> more existing processes that did page-outs earlier.
>
> (Expect my descriptions to be over simplified, but hopefully
> pointing in the right general direction.)
>
> > > Please educate me.
> > >
>
>
> ==> Mark Millard
> marklmi at yahoo.com
> ( markmi at dsl-only.net is
> going away in 2018-Feb, late)
>
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at
freebsd.org"
>


-- 
brandon s allbery kf8nh                               sine nomine associates
allbery.b at gmail.com                                  ballbery at
sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

04.02.2018 5:09, Mark Millard via freebsd-stable wrote:
> I do not know if a W after the first letter in state (STAT) for
> "ps auxww" track the kernel-stacks' resident-vs-not status
for the
> process or not. (Matching your not sure status.)
A process has specific flag P_INMEM that is normally 1 but is cleader by a
kernel
when "swapout" procedure is executed for a process. Such process
cannot get CPU time etc.
top shows process name like "<name>" instead of just
"name" in that case.