On 18/06/2018 09:08, Erich Dollansky wrote:> Hi, > > On Sun, 17 Jun 2018 23:19:02 +0100 > tech-lists <tech-lists at zyxst.net> wrote: > >> freebsd-11-stable r333874, ZFS raidz1-0 (3x4TB disks), 128GB RAM, >> Swap: 4096M Total, 3502M Used, 594M Free, 85% Inuse > > this might not be related but I noticed that your swap space is small > compared to RAM size. I noticed on a much smaller Raspberry Pi, that it > runs into trouble when there is no swap even there is enough RAM > available. Is it easily possible for you to add some GB of swap space > and let the machine run then? > > How much swap do the other machines have?Hi, Yes, the machine with the problem uses the default 4GB swap. That's all the swap it has. The machine without issue has a swapfile installed on a SSD in addition to the default 4GB swap. problematic machine: Device 512-blocks Used Avail Capacity /dev/ada0p3 8388608 3.3G 714M 83% machine without a problem, it has swapfile installed: Device 512-blocks Used Avail Capacity /dev/ada0s1b 8262248 1.7G 2.2G 44% /dev/md0 65536000 1.9G 29G 6% Total 73798248 3.7G 32G 10% I added the swapfile a long time ago on this machine due to the same issue. But my problem isn't so much an out of swapspace problem; all this is, is a symptom. My problem is "why is it swapping out at all on a 128GB system and why is what's swapped out not being swapped back in again". thanks, -- J.
On Mon, Jun 18, 2018 at 7:27 AM, tech-lists <tech-lists at zyxst.net> wrote:> On 18/06/2018 09:08, Erich Dollansky wrote: > >> Hi, >> >> On Sun, 17 Jun 2018 23:19:02 +0100 >> tech-lists <tech-lists at zyxst.net> wrote: >> >> freebsd-11-stable r333874, ZFS raidz1-0 (3x4TB disks), 128GB RAM, >>> Swap: 4096M Total, 3502M Used, 594M Free, 85% Inuse >>> >> >> this might not be related but I noticed that your swap space is small >> compared to RAM size. I noticed on a much smaller Raspberry Pi, that it >> runs into trouble when there is no swap even there is enough RAM >> available. Is it easily possible for you to add some GB of swap space >> and let the machine run then? >> >> How much swap do the other machines have? >> > > Hi, > > Yes, the machine with the problem uses the default 4GB swap. That's all > the swap it has. The machine without issue has a swapfile installed on a > SSD in addition to the default 4GB swap. > > problematic machine: > Device 512-blocks Used Avail Capacity > /dev/ada0p3 8388608 3.3G 714M 83% > > machine without a problem, it has swapfile installed: > Device 512-blocks Used Avail Capacity > /dev/ada0s1b 8262248 1.7G 2.2G 44% > /dev/md0 65536000 1.9G 29G 6% > Total 73798248 3.7G 32G 10% > > I added the swapfile a long time ago on this machine due to the same issue. > > But my problem isn't so much an out of swapspace problem; all this is, is > a symptom. My problem is "why is it swapping out at all on a 128GB system > and why is what's swapped out not being swapped back in again". >What is the output of sysctl vm.overcommit? If this system is intended on being a VM host, then why don't you limit ARC to something reasonable like Total Mem - Projected VM Mem - Overhead = Ideal ARC . -- Adam
On Mon, Jun 18, 2018 at 5:27 AM, tech-lists <tech-lists at zyxst.net> wrote:> On 18/06/2018 09:08, Erich Dollansky wrote: > >> Hi, >> >> On Sun, 17 Jun 2018 23:19:02 +0100 >> tech-lists <tech-lists at zyxst.net> wrote: >> >> freebsd-11-stable r333874, ZFS raidz1-0 (3x4TB disks), 128GB RAM, >>> Swap: 4096M Total, 3502M Used, 594M Free, 85% Inuse >>> >> >> this might not be related but I noticed that your swap space is small >> compared to RAM size. I noticed on a much smaller Raspberry Pi, that it >> runs into trouble when there is no swap even there is enough RAM >> available. Is it easily possible for you to add some GB of swap space >> and let the machine run then? >> >> How much swap do the other machines have? >> > > Hi, > > Yes, the machine with the problem uses the default 4GB swap. That's all > the swap it has. The machine without issue has a swapfile installed on a > SSD in addition to the default 4GB swap. > > problematic machine: > Device 512-blocks Used Avail Capacity > /dev/ada0p3 8388608 3.3G 714M 83% > > machine without a problem, it has swapfile installed: > Device 512-blocks Used Avail Capacity > /dev/ada0s1b 8262248 1.7G 2.2G 44% > /dev/md0 65536000 1.9G 29G 6% > Total 73798248 3.7G 32G 10% > > I added the swapfile a long time ago on this machine due to the same issue. > > But my problem isn't so much an out of swapspace problem; all this is, is > a symptom. My problem is "why is it swapping out at all on a 128GB system > and why is what's swapped out not being swapped back in again". > > thanks, > -- > J.Small correction. Your problem is "why is it swapping out at all on a 128GB system ." Once pages are swapped out, they are never swapped back in until/unless they are needed. There is no reason to waste time/disk activity to swap pages back into memory unless they are required. RAM is always more valuable than swap. Ir is easy to write a process that eats up a large amount of memory, then goes idle without freeing it. The memory will get pages out, fill swap, and, unless the process terminates or becomes active, will consume up a great deal of swap space "forever". Firefox is a good example of this. I have to restart it every day or two and occasionally will run out of swap which results in a nearly deadlocked system. It can take many minutes to just kill firefox. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman at gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
Hi, On Mon, 18 Jun 2018 13:27:23 +0100 tech-lists <tech-lists at zyxst.net> wrote:> On 18/06/2018 09:08, Erich Dollansky wrote: > > > > On Sun, 17 Jun 2018 23:19:02 +0100 > > tech-lists <tech-lists at zyxst.net> wrote: > > > >> freebsd-11-stable r333874, ZFS raidz1-0 (3x4TB disks), 128GB RAM, > >> Swap: 4096M Total, 3502M Used, 594M Free, 85% Inuse > > > > this might not be related but I noticed that your swap space is > > small compared to RAM size. I noticed on a much smaller Raspberry > > Pi, that it runs into trouble when there is no swap even there is > > enough RAM available. Is it easily possible for you to add some GB > > of swap space and let the machine run then? > > > > How much swap do the other machines have? > > Yes, the machine with the problem uses the default 4GB swap. That's > all the swap it has. The machine without issue has a swapfile > installed on a SSD in addition to the default 4GB swap. > > problematic machine: > Device 512-blocks Used Avail Capacity > /dev/ada0p3 8388608 3.3G 714M 83% > > machine without a problem, it has swapfile installed: > Device 512-blocks Used Avail Capacity > /dev/ada0s1b 8262248 1.7G 2.2G 44% > /dev/md0 65536000 1.9G 29G 6% > Total 73798248 3.7G 32G 10% > > I added the swapfile a long time ago on this machine due to the same > issue.so, the same effect as on a small Raspberry. It seems that you also use a memory disk for swap. Mine is backed by a file via NFS.> > But my problem isn't so much an out of swapspace problem; all this > is, is a symptom. My problem is "why is it swapping out at all on a > 128GB system and why is what's swapped out not being swapped back in > again". >I wondered even on the small Raspberry about this. The Raspberries come with 1GB of RAM. Running just a compilation should never be the problem but sometimes it is. A very long time ago - and not on FreeBSD but maybe on a real BSD - I worked with a system that swapped pages out just to bring it back as one contiguous block. This made a difference those days. I do not know if the code made it out of the university I was working at. I just imagine now that the code made it out and is still in use with the opposite effect. Erich