On Tue, Sep 22, 2020 at 07:31:07PM +0200, Peter wrote:> On Tue, Sep 22, 2020 at 12:33:19PM -0400, Mark Johnston wrote:
>
> ! On Tue, Sep 22, 2020 at 06:08:01PM +0200, Peter wrote:
>
> ! > my machine should use about 3-4, maybe 5 GB swapspace. Today I
found
> ! > it suddenly uses 8 GB (which is worryingly near the configured 10G).
> ! >
> ! > I stopped all the big suckers - nothing found.
> ! > I stopped all the jails - no success.
> ! > I brought it down to singleuser: it tried to swapoff, but failed.
> ! >
> ! > I unmounted all filesystems, exported all pools, detached all geli,
> ! > and removed most of the netgraphs. Swap is still occupied.
> ! > ! > Machine is now running only the init and a shell processes,
has
> ! > almost no filesystems mounted, has mostly native networks only, and
> ! > this still occupies 3 GB of swap which cannot be released.
> ! >
> ! > What is going on, what is doing this, and how can I get this
swapspace
> ! > released??
> !
> ! Do you have any shared memory segments lingering? ipcs -a will show
> ! SysV shared memory usage.
>
> I have four small shmem segments from four postgres clusters running.
> These should cleanly disappear when the clusters are stopped, and
> they are very small.
>
> Shared Memory:
> T ID KEY MODE OWNER GROUP CREATOR CGROUP
NATTCH SEGSZ CPID LPID ATIME DTIME CTIME
> m 65536 5432001 --rw------- postgres postgres postgres postgres
7 48 4793 4793 6:09:34 18:00:31 6:09:34
> m 65537 0 --rw------- postgres postgres postgres postgres
11 48 6268 6268 6:09:42 10:48:27 6:09:42
> m 65538 0 --rw------- postgres postgres postgres postgres
5 48 6968 6968 6:09:46 18:28:36 6:09:46
> m 65539 0 --rw------- postgres postgres postgres postgres
6 48 6992 6992 6:09:47 3:38:34 6:09:47
>
> ! For POSIX shared memory, in 11.4 we do not
> ! have any good way of listing objects, but "vmstat -m | grep
shmfd" will
> ! at least show whether any are allocated.
>
> There is something, and I don't know who owns that:
> $ vmstat -m | grep shmfd
> shmfd 13 14K - 473 64,256,1024,8192
>
> But that doesn't look big either.
That is just the amount of kernel memory used to track a set of objects,
not the actual object sizes. Unfortunately, in 11 I don't think there's
any way to enumerate them other than running kgdb and examining the
shm_dictionary hash table.
> Furthermore, this machine is running for quite some time already; it
> was running as i386 (with ZFS) until very recently, and I know quite
> well what is using much memory: these 3 GB were illegitimate; they
> came from nothing I did install. And they are new; this has not
> happened before.
>
> ! If those don't turn anything
> ! up then it's possible that there's a swap leak. Do you use any
DRM
> ! graphics drivers on this system?
>
> Probably yes. There is no graphics used at all; it just uses "device
> vt" in text mode, but it uses i5-3570T CPU (IvyBridge HD2500) graphics
> for that, and the driver is "drm2" and "i915drm" from
/usr/src/sys (not
> those from ports).
> Not sure how that would account for 3 GB, unless there is indeed some
> leak.
I think I see a possible problem in i915, though I'm not sure if you'd
trigger it just by using vt(4). It should be fixed in later FreeBSD
versions, but is still a problem in 11. Here's a (untested) patch:
Index: sys/dev/drm2/i915/i915_gem.c
==================================================================---
sys/dev/drm2/i915/i915_gem.c (revision 365772)
+++ sys/dev/drm2/i915/i915_gem.c (working copy)
@@ -1863,6 +1863,8 @@ i915_gem_object_truncate(struct drm_i915_gem_objec
vm_obj = obj->base.vm_obj;
VM_OBJECT_WLOCK(vm_obj);
vm_object_page_remove(vm_obj, 0, 0, false);
+ if (vm_obj->type == OBJT_SWAP)
+ swap_pager_freespace(vm_obj, 0, vm_obj->size);
VM_OBJECT_WUNLOCK(vm_obj);
i915_gem_object_free_mmap_offset(obj);