$ swap -s total: 473164k bytes allocated + 388916k reserved = 862080k used, 6062060k available $ swap -l swapfile dev swaplo blocks free /dev/dsk/c6t0d0s1 215,1 8 12594952 12594952 Can someone please do the math for me here? I am not able to figure the total. What is "473164k bytes allocated"? Where is it allocated? In some hidden zfs swap FS in my root pool? What''s the magic behind the number 473164k? What is "388916k reserved"? 862080k+6062060k != 12594952/2 - So, where did the rest of it come from? I just configured one device in /etc/vfstab. -- This message posted from opensolaris.org
On Wed, Jun 9, 2010 at 8:17 PM, devsk <funtoos at yahoo.com> wrote:> $ swap -s > total: 473164k bytes allocated + 388916k reserved = 862080k used, 6062060k available > > $ swap -l > swapfile ? ? ? ? ? ? dev ? ?swaplo ? blocks ? ? free > /dev/dsk/c6t0d0s1 ? 215,1 ? ? ? ? 8 12594952 12594952 > > Can someone please do the math for me here? I am not able to figure the total. > > What is "473164k bytes allocated"? Where is it allocated? In some hidden zfs swap FS in my root pool? > What''s the magic behind the number 473164k? > What is "388916k reserved"? > 862080k+6062060k != 12594952/2 - So, where did the rest of it come from? I just configured one device in /etc/vfstab. > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >man swap " These numbers include swap space from all configured swap areas as listed by the -l option, as well swap space in the form of physical memory."
On 6/9/2010 7:20 PM, Greg Eanes wrote:> On Wed, Jun 9, 2010 at 8:17 PM, devsk<funtoos at yahoo.com> wrote: > >> $ swap -s >> total: 473164k bytes allocated + 388916k reserved = 862080k used, 6062060k available >> >> $ swap -l >> swapfile dev swaplo blocks free >> /dev/dsk/c6t0d0s1 215,1 8 12594952 12594952 >> >> Can someone please do the math for me here? I am not able to figure the total. >> >> What is "473164k bytes allocated"? Where is it allocated? In some hidden zfs swap FS in my root pool? >> What''s the magic behind the number 473164k? >> What is "388916k reserved"? >> 862080k+6062060k != 12594952/2 - So, where did the rest of it come from? I just configured one device in /etc/vfstab. >> -- >> > > man swap > > " These numbers include swap space from all configured > swap areas as listed by the -l option, as well swap > space in the form of physical memory." >This is also a reasonable explanation of what the output of ''swap -s'' actually means. http://www.softpanorama.org/Solaris/Processes_and_memory/swap_space_management.shtml Look about half-way down the page, under "Monitoring Swap Resources". The whole page is worth a read, though. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA
Erik, That doesn''t explain anything. More of the same that I found in man page. What is swap allocated in physical memory? I have hard time wrapping my arms around that. Is it something like swap cache in Linux? If its disk-backed, where is the actual location of the backing store? And the numbers? 473164k is not same as 256MB as per the table in that page. If you can explain the individual numbers and how they add up across ''swap -s'', ''swap -l'' and ''top -b'', that would be great! -devsk ________________________________ From: Erik Trimble <erik.trimble at oracle.com> Cc: devsk <funtoos at yahoo.com>; zfs-discuss at opensolaris.org Sent: Wed, June 9, 2010 7:41:22 PM Subject: Re: [zfs-discuss] swap - where is it coming from? On 6/9/2010 7:20 PM, Greg Eanes wrote:> On Wed, Jun 9, 2010 at 8:17 PM, devsk<funtoos at yahoo.com> wrote: > >> $ swap -s >> total: 473164k bytes allocated + 388916k reserved = 862080k used, 6062060k available >> >> $ swap -l >> swapfile dev swaplo blocks free >> /dev/dsk/c6t0d0s1 215,1 8 12594952 12594952 >> >> Can someone please do the math for me here? I am not able to figure the total. >> >> What is "473164k bytes allocated"? Where is it allocated? In some hidden zfs swap FS in my root pool? >> What''s the magic behind the number 473164k? >> What is "388916k reserved"? >> 862080k+6062060k != 12594952/2 - So, where did the rest of it come from? I just configured one device in /etc/vfstab. >> -- >> > > man swap > > " These numbers include swap space from all configured > swap areas as listed by the -l option, as well swap > space in the form of physical memory." >This is also a reasonable explanation of what the output of ''swap -s'' actually means. http://www.softpanorama.org/Solaris/Processes_and_memory/swap_space_management.shtml Look about half-way down the page, under "Monitoring Swap Resources". The whole page is worth a read, though. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100609/983758b2/attachment.html>
Think of swap space as "current total available virtual memory space available for program use". It is composed of unreserved RAM (that is, RAM not currently reserved for usage other than program memory - things such as file caching, certain ZFS functionality, kernel usage) PLUS whatever disk space you have allocated for usage as backing store. Thus, the used + available of ''swap -s'' will: (1) *always* exceed the amount of disk space allocated to swap (2) *never* be equal to or more than total ram + disk swap Re-read the section on"Swap Space and Virtual Memory" for particulars on how Solaris does virtual memory mapping, and the concept of Virtual Swap Space, which is what ''swap -s'' is really reporting on. The 256MB mentioned in the "How Swap Space is Allocated" section is the MINIMUM DEFAULT disk swap space. These days, we don''t have disk swap slices that small, so it''s safe to ignore. You can think of the "allocated" swap as virtual addresses that are mapped to a place on the physical swap device(s) (whether or not they''ve actually been written to disk is another question), and the "reserved" amounts as that which hasn''t actually been malloc()''d for use, but the application has indicated that it might use some additional virtual address space. -Erik On 6/9/2010 8:46 PM, devsk wrote:> Erik, > > That doesn''t explain anything. More of the same that I found in man > page. What is swap allocated in physical memory? I have hard time > wrapping my arms around that. Is it something like swap cache in > Linux? If its disk-backed, where is the actual location of the backing > store? > > And the numbers? 473164k is not same as 256MB as per the table in that > page. If you can explain the individual numbers and how they add up > across ''swap -s'', ''swap -l'' and ''top -b'', that would be great! > > -devsk > > > ------------------------------------------------------------------------ > *From:* Erik Trimble <erik.trimble at oracle.com> > *Cc:* devsk <funtoos at yahoo.com>; zfs-discuss at opensolaris.org > *Sent:* Wed, June 9, 2010 7:41:22 PM > *Subject:* Re: [zfs-discuss] swap - where is it coming from? > > On 6/9/2010 7:20 PM, Greg Eanes wrote: > > On Wed, Jun 9, 2010 at 8:17 PM, devsk<funtoos at yahoo.com > <mailto:funtoos at yahoo.com>> wrote: > > > >> $ swap -s > >> total: 473164k bytes allocated + 388916k reserved = 862080k used, > 6062060k available > >> > >> $ swap -l > >> swapfile dev swaplo blocks free > >> /dev/dsk/c6t0d0s1 215,1 8 12594952 12594952 > >> > >> Can someone please do the math for me here? I am not able to figure > the total. > >> > >> What is "473164k bytes allocated"? Where is it allocated? In some > hidden zfs swap FS in my root pool? > >> What''s the magic behind the number 473164k? > >> What is "388916k reserved"? > >> 862080k+6062060k != 12594952/2 - So, where did the rest of it come > from? I just configured one device in /etc/vfstab. > >> -- > >> > > > > man swap > > > > " These numbers include swap space from all configured > > swap areas as listed by the -l option, as well swap > > space in the form of physical memory." > > > > This is also a reasonable explanation of what the output of ''swap -s'' > actually means. > > http://www.softpanorama.org/Solaris/Processes_and_memory/swap_space_management.shtml > > Look about half-way down the page, under "Monitoring Swap Resources". > The whole page is worth a read, though. > > -- > Erik Trimble > Java System Support > Mailstop: usca22-123 > Phone: x17195 > Santa Clara, CA > >-- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100609/6d6a75e6/attachment.html>
> Re-read the section on"Swap Space and Virtual Memory" for particulars on > how Solaris does virtual memory mapping, and the concept of Virtual Swap > Space, which is what ''swap -s'' is really reporting on.The Solaris Internals book is awesome for this sort of thing. A bit over the top in detail but awesome regardless. -- Dennis
Swap is perhaps the wrong name; it is really "virtual memory"; virtual memory consists of real memory and swap on disk. In Solaris, a page either exists on the physical swap device or in memory. Of course, not all memory is available as the kernel and other caches use a large part of the memory. When no swap based disk is in use, then there is sufficient free memory; reserved is pages reserved, e.g., fork, (pages to copy when copy-on-write happens) or allocated memory but not written to. Casper
On Thu, 10 Jun 2010, Casper.Dik at Sun.COM wrote:> > Swap is perhaps the wrong name; it is really "virtual memory"; virtual > memory consists of real memory and swap on disk. In Solaris, a page > either exists on the physical swap device or in memory. Of course, not > all memory is available as the kernel and other caches use a large part > of the memory.Don''t forget that virtual memory pages may also come from memory mapped files from the filesystem. However, it seems that zfs is effectively diminishing this. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
>On Thu, 10 Jun 2010, Casper.Dik at Sun.COM wrote: >> >> Swap is perhaps the wrong name; it is really "virtual memory"; virtual >> memory consists of real memory and swap on disk. In Solaris, a page >> either exists on the physical swap device or in memory. Of course, not >> all memory is available as the kernel and other caches use a large part >> of the memory. > >Don''t forget that virtual memory pages may also come from memory >mapped files from the filesystem. However, it seems that zfs is >effectively diminishing this.I should have said "anonymous" virtual memory. Casper
On 10 Jun 2010, at 19:20, Bob Friesenhahn <bfriesen at simple.dallas.tx.us> wrote:> On Thu, 10 Jun 2010, Casper.Dik at Sun.COM wrote: >> >> Swap is perhaps the wrong name; it is really "virtual memory"; >> virtual >> memory consists of real memory and swap on disk. In Solaris, a page >> either exists on the physical swap device or in memory. Of course, >> not >> all memory is available as the kernel and other caches use a large >> part >> of the memory. > > Don''t forget that virtual memory pages may also come from memory > mapped files from the filesystem. However, it seems that zfs is > effectively diminishing this.Processes are largely filesystem agnostic. For example, both ZFS and UFS provide implementations for open(2), close(2), stat(2), read(2), write(2) and mmap(2). Every process is a consumer of mmap(2) because dynamic linking is implemented using memory mapped files. However ZFS does diminish the usefulness of memory mapped files for I/ O because the current implementation performs so poorly. ZFS is the first major filesystem for Solaris that majors on a cache (the ARC) distinct from the Solaris page cache. Today, memory mapped files use the Solaris page cache, even for the ZFS implementation of mmap(2). A mapped ZFS file will be cached twice - which is one performance hit. Another is that the caches have to be kept in sync, so that consumers if read(2), write(2) and mmap(2) see a consistent view of the same file. Linux already had a head start on mapped file performance compared to UFS on Solaris. With ZFS that gap got a lot bigger. Applications like kdb+ that are heavily dependent on high performance mapped files I/O are currently not viable with ZFS on Solaris (and considerably slower with UFS than on Linux).