miyamoto moesasji
2011-Jan-01 23:22 UTC
tmpfs runs out of space on 8.2pre-release, zfs related?
In setting up tmpfs (so not tmpmfs) on a machine that is using zfs(v15, zfs v4) on 8.2prerelease I run out of space on the tmpfs when copying a file of ~4.6 GB file from the zfs-filesystem to the memory disk. This machine has 8GB of memory backed by swap on the harddisk, so I expected the file to copy to memory without problems. Below in detail what happens: upon rebooting the machine the tmpfs has 8GB available as can be seen below: --- hge@PulsarX4:~/ > df -hi /tmp Filesystem Size Used Avail Capacity iused ifree %iused Mounted on tmpfs 8.2G 12K 8.2G 0% 19 39M 0% /tmp --- Subsequently copying a ~4.6GB file from a location in the zfs-pool to the memory filesystem fails with no more space left message --- hge@PulsarX4:~/ > cp ~/temp/large.iso /tmp/large_file cp: /tmp/large_file: No space left on device --- After this the tmpfs has shrunk to just 2.7G, obviously much less than the 8.2G available before the copy-operation. At the same time there are still free inodes left, so that does not appear to be the problem. Output of the df after the copy: --- hge@PulsarX4:~/ > df -hi /tmp Filesystem Size Used Avail Capacity iused ifree %iused Mounted on tmpfs 2.7G 2.7G 1.4M 100% 19 6.4k 0% /tmp --- A quick search shows the following bug-report for solaris: http://bugs.opensolaris.org/bugdatabase/view_bug.do;jsessionid=e4ae9c32983000ef651e38edbba1?bug_id=6804661This appears closely related as here I also try to copy a file >50% of memory to the tmpfs and the way to reproduce appears identical to what I did here. As it might help spot the problem: below the information on the zfs ARC size obtained from the output of zfs-stats. This gives: Before the copy: --- System Memory Statistics: Physical Memory: 8161.74M Kernel Memory: 511.64M DATA: 94.27% 482.31M TEXT: 5.73% 29.33M ARC Size: Current Size (arcsize): 5.88% 404.38M Target Size (Adaptive, c): 100.00% 6874.44M Min Size (Hard Limit, c_min): 12.50% 859.31M Max Size (High Water, c_max): ~8:1 6874.44M --- After the copy: --- System Memory Statistics: Physical Memory: 8161.74M Kernel Memory: 3326.98M DATA: 99.12% 3297.65M TEXT: 0.88% 29.33M ARC Size: Current Size (arcsize): 46.99% 3230.55M Target Size (Adaptive, c): 100.00% 6874.44M Min Size (Hard Limit, c_min): 12.50% 859.31M Max Size (High Water, c_max): ~8:1 6874.44M --- Unfortunately I have difficulties interpreting this further than this, so suggestions how to prevent this behavior (or further trouble shoot this) would be appreciated as my feeling is that this should not happen.
miyamoto moesasji
2011-Jan-02 08:41 UTC
tmpfs runs out of space on 8.2pre-release, zfs related?
miyamoto moesasji <miyamoto.31b <at> gmail.com> writes:> > In setting up tmpfs (so not tmpmfs) on a machine that is using > zfs(v15, zfs v4) on 8.2prerelease I run out of space on the tmpfs when > copying a file of ~4.6 GB file from the zfs-filesystem to the memory > disk. This machine has 8GB of memory backed by swap on the harddisk, > so I expected the file to copy to memory without problems. >....this is in fact worse than I first thought. After leaving the machine running overnight the tmpfs is reduced to a size of 4K, which shows that tmpfs is in fact completely unusable for me. See the output of df: --- hge@PulsarX4:~/ > df -hi /tmp Filesystem Size Used Avail Capacity iused ifree %iused Mounted on tmpfs 4.0K 4.0K 0B 100% 18 0 100% /tmp --- Relevant zfs-stats info: --- System Memory Statistics: Physical Memory: 8161.74M Kernel Memory: 4117.40M DATA: 99.29% 4088.07M TEXT: 0.71% 29.33M ARC Size: Current Size (arcsize): 63.58% 4370.60M Target Size (Adaptive, c): 100.00% 6874.44M Min Size (Hard Limit, c_min): 12.50% 859.31M Max Size (High Water, c_max): ~8:1 6874.44M --- I'm not sure what triggered this further reduction in size; but the above 4K size is probably important to show how dramatic this goes wrong.
On 01/02/11 09:41, miyamoto moesasji wrote:> miyamoto moesasji<miyamoto.31b<at> gmail.com> writes: > >> >> In setting up tmpfs (so not tmpmfs) on a machine that is using >> zfs(v15, zfs v4) on 8.2prerelease I run out of space on the tmpfs when >> copying a file of ~4.6 GB file from the zfs-filesystem to the memory >> disk. This machine has 8GB of memory backed by swap on the harddisk, >> so I expected the file to copy to memory without problems. >> > > ....this is in fact worse than I first thought. After leaving the > machine running overnight the tmpfs is reduced to a size of 4K, which > shows that tmpfs is in fact > completely unusable for me. See the output of df: > --- > hge@PulsarX4:~/> df -hi /tmp > Filesystem Size Used Avail Capacity iused ifree %iused Mounted on > tmpfs 4.0K 4.0K 0B 100% 18 0 100% /tmpThis is a known problem. So far, no solution has been offered, which means that effectively, tmpfs cannot be used with ZFS on the same system.