I've seen people discuss about this and it happened to me as well. Freebsd 8.1 stable 25th september amd64 This server has one single boot disk with ufs + 1 array (hardware, 3ware) which hold 2,7TB data. I've formatted the latter with ZFS. Things seemed to work ok until I upgraded the system. One day the server jails didn't boot anymore. I checked and the array occupation was 100% with 0 byte free. This sounded strange, so I found a hidden file under /var/amavis/.spamassassin. I removed the dir, started again, but after a while it grew again until the system became unusable. This is how I solved: I moved the amavis partition under UFS, then mounted with nullfs that dir under /var in the zfs jail. I solved in the sense that it didn't grow anymore, but the fs occupation stayed very near 100%, with only 1GB free space. If I launch du -s under /zfs (host machine, not jail) I get a total space of about 750GB, but df -h always turns up with 2,7TB space occupied. I tried a zpool upgrade zfs and zpool scrub zfs, but to no avail. What should I do. short of moving the data, destroying the pool and creating it again (which I can't very easily do)? Thanks.
On Fri, Oct 22, 2010 at 03:03:53PM +0200, Giulio Ferro wrote:> [...snip; focusing specifically on this piece...] > > If I launch du -s under /zfs (host machine, not jail) I get a total space > of about 750GB, but df -h always turns up with 2,7TB space occupied.Is the ZFS filesystem (not pool) using compression? -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
On 10/22/2010 03:54 PM, Jeremy Chadwick wrote:> On Fri, Oct 22, 2010 at 03:03:53PM +0200, Giulio Ferro wrote: >> [...snip; focusing specifically on this piece...] >> >> If I launch du -s under /zfs (host machine, not jail) I get a total space >> of about 750GB, but df -h always turns up with 2,7TB space occupied. > > Is the ZFS filesystem (not pool) using compression? >Nope.
On 10/22/10 15:03, Giulio Ferro wrote:> I've seen people discuss about this and it happened to me as well. > > Freebsd 8.1 stable 25th september amd64 > This server has one single boot disk with ufs + 1 array (hardware, 3ware) > which hold 2,7TB data. I've formatted the latter with ZFS. > > Things seemed to work ok until I upgraded the system. One day the server > jails didn't boot anymore. I checked and the array occupation was 100% with > 0 byte free. > > This sounded strange, so I found a hidden file under > /var/amavis/.spamassassin. > I removed the dir, started again, but after a while it grew again until > the system became unusable. > > This is how I solved: I moved the amavis partition under UFS, then > mounted with nullfs that dir under /var in the zfs jail. > > I solved in the sense that it didn't grow anymore, but the fs occupation > stayed very near 100%, with only 1GB free space. > > If I launch du -s under /zfs (host machine, not jail) I get a total space > of about 750GB, but df -h always turns up with 2,7TB space occupied. > > I tried a zpool upgrade zfs and zpool scrub zfs, but to no avail. > > What should I do. short of moving the data, destroying the pool and > creating it again (which I can't very easily do)?I had the same problem with extremely large files in /var/amavis/.spamassassin after upgrading from 7.x to 8.1 (zpool mirror, no compression, zpool upgraded to v14, zfs filesystems upgraded to v3). This worked for me as a fix/workaround: zfs create pool/newamavisjail rm /pool/oldamavisjail/var/amavis/.spamassassin/[hugefiles] rsync ... /pool/oldamavisjail /pool/newamavisjail portsnap fetch update; portupgrade -af (in newamavisjail) restart newamavisjail I don't think I permanently lost disk space from the pool because of this. I guess you have checked that there are no snapshots referring to the huge files? (zfs list and zfs list -t snapshot) -- Erik