Hi there, I have a small backup server (8.2-STABLE). It boots from ufs and has a zfs pool for backups that consists of 8 drives configured as 4 mirrored devices, totally around 2.5 TB. Been working great, no issues, until the past few days when remote rsyncs to it have started to get very slow (it's only at around %50 capacity). Rebooting it helps for a while, then it gets slow again. But this isn't the problem now... After the last reboot, it froze while booting right at the point where the file system gets mounted. No errors, it just doesn't proceed past the ZFS version message. I rebooted single user and tried to access it with "zpool status", and the command hangs in the same way. Any attempt to access it ("zfs list", for example) does the same thing. The disks themselves seem fine. They are all connected to a pair of Adaptec RAID controllers (configured as individual drives, with mirroring handled by zfs) and the controller software shows them all to be intact. I disabled zfs in rc.conf and was able to boot, but I can't access the pool. Any ideas on how to diagnose and hopefully repair this? Mark
On Mon, Mar 28, 2011 at 1:17 PM, Mark Morley <mark@islandnet.com> wrote:> Hi there, > > I have a small backup server (8.2-STABLE). ?It boots from ufs and has a zfs pool for backups that consists of 8 drives configured as 4 mirrored devices, totally around 2.5 TB. > > Been working great, no issues, until the past few days when remote rsyncs to it have started to get very slow (it's only at around %50 capacity). ?Rebooting it helps for a while, then it gets slow again. ?But this isn't the problem now... > > After the last reboot, it froze while booting right at the point where the file system gets mounted. ?No errors, it just doesn't proceed past the ZFS version message. > > I rebooted single user and tried to access it with "zpool status", and the command hangs in the same way. ?Any attempt to access it ("zfs list", for example) does the same thing. > > The disks themselves seem fine. ?They are all connected to a pair of Adaptec RAID controllers (configured as individual drives, with mirroring handled by zfs) and the controller software shows them all to be intact. > > I disabled zfs in rc.conf and was able to boot, but I can't access the pool. > > Any ideas on how to diagnose and hopefully repair this? >Your going to need to download a recent -CURRENT ISO that contans zfs v28, then you can try to recover the pool as outlined in this post http://opensolaris.org/jive/message.jspa?messageID=445269 zpool import -nfF -R /mnt rpool If it reports that it can get back to good pool state, then do actual import with zpool import -fF -R /mnt rpool In case first command cannot rewind to older state, try to add -X option: zpool import -nfFX -R /mnt rpool and if it says that it can recover your pool with some data loss and you are ok with it, then do actual import zpool import -fFX -R /mnt rpool Note: I haven't given this a try on my system yet, let me know how it goes ;-). Scot
On Mon, 28 Mar 2011 13:38:54 -0500 Scot Hetzel wrote: On Mon, Mar 28, 2011 at 1:17 PM, Mark Morley wrote:> Hi there, > > I have a small backup server (8.2-STABLE). ?It boots from ufs and has azfs pool for backups that consists of 8 drives configured as 4 mirrored devices, totally around 2.5 TB. > > Been working great, no issues, until the past few days when remote rsyncsto it have started to get very slow (it's only at around %50 capacity). ?Rebooting it helps for a while, then it gets slow again. ?But this isn't the problem now... > > After the last reboot, it froze while booting right at the point where the file system gets mounted. ?No errors, it just doesn't proceed past the ZFS version message. > > I rebooted single user and tried to access it with "zpool status", and the command hangs in the same way. ?Any attempt to access it ("zfs list", for example) does the same thing. > > The disks themselves seem fine. ?They are all connected to a pair of Adaptec RAID controllers (configured as individual drives, with mirroring handled by zfs) and the controller software shows them all to be intact. > > I disabled zfs in rc.conf and was able to boot, but I can't access the pool. > > Any ideas on how to diagnose and hopefully repair this? >Your going to need to download a recent -CURRENT ISO that contans zfs v28, then you can try to recover the pool as outlined in this post http://opensolaris.org/jive/message.jspa?messageID=445269 Well, what I did was rebuild world and kernel top 9.0-CURRENT and reboot. It was able to see and access the zfs file system immediately without having to import it. I did a zpool upgrade to v28 and all seems well so far. Mark