Folks,
my ZIL/cache SSD apparently just died. Rebooting the system with the SATA M.2
SSD hung, so I removed the card from the system.
On powerup, loader acts normally, all four SATA disks (main Raid-Z1 devices) are
all probed successfully, but mount root fails:
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device
ada0: Serial Number Z5Q7K0RIFFRC
ada0: 600.000MB/s transfers (SATA 3.x, UDMA5, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 4769307MB (9767541168 512 byte sectors)
ada1 at ahcich2 bus 0 scbus2 target 0 lun 0
ada1: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device
ada1: Serial Number Y5PIK0A2FFRC
ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 4769307MB (9767541168 512 byte sectors)
ada2 at ahcich3 bus 0 scbus3 target 0 lun 0
ada2: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device
ada2: Serial Number 36D2K0VZFFRC
ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 4769307MB (9767541168 512 byte sectors)
ada3 at ahcich4 bus 0 scbus4 target 0 lun 0
ada3: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device
ada3: Serial Number Z5SDK0J3FFRC
ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 4769307MB (9767541168 512 byte sectors)
pass4 at ahciem0 bus 0 scbus6 target 0 lun 0
pass4: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
Trying to mount root from zfs:p2/be/11 []...
GEOM_MIRROR: Device mirror/p2swap launched (4/4).
random: unblocking device.
Mounting from zfs:p2/be/11 failed with error 6; retrying for 3 more seconds
Mounting from zfs:p2/be/11 failed with error 6.
Loader variables:
vfs.root.mountfrom=zfs:p2/be/11
Manual root filesystem specification:
<fstype>:<device> [options]
Mount <device> using filesystem <fstype>
and with the specified (optional) option list.
eg. ufs:/dev/da0s1a
zfs:tank
cd9660:/dev/cd0 ro
(which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)
? List valid disk boot devices
. Yield 1 second (for background tasks)
<empty line> Abort manual input
mountroot>
Is there an easy way to boot into single user mode and remove the ZIL/cache
devices that are not there anymore? Or do I need a USB key to boot off of and
zfs import the pool first?
Thanks,
Stefan
--
Stefan Bethke <stb at lassitu.de> Fon +49 151 14070811
On Sun, Jul 29, 2018 at 4:43 AM, Stefan Bethke <stb at lassitu.de> wrote:> Folks, > > my ZIL/cache SSD apparently just died. Rebooting the system with the SATA > M.2 SSD hung, so I removed the card from the system. > > On powerup, loader acts normally, all four SATA disks (main Raid-Z1 > devices) are all probed successfully, but mount root fails: > > ZFS filesystem version: 5 > ZFS storage pool version: features support (5000) > Timecounters tick every 1.000 msec > ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 > ada0: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device > ada0: Serial Number Z5Q7K0RIFFRC > ada0: 600.000MB/s transfers (SATA 3.x, UDMA5, PIO 8192bytes) > ada0: Command Queueing enabled > ada0: 4769307MB (9767541168 512 byte sectors) > ada1 at ahcich2 bus 0 scbus2 target 0 lun 0 > ada1: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device > ada1: Serial Number Y5PIK0A2FFRC > ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > ada1: Command Queueing enabled > ada1: 4769307MB (9767541168 512 byte sectors) > ada2 at ahcich3 bus 0 scbus3 target 0 lun 0 > ada2: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device > ada2: Serial Number 36D2K0VZFFRC > ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > ada2: Command Queueing enabled > ada2: 4769307MB (9767541168 512 byte sectors) > ada3 at ahcich4 bus 0 scbus4 target 0 lun 0 > ada3: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device > ada3: Serial Number Z5SDK0J3FFRC > ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > ada3: Command Queueing enabled > ada3: 4769307MB (9767541168 512 byte sectors) > pass4 at ahciem0 bus 0 scbus6 target 0 lun 0 > pass4: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device > Trying to mount root from zfs:p2/be/11 []... > GEOM_MIRROR: Device mirror/p2swap launched (4/4). > random: unblocking device. > Mounting from zfs:p2/be/11 failed with error 6; retrying for 3 more seconds > Mounting from zfs:p2/be/11 failed with error 6. > > Loader variables: > vfs.root.mountfrom=zfs:p2/be/11 > > Manual root filesystem specification: > <fstype>:<device> [options] > Mount <device> using filesystem <fstype> > and with the specified (optional) option list. > > eg. ufs:/dev/da0s1a > zfs:tank > cd9660:/dev/cd0 ro > (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /) > > ? List valid disk boot devices > . Yield 1 second (for background tasks) > <empty line> Abort manual input > > mountroot> > > Is there an easy way to boot into single user mode and remove the > ZIL/cache devices that are not there anymore? Or do I need a USB key to > boot off of and zfs import the pool first? > > > Thanks, > Stefan > > -- > Stefan Bethke <stb at lassitu.de> Fon +49 151 14070811A missing L2ARC is no problem, but a missing ZIL may mean filesystem corruption. Hopefully the ZIL was empty when it failed. I would try booting from ZFS and removing the offending device. Good luck. -Alan
On Sun, Jul 29, 2018, 3:46 AM Stefan Bethke, <stb at lassitu.de> wrote:> Folks, > > my ZIL/cache SSD apparently just died. Rebooting the system with the SATA > M.2 SSD hung, so I removed the card from the system. > > On powerup, loader acts normally, all four SATA disks (main Raid-Z1 > devices) are all probed successfully, but mount root fails: > > ZFS filesystem version: 5 > ZFS storage pool version: features support (5000) > Timecounters tick every 1.000 msec > ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 > ada0: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device > ada0: Serial Number Z5Q7K0RIFFRC > ada0: 600.000MB/s transfers (SATA 3.x, UDMA5, PIO 8192bytes) > ada0: Command Queueing enabled > ada0: 4769307MB (9767541168 512 byte sectors) > ada1 at ahcich2 bus 0 scbus2 target 0 lun 0 > ada1: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device > ada1: Serial Number Y5PIK0A2FFRC > ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > ada1: Command Queueing enabled > ada1: 4769307MB (9767541168 512 byte sectors) > ada2 at ahcich3 bus 0 scbus3 target 0 lun 0 > ada2: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device > ada2: Serial Number 36D2K0VZFFRC > ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > ada2: Command Queueing enabled > ada2: 4769307MB (9767541168 512 byte sectors) > ada3 at ahcich4 bus 0 scbus4 target 0 lun 0 > ada3: <TOSHIBA MD04ACA50D FS2A> ATA8-ACS SATA 3.x device > ada3: Serial Number Z5SDK0J3FFRC > ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > ada3: Command Queueing enabled > ada3: 4769307MB (9767541168 512 byte sectors) > pass4 at ahciem0 bus 0 scbus6 target 0 lun 0 > pass4: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device > Trying to mount root from zfs:p2/be/11 []... > GEOM_MIRROR: Device mirror/p2swap launched (4/4). > random: unblocking device. > Mounting from zfs:p2/be/11 failed with error 6; retrying for 3 more seconds > Mounting from zfs:p2/be/11 failed with error 6. > > Loader variables: > vfs.root.mountfrom=zfs:p2/be/11 > > Manual root filesystem specification: > <fstype>:<device> [options] > Mount <device> using filesystem <fstype> > and with the specified (optional) option list. > > eg. ufs:/dev/da0s1a > zfs:tank > cd9660:/dev/cd0 ro > (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /) > > ? List valid disk boot devices > . Yield 1 second (for background tasks) > <empty line> Abort manual input > > mountroot> > > Is there an easy way to boot into single user mode and remove the > ZIL/cache devices that are not there anymore? Or do I need a USB key to > boot off of and zfs import the pool first? > > > Thanks, > Stefan >Boot off USB or CD, then try to manually import the pool. There's an option to "zpool import" to ignore missing log devices. Once it's imported, you can remove/detach the missing device. Then you should be able to boot again. Cheers, Freddie Typos courtesy of my phone's keyboard.>