zfs discuss - Aug 2007 - Firewire zpool transport rejected fatal error, 6560174

I think I have ran into this bug, 6560174, with a firewire drive. Running build
64a, formatted entire external firewire drive as a zpool. Worked fine with
smaller zfs send operations. Tried zfs send on a 10GB filesystem and it seemed
to hang (though there is no notice of progress with zfs send). Let it run
overnight and still nothing (firewire 400). Both ''ls'' into the
drive and ''format'' froze. The system would not reboot nicely
either.

Rebooting with the drive not plugged in I get the error that a zpool cannot be
found. Zpool status -x suggests that I can reattach the drive and run
''zpool online <device> <drive>'', but that states
that the pool is not available. Booting with the drive plugged in I get the
''transport rejected fatal error'' message and the system stops
booting.

Is there a way to reformat and try to use this drive again? Can I recover from
this and use the information on the drive? Are firewire drives not a good idea
for backup/recover using ZFS?

thanks,
 
 
This message posted from opensolaris.org

> I think I have ran into this bug, 6560174, with a firewire drive. 
And 6560174 might be a duplicate of 6445725
 
 
This message posted from opensolaris.org

> And 6560174 might be a duplicate of 6445725
I see what you mean. Unfortunately there does not look to be a work-around. 

It is beginning to sound like firewire drives are not a safe alternative for
backup? This is unfortunate when you have an Ultra20 with only 2 disks.

Is there a way to destroy the pool on the device and start over?
 
 
This message posted from opensolaris.org

> > And 6560174 might be a duplicate of 6445725
> 
> I see what you mean. Unfortunately there does not
> look to be a work-around. 
Nope, no work-around.  This is a scsa1394 bug; it
has some issues when it is used from interrupt context.

I have some source code diffs, that are supposed to
fix the issue, see this thread:

    http://www.opensolaris.org/jive/thread.jspa?messageID=46190
 
 
This message posted from opensolaris.org

> Nope, no work-around.  
OK. Then I have 3 questions:

1) How do I destroy the pool that was on the firewire drive? (So that zfs stops
complaining about it)

2) How can I reformat the firewire drive? Does this need to be done on a
non-Solaris OS?

3) Can your code diffs be integrated into the OS on my end to use this drive,
and if so, how?

thanks
 
 
This message posted from opensolaris.org

aric wrote:
>> Nope, no work-around.  
> 
> OK. Then I have 3 questions:
> 
> 1) How do I destroy the pool that was on the firewire drive? (So that zfs
stops complaining about it)
`zpool destroy <poolname>`  this works even if the drive isn''t
connected.
> 2) How can I reformat the firewire drive? Does this need to be done on a
non-Solaris OS?
format(1M) on Solaris should work okay even with this problem.
> 3) Can your code diffs be integrated into the OS on my end to use this
drive, and if so, how?
I believe the bug is still being worked on, right Jurgen ?

-- 
Darren J Moffat

> > Nope, no work-around.  
> 
> OK. Then I have 3 questions:
> 
> 1) How do I destroy the pool that was on the firewire
> drive? (So that zfs stops complaining about it)
Even if the drive is disconnected, it should be possible
to "zpool export" it, so that the OS forgets about it
and doesn''t try to mount from that pool during the next
boot.

 
> 2) How can I reformat the firewire drive? Does this
> need to be done on a non-Solaris OS?
When 6445725 is fixed, it should be possible to reformat
and / or use it with Solaris.

> 3) Can your code diffs be integrated into the OS on
> my end to use this drive, and if so, how?
Sure.  You need the opensolaris "ON Source", unpack them,
apply the patch from the website using something like
"gpatch -p0 < scsa1394-mkfs-hang2-alt" and build everything
using the "nightly" command.

You''ll also need to install the "ON Specific Build Tools"
package,
the "ON Binary-Only Components", and the correct Studio 11 compiler
for building the opensolaris sources.

Here are some detailed instuctions on building the opensolaris sources:

    http://www.blastwave.org/articles/BLS-0050/index.html


Unfortunately, the sources for your installed version (build_64a) are
missing on http://dlc.sun.com/osol/on/downloads ; there are sources
for build 63 and 65, but not for 64a .

You could pick a newer release of the opensolaris sources (the latest
available for download is build_69), patch the sources and compile them,
and upgrade your installation to that newer release, using the "bfu" 
command.


Or pick a slightly newer release than 64a, patch & compile (make sure
to compile as a "release" build) , and just replace the firewire
kernel driver
modules that are affected by the bugfix, "scsa1394" and
"sbp":

usr/src/uts/intel/scsa1394/obj32/scsa1394 -> /kernel/drv/scsa1394
usr/src/uts/intel/scsa1394/obj64/scsa1394 -> /kernel/drv/amd64/scsa1394
usr/src/uts/intel/sbp2/obj32/sbp2 -> /kernel/misc/sbp2
usr/src/uts/intel/sbp2/obj64/sbp2 -> /kernel/misc/amd64/sbp2
 
 
This message posted from opensolaris.org

> > 3) Can your code diffs be integrated into the OS on my end to use this
drive, and if so, how?
> 
> I believe the bug is still being worked on, right J?rgen ?
The opensolaris sponsor process for fixing bug 6445725 seems
to got stuck.  I ping''ed Alan P. on the state of that bug...
 
 
This message posted from opensolaris.org

J?rgen Keil wrote:
>>> And 6560174 might be a duplicate of 6445725
>> I see what you mean. Unfortunately there does not
>> look to be a work-around. 
> 
> Nope, no work-around.  This is a scsa1394 bug; it
> has some issues when it is used from interrupt context.
> 
> I have some source code diffs, that are supposed to
> fix the issue, see this thread:
> 
>     http://www.opensolaris.org/jive/thread.jspa?messageID=46190
By coincidence, I spent some time dtracing 6560174 yesterday afternoon on 
b62, and these bugs are indeed duplicates. I never noticed 6445725 because my 
system wasn''t hanging but as the notes say, the fix for 6434435 changes
the
problem, and instead the error that gets propogated back from t1394_write() 
causes "transport rejected" messages.

I see your proposed fix (which looks very plausible) is dated over a year 
ago... Have you heard anything on when it might get integrated?

Pete.

> By coincidence, I spent some time dtracing 6560174 yesterday afternoon on 
> b62, and these bugs are indeed duplicates. I never noticed 6445725 because
my
> system wasn''t hanging but as the notes say, the fix for 6434435
changes the
> problem, and instead the error that gets propogated back from t1394_write()
> causes "transport rejected" messages.
Yes, I had filed two bugs (6445725 / 6434435) a  year ago and started the
opensolaris request-sponsor process for both.  The fix for 6434435 has been
integrated, but 6445725 is stuck somehow.  
> I see your proposed fix (which looks very plausible) is dated over a year 
> ago... Have you heard anything on when it might get integrated?
No, nothing.

I did send Alan Perry (@sun.com) a mail last friday, asking about the state
of bug 6445725 and my suggested fix, but so far received no reply...
 
 
This message posted from opensolaris.org

Thank you very much for this input. I eventually upgraded to snv_69 and did the
ON build of 69 with your patch. I copied to patched kernels over and have now
re-imported the defunct pool. The pool is working after a quick
''resilvering''. Thanks very much!
 
 
This message posted from opensolaris.org

This bug is still not integrated? To upgrade to a community release I still have
to patch and compile the kernel? How can this bug fix be integrated with the
code?
 
 
This message posted from opensolaris.org

Regarding the patches for this bug (6445725) I applied them to a nightly build
of snv 77, where they now appear to reside in usr/src/uts/intel/sbp2/debug64
rather than obj64 after successful build. I then copied them over the kernels in
the community release of b77.

usr/src/uts/intel/scsa1394/debug32/scsa1394 -> /kernel/drv/scsa1394
usr/src/uts/intel/scsa1394/debug64/scsa1394 -> /kernel/drv/amd64/scsa1394
usr/src/uts/intel/sbp2/debug32/sbp2 -> /kernel/misc/sbp2
usr/src/uts/intel/sbp2/debug64/sbp2 -> /kernel/misc/amd64/sbp2

I am wondering if anyone has tried patching and replacing these kernels in b77
and if so if they have experienced any system freezing after doing so?

I had to reboot into b69 because I had two hard freezes (only mouse moving,
keyboard unresponsive) in b77. I am running on an Ultra 20, so I thought that
the freeze may be due to the patch, since I would expect that not to happen on
Sun hardware with the standard community release.

thanks for any pointers 

ps. I still need the patch, otherwise my firewire drive is inaccessible
 
 
This message posted from opensolaris.org

I can''t believe its almost a year later, with a patch provided, and
this bug is still not fixed.

For those of us that cant recompile the sources, it makes solaris useless if we
want to use a firewire drive.

--m
 
 
This message posted from opensolaris.org

On Sat, 31 May 2008, Matt Cowger wrote:
> I can''t believe its almost a year later, with a patch provided,
and
> this bug is still not fixed.
>
> For those of us that cant recompile the sources, it makes solaris 
> useless if we want to use a firewire drive.
Can you provide the history behind this?  I was planning to use a 
firewire drive with ZFS.

Bob
=====================================Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

On Sat, 31 May 2008, Matt Cowger wrote:
> Sure.  Here''s the thread discussing the overall issue:
>
> http://www.opensolaris.org/jive/thread.jspa?threadID=36365&tstart=0
I found that via Google as well as this bug entry

http://bugs.opensolaris.org/view_bug.do?bug_id=6560174

This is really unfortunate since (in my experience) Firewire offers 
better peformance than USB2 for external disk access.

Bob
=====================================Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

Anyone willing to provide the modified kernel binaries for opensolaris2008.05?
 
 
This message posted from opensolaris.org

You should contact the storage community and hopefully get a hold of the
responsible engineer.  This is a firewire bug, not a ZFS bug.

- Eric

On Sat, May 31, 2008 at 03:56:52PM -0700, Matt Cowger
wrote:
> Anyone willing to provide the modified kernel binaries for
opensolaris2008.05?
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
--
Eric Schrock, Fishworks                        http://blogs.sun.com/eschrock