freebsd stable - Jan 2012 - FreeBSD 9 crash/deadlock when dump(8)ing file system with journaling enabled.

A few days ago I discovered that running dump(8) on a file system which 
has journaled soft updates enabled causes the machine to become 
unresponsive.  This is using FreeBSD 9.0-R.   I can still interact with 
the system a little bit (typing echos back) but never get a response to 
any action.   It seems like there is an I/O issue (deadlock or overload) 
and CPU usage spikes as well.  Resetting the machine seems to be the 
only solution once this occurs (CTRL-C, CTRL-ALT-DEL, logging in again, 
etc all fail).

Disclaimer: I have only tested this as a VM under ESXi 5.0 as I don't 
have access to any physical servers with 9.0 on them that I can test 
with (I will soon though).

As a side note this is the same issue reported here:
http://forums.freebsd.org/showthread.php?t=25787

For now I've turned off journaling (soft updates seem fine) and that 
works around the issue.

Let me know if I can provide more details etc!

-- 

Adam Strohl
A-Team Systems

On Mon, Jan 30, 2012 at 06:12:20PM +0700, Adam Strohl
wrote:
> A few days ago I discovered that running dump(8) on a file system
> which has journaled soft updates enabled causes the machine to
> become unresponsive.  This is using FreeBSD 9.0-R.   I can still
> interact with the system a little bit (typing echos back) but never
> get a response to any action.   It seems like there is an I/O issue
> (deadlock or overload) and CPU usage spikes as well.  Resetting the
> machine seems to be the only solution once this occurs (CTRL-C,
> CTRL-ALT-DEL, logging in again, etc all fail).
> 
> Disclaimer: I have only tested this as a VM under ESXi 5.0 as I
> don't have access to any physical servers with 9.0 on them that I
> can test with (I will soon though).
> 
> As a side note this is the same issue reported here:
> http://forums.freebsd.org/showthread.php?t=25787
> 
> For now I've turned off journaling (soft updates seem fine) and that
> works around the issue.
> 
> Let me know if I can provide more details etc!
I'm not sure, but this may be an after-effect of known problems right
now with SU+J on 9.0.  It would help if you could state if you're using
"dump -L" or not.

I've seen the "deadlock" behaviour you describe on older FreeBSD
versions (dating back to at least 7.x) when using "dump -L", which
generates a fs snapshot.  Obviously 7.x does not have SU+J, so I'm a
little surprised disabling journalling fixes the problem for you.  This
type of problem is exactly why we have avoided use of filesystem
snapshots on our systems for years upon years and why we use
rsnapshot/rsync instead of dump.  I'm sure if you Google you can find
old posts from me talking about the problem.

You may also be interested in the thread on freebsd-fs titled "kernel:
panic: softdep_sync_buf: Unknown type jnewblk".  I realise that's a
panic vs.  the deadlock you're seeing.  You will need to search the
mailing list for that subject, since not everyone's mail clients support
Reference headers thus replies are scattered all over:

http://lists.freebsd.org/pipermail/freebsd-fs/2012-January/thread.html

The most important thing to note is probably this:

http://lists.freebsd.org/pipermail/freebsd-fs/2012-January/013573.html

-- 
| Jeremy Chadwick                                 jdc@parodius.com |
| Parodius Networking                     http://www.parodius.com/ |
| UNIX Systems Administrator                 Mountain View, CA, US |
| Making life hard for others since 1977.             PGP 4BD6C0CB |

Hi, what would be the best choice?

1. turn off SUJ and dump using -L flag and the turn it on again (but I'm
affraid to destroy the system this way - is this possible?)
2. dump the unmounted partitions with a live cd, per example

Sorry for top posting, mobile mail client problem.

Em 30/01/2012 12:56, "Ivan Voras" <ivoras@freebsd.org>escreveu:

On 30/01/2012 13:06, Jeremy Chadwick wrote:
>> For now I've turned off journaling (soft updates see...
It's a known bug: SU+J currently deadlocks when used with UFS snapshots.




_______________________________________________
freebsd-stable@freebsd.org mailing list
http://li...