similar to: ZFS Locking Up periodically

Now at 36 hours since zdb process start and: PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP 827 root 4936M 4931M sleep 59 0 0:50:47 0.2% zdb/209 Idling at 0.2% processor for nearly the past 24 hours... feels very stuck. Thoughts on how to determine where and why? -- This message posted from opensolaris.org

on Solaris 10 5.10 Generic_118822-23 sun4v sparc SUNW,Sun-Fire-T200 I run #iostat -xn 5 to monitor the IO statistics on SF T2000 server. The system also have a heavy IO load, for some reason iostat donot refresh (no any update). It seems like iostat is calling pause() and stucked there. Also my HBA driver''s interrupt stack trace indicates there is a lot of swtch(), the overall IOPS

I just had a quick play with gzip compression on a filesystem and the result was the machine grinding to a halt while copying some large (.wav) files to it from another filesystem in the same pool. The system became very unresponsive, taking several seconds to echo keystrokes. The box is a maxed out AMD QuadFX, so it should have plenty of grunt for this. Comments? Ian

Hi, There seems to be a few threads about zpool hang, do we have a workaround to resolve the hang issue without rebooting ? In my case, I have a pool with disks from external LUNs via a fiber cable. When the cable is unplugged while there is IO in the pool, All zpool related command hang (zpool status, zpool list, etc.), put the cable back does not solve the problem. Eventually, I

Hi All, I used the following script from the dtrace manual to measure how long a particular process runs on the cpu. Modified it to look at the exact timestamps: [i] sched:::on-cpu { self->ts = timestamp; printf("on=%d\n", self->ts) } sched:::off-cpu /self->ts/ { printf("off=%d\n", timestamp) self->ts = 0; } [/i] The output looks like this: [i]on=24591 off=24603

It seems we are hitting a boundary with zfs send/receive over a network link (10Gb/s). We can see peak values of up to 150MB/s, but on average about 40-50MB/s are replicated. This is far away from the bandwidth that a 10Gb link can offer. Is it possible, that ZFS is giving replication a too low priority/throttling it too much?

Hi, I am profiling some workloads for the voluntary and involuntary context switches. I am interested in finding out the reasons causing these two types of context switches. As far as I understand, involuntary context switch happens on expiration of time slice or when a higher priority process comes in. While the voluntary switch generally happens when a process is waiting for I/O etc. So to

Customer has a Thumper running: SunOS x4501 5.10 Generic_120012-14 i86pc i386 i86pc where running "zpool detech disk c6t7d0" to detech a mirror causes zpool command to hang with following kernel stack trace: PC: _resume_from_idle+0xf8 CMD: zpool detach disk1 c6t7d0 stack pointer for thread fffffe84d34b4920: fffffe8001c30c10 [ fffffe8001c30c10 _resume_from_idle+0xf8() ]

The vendor wanted to come in and replace an HDD in the 2nd X4500, as it was "constantly busy", and since our x4500 has always died miserably in the past when a HDD dies, they wanted to replace it before the HDD actually died. The usual was done, HDD replaced, resilvering started and ran for about 50 minutes. Then the system hung, same as always, all ZFS related commands would just

I have a ZFS pool that has been corrupted. The pool contains a single device which was actually a file on UFS. The machine was accidentally halted and now the pool is corrupt. There are (of course) no backups and I''ve been asked to recover the pool. The system panics when trying to do anything with the pool. root@:/$ zpool status panic[cpu1]/thread=fffffe8000758c80: assertion failed:

After multiple power outages caused by storms coming through, I can no longer access /dev/zvol/dsk/poolname, which are hold l2arc and slog devices in another pool I don''t think this is related, since I the pools are ofline pending access to the volumes. I tried running find /dev/zvol/dsk/poolname -type f and here is the stack, hopefully this someone a hint at what the issue is, I have

Hello, i'm experiencing periodic kernel panics on a server with FreeBSD 7.0-STABLE #0: Tue May 20 19:09:43 CEST 2008. My big problem is that the system is not performing memory dumping and/or automatic reoboot, it just stays there. Here' console output: em0: watchdog timeout -- resetting kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 0;

Hi all. One of our server had a panic and now can''t mount the zpool anymore! Here is what I get at boot: Mar 21 11:09:17 SERVER142 ^Mpanic[cpu1]/thread=ffffffff90878200: Mar 21 11:09:17 SERVER142 genunix: [ID 603766 kern.notice] assertion failed: ss->ss_start <= start (0x670000b800 <= 0x67 00009000), file: ../../common/fs/zfs/space_map.c, line: 126 Mar 21 11:09:17 SERVER142

Hi, System: Netra 1405, 4x450Mhz, 4GB RAM and 2x146GB (root pool) and 2x146GB (space pool). snv_98. After a panic the system hangs on boot and manual attempts to mount (at least) one dataset in single user mode, hangs. The Panic: Dec 27 04:42:11 base ^Mpanic[cpu0]/thread=300021c1a20: Dec 27 04:42:11 base unix: [ID 521688 kern.notice] [AFT1] errID 0x00167f73.1c737868 UE Error(s) Dec 27

Hi there We have a pair of servers running FreeBSD 9.1-RC3 that act as transparent layer 7 loadbalancer (relayd) and pop/imap proxy (dovecot). Only one of them is active at a given time, it's a failover setup. From time to time the active one gets in a state in which the 'thread taskq' thread uses up 100% of one cpu on its own, like here: ---- PID USERNAME PRI NICE SIZE

Hello ZFS, System was rebooted and after reboot server again System is snv_39, SPARC, T2000 bash-3.00# ptree 7 /lib/svc/bin/svc.startd -s 163 /sbin/sh /lib/svc/method/fs-local 254 /usr/sbin/zfs mount -a [...] bash-3.00# zfs list|wc -l 46 Using df I can see most file systems are already mounted. > ::ps!grep zfs R 254 163 7 7 0 0x4a004000

Hi, I am fairly new to DTrace, and wanted to ask something regarding synchronization objects. I have a multithreaded C++ program in which I use semaphores (POSIX version) for synchronization. I was interested in using DTrace to figure out the total time for which each thread in my application sleeps, blocked on a semaphore, i.e., as a result of sem_wait() call. In order to measure this, I

Hi All, I believe this has been asked before, but I wasn?t able to find too much information about the subject. Long story short, I was moving data around on a storage zpool of mine and a zfs destroy <filesystem> hung (or so I thought). This pool had dedup turned on at times while imported as well; it?s running on a Nexenta Core 3.0.1 box (snv_134f). The first time the machine was

Hi. T2000 1.2GHz 8-core, 32GB RAM, S10U3, zil_disable=1. Command ''zpool export f3-2'' is hung for 30 minutes now and still is going. Nothing else is running on the server. I can see one CPU being 100% in SYS like: bash-3.00# mpstat 1 [...] CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 0 0 67 220 110 20 0 0 0 0

this is 6.0-STABLE as for Mar 17. KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.1-PRERELEASE #2: Fri Mar 17 11:05:32 UTC 2006 vlad@host:/usr/src/sys/amd64/compile/DEF_WEB Timecounter