I know this has been reported already, but I want to give a "me too". After installing a new world and kernel from the tree yesterday afternoon, I let my system run all night. This morning everything was extremely sluggish and unresponsive. According to top, which I thankfully left running, processes were going in and out of "*kmem_" (obviously truncated). CPU usage was 80+% system and load averages were around 5.4. The only changes I made to my system besides upgrading were to include the options KDB, DDB and STACK in my kernel for zfs functionality. Unfortunately, I cannot try without those options since my root is zfs. Booting a kernel from 8/20 works fine.
Wes Morgan wrote:> I know this has been reported already, but I want to give a "me too". > After installing a new world and kernel from the tree yesterday > afternoon, I let my system run all night. This morning everything was > extremely sluggish and unresponsive. According to top, which I > thankfully left running, processes were going in and out of "*kmem_" > (obviously truncated). CPU usage was 80+% system and load averages were > around 5.4. The only changes I made to my system besides upgrading were > to include the options KDB, DDB and STACK in my kernel for zfs > functionality. Unfortunately, I cannot try without those options since > my root is zfs. Booting a kernel from 8/20 works fine.Same here. Running root on ZFS. Yesterday I hosed my system and recovered by booting FreeSBIE, downloading the latest amd64-bootonly.iso copying /boot/kernel/*, tweaking loader.conf for the GENERIC kernel and I was back in business. First thing was to csup again, add STACK to my custom kernel (which is much like generic with all the drivers unrelated to my hardware removed) and build/install world. All looked good except running "systat -vmstat 1" shows Sys using 100% of CPU after a few minutes. I'm not even sure if it's related to anything specific running on the box or not. Just gets progressively more sluggish within minutes of booting and all the cycles seem to be spinning in the Sys category with almost nothing left for userland or anything else. The network stack seems to be routing traffic well enough though. I can still stream music and video and send emails such as this one routed via the box in question. But otherwise just about any load seems to send %Sys into the stratosphere. -- Yarema http://yds.CoolRat.org/zfsboot.shtml
On Sun, Aug 31, 2008 at 06:35:16AM -0500, Wes Morgan wrote:> I know this has been reported already, but I want to give a "me too". > After installing a new world and kernel from the tree yesterday afternoon, > I let my system run all night. This morning everything was extremely > sluggish and unresponsive. According to top, which I thankfully left > running, processes were going in and out of "*kmem_" (obviously > truncated). CPU usage was 80+% system and load averages were around 5.4. > The only changes I made to my system besides upgrading were to include the > options KDB, DDB and STACK in my kernel for zfs functionality. > Unfortunately, I cannot try without those options since my root is zfs. > Booting a kernel from 8/20 works fine.That's not good news. :-( I think I'll have to look at a partial back out of the ZFS changes now. -- John Birrell
On Sun, Aug 31, 2008 at 06:35:16AM -0500, Wes Morgan wrote:> I know this has been reported already, but I want to give a "me too". > After installing a new world and kernel from the tree yesterday afternoon, > I let my system run all night. This morning everything was extremely > sluggish and unresponsive. According to top, which I thankfully left > running, processes were going in and out of "*kmem_" (obviously > truncated). CPU usage was 80+% system and load averages were around 5.4. > The only changes I made to my system besides upgrading were to include the > options KDB, DDB and STACK in my kernel for zfs functionality. > Unfortunately, I cannot try without those options since my root is zfs. > Booting a kernel from 8/20 works fine.Wes, will you please try removing "#define KMEM_DEBUG" from: src/sys/cddl/compat/opensolaris/kern/opensolaris_kmem.c This is the likely cause of the performance problems you are seeing. It is also the reason why you needed to add DDB, DBB and stack to the kernel. The quickest way to try this is to just build the 'opensolaris' kernel module: (keep a copy of your current /boot/kernel) cd src/sys/modules/opensolaris make obj && make depend && make all && make install -- John Birrell