Hi folks,
For the records, I was running some tests yesterday on top of a
9.0-RELEASE, amd64, kernel when the box hanged. At the time of the
hang, the box was running a process with about 2800 threads with heavy
IPC between 1400 writers and 1400 readers. The box was in single user
mode (/bin/sh coming from FreeBSD 7.4-STABLE). Here is the beginning
of the dmesg:
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012
root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
CPU: Intel(R) Atom(TM) CPU D510 @ 1.66GHz (1666.70-MHz K8-class CPU)
Origin = "GenuineIntel" Id = 0x106ca Family = 6 Model = 1c
Stepping = 10
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x40e31d<SSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE>
AMD Features=0x20000800<SYSCALL,LM>
AMD Features2=0x1<LAHF>
TSC: P-state invariant, performance statistics
real memory = 2137587712 (2038 MB)
avail memory = 2037841920 (1943 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <070611 APIC1125>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads
cpu0 (BSP): APIC ID: 0
cpu1 (AP/HT): APIC ID: 1
cpu2 (AP): APIC ID: 2
cpu3 (AP/HT): APIC ID: 3
I will restart the test and see if this happens again.
regards,
- Arnaud
Hi, On Tue, Feb 14, 2012 at 11:41 AM, Arnaud Lacombe <lacombar@gmail.com> wrote:> Hi folks, > > For the records, I was running some tests yesterday on top of a > 9.0-RELEASE, amd64, kernel when the box hanged. At the time of the > hang, the box was running a process with about 2800 threads with heavy > IPC between 1400 writers and 1400 readers. The box was in single user > mode (/bin/sh coming from FreeBSD 7.4-STABLE). Here is the beginning > of the dmesg: >This happened a second time, now with FreeBSD 8.2-RELEASE. Complete machine hang. The machine was running about 4000 threads in a single process, all the other condition are the same. - Arnaud> Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > ? ? ? ?The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 9.0-RELEASE #0: Tue Jan ?3 07:46:30 UTC 2012 > ? ?root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 > CPU: Intel(R) Atom(TM) CPU D510 ? @ 1.66GHz (1666.70-MHz K8-class CPU) > ?Origin = "GenuineIntel" ?Id = 0x106ca ?Family = 6 ?Model = 1c ?Stepping = 10 > ?Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > ?Features2=0x40e31d<SSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE> > ?AMD Features=0x20000800<SYSCALL,LM> > ?AMD Features2=0x1<LAHF> > ?TSC: P-state invariant, performance statistics > real memory ?= 2137587712 (2038 MB) > avail memory = 2037841920 (1943 MB) > Event timer "LAPIC" quality 400 > ACPI APIC Table: <070611 APIC1125> > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs > FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads > ?cpu0 (BSP): APIC ID: ?0 > ?cpu1 (AP/HT): APIC ID: ?1 > ?cpu2 (AP): APIC ID: ?2 > ?cpu3 (AP/HT): APIC ID: ?3 > > I will restart the test and see if this happens again. > > regards, > ?- Arnaud
Hi, [for the record...] On Tue, Feb 14, 2012 at 11:41 AM, Arnaud Lacombe <lacombar@gmail.com> wrote:> Hi folks, > > For the records, I was running some tests yesterday on top of a > 9.0-RELEASE, amd64, kernel when the box hanged. At the time of the > hang, the box was running a process with about 2800 threads with heavy > IPC between 1400 writers and 1400 readers. The box was in single user > mode (/bin/sh coming from FreeBSD 7.4-STABLE). Here is the beginning > of the dmesg: > > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > ? ? ? ?The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 9.0-RELEASE #0: Tue Jan ?3 07:46:30 UTC 2012 > ? ?root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 > CPU: Intel(R) Atom(TM) CPU D510 ? @ 1.66GHz (1666.70-MHz K8-class CPU) > ?Origin = "GenuineIntel" ?Id = 0x106ca ?Family = 6 ?Model = 1c ?Stepping = 10 > ?Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > ?Features2=0x40e31d<SSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE> > ?AMD Features=0x20000800<SYSCALL,LM> > ?AMD Features2=0x1<LAHF> > ?TSC: P-state invariant, performance statistics > real memory ?= 2137587712 (2038 MB) > avail memory = 2037841920 (1943 MB) > Event timer "LAPIC" quality 400 > ACPI APIC Table: <070611 APIC1125> > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs > FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads > ?cpu0 (BSP): APIC ID: ?0 > ?cpu1 (AP/HT): APIC ID: ?1 > ?cpu2 (AP): APIC ID: ?2 > ?cpu3 (AP/HT): APIC ID: ?3 > > I will restart the test and see if this happens again. >I reproduced the previous problem on 10-CURRENT from r233917, on the following platform (here running 8.2-RELEASE): FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Atom(TM) CPU D525 @ 1.80GHz (1800.01-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x106ca Family = 6 Model = 1c Stepping = 10 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x40e31d<SSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE> AMD Features=0x20100800<SYSCALL,NX,LM> AMD Features2=0x1<LAHF> TSC: P-state invariant real memory = 2136539136 (2037 MB) avail memory = 2043772928 (1949 MB) ACPI APIC Table: <010312 APIC0947> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP/HT): APIC ID: 3 Complete system freeze while running about 2400 threads. I had to power cycle the system to get it back alive. I discussed a way to debug this with attilio@ on freebsd-stable@, but still did not had time to implement it. regards, - Arnaud