freebsd stable - Mar 2005 - FreeBSD 5.3 SMP freezes with MySQL 4.1

Hi,

I have a dell pe2650 box with dual xeon 2.4G, disabled the HTT, 
installed 5.3-RELEASE, enabled SMP, mysql 4.1.10a built from
the port. 

If the mysql connections is high (e.g. over 100 connections), 
the system will freezes in several minutes, and the fatal alway 
indicated to the mysqld process. When I build the kernel without 
SMP, the system is stable. I have try 5.3-p5 and 5.4-PRE, the 
problem is still there.

It's my configuration mistake or will be solved in 5.4-RELEASE ?

ÀîÒã¸Õ wrote:
>Hi,
>
>I have a dell pe2650 box with dual xeon 2.4G, disabled the HTT, 
>installed 5.3-RELEASE, enabled SMP, mysql 4.1.10a built from
>the port. 
>
>If the mysql connections is high (e.g. over 100 connections), 
>the system will freezes in several minutes, and the fatal alway 
>indicated to the mysqld process. 
>
Does it actually crash or juyst hangs?
Which thread options did you use in the port config?
Please post your dmseg, mysql log, mysql slow query log.
>When I build the kernel without 
>SMP, the system is stable. I have try 5.3-p5 and 5.4-PRE, the 
>problem is still there.
>
>It's my configuration mistake or will be solved in 5.4-RELEASE ?
>  
>
I'm currently running a mysql 4.10 on a heavy loaded 5.3-RELEASE-p5
production SMP server with no problems at all.

U.
>  
>
>------------------------------------------------------------------------
>
>_______________________________________________
>freebsd-stable@freebsd.org mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-stable
>To unsubscribe, send any mail to
"freebsd-stable-unsubscribe@freebsd.org"
>  
>

Young Lee wrote:
> the system is hang and throw a fatal 12 pannic.
> 
> I build mysql41-server from port without any parameters, and the
> thread lib use libpthread.so.1, use ldd to look details as below
> 
> admin# ldd /usr/local/libexec/mysqld
> /usr/local/libexec/mysqld:
>         libz.so.2 => /lib/libz.so.2 (0x28422000)
>         libwrap.so.3 => /usr/lib/libwrap.so.3 (0x28432000)
>         libcrypt.so.2 => /lib/libcrypt.so.2 (0x2843a000)
>         libstdc++.so.4 => /usr/lib/libstdc++.so.4 (0x28452000)
>         libm.so.3 => /lib/libm.so.3 (0x28524000)
>         libpthread.so.1 => /usr/lib/libpthread.so.1 (0x2853f000)
>         libc.so.5 => /lib/libc.so.5 (0x28563000)
> 
i wonder why should it uses libwrap and libstdc++...

did you try compiling it optimized? or with linuxthreads?

> I use the original SMP kernel configuration file:
> 
> admin# cat /usr/src/sys/i386/conf/SMP
> include GENERIC
> ident           SMP-GENERIC
> options         SMP

did you add "options 	SMP" in your kernel conf?
I can see the kernel loading 1 CPU

should look like that:

<snip>
...
Copyright (c) 1992-2004 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD 5.3-RELEASE-p5 #2: Tue Feb 22 16:47:08 UTC 2005
    root@localhost.localdomain:/usr/obj/usr/src/sys/GENERIC
ACPI APIC Table: <HP     00000083>
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.20GHz (3200.13-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf34  Stepping = 4

Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Hyperthreading: 2 logical CPUs
real memory  = 2147430400 (2047 MB)
avail memory = 2100178944 (2002 MB)
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  6
 cpu3 (AP): APIC ID:  7
 ioapic0 <Version 2.0> irqs 0-23 on motherboard
 ioapic1 <Version 2.0> irqs 24-47 on motherboard
...
</snip>

> 
> -------------------- dmesg with up below ---------------------
> Copyright (c) 1992-2005 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>         The Regents of the University of California. All rights reserved.
> FreeBSD 5.4-PRERELEASE #2: Thu Mar 24 19:16:17 CST 2005 
>     root@admin.smartdove.com:/usr/obj/usr/src/sys/GENERIC
> Timecounter "i8254" frequency 1193182 Hz quality 0
> CPU: Intel(R) Xeon(TM) CPU 2.40GHz (2387.28-MHz 686-class CPU)
>   Origin = "GenuineIntel"  Id = 0xf29  Stepping = 9
>  
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,
> MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
> real memory  = 2147352576 (2047 MB)
> avail memory = 2095882240 (1998 MB)
> ACPI APIC Table: <DELL   PE2650  >
> ioapic0: Changing APIC ID to 8
> ioapic1: Changing APIC ID to 9
> ioapic2: Changing APIC ID to 10
> MADT: Forcing active-low polarity and level trigger for SCI
> ioapic0 <Version 1.1> irqs 0-15 on motherboard
> ioapic1 <Version 1.1> irqs 16-31 on motherboard
> ioapic2 <Version 1.1> irqs 32-47 on motherboard
> npx0: <math processor> on motherboardusing
> npx0: INT 16 interfaceusing
> acpi0: <DELL PE2650> on motherboard
> acpi0: Power Button (fixed)
> Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000
> acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
> cpu0: <ACPI CPU> on acpi0
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pci0: <ACPI PCI bus> on pcib0
> pci0: <unknown> at device 4.0 (no driver attached)
> pci0: <unknown> at device 4.1 (no driver attached)
> pci0: <unknown> at device 4.2 (no driver attached)
> pci0: <display, VGA> at device 14.0 (no driver attached)
> atapci0: <ServerWorks CSB5 UDMA100 controller> port
0x8b0-0x8bf,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 1
> 5.1 on pci0
> ata0: channel #0 on atapci0
> ata1: channel #1 on atapci0
> ohci0: <OHCI (generic) USB controller> mem 0xfe100000-0xfe100fff irq
5 at device 15.2 on pci0
> usb0: OHCI version 1.0, legacy support
> usb0: SMM does not respond, resetting
> usb0: <OHCI (generic) USB controller> on ohci0
> usb0: USB revision 1.0
> uhub0: (0x1166) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
> uhub0: 4 ports with 4 removable, self powered
> isab0: <PCI-ISA bridge> at device 15.3 on pci0
> isa0: <ISA bus> on isab0
> pcib1: <ACPI Host-PCI bridge> on acpi0
> pci4: <ACPI PCI bus> on pcib1
> pcib2: <ACPI PCI-PCI bridge> at device 8.0 on pci4
> pci5: <ACPI PCI bus> on pcib2
> aac0: <Dell PERC 3/Di> mem 0xf8000000-0xfbffffff irq 30 at device 8.1
on pci4
> aac0: i960RX 100MHz, 54MB cache memory, optional battery present
> aac0: Kernel 2.8-0, Build 6082, S/N a438d3
> aac0: Supported
Options=275c<WCACHE,DATA64,HOSTTIME,WINDOW4GB,SOFTERR,NORECOND,SGMAP64>
> pcib3: <ACPI Host-PCI bridge> on acpi0
> pci3: <ACPI PCI bus> on pcib3
> bge0: <Broadcom BCM5703 Gigabit Ethernet, ASIC rev. 0x1002> mem
0xfcf10000-0xfcf1ffff irq 28 at device 6.0 on p
> ci3
> miibus0: <MII bus> on bge0
> brgphy0: <BCM5703 10/100/1000baseTX PHY> on miibus0
> brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
1000baseTX-FDX, auto
> bge0: Ethernet address: 00:0d:56:ba:1f:6b
> bge1: <Broadcom BCM5703 Gigabit Ethernet, ASIC rev. 0x1002> mem
0xfcf00000-0xfcf0ffff irq 29 at device 8.0 on p
> ci3
> miibus1: <MII bus> on bge1
> brgphy1: <BCM5703 10/100/1000baseTX PHY> on miibus1
> brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
1000baseTX-FDX, auto
> bge1: Ethernet address: 00:0d:56:ba:1f:6c
> pcib4: <ACPI Host-PCI bridge> on acpi0
> pci2: <ACPI PCI bus> on pcib4
> pcib5: <ACPI Host-PCI bridge> on acpi0
> pci1: <ACPI PCI bus> on pcib5
> fdc0: <floppy drive controller> port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on
acpi0
> fd0: <1440-KB 3.5" drive> on fdc0 drive 0
> atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10
on acpi0
> sio0: type 16550A
> sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
> orm0: <ISA Option ROMs> at iomem
0xec000-0xeffff,0xc8000-0xcbfff,0xc0000-0xc7fff on isa0
> pmtimer0 on isa0
> ppc0: parallel port not found.
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=0x300>
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on
isa0
> Timecounter "TSC" frequency 2387278408 Hz quality 800
> Timecounters tick every 10.000 msec
> acd0: CDROM <TEAC CD-ROM CD-224E/K.9A> at ata0-master UDMA33
> aacd0: <RAID 5> on aac0
> aacd0: 104137MB (213274368 sectors)
> Mounting root from ufs:/dev/aacd0s1a
> IP Filter: v3.4.35 initialized.  Default = pass all, Logging = enabled
> -------------------- dmesg end -------------------------------
> 
> 

-- 
Uzi Klein
Software Development Manager
BMBY Software Systems Ltd
2 Hataasia Street, Yokneam, Israel
Email:   uzi@bmby.com
Phone:  +972 4  959 79 89
Mobile: +972 52 374 53 34
Fax:    +972 3  617 93 36
Web:     http://www.bmby.com

On Mar 29, 2005, at 1:17 AM, ~{@nRc8U~} wrote:
> I have a dell pe2650 box with dual xeon 2.4G, disabled the HTT,
> installed 5.3-RELEASE, enabled SMP, mysql 4.1.10a built from
> the port.
I am observing similar problems with high load (not necessarily high 
number of connections) with 5.4-PRERELEASE from March 22 on a Tyan K8SR 
motherboard dual Opteron system running PostgreSQL 8.

I am not ready to rule out hardware problem, though, as I have had a 
fair number of ethernet port timout+reset reported by the kernel.  I 
shall try disabling SMP to see what happens.  It is times like these I 
wish there was an equivalent to the Dell diagnostics suite for other 
machines :-(  If only Dell would sell Opteron!

On Tue, 29 Mar 2005, [GB2312] =C0=EE=D2=E3=B8=D5 wrote:
> Hi,
>
> I have a dell pe2650 box with dual xeon 2.4G, disabled the HTT,
> installed 5.3-RELEASE, enabled SMP, mysql 4.1.10a built from
> the port.
>
> If the mysql connections is high (e.g. over 100 connections),
> the system will freezes in several minutes, and the fatal alway
> indicated to the mysqld process. When I build the kernel without
> SMP, the system is stable. I have try 5.3-p5 and 5.4-PRE, the
> problem is still there.
>
> It's my configuration mistake or will be solved in 5.4-RELEASE ?
There's just not enough information to tell. When you say
"freeze", is the
machine totally unresponsive, or does the console allow you to type
characters? Can you ping the system when it appears frozen? Does the
system eventually recover if the load is taken away?

--=20
Doug White                    |  FreeBSD: The Power to Serve
dwhite@gumbysoft.com          |  www.FreeBSD.org