Jim King
2007-Apr-28 11:50 UTC
[CentOS] How to diagnose "Unable to handle kernel paging request" kernel panic?
Does anyone know how to diagnose the error "Unable to handle kernel paging request at virtual address xxxxxx"? I got this error (followed by a panic and a crash) on a server running CentOS 4.4 with the 2.6.9-42.0.8.ELsmp kernel. The machine is based on an Intel 975XBX2 motherboard (975X chipset). I have built several of these and they have been extremely stable. This one is an Asterisk PBX. It was up for over a month before this crash. Here is the exact kernel build info, from the restart log: "Linux version 2.6.9-42.0.8.ELsmp (mockbuild at builder6.centos.org) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-3)) #1 SMP Tue Jan 30 12:33:47 EST 2007" Does the call trace indicate something to do with the ext3 file system? I am not running RAID on this machine, it is just a single drive with LVM. Any help that the list could be would be really great. Thanks, Jim Here is the /var/log/messages entry: Apr 25 11:29:25 amd kernel: Unable to handle kernel paging request at virtual address 7c4ad43d Apr 25 11:29:25 amd kernel: printing eip: Apr 25 11:29:25 amd kernel: f8892ad2 Apr 25 11:29:25 amd kernel: *pde = 00000000 Apr 25 11:29:25 amd kernel: Oops: 0002 [#1] Apr 25 11:29:25 amd kernel: SMP Apr 25 11:29:25 amd kernel: Modules linked in: zttranscode(U) wctdm24xxp(U) zaptel(U) md5 ipv6 parport_pc lp parport autofs4 sunrpc crc_ccitt button battery ac hw_random sky2 e1000 floppy dm_snapshot dm_zero dm_mirror ext3 jbd dm_mod ata_piix libata sd_mod scsi_mod Apr 25 11:29:25 amd kernel: CPU: 0 Apr 25 11:29:25 amd kernel: EIP: 0060:[<f8892ad2>] Not tainted VLI Apr 25 11:29:25 amd kernel: EFLAGS: 00010202 (2.6.9-42.0.8.ELsmp) Apr 25 11:29:25 amd kernel: EIP is at journal_grab_journal_head +0x26/0x42 [jbd] Apr 25 11:29:25 amd kernel: eax: 1e14ae0f ebx: cc9e5284 ecx: 7c4ad439 edx: cc9e5284 Apr 25 11:29:25 amd kernel: esi: c1114300 edi: 00000000 ebp: c89e5284 esp: f7cfee24 Apr 25 11:29:25 amd kernel: ds: 007b es: 007b ss: 0068 Apr 25 11:29:25 amd kernel: Process kswapd0 (pid: 51, threadinfo=f7cfe000 task=f7d276f0) Apr 25 11:29:25 amd kernel: Stack: f888d9f0 00000000 f7ec5a00 f88c44d3 000000d0 f3c4f678 f7cfef58 c015d323 Apr 25 11:29:25 amd kernel: f3c4f678 c1114300 c0149c7f 00000001 00000001 00000010 00000000 f7cfeec8 Apr 25 11:29:25 amd kernel: f7cfee64 f7cfee64 00000002 00000001 c11142c0 c11142e0 c1114140 c1114160 Apr 25 11:29:25 amd kernel: Call Trace: Apr 25 11:29:25 amd kernel: [<f888d9f0>] journal_try_to_free_buffers +0x5d/0xc5 [jbd] Apr 25 11:29:25 amd kernel: [<f88c44d3>] ext3_releasepage+0x0/0x54 [ext3] Apr 25 11:29:25 amd kernel: [<c015d323>] try_to_release_page+0x34/0x46 Apr 25 11:29:25 amd kernel: [<c0149c7f>] shrink_list+0x293/0x3ed Apr 25 11:29:25 amd kernel: [<c0149fb6>] shrink_cache+0x1dd/0x34d Apr 25 11:29:25 amd kernel: [<c0144e78>] get_writeback_state+0x30/0x35 Apr 25 11:29:25 amd kernel: [<c0180c1e>] mb_cache_shrink_fn+0x56/0x131 Apr 25 11:29:25 amd kernel: [<c0149776>] shrink_slab+0x7e/0x161 Apr 25 11:29:25 amd kernel: [<c014a674>] shrink_zone+0xa7/0xb6 Apr 25 11:29:25 amd kernel: [<c014aa6f>] balance_pgdat+0x1c5/0x30e Apr 25 11:29:25 amd kernel: [<c02d27c9>] schedule+0x86d/0x8db Apr 25 11:29:25 amd kernel: [<c0120420>] prepare_to_wait+0x12/0x4c Apr 25 11:29:25 amd kernel: [<c014ac82>] kswapd+0xca/0xcc Apr 25 11:29:25 amd kernel: [<c01204f5>] autoremove_wake_function +0x0/0x2d Apr 25 11:29:25 amd kernel: [<c02d47f2>] ret_from_fork+0x6/0x14 Apr 25 11:29:25 amd kernel: [<c01204f5>] autoremove_wake_function +0x0/0x2d Apr 25 11:29:25 amd kernel: [<c014abb8>] kswapd+0x0/0xcc Apr 25 11:29:25 amd kernel: [<c01041f5>] kernel_thread_helper+0x5/0xb Apr 25 11:29:25 amd kernel: Code: 5b 5e 5f 5d c3 89 c2 31 c9 eb 0b f3 90 8b 02 a9 00 00 10 00 75 f5 f0 0f ba 2a 14 19 c0 85 c0 75 ec 8b 02 f6 c4 20 74 06 8b 4a 28 <ff> 41 04 8b 02 a9 00 00 10 00 75 08 0f 0b 11 02 58 42 89 f8 f0 Apr 25 11:29:25 amd kernel: <0>Fatal exception: panic in 5 seconds
Steven Haigh
2007-Apr-28 12:16 UTC
[CentOS] How to diagnose "Unable to handle kernel paging request" kernel panic?
On 28/04/2007, at 9:50 PM, Jim King wrote:> Does anyone know how to diagnose the error "Unable to handle kernel > paging request at virtual address xxxxxx"? > > I got this error (followed by a panic and a crash) on a server > running CentOS 4.4 with the 2.6.9-42.0.8.ELsmp kernel. The machine > is based on an Intel 975XBX2 motherboard (975X chipset). I have > built several of these and they have been extremely stable. This > one is an Asterisk PBX. It was up for over a month before this crash. > > Here is the exact kernel build info, from the restart log: "Linux > version 2.6.9-42.0.8.ELsmp (mockbuild at builder6.centos.org) (gcc > version 3.4.6 20060404 (Red Hat 3.4.6-3)) #1 SMP Tue Jan 30 > 12:33:47 EST 2007" > > Does the call trace indicate something to do with the ext3 file > system? I am not running RAID on this machine, it is just a single > drive with LVM. > > Any help that the list could be would be really great.<snip> Do you have a TDM400 card in these machines by chance? If so, are you running Zaptel 1.4.2? I had this same crash happen using zaptel 1.4.2. Updating to 1.4.2.1 fixed this. -- Steven Haigh Email: netwiz at crc.id.au Web: http://www.crc.id.au Phone: (03) 9017 0597 - 0404 087 474