Hi Genius, Good Day !!!!!!!!!!!!!!!!!! I am Daniel. My OSS getting automatically rebooted again and again . kindly help to me Its showing the below error messages *kernel: LustreError: 23351:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error (-19) req at ffff810400e24400 x1353488904620274/t0 o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292738958 ref 1 fl Interpret:/0/0 rc -19/0 kernel: LustreError: 137-5: UUID ''south-ost7_UUID'' is not available for connect (no target) kernel: LustreError: 23284:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error (-19) req at ffff8101124c7c00 x1353488904620359/t0 o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292739025 ref 1 fl Interpret:/0/0 rc -19/0 * Regards, Daniel A -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-devel/attachments/20101221/c097948b/attachment.html
Hi Genius, Good Day !!!!!!!!!!!!!!!!!! I am Daniel. My OSS getting automatically rebooted again and again . kindly help to me Its showing the below error messages *kernel: LustreError: 23351:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error (-19) req at ffff810400e24400 x1353488904620274/t0 o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292738958 ref 1 fl Interpret:/0/0 rc -19/0 kernel: LustreError: 137-5: UUID ''south-ost7_UUID'' is not available for connect (no target) kernel: LustreError: 23284:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error (-19) req at ffff8101124c7c00 x1353488904620359/t0 o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292739025 ref 1 fl Interpret:/0/0 rc -19/0 * Regards, Daniel A -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-devel/attachments/20101221/7e7a2091/attachment.html
Hi Genius, Good Day !!!!!!!!!!!!!!!!!! I am Daniel. My OSS getting automatically rebooted again and again . kindly help to me Its showing the below error messages *kernel: LustreError: 23351:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error (-19) req at ffff810400e24400 x1353488904620274/t0 o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292738958 ref 1 fl Interpret:/0/0 rc -19/0 kernel: LustreError: 137-5: UUID ''south-ost7_UUID'' is not available for connect (no target) kernel: LustreError: 23284:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error (-19) req at ffff8101124c7c00 x1353488904620359/t0 o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292739025 ref 1 fl Interpret:/0/0 rc -19/0 * Regards, Daniel A -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101221/e13ef8ae/attachment.html
Jeff Johnson
2010-Dec-21 06:15 UTC
[Lustre-discuss] Fwd: Reg /// OSS rebooted automatically
Daniel, It looks like your OST backend storage device may be having an issue. I would check the health and stability of the backend storage device or raid you are using for an OST device. It wouldn''t likely cause a system reboot of your OSS system. There may be more problems, hardware and/or OS related that are causing the system to reboot in addition to Lustre complaining that it can''t find the OST storage device. Others here on the list will likely give you a more detailed answer. The storage device is the place i would look first. --Jeff -- ------------------------------ Jeff Johnson Manager Aeon Computing jeff.johnson at aeoncomputing.com www.aeoncomputing.com t: 858-412-3810 x101 f: 858-412-3845 m: 619-204-9061 4905 Morena Boulevard, Suite 1313 - San Diego, CA 92117 On Mon, Dec 20, 2010 at 9:43 PM, Daniel Raj <danielraj2006 at gmail.com> wrote:> > > > Hi Genius, > > > Good Day !!!!!!!!!!!!!!!!!! > > > I am Daniel. My OSS getting automatically rebooted again and again . > kindly help to me > > Its showing the below error messages > > > *kernel: LustreError: 23351:0:(ldlm_lib.c:1892:target_send_reply_msg()) > @@@ processing error (-19) req at ffff810400e24400 x1353488904620274/t0 > o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292738958 ref 1 fl Interpret:/0/0 rc > -19/0 > kernel: LustreError: 137-5: UUID ''south-ost7_UUID'' is not available for > connect (no target) > kernel: LustreError: 23284:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ > processing error (-19) req at ffff8101124c7c00 x1353488904620359/t0 > o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292739025 ref 1 fl Interpret:/0/0 rc > -19/0 > * > > Regards, > > Daniel A > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101220/f09143bf/attachment.html
Hi Jeff, Thanks for your reply *Storage information *: DL380G5 == OSS + 16GB Ram OS == SFS G3.2-2 + centos 5.3 + lustre 1.8.3 MSA60 box == OST RAID 6 Regards, Daniel A On Tue, Dec 21, 2010 at 11:45 AM, Jeff Johnson < jeff.johnson at aeoncomputing.com> wrote:> Daniel, > > It looks like your OST backend storage device may be having an issue. I > would check the health and stability of the backend storage device or raid > you are using for an OST device. It wouldn''t likely cause a system reboot of > your OSS system. There may be more problems, hardware and/or OS related that > are causing the system to reboot in addition to Lustre complaining that it > can''t find the OST storage device. > > Others here on the list will likely give you a more detailed answer. The > storage device is the place i would look first. > > --Jeff > > -- > ------------------------------ > Jeff Johnson > Manager > Aeon Computing > > jeff.johnson at aeoncomputing.com > www.aeoncomputing.com > t: 858-412-3810 x101 f: 858-412-3845 > m: 619-204-9061 > > 4905 Morena Boulevard, Suite 1313 - San Diego, CA 92117 > > > On Mon, Dec 20, 2010 at 9:43 PM, Daniel Raj <danielraj2006 at gmail.com>wrote: > >> >> >> >> Hi Genius, >> >> >> Good Day !!!!!!!!!!!!!!!!!! >> >> >> I am Daniel. My OSS getting automatically rebooted again and again . >> kindly help to me >> >> Its showing the below error messages >> >> >> *kernel: LustreError: 23351:0:(ldlm_lib.c:1892:target_send_reply_msg()) >> @@@ processing error (-19) req at ffff810400e24400 x1353488904620274/t0 >> o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292738958 ref 1 fl Interpret:/0/0 rc >> -19/0 >> kernel: LustreError: 137-5: UUID ''south-ost7_UUID'' is not available for >> connect (no target) >> kernel: LustreError: 23284:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ >> processing error (-19) req at ffff8101124c7c00 x1353488904620359/t0 >> o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292739025 ref 1 fl Interpret:/0/0 rc >> -19/0 >> * >> >> Regards, >> >> Daniel A >> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101221/6dc14f52/attachment.html
Jeff Johnson
2010-Dec-21 07:00 UTC
[Lustre-discuss] Fwd: Reg /// OSS rebooted automatically
Daniel, Check the health and stability of your raid-6 volume. Make sure the raid is healthy and online. Use whatever monitor utility came with your raid card or check /proc/mdstat if it''s a Linux mdraid. Check /var/log/messages for error messages from your raid or other hardware. --Jeff ---mobile signature--- Jeff Johnson - Aeon Computing jeff.johnson at aeoncomputing.com On Dec 20, 2010, at 22:27, Daniel Raj <danielraj2006 at gmail.com> wrote:> Hi Jeff, > > > Thanks for your reply > > Storage information : > > > DL380G5 == OSS + 16GB Ram > OS == SFS G3.2-2 + centos 5.3 + lustre 1.8.3 > MSA60 box == OST > RAID 6 > > > Regards, > > Daniel A > > On Tue, Dec 21, 2010 at 11:45 AM, Jeff Johnson <jeff.johnson at aeoncomputing.com> wrote: > Daniel, > > It looks like your OST backend storage device may be having an issue. I would check the health and stability of the backend storage device or raid you are using for an OST device. It wouldn''t likely cause a system reboot of your OSS system. There may be more problems, hardware and/or OS related that are causing the system to reboot in addition to Lustre complaining that it can''t find the OST storage device. > > Others here on the list will likely give you a more detailed answer. The storage device is the place i would look first. > > --Jeff > > -- > ------------------------------ > Jeff Johnson > Manager > Aeon Computing > > jeff.johnson at aeoncomputing.com > www.aeoncomputing.com > t: 858-412-3810 x101 f: 858-412-3845 > m: 619-204-9061 > > 4905 Morena Boulevard, Suite 1313 - San Diego, CA 92117 > > > On Mon, Dec 20, 2010 at 9:43 PM, Daniel Raj <danielraj2006 at gmail.com> wrote: > > > > Hi Genius, > > > Good Day !!!!!!!!!!!!!!!!!! > > > I am Daniel. My OSS getting automatically rebooted again and again . kindly help to me > > Its showing the below error messages > > > kernel: LustreError: 23351:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error (-19) req at ffff810400e24400 x1353488904620274/t0 o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292738958 ref 1 fl Interpret:/0/0 rc -19/0 > kernel: LustreError: 137-5: UUID ''south-ost7_UUID'' is not available for connect (no target) > kernel: LustreError: 23284:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error (-19) req at ffff8101124c7c00 x1353488904620359/t0 o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292739025 ref 1 fl Interpret:/0/0 rc -19/0 > > > Regards, > > Daniel A > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101220/0481fed1/attachment-0001.html
Hi Jeff,
*messages :*
Dec 19 04:03:03 cluster syslogd 1.4.1: restart.
Dec 19 04:18:20 cluster kernel: Lustre:
23278:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
d957783f-e60b-07b0-2c86-ecfbc7eb57b6 reconnecting
Dec 19 04:18:20 cluster kernel: Lustre:
23278:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 04:19:43 cluster kernel: Lustre:
23316:0:(ldlm_lib.c:575:target_handle_reconnect()) dan4-OST0006:
487f7476-b6de-72f6-adb5-9ee7dc77bc43 reconnecting
Dec 19 04:19:49 cluster kernel: Lustre:
23300:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
d957783f-e60b-07b0-2c86-ecfbc7eb57b6 reconnecting
Dec 19 04:19:49 cluster kernel: Lustre:
23300:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 4 previous
similar messages
Dec 19 04:30:05 cluster kernel: Lustre:
23308:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
d957783f-e60b-07b0-2c86-ecfbc7eb57b6 reconnecting
Dec 19 04:30:05 cluster kernel: LustreError: 137-5: UUID
''cluster-ost7_UUID''
is not available for connect (no target)
Dec 19 04:30:05 cluster kernel: LustreError:
23290:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8103fd722c00 x1355442914715019/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292713305 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 04:30:05 cluster kernel: LustreError:
23290:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 04:30:05 cluster kernel: Lustre:
23308:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 04:30:29 cluster kernel: Lustre:
23355:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
d957783f-e60b-07b0-2c86-ecfbc7eb57b6 reconnecting
Dec 19 04:30:29 cluster kernel: Lustre:
23355:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 04:38:53 cluster kernel: Lustre:
23273:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 04:38:53 cluster kernel: Lustre:
23273:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 04:41:23 cluster kernel: Lustre:
23340:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 04:41:23 cluster kernel: Lustre:
23340:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 04:41:43 cluster kernel: Lustre:
23261:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
36a78046-7285-b0ed-6784-2e7ea2856d77 reconnecting
Dec 19 04:41:43 cluster kernel: Lustre:
23261:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 04:45:46 cluster kernel: Lustre:
23275:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
e0c295fb-5f86-338a-1564-cc553519d5c1 reconnecting
Dec 19 04:45:46 cluster kernel: Lustre:
23275:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 04:47:25 cluster kernel: Lustre:
23353:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 04:47:25 cluster kernel: Lustre:
23371:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0007:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 04:47:25 cluster kernel: Lustre:
23353:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 04:47:31 cluster kernel: LustreError: 137-5: UUID
''cluster-ost7_UUID''
is not available for connect (no target)
Dec 19 04:47:31 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 04:47:31 cluster kernel: LustreError:
23296:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8100b39b6800 x1353489081275843/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292714351 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 04:47:31 cluster kernel: LustreError:
23296:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 04:47:35 cluster kernel: LustreError: 137-5: UUID
''cluster-ost51_UUID'' is not available for connect (no target)
Dec 19 04:47:35 cluster kernel: LustreError:
23341:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff81041a93f050 x1353489081275844/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292714355 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 04:47:35 cluster kernel: LustreError: Skipped 2 previous similar
messages
Dec 19 04:49:55 cluster kernel: Lustre:
23305:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
6c537da9-d2da-dff0-b81e-75c61a79537e reconnecting
Dec 19 04:49:55 cluster kernel: Lustre:
23368:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0007:
6c537da9-d2da-dff0-b81e-75c61a79537e reconnecting
Dec 19 04:49:55 cluster kernel: Lustre:
23305:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 04:56:18 cluster kernel: LustreError: 137-5: UUID
''cluster-ost7_UUID''
is not available for connect (no target)
Dec 19 04:56:18 cluster kernel: LustreError:
23300:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8100345c1000 x1353489081276757/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292714878 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 04:56:18 cluster kernel: LustreError:
23300:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 2 previous similar
messages
Dec 19 04:56:27 cluster kernel: LustreError: 137-5: UUID
''cluster-ost8_UUID''
is not available for connect (no target)
Dec 19 04:56:27 cluster kernel: LustreError:
23377:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8100b224f400 x1353489081276769/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292714887 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 04:56:27 cluster kernel: LustreError: Skipped 2 previous similar
messages
Dec 19 05:00:46 cluster kernel: Lustre:
23292:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 05:00:46 cluster kernel: Lustre:
23292:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 05:01:03 cluster kernel: LustreError: 137-5: UUID
''cluster-ost7_UUID''
is not available for connect (no target)
Dec 19 05:01:03 cluster kernel: LustreError:
23317:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8100345c1000 x1353489081277216/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292715163 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 05:01:03 cluster kernel: LustreError:
23317:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 2 previous similar
messages
Dec 19 05:01:08 cluster kernel: LustreError: 137-5: UUID
''cluster-ost8_UUID''
is not available for connect (no target)
Dec 19 05:01:08 cluster kernel: LustreError:
23345:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8100b3f63c00 x1353489081277244/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292715168 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 05:01:08 cluster kernel: LustreError: Skipped 2 previous similar
messages
Dec 19 05:07:32 cluster kernel: Lustre:
23271:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
e0c295fb-5f86-338a-1564-cc553519d5c1 reconnecting
Dec 19 05:07:32 cluster kernel: Lustre:
23271:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 6 previous
similar messages
Dec 19 05:22:33 cluster kernel: Lustre:
23258:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
6c537da9-d2da-dff0-b81e-75c61a79537e reconnecting
Dec 19 05:22:33 cluster kernel: Lustre:
23258:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 05:49:45 cluster kernel: Lustre:
23306:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
e0c295fb-5f86-338a-1564-cc553519d5c1 reconnecting
Dec 19 05:49:45 cluster kernel: Lustre:
23306:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 7 previous
similar messages
Dec 19 06:04:53 cluster kernel: Lustre:
23299:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 06:04:53 cluster kernel: Lustre:
23299:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 7 previous
similar messages
Dec 19 06:42:30 cluster kernel: Lustre:
23349:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 06:42:30 cluster kernel: Lustre:
23349:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 06:49:25 cluster kernel: Lustre:
23275:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 06:49:25 cluster kernel: Lustre:
23275:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 06:49:27 cluster kernel: LustreError: 137-5: UUID
''cluster-ost7_UUID''
is not available for connect (no target)
Dec 19 06:49:27 cluster kernel: LustreError:
23266:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff81041a946850 x1353489081288957/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292721667 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 06:49:27 cluster kernel: LustreError:
23266:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 2 previous similar
messages
Dec 19 06:49:27 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 06:59:42 cluster kernel: Lustre:
23373:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
36a78046-7285-b0ed-6784-2e7ea2856d77 reconnecting
Dec 19 06:59:42 cluster kernel: Lustre:
23373:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 07:24:57 cluster kernel: Lustre:
23351:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 07:24:57 cluster kernel: Lustre:
23351:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 07:24:59 cluster kernel: LustreError: 137-5: UUID
''cluster-ost51_UUID'' is not available for connect (no target)
Dec 19 07:24:59 cluster kernel: LustreError:
23309:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8100b22cc000 x1353489081292967/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292723799 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 07:24:59 cluster kernel: LustreError:
23309:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 07:24:59 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 07:27:05 cluster kernel: Lustre: There was an unexpected network
error while writing to 172.32.0.108: -110.
Dec 19 07:30:13 cluster kernel: Lustre: dan3-OST0007: haven''t heard
from
client 1bf012e7-5fd4-d475-7569-98cf491c5f8a (at 172.32.0.108 at tcp) in 227
seconds. I think it''s dead, and I am evicting it.
Dec 19 07:30:13 cluster kernel: Lustre: Skipped 3 previous similar messages
Dec 19 07:32:29 cluster kernel: Lustre:
23269:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 07:32:29 cluster kernel: Lustre:
23269:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 07:46:21 cluster kernel: Lustre:
23325:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 07:46:21 cluster kernel: Lustre:
23325:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 07:50:07 cluster kernel: Lustre:
23318:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
d957783f-e60b-07b0-2c86-ecfbc7eb57b6 reconnecting
Dec 19 07:50:07 cluster kernel: Lustre:
23318:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 08:05:51 cluster kernel: Lustre:
23262:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 08:05:51 cluster kernel: Lustre:
23260:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0007:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 08:05:51 cluster kernel: Lustre:
23262:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 1 previous
similar message
Dec 19 08:13:21 cluster kernel: Lustre:
23272:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0007:
6c537da9-d2da-dff0-b81e-75c61a79537e reconnecting
Dec 19 08:13:21 cluster kernel: Lustre:
23272:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 1 previous
similar message
Dec 19 08:14:51 cluster kernel: Lustre:
23259:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
6c537da9-d2da-dff0-b81e-75c61a79537e reconnecting
Dec 19 08:14:51 cluster kernel: Lustre:
23259:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 08:15:19 cluster kernel: Lustre:
23280:0:(ldlm_lib.c:575:target_handle_reconnect()) dan4-OST0007:
83b5fa13-7c6d-728f-f5c5-4af45debf629 reconnecting
Dec 19 08:15:19 cluster kernel: Lustre:
23303:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0007:
6c537da9-d2da-dff0-b81e-75c61a79537e reconnecting
Dec 19 08:15:19 cluster kernel: Lustre:
23303:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 08:15:41 cluster kernel: LustreError: 137-5: UUID
''cluster-ost8_UUID''
is not available for connect (no target)
Dec 19 08:15:41 cluster kernel: LustreError:
23289:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8103fd722400 x1353488909210458/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292726841 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 08:15:41 cluster kernel: LustreError:
23289:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 08:15:41 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 08:19:36 cluster kernel: Lustre:
23357:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
36a78046-7285-b0ed-6784-2e7ea2856d77 reconnecting
Dec 19 08:19:36 cluster kernel: Lustre:
23357:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 08:20:43 cluster kernel: Lustre: dan4-OST0007: haven''t heard
from
client 83b5fa13-7c6d-728f-f5c5-4af45debf629 (at 172.22.0.121 at tcp) in 227
seconds. I think it''s dead, and I am evicting it.
Dec 19 08:20:43 cluster kernel: Lustre: Skipped 3 previous similar messages
Dec 19 08:20:54 cluster kernel: Lustre: dan3-OST0006: haven''t heard
from
client 6c537da9-d2da-dff0-b81e-75c61a79537e (at 172.22.0.121 at tcp) in 227
seconds. I think it''s dead, and I am evicting it.
Dec 19 08:20:54 cluster kernel: Lustre: Skipped 2 previous similar messages
Dec 19 08:26:09 cluster kernel: Lustre:
23361:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
d957783f-e60b-07b0-2c86-ecfbc7eb57b6 reconnecting
Dec 19 08:26:09 cluster kernel: Lustre:
23361:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 08:35:25 cluster kernel: Lustre:
23311:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 08:35:25 cluster kernel: Lustre:
23311:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 08:48:52 cluster kernel: Lustre:
23309:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
d957783f-e60b-07b0-2c86-ecfbc7eb57b6 reconnecting
Dec 19 08:48:52 cluster kernel: Lustre:
23309:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 09:10:17 cluster kernel: Lustre:
23339:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
e4e6ae31-04e5-6d20-284b-eb4f8b4e4c32 reconnecting
Dec 19 09:10:17 cluster kernel: Lustre:
23339:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 09:10:23 cluster kernel: Lustre:
23313:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
d957783f-e60b-07b0-2c86-ecfbc7eb57b6 reconnecting
Dec 19 09:10:23 cluster kernel: Lustre:
23313:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 09:15:09 cluster kernel: Lustre:
23276:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 09:15:09 cluster kernel: Lustre:
23276:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 09:18:50 cluster kernel: Lustre:
23332:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 09:18:50 cluster kernel: Lustre:
23332:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 09:36:17 cluster kernel: Lustre:
23364:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 09:36:17 cluster kernel: Lustre:
23364:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 09:49:58 cluster kernel: Lustre:
23363:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
e4e6ae31-04e5-6d20-284b-eb4f8b4e4c32 reconnecting
Dec 19 09:49:58 cluster kernel: Lustre:
23363:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 09:56:46 cluster kernel: Lustre:
23267:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
e0c295fb-5f86-338a-1564-cc553519d5c1 reconnecting
Dec 19 09:56:46 cluster kernel: Lustre:
23267:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 09:59:59 cluster kernel: Lustre:
23336:0:(ldlm_lib.c:575:target_handle_reconnect()) dan4-OST0006:
ff151d6a-5ef7-087d-8daa-09aed6ee250b reconnecting
Dec 19 09:59:59 cluster kernel: Lustre:
23336:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 10:00:34 cluster kernel: Lustre:
23263:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
e4e6ae31-04e5-6d20-284b-eb4f8b4e4c32 reconnecting
Dec 19 10:00:34 cluster kernel: Lustre:
23263:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 10:20:42 cluster kernel: Lustre:
23302:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
e0c295fb-5f86-338a-1564-cc553519d5c1 reconnecting
Dec 19 10:20:42 cluster kernel: Lustre:
23302:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 10:37:41 cluster kernel: Lustre:
23293:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 10:37:41 cluster kernel: Lustre:
23293:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 10:40:29 cluster kernel: Lustre:
23382:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
e4e6ae31-04e5-6d20-284b-eb4f8b4e4c32 reconnecting
Dec 19 10:40:29 cluster kernel: Lustre:
23382:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 11:00:21 cluster kernel: Lustre:
23271:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
e4e6ae31-04e5-6d20-284b-eb4f8b4e4c32 reconnecting
Dec 19 11:00:21 cluster kernel: Lustre:
23271:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 11:18:34 cluster kernel: Lustre:
23257:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
12406f21-0627-0afa-2101-5da03d74c298 reconnecting
Dec 19 11:18:34 cluster kernel: Lustre:
23257:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 11:21:42 cluster kernel: Lustre:
23321:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
36a78046-7285-b0ed-6784-2e7ea2856d77 reconnecting
Dec 19 11:21:42 cluster kernel: LustreError: 137-5: UUID
''cluster-ost7_UUID''
is not available for connect (no target)
Dec 19 11:21:42 cluster kernel: LustreError:
23282:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff810349a21000 x1353488904616992/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292738002 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 11:21:42 cluster kernel: LustreError:
23282:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 11:21:42 cluster kernel: Lustre:
23321:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 11:22:14 cluster kernel: Lustre:
23344:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
36a78046-7285-b0ed-6784-2e7ea2856d77 reconnecting
Dec 19 11:22:14 cluster kernel: Lustre:
23344:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 11:25:24 cluster kernel: Lustre:
23349:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
36a78046-7285-b0ed-6784-2e7ea2856d77 reconnecting
Dec 19 11:25:24 cluster kernel: Lustre:
23349:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 3 previous
similar messages
Dec 19 11:25:27 cluster kernel: Lustre:
23349:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
36a78046-7285-b0ed-6784-2e7ea2856d77 reconnecting
Dec 19 11:25:27 cluster kernel: Lustre:
23349:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 1 previous
similar message
Dec 19 11:26:00 cluster kernel: LustreError: 137-5: UUID
''cluster-ost51_UUID'' is not available for connect (no target)
Dec 19 11:26:00 cluster kernel: LustreError:
23313:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff81041b261050 x1353488904617703/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292738260 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 11:26:00 cluster kernel: LustreError:
23313:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 11:26:00 cluster kernel: LustreError: Skipped 6 previous similar
messages
Dec 19 11:26:46 cluster kernel: LustreError: 137-5: UUID
''cluster-ost7_UUID''
is not available for connect (no target)
Dec 19 11:26:46 cluster kernel: LustreError:
23293:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8103fe823c00 x1353488904617805/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292738306 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 11:26:46 cluster kernel: LustreError:
23293:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 11:26:46 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 11:28:04 cluster kernel: Lustre:
23346:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
36a78046-7285-b0ed-6784-2e7ea2856d77 reconnecting
Dec 19 11:28:04 cluster kernel: Lustre:
23346:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 11:31:12 cluster kernel: Lustre:
23312:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
36a78046-7285-b0ed-6784-2e7ea2856d77 reconnecting
Dec 19 11:31:12 cluster kernel: Lustre:
23297:0:(ldlm_lib.c:575:target_handle_reconnect()) dan4-OST0006:
ff151d6a-5ef7-087d-8daa-09aed6ee250b reconnecting
Dec 19 11:31:12 cluster kernel: Lustre:
23297:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 1 previous
similar message
Dec 19 11:31:12 cluster kernel: Lustre:
23312:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 1 previous
similar message
Dec 19 11:34:24 cluster kernel: LustreError: 137-5: UUID
''cluster-ost7_UUID''
is not available for connect (no target)
Dec 19 11:34:24 cluster kernel: LustreError:
23287:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8100b28c1000 x1353488904619366/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292738764 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 11:34:24 cluster kernel: LustreError:
23287:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 11:34:24 cluster kernel: Lustre:
23287:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
36a78046-7285-b0ed-6784-2e7ea2856d77 reconnecting
Dec 19 11:34:24 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 11:35:29 cluster kernel: LustreError: 137-5: UUID
''cluster-ost51_UUID'' is not available for connect (no target)
Dec 19 11:35:29 cluster kernel: LustreError:
23304:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff810329105800 x1353488904619546/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292738829 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 11:35:29 cluster kernel: LustreError:
23304:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 11:35:29 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 11:37:38 cluster kernel: LustreError: 137-5: UUID
''cluster-ost7_UUID''
is not available for connect (no target)
Dec 19 11:37:38 cluster kernel: LustreError:
23351:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff810400e24400 x1353488904620274/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292738958 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 11:37:38 cluster kernel: LustreError:
23351:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 11:37:38 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 11:38:45 cluster kernel: LustreError: 137-5: UUID
''cluster-ost7_UUID''
is not available for connect (no target)
Dec 19 11:38:45 cluster kernel: LustreError:
23284:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8101124c7c00 x1353488904620359/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292739025 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 11:38:45 cluster kernel: LustreError:
23284:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 11:41:04 cluster kernel: LustreError: 137-5: UUID
''cluster-ost8_UUID''
is not available for connect (no target)
Dec 19 11:41:04 cluster kernel: LustreError:
23324:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff8100b39bac00 x1353488904620700/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292739164 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 11:41:04 cluster kernel: LustreError: Skipped 2 previous similar
messages
Dec 19 11:42:26 cluster kernel: LustreError: 137-5: UUID
''cluster-ost8_UUID''
is not available for connect (no target)
Dec 19 11:42:26 cluster kernel: LustreError:
23330:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff810049702400 x1353488904620861/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292739246 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 11:42:26 cluster kernel: LustreError:
23330:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 11:42:26 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 11:44:05 cluster kernel: LustreError: 137-5: UUID
''cluster-ost8_UUID''
is not available for connect (no target)
Dec 19 11:44:05 cluster kernel: LustreError:
23292:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff810283dee000 x1353488904620989/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292739345 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 11:44:05 cluster kernel: LustreError:
23292:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 11:44:05 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 11:48:11 cluster syslogd 1.4.1: restart.
Dec 19 11:48:11 cluster kernel: klogd 1.4.1, log source = /proc/kmsg
started.
Dec 19 11:48:11 cluster kernel: Linux version
2.6.18-164.11.1.el5_lustre.1.8.3 (lbuild at x86-build-1) (gcc version 4.1.2
20080704 (Red Hat 4.1.2-46)) #1 SMP Fri Apr 9 18:00:39 MDT 2010
Dec 19 11:48:11 cluster kernel: Command line: ro root=LABEL=/ rhgb quiet
irqpoll maxcpus=1 reset_devices memmap=exactmap
memmap=640K at 0Kmemmap=5276K at 16384Kmemmap=125140K at
22300Kelfcorehdr=147440K
memmap=32K#3144016K
Dec 19 11:48:11 cluster kernel: BIOS-provided physical RAM map:
Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000000010000 -
000000000009ec00 (usable)
Dec 19 11:48:11 cluster kernel: BIOS-e820: 000000000009ec00 -
00000000000a0000 (reserved)
Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000000100000 -
00000000bfe54000 (usable)
Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe54000 -
00000000bfe5c000 (ACPI data)
Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe5c000 -
00000000bfe5d000 (usable)
Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe5d000 -
00000000c0000000 (reserved)
Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000e0000000 -
00000000f0000000 (reserved)
Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000fec00000 -
00000000fed00000 (reserved)
Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000fee00000 -
00000000fee10000 (reserved)
Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000ffc00000 -
0000000100000000 (reserved)
Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000100000000 -
000000043ffff000 (usable)
Dec 19 11:48:11 cluster kernel: user-defined physical RAM map:
Dec 19 11:48:11 cluster kernel: user: 0000000000000000 - 00000000000a0000
(usable)
Dec 19 11:48:12 cluster kernel: user: 0000000001000000 - 0000000001527000
(usable)
Dec 19 11:48:12 cluster kernel: user: 00000000015c7000 - 0000000008ffc000
(usable)
Dec 19 11:48:12 cluster kernel: user: 00000000bfe54000 - 00000000bfe5c000
(ACPI data)
Dec 19 11:48:12 cluster kernel: DMI 2.4 present.
Dec 19 11:48:12 cluster rpc.statd[5886]: Version 1.0.9 Starting
Dec 19 11:48:12 cluster rpc.statd[5886]: statd running as root. chown
/var/lib/nfs/statd/sm to choose different user
Dec 19 11:48:12 cluster kernel: No NUMA configuration found
Dec 19 11:48:12 cluster kernel: Faking a node at
0000000000000000-0000000008ffc000
Dec 19 11:48:12 cluster kernel: Bootmem setup node 0
0000000000000000-0000000008ffc000
Dec 19 11:48:12 cluster kernel: Memory for crash kernel (0x0 to 0x0)
notwithin permissible range
Dec 19 11:48:12 cluster kernel: disabling kdump
Dec 19 11:48:12 cluster kernel: ACPI: PM-Timer IO Port: 0x908
Dec 19 11:48:12 cluster kernel: ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00]
enabled)
Dec 19 11:48:12 cluster kernel: Processor #0 6:15 APIC version 20
Dec 19 11:48:12 cluster kernel: ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04]
disabled)
Dec 19 11:48:12 cluster kernel: ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02]
disabled)
Dec 19 11:48:12 cluster kernel: ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06]
enabled)
Dec 19 11:48:12 cluster kernel: Processor #6 6:15 APIC version 20
Dec 19 11:48:12 cluster kernel: ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01]
enabled)
Dec 19 11:48:12 cluster kernel: Processor #1 6:15 APIC version 20
Dec 19 11:48:12 cluster kernel: ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05]
disabled)
Dec 19 11:48:12 cluster kernel: ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03]
disabled)
Dec 19 11:48:12 cluster kernel: ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07]
enabled)
Dec 19 11:48:12 cluster kernel: Processor #7 6:15 APIC version 20
Dec 19 11:48:12 cluster kernel: ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl
lint[0x1])
Dec 19 11:48:12 cluster kernel: ACPI: IOAPIC (id[0x08] address[0xfec00000]
gsi_base[0])
Dec 19 11:48:12 cluster kernel: IOAPIC[0]: apic_id 8, version 32, address
0xfec00000, GSI 0-23
Dec 19 11:48:12 cluster kernel: ACPI: IOAPIC (id[0x09] address[0xfec82000]
gsi_base[24])
Dec 19 11:48:12 cluster kernel: IOAPIC[1]: apic_id 9, version 32, address
0xfec82000, GSI 24-47
Dec 19 11:48:12 cluster kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0
global_irq 2 high edge)
Dec 19 11:48:12 cluster kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9
global_irq 9 high level)
Dec 19 11:48:12 cluster kernel: Setting APIC routing to physical flat
Dec 19 11:48:12 cluster kernel: ACPI: HPET id: 0x8086a201 base: 0xfed00000
Dec 19 11:48:12 cluster kernel: Using ACPI (MADT) for SMP configuration
information
Dec 19 11:48:12 cluster kernel: Nosave address range: 00000000000a0000 -
0000000001000000
Dec 19 11:48:12 cluster kernel: Nosave address range: 0000000001527000 -
00000000015c7000
Dec 19 11:48:13 cluster kernel: Allocating PCI resources starting at
10000000 (gap: 8ffc000:b6e58000)
Dec 19 11:48:13 cluster kernel: SMP: Allowing 8 CPUs, 4 hotplug CPUs
Dec 19 11:48:13 cluster kernel: Built 1 zonelists. Total pages: 32251
Dec 19 11:48:13 cluster kernel: Kernel command line: ro root=LABEL=/ rhgb
quiet irqpoll maxcpus=1 reset_devices memmap=exactmap
memmap=640K at 0Kmemmap=5276K at 16384Kmemmap=125140K at
22300Kelfcorehdr=147440K
memmap=32K#3144016K
Dec 19 11:48:13 cluster kernel: Misrouted IRQ fixup and polling support
enabled
Dec 19 11:48:13 cluster kernel: This may significantly impact system
performance
Dec 19 11:48:13 cluster kernel: Initializing CPU#0
Dec 19 11:48:13 cluster kernel: PID hash table entries: 512 (order: 9, 4096
bytes)
Dec 19 11:48:13 cluster kernel: irq 146, desc: ffffffff803e3580, depth: 1,
count: 0, unhandled: 0
Dec 19 11:48:13 cluster kernel: ->handle_irq(): ffffffff800ba32b,
handle_bad_irq+0x0/0x1f6
Dec 19 11:48:13 cluster kernel: ->chip(): ffffffff8030fee0,
0xffffffff8030fee0
Dec 19 11:48:13 cluster kernel: ->action(): 0000000000000000
Dec 19 11:48:13 cluster kernel: IRQ_DISABLED set
Dec 19 11:48:13 cluster kernel: unexpected IRQ trap at vector 92
Dec 19 11:48:13 cluster kernel: Console: colour VGA+ 80x25
Dec 19 11:48:13 cluster kernel: Dentry cache hash table entries: 16384
(order: 5, 131072 bytes)
Dec 19 11:48:13 cluster kernel: Inode-cache hash table entries: 8192 (order:
4, 65536 bytes)
Dec 19 11:48:13 cluster kernel: Checking aperture...
Dec 19 11:48:13 cluster kernel: ACPI: DMAR not present
Dec 19 11:48:13 cluster kernel: Memory: 119188k/147440k available (2553k
kernel code, 11868k reserved, 1293k data, 208k init)
Dec 19 11:48:13 cluster kernel: Calibrating delay loop (skipped), value
calculated using timer frequency.. 5999.99 BogoMIPS (lpj=2999999)
Dec 19 11:48:13 cluster kernel: Security Framework v1.0.0 initialized
Dec 19 11:48:13 cluster kernel: SELinux: Initializing.
Dec 19 11:48:13 cluster kernel: selinux_register_security: Registering
secondary module capability
Dec 19 11:48:13 cluster kernel: Capability LSM initialized as secondary
Dec 19 11:48:13 cluster S14daninit: started
Dec 19 11:48:13 cluster kernel: Mount-cache hash table entries: 256
Dec 19 11:48:13 cluster kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Dec 19 11:48:13 cluster kernel: CPU: L2 cache: 4096K
Dec 19 11:48:13 cluster kernel: using mwait in idle threads.
Dec 19 11:48:13 cluster kernel: CPU: Physical Processor ID: 3
Dec 19 11:48:13 cluster kernel: CPU: Processor Core ID: 0
Dec 19 11:48:13 cluster kernel: SMP alternatives: switching to UP code
Dec 19 11:48:13 cluster kernel: ACPI: Core revision 20060707
Dec 19 11:48:13 cluster kernel: Using local APIC timer interrupts.
Dec 19 11:48:13 cluster kernel: Detected 20.833 MHz APIC timer.
Dec 19 11:48:13 cluster kernel: Brought up 1 CPUs
Dec 19 11:48:13 cluster kernel: testing NMI watchdog ... OK.
Dec 19 11:48:13 cluster kernel: time.c: Using 14.318180 MHz WALL HPET GTOD
HPET/TSC timer.
Dec 19 11:48:13 cluster kernel: time.c: Detected 2999.999 MHz processor.
Dec 19 11:48:14 cluster kernel: checking if image is initramfs... it is
Dec 19 11:48:14 cluster kernel: irq 170, desc: ffffffff803e4d80, depth: 1,
count: 0, unhandled: 0
Dec 19 11:48:14 cluster kernel: ->handle_irq(): ffffffff800ba32b,
handle_bad_irq+0x0/0x1f6
Dec 19 11:48:14 cluster kernel: ->chip(): ffffffff8030fee0,
0xffffffff8030fee0
Dec 19 11:48:14 cluster kernel: ->action(): 0000000000000000
Dec 19 11:48:14 cluster kernel: IRQ_DISABLED set
Dec 19 11:48:14 cluster kernel: unexpected IRQ trap at vector aa
Dec 19 11:48:14 cluster kernel: Freeing initrd memory: 2352k freed
Dec 19 11:48:14 cluster kernel: NET: Registered protocol family 16
Dec 19 11:48:14 cluster kernel: ACPI: bus type pci registered
Dec 19 11:48:14 cluster kernel: PCI: BIOS Bug: MCFG area at e0000000 is not
E820-reserved
Dec 19 11:48:14 cluster kernel: PCI: Not using MMCONFIG.
Dec 19 11:48:14 cluster kernel: PCI: Using configuration type 1
Dec 19 11:48:14 cluster kernel: ACPI: Interpreter enabled
Dec 19 11:48:14 cluster kernel: ACPI: Using IOAPIC for interrupt routing
Dec 19 11:48:15 cluster kernel: ACPI: No dock devices found.
Dec 19 11:48:15 cluster kernel: ACPI: PCI Root Bridge [PCI0] (0000:00)
Dec 19 11:48:17 cluster kernel: PCI: Ignoring BAR0-3 of IDE controller
0000:00:1f.1
Dec 19 11:48:19 cluster kernel: PCI: PXH quirk detected, disabling MSI for
SHPC device
Dec 19 11:48:21 cluster kernel: PCI: Transparent bridge - 0000:00:1e.0
Dec 19 11:48:43 cluster kernel: ACPI: PCI Interrupt Link [LNKA] (IRQs *5 7
10 11), disabled.
Dec 19 11:48:43 cluster kernel: ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *7
10 11), disabled.
Dec 19 11:48:43 cluster kernel: ACPI: PCI Interrupt Link [LNKC] (IRQs 5 7
*10 11), disabled.
Dec 19 11:48:43 cluster kernel: ACPI: PCI Interrupt Link [LNKD] (IRQs 5 7
*10 11), disabled.
Dec 19 11:48:43 cluster kernel: ACPI: PCI Interrupt Link [LNKE] (IRQs 5 7 10
11) *0, disabled.
Dec 19 11:48:43 cluster kernel: ACPI: PCI Interrupt Link [LNKF] (IRQs *5 7
10 11), disabled.
Dec 19 11:48:43 cluster kernel: ACPI: PCI Interrupt Link [LNKG] (IRQs 5 7
*10 11), disabled.
Dec 19 11:48:43 cluster kernel: ACPI: PCI Interrupt Link [LNKH] (IRQs 5 *7
10 11), disabled.
Dec 19 11:48:43 cluster kernel: Linux Plug and Play Support v0.97 (c) Adam
Belay
Dec 19 11:48:43 cluster kernel: pnp: PnP ACPI init
Dec 19 11:48:43 cluster kernel: pnp: PnP ACPI: found 10 devices
Dec 19 11:48:43 cluster kernel: usbcore: registered new driver usbfs
Dec 19 11:48:43 cluster kernel: usbcore: registered new driver hub
Dec 19 11:48:45 cluster kernel: PCI: Using ACPI for IRQ routing
Dec 19 11:48:47 cluster kernel: PCI: If a device doesn''t work, try
"pci=routeirq". If it helps, post a report
Dec 19 11:48:48 cluster kernel: NetLabel: Initializing
Dec 19 11:48:49 cluster kernel: NetLabel: domain hash size = 128
Dec 19 11:48:53 cluster kernel: NetLabel: protocols = UNLABELED CIPSOv4
Dec 19 11:48:54 cluster kernel: NetLabel: unlabeled traffic allowed by
default
Dec 19 11:48:55 cluster kernel: hpet0: at MMIO 0xfed00000 (virtual
0xffffffffff5fe000), IRQs 2, 8, 0
Dec 19 11:48:56 cluster kernel: hpet0: 3 64-bit timers, 14318180 Hz
Dec 19 11:48:58 cluster kernel: ACPI: DMAR not present
Dec 19 11:48:58 cluster kernel: PCI-GART: No AMD northbridge found.
Dec 19 11:48:59 cluster kernel: pnp: 00:01: ioport range 0x408-0x40f has
been reserved
Dec 19 11:49:01 cluster kernel: pnp: 00:01: ioport range 0x4d0-0x4d1 has
been reserved
Dec 19 11:49:02 cluster kernel: pnp: 00:01: ioport range 0x700-0x71f has
been reserved
Dec 19 11:49:03 cluster kernel: PCI: Bridge: 0000:0a:00.0
Dec 19 11:49:06 cluster kernel: IO window: disabled.
Dec 19 11:49:08 cluster kernel: MEM window: disabled.
Dec 19 11:49:09 cluster kernel: PREFETCH window: disabled.
Dec 19 11:49:09 cluster kernel: PCI: Bridge: 0000:0a:01.0
Dec 19 11:49:10 cluster kernel: IO window: disabled.
Dec 19 11:49:12 cluster kernel: MEM window: disabled.
Dec 19 11:49:14 cluster kernel: PREFETCH window: disabled.
Dec 19 11:49:15 cluster kernel: PCI: Bridge: 0000:0a:02.0
Dec 19 11:49:17 cluster kernel: IO window: disabled.
Dec 19 11:49:18 cluster kernel: MEM window: disabled.
Dec 19 11:49:21 cluster kernel: PREFETCH window: disabled.
Dec 19 11:49:22 cluster kernel: PCI: Bridge: 0000:09:00.0
Dec 19 11:49:24 cluster kernel: IO window: disabled.
Dec 19 11:49:26 cluster kernel: MEM window: disabled.
Dec 19 11:49:28 cluster kernel: PREFETCH window: disabled.
Dec 19 11:49:31 cluster kernel: PCI: Bridge: 0000:09:00.3
Dec 19 11:49:33 cluster kernel: IO window: disabled.
Dec 19 11:49:34 cluster kernel: MEM window: disabled.
Dec 19 11:49:36 cluster kernel: PREFETCH window: disabled.
Dec 19 11:49:39 cluster kernel: PCI: Bridge: 0000:00:02.0
Dec 19 11:49:40 cluster kernel: IO window: disabled.
Dec 19 11:49:42 cluster kernel: MEM window: fdc00000-fdcfffff
Dec 19 11:49:44 cluster kernel: PREFETCH window: disabled.
Dec 19 11:49:45 cluster kernel: PCI: Bridge: 0000:00:03.0
Dec 19 11:49:48 cluster kernel: IO window: 4000-4fff
Dec 19 11:49:49 cluster kernel: MEM window: fda00000-fdbfffff
Dec 19 11:49:51 cluster kernel: PREFETCH window
0x00000000c2000000-0x00000000c20fffff
Dec 19 11:49:54 cluster kernel: PCI: Bridge: 0000:12:01.0
Dec 19 11:49:55 cluster kernel: IO window: 5000-5fff
Dec 19 11:49:56 cluster kernel: MEM window: fdd00000-fddfffff
Dec 19 11:49:57 cluster kernel: PREFETCH window: disabled.
Dec 19 11:49:58 cluster kernel: PCI: Bridge: 0000:12:02.0
Dec 19 11:50:00 cluster kernel: IO window: 6000-6fff
Dec 19 11:50:01 cluster kernel: MEM window: fde00000-fdefffff
Dec 19 11:50:03 cluster kernel: PREFETCH window: disabled.
Dec 19 11:50:05 cluster kernel: PCI: Bridge: 0000:11:00.0
Dec 19 11:50:07 cluster kernel: IO window: 5000-6fff
Dec 19 11:50:09 cluster kernel: MEM window: fdd00000-fdefffff
Dec 19 11:50:12 cluster kernel: PREFETCH window: disabled.
Dec 19 11:50:14 cluster kernel: PCI: Bridge: 0000:00:04.0
Dec 19 11:50:16 cluster kernel: IO window: 5000-6fff
Dec 19 11:50:19 cluster kernel: MEM window: fdd00000-fdefffff
Dec 19 11:50:20 cluster kernel: PREFETCH window: disabled.
Dec 19 11:50:22 cluster kernel: PCI: Bridge: 0000:00:05.0
Dec 19 11:50:24 cluster kernel: IO window: disabled.
Dec 19 11:50:25 cluster kernel: MEM window: disabled.
Dec 19 11:50:27 cluster kernel: PREFETCH window: disabled.
Dec 19 11:50:28 cluster kernel: PCI: Bridge: 0000:00:06.0
Dec 19 11:50:29 cluster kernel: IO window: disabled.
Dec 19 11:50:32 cluster kernel: MEM window: fdf00000-fdffffff
Dec 19 11:50:34 cluster kernel: PREFETCH window
0x00000000d0000000-0x00000000dfffffff
Dec 19 11:50:37 cluster kernel: PCI: Bridge: 0000:00:07.0
Dec 19 11:50:39 cluster kernel: IO window: disabled.
Dec 19 11:50:40 cluster kernel: MEM window: disabled.
Dec 19 11:50:41 cluster kernel: PREFETCH window: disabled.
Dec 19 11:50:43 cluster kernel: PCI: Bridge: 0000:02:00.0
Dec 19 11:50:45 cluster kernel: IO window: disabled.
Dec 19 11:50:46 cluster kernel: MEM window: f8000000-f9ffffff
Dec 19 11:50:48 cluster kernel: PREFETCH window
0x00000000c2100000-0x00000000c21fffff
Dec 19 11:50:49 cluster kernel: PCI: Bridge: 0000:00:1c.0
Dec 19 11:50:51 cluster kernel: IO window: disabled.
Dec 19 11:50:52 cluster kernel: MEM window: f8000000-f9ffffff
Dec 19 11:50:54 cluster kernel: PREFETCH window
0x00000000c2100000-0x00000000c21fffff
Dec 19 11:50:56 cluster kernel: PCI: Bridge: 0000:04:00.0
Dec 19 11:50:57 cluster kernel: IO window: disabled.
Dec 19 11:50:59 cluster kernel: MEM window: fa000000-fbffffff
Dec 19 11:51:00 cluster kernel: PREFETCH window
0x00000000c2200000-0x00000000c22fffff
Dec 19 11:51:01 cluster kernel: PCI: Bridge: 0000:00:1c.1
Dec 19 11:51:03 cluster kernel: IO window: disabled.
Dec 19 11:51:05 cluster kernel: MEM window: fa000000-fbffffff
Dec 19 11:51:06 cluster kernel: PREFETCH window
0x00000000c2200000-0x00000000c22fffff
Dec 19 11:51:07 cluster kernel: PCI: Bridge: 0000:00:1e.0
Dec 19 11:51:09 cluster kernel: IO window: 2000-3fff
Dec 19 11:51:12 cluster kernel: MEM window: f7e00000-f7ffffff
Dec 19 11:51:14 cluster kernel: PREFETCH window
0x00000000c8000000-0x00000000cfffffff
Dec 19 11:51:17 cluster kernel: GSI 16 sharing vector 0xA9 and IRQ 16
Dec 19 11:51:17 cluster kernel: ACPI: PCI Interrupt 0000:0a:00.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:51:17 cluster kernel: GSI 17 sharing vector 0xB1 and IRQ 17
Dec 19 11:51:17 cluster kernel: ACPI: PCI Interrupt 0000:0a:01.0[A] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:51:17 cluster kernel: GSI 18 sharing vector 0xB9 and IRQ 18
Dec 19 11:51:17 cluster kdump: saved a vmcore to /var/crash/2010-12-19-11:48
Dec 19 11:51:17 cluster kernel: ACPI: PCI Interrupt 0000:0a:02.0[A] -> GSI
18 (level, low) -> IRQ 185
Dec 19 11:51:17 cluster shutdown[6007]: shutting down for system reboot
Dec 19 11:51:17 cluster kernel: ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:51:17 cluster kernel: ACPI: PCI Interrupt 0000:00:1c.1[B] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:51:17 cluster kernel: NET: Registered protocol family 2
Dec 19 11:51:17 cluster kernel: IP route cache hash table entries: 1024
(order: 1, 8192 bytes)
Dec 19 11:51:17 cluster kernel: TCP established hash table entries: 4096
(order: 4, 65536 bytes)
Dec 19 11:51:17 cluster kernel: TCP bind hash table entries: 2048 (order: 3,
32768 bytes)
Dec 19 11:51:17 cluster kernel: TCP: Hash tables configured (established
4096 bind 2048)
Dec 19 11:51:17 cluster kernel: TCP reno registered
Dec 19 11:51:17 cluster kernel: audit: initializing netlink socket
(disabled)
Dec 19 11:51:17 cluster kernel: type=2000 audit(1292759089.234:1):
initialized
Dec 19 11:51:17 cluster kernel: Total HugeTLB memory allocated, 0
Dec 19 11:51:17 cluster kernel: VFS: Disk quotas dquot_6.5.1
Dec 19 11:51:17 cluster kernel: Dquot-cache hash table entries: 512 (order
0, 4096 bytes)
Dec 19 11:51:17 cluster kernel: Initializing Cryptographic API
Dec 19 11:51:17 cluster kernel: alg: No test for crc32c (crc32c-generic)
Dec 19 11:51:17 cluster kernel: ksign: Installing public key data
Dec 19 11:51:17 cluster kernel: Loading keyring
Dec 19 11:51:17 cluster kernel: io scheduler noop registered
Dec 19 11:51:17 cluster kernel: io scheduler anticipatory registered
Dec 19 11:51:17 cluster kernel: io scheduler deadline registered (default)
Dec 19 11:51:17 cluster kernel: io scheduler cfq registered
Dec 19 11:51:18 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:51:18 cluster last message repeated 5 times
Dec 19 11:51:18 cluster kernel: ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:51:18 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:51:18 cluster kernel: ACPI: PCI Interrupt 0000:00:1c.1[B] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:51:18 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:51:18 cluster kernel: ACPI: PCI Interrupt 0000:0a:00.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:51:18 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:51:18 cluster kernel: ACPI: PCI Interrupt 0000:0a:01.0[A] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:51:18 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:51:18 cluster kernel: ACPI: PCI Interrupt 0000:0a:02.0[A] -> GSI
18 (level, low) -> IRQ 185
Dec 19 11:51:18 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:51:18 cluster kernel: pci_hotplug: PCI Hot Plug PCI Core version:
0.5
Dec 19 11:51:18 cluster kernel: ACPI (exconfig-0456): Dynamic SSDT Load -
OemId [HP ] OemTableId [ CPU0CST] [20060707]
Dec 19 11:51:18 cluster kernel: ACPI (exconfig-0456): Dynamic SSDT Load -
OemId [HP ] OemTableId [ CPU1CST] [20060707]
Dec 19 11:51:18 cluster kernel: ACPI (exconfig-0456): Dynamic SSDT Load -
OemId [HP ] OemTableId [ CPU6CST] [20060707]
Dec 19 11:51:18 cluster kernel: ACPI (exconfig-0456): Dynamic SSDT Load -
OemId [HP ] OemTableId [ CPU7CST] [20060707]
Dec 19 11:51:18 cluster kernel: ACPI: Thermal Zone [THM0] (8 C)
Dec 19 11:51:18 cluster kernel: Real Time Clock Driver v1.12ac
Dec 19 11:51:18 cluster kernel: Non-volatile memory driver v1.2
Dec 19 11:51:18 cluster kernel: Linux agpgart interface v0.101 (c) Dave
Jones
Dec 19 11:51:18 cluster kernel: Serial: 8250/16550 driver $Revision: 1.90 $
4 ports, IRQ sharing enabled
Dec 19 11:51:18 cluster kernel: serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is
a 16550A
Dec 19 11:51:18 cluster kernel: serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is
a 16550A
Dec 19 11:51:18 cluster kernel: 00:09: ttyS0 at I/O 0x3f8 (irq = 4) is a
16550A
Dec 19 11:51:18 cluster kernel: brd: module loaded
Dec 19 11:51:18 cluster kernel: Uniform Multi-Platform E-IDE driver
Revision: 7.00alpha2
Dec 19 11:51:18 cluster kernel: ide: Assuming 33MHz system bus speed for PIO
modes; override with idebus=xx
Dec 19 11:51:18 cluster kernel: ESB2: IDE controller at PCI slot
0000:00:1f.1
Dec 19 11:51:18 cluster kernel: ACPI: PCI Interrupt 0000:00:1f.1[A] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:51:18 cluster kernel: ESB2: chipset revision 9
Dec 19 11:51:18 cluster kernel: ESB2: not 100% native mode: will probe irqs
later
Dec 19 11:51:18 cluster kernel: ide0: BM-DMA at 0x0500-0x0507, BIOS
settings: hda:pio, hdb:pio
Dec 19 11:51:18 cluster kernel: ide1: BM-DMA at 0x0508-0x050f, BIOS
settings: hdc:pio, hdd:pio
Dec 19 11:51:18 cluster kernel: ide-floppy driver 0.99.newide
Dec 19 11:51:18 cluster kernel: usbcore: registered new driver hiddev
Dec 19 11:51:18 cluster kernel: usbcore: registered new driver usbhid
Dec 19 11:51:18 cluster kernel: drivers/usb/input/hid-core.c: v2.6:USB HID
core driver
Dec 19 11:51:18 cluster kernel: PNP: PS/2 Controller
[PNP0303:KBD,PNP0f0e:PS2M] at 0x60,0x64 irq 1,12
Dec 19 11:51:18 cluster kernel: serio: i8042 KBD port at 0x60,0x64 irq 1
Dec 19 11:51:18 cluster kernel: serio: i8042 AUX port at 0x60,0x64 irq 12
Dec 19 11:51:18 cluster kernel: mice: PS/2 mouse device common for all mice
Dec 19 11:51:18 cluster kernel: md: md driver 0.90.3 MAX_MD_DEVS=256,
MD_SB_DISKS=27
Dec 19 11:51:18 cluster kernel: md: bitmap version 4.39
Dec 19 11:51:18 cluster kernel: TCP bic registered
Dec 19 11:51:18 cluster kernel: Initializing IPsec netlink socket
Dec 19 11:51:18 cluster kernel: NET: Registered protocol family 1
Dec 19 11:51:18 cluster kernel: NET: Registered protocol family 17
Dec 19 11:51:18 cluster kernel: ACPI: (supports S0 S4 S5)
Dec 19 11:51:18 cluster kernel: Initalizing network drop monitor service
Dec 19 11:51:18 cluster kernel: Freeing unused kernel memory: 208k freed
Dec 19 11:51:19 cluster kernel: Write protecting the kernel read-only data:
498k
Dec 19 11:51:19 cluster kernel: SCSI subsystem initialized
Dec 19 11:51:19 cluster kernel: HP CISS Driver (v 3.6.20-RH3)
Dec 19 11:51:19 cluster kernel: cciss: using PCI PM to reset controller
Dec 19 11:51:19 cluster kernel: cciss: resetting MSI-X
Dec 19 11:51:19 cluster kernel: cciss: controller message 03:00 succeeded
Dec 19 11:51:19 cluster kernel: ACPI: PCI Interrupt 0000:06:00.0[A] -> GSI
18 (level, low) -> IRQ 185
Dec 19 11:51:19 cluster kernel: cciss0: <0x3230> at PCI 0000:06:00.0 IRQ
114
using DAC
Dec 19 11:51:19 cluster kernel: blocks= 143305920 block_size= 512
Dec 19 11:51:19 cluster kernel: heads= 255, sectors= 32, cylinders17562
Dec 19 11:51:19 cluster kernel:
Dec 19 11:51:19 cluster kernel: blocks= 143305920 block_size= 512
Dec 19 11:51:19 cluster kernel: heads= 255, sectors= 32, cylinders17562
Dec 19 11:51:19 cluster kernel:
Dec 19 11:51:19 cluster kernel: cciss/c0d0: p1 p2 p3
Dec 19 11:51:19 cluster kernel: cciss: using PCI PM to reset controller
Dec 19 11:51:19 cluster kernel: cciss: controller message 03:00 succeeded
Dec 19 11:51:19 cluster kernel: GSI 19 sharing vector 0x82 and IRQ 19
Dec 19 11:51:19 cluster kernel: ACPI: PCI Interrupt 0000:13:04.0[A] -> GSI
25 (level, low) -> IRQ 130
Dec 19 11:51:19 cluster kernel: cciss1: <0x46> at PCI 0000:13:04.0 IRQ 130
using DAC
Dec 19 11:51:19 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:19 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:19 cluster kernel:
Dec 19 11:51:19 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:19 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:19 cluster kernel:
Dec 19 11:51:19 cluster kernel: cciss/c1d0: p1 p2 p3 p4
Dec 19 11:51:19 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:19 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:19 cluster kernel:
Dec 19 11:51:19 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:19 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:19 cluster kernel:
Dec 19 11:51:19 cluster kernel: cciss/c1d1: unknown partition table
Dec 19 11:51:19 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:19 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:19 cluster kernel:
Dec 19 11:51:19 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:19 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:19 cluster kernel:
Dec 19 11:51:19 cluster kernel: cciss/c1d2: unknown partition table
Dec 19 11:51:19 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:19 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:19 cluster kernel:
Dec 19 11:51:19 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:19 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:19 cluster kernel:
Dec 19 11:51:19 cluster kernel: cciss/c1d3: unknown partition table
Dec 19 11:51:19 cluster kernel: cciss: using PCI PM to reset controller
Dec 19 11:51:19 cluster kernel: cciss: controller message 03:00 succeeded
Dec 19 11:51:19 cluster kernel: GSI 20 sharing vector 0x8A and IRQ 20
Dec 19 11:51:20 cluster kernel: ACPI: PCI Interrupt 0000:13:05.0[A] -> GSI
24 (level, low) -> IRQ 138
Dec 19 11:51:20 cluster kernel: cciss2: <0x46> at PCI 0000:13:05.0 IRQ 138
using DAC
Dec 19 11:51:20 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: cciss/c2d0: unknown partition table
Dec 19 11:51:20 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: cciss/c2d1: unknown partition table
Dec 19 11:51:20 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: cciss/c2d2: unknown partition table
Dec 19 11:51:20 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: cciss/c2d3: unknown partition table
Dec 19 11:51:20 cluster kernel: cciss: using PCI PM to reset controller
Dec 19 11:51:20 cluster kernel: cciss: controller message 03:00 succeeded
Dec 19 11:51:20 cluster kernel: GSI 21 sharing vector 0x92 and IRQ 21
Dec 19 11:51:20 cluster kernel: ACPI: PCI Interrupt 0000:14:04.0[A] -> GSI
31 (level, low) -> IRQ 146
Dec 19 11:51:20 cluster kernel: cciss3: <0x46> at PCI 0000:14:04.0 IRQ 146
using DAC
Dec 19 11:51:20 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: cciss/c3d0: unknown partition table
Dec 19 11:51:20 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:20 cluster kernel:
Dec 19 11:51:20 cluster kernel: cciss/c3d1: unknown partition table
Dec 19 11:51:20 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:20 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: cciss/c3d2: unknown partition table
Dec 19 11:51:21 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: cciss/c3d3: unknown partition table
Dec 19 11:51:21 cluster kernel: cciss: using PCI PM to reset controller
Dec 19 11:51:21 cluster kernel: cciss: controller message 03:00 succeeded
Dec 19 11:51:21 cluster kernel: GSI 22 sharing vector 0x9A and IRQ 22
Dec 19 11:51:21 cluster kernel: ACPI: PCI Interrupt 0000:14:05.0[A] -> GSI
30 (level, low) -> IRQ 154
Dec 19 11:51:21 cluster kernel: cciss4: <0x46> at PCI 0000:14:05.0 IRQ 154
using DAC
Dec 19 11:51:21 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: cciss/c4d0: unknown partition table
Dec 19 11:51:21 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: cciss/c4d1: unknown partition table
Dec 19 11:51:21 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: cciss/c4d2: unknown partition table
Dec 19 11:51:21 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: cciss/c4d3: unknown partition table
Dec 19 11:51:21 cluster kernel: blocks= 143305920 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders17562
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: cciss/c0d0: p1 p2 p3
Dec 19 11:51:21 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:21 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:21 cluster kernel:
Dec 19 11:51:21 cluster kernel: cciss/c1d0: p1 p2 p3 p4
Dec 19 11:51:21 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c1d1: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c1d2: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c1d3: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c2d0: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c2d1: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c2d2: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c2d3: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c3d0: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c3d1: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c3d2: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c3d3: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c4d0: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c4d1: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:51:22 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:51:22 cluster kernel:
Dec 19 11:51:22 cluster kernel: cciss/c4d2: unknown partition table
Dec 19 11:51:22 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:51:23 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:51:23 cluster kernel:
Dec 19 11:51:23 cluster kernel: cciss/c4d3: unknown partition table
Dec 19 11:51:23 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 11:51:23 cluster kernel: EXT3 FS on cciss/c0d0p3, internal journal
Dec 19 11:51:23 cluster kernel: EXT3-fs: mounted filesystem with ordered
data mode.
Dec 19 11:51:23 cluster kernel: SELinux: Disabled at runtime.
Dec 19 11:51:23 cluster kernel: type=1404 audit(1292739467.057:2): selinux=0
auid=4294967295 ses=4294967295
Dec 19 11:51:23 cluster kernel: GSI 23 sharing vector 0xA2 and IRQ 23
Dec 19 11:51:23 cluster kernel: ACPI: PCI Interrupt 0000:01:04.2[B] -> GSI
22 (level, low) -> IRQ 162
Dec 19 11:51:23 cluster kernel: USB Universal Host Controller Interface
driver v3.0
Dec 19 11:51:23 cluster kernel: ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.0: UHCI Host Controller
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.0: new USB bus
registered, assigned bus number 1
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.0: irq 169, io base
0x00001000
Dec 19 11:51:23 cluster kernel: usb usb1: configuration #1 chosen from 1
choice
Dec 19 11:51:23 cluster kernel: hub 1-0:1.0: USB hub found
Dec 19 11:51:23 cluster kernel: hub 1-0:1.0: 2 ports detected
Dec 19 11:51:23 cluster kernel: intel_rng: FWH not detected
Dec 19 11:51:23 cluster kernel: ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.1: UHCI Host Controller
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.1: new USB bus
registered, assigned bus number 2
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.1: irq 177, io base
0x00001020
Dec 19 11:51:23 cluster kernel: usb usb2: configuration #1 chosen from 1
choice
Dec 19 11:51:23 cluster kernel: hub 2-0:1.0: USB hub found
Dec 19 11:51:23 cluster kernel: hub 2-0:1.0: 2 ports detected
Dec 19 11:51:23 cluster kernel: ACPI: PCI Interrupt 0000:00:1d.2[C] -> GSI
18 (level, low) -> IRQ 185
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.2: UHCI Host Controller
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.2: new USB bus
registered, assigned bus number 3
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.2: irq 185, io base
0x00001040
Dec 19 11:51:23 cluster kernel: usb usb3: configuration #1 chosen from 1
choice
Dec 19 11:51:23 cluster kernel: hub 3-0:1.0: USB hub found
Dec 19 11:51:23 cluster kernel: hub 3-0:1.0: 2 ports detected
Dec 19 11:51:23 cluster kernel: GSI 24 sharing vector 0xAA and IRQ 24
Dec 19 11:51:23 cluster kernel: ACPI: PCI Interrupt 0000:00:1d.3[D] -> GSI
19 (level, low) -> IRQ 170
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.3: UHCI Host Controller
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.3: new USB bus
registered, assigned bus number 4
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:00:1d.3: irq 170, io base
0x00001060
Dec 19 11:51:23 cluster kernel: usb usb4: configuration #1 chosen from 1
choice
Dec 19 11:51:23 cluster kernel: hub 4-0:1.0: USB hub found
Dec 19 11:51:23 cluster kernel: hub 4-0:1.0: 2 ports detected
Dec 19 11:51:23 cluster kernel: EDAC MC: Ver: 2.0.1 Apr 9 2010
Dec 19 11:51:23 cluster kernel: input: PC Speaker as /class/input/input0
Dec 19 11:51:23 cluster kernel: ACPI: PCI Interrupt 0000:01:04.4[B] -> GSI
22 (level, low) -> IRQ 162
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:01:04.4: UHCI Host Controller
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:01:04.4: new USB bus
registered, assigned bus number 5
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:01:04.4: port count
misdetected? forcing to 2 ports
Dec 19 11:51:23 cluster kernel: uhci_hcd 0000:01:04.4: irq 162, io base
0x00003800
Dec 19 11:51:23 cluster kernel: usb usb5: configuration #1 chosen from 1
choice
Dec 19 11:51:23 cluster kernel: hub 5-0:1.0: USB hub found
Dec 19 11:51:23 cluster kernel: hub 5-0:1.0: 2 ports detected
Dec 19 11:51:23 cluster kernel: shpchp: Standard Hot Plug PCI Controller
Driver version: 0.4
Dec 19 11:51:23 cluster kernel: ib_mthca: Mellanox InfiniBand HCA driver
v1.0 (April 4, 2008)
Dec 19 11:51:23 cluster kernel: ib_mthca: Initializing 0000:17:00.0
Dec 19 11:51:24 cluster kernel: ACPI: PCI Interrupt 0000:17:00.0[A] -> GSI
19 (level, low) -> IRQ 170
Dec 19 11:51:24 cluster kernel: Floppy drive(s): fd0 is 1.44M
Dec 19 11:51:24 cluster kernel: usb 5-1: new full speed USB device using
uhci_hcd and address 2
Dec 19 11:51:24 cluster kernel: usb 5-1: configuration #1 chosen from 1
choice
Dec 19 11:51:24 cluster kernel: input: HP Virtual Keyboard as
/class/input/input1
Dec 19 11:51:24 cluster kernel: input: USB HID v1.01 Keyboard [HP Virtual
Keyboard] on usb-0000:01:04.4-1
Dec 19 11:51:24 cluster kernel: input: HP Virtual Keyboard as
/class/input/input2
Dec 19 11:51:24 cluster kernel: input: USB HID v1.01 Mouse [HP Virtual
Keyboard] on usb-0000:01:04.4-1
Dec 19 11:51:24 cluster kernel: ACPI: PCI Interrupt 0000:00:1d.7[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:51:24 cluster kernel: ehci_hcd 0000:00:1d.7: EHCI Host Controller
Dec 19 11:51:24 cluster kernel: ehci_hcd 0000:00:1d.7: new USB bus
registered, assigned bus number 6
Dec 19 11:51:24 cluster kernel: ehci_hcd 0000:00:1d.7: debug port 1
Dec 19 11:51:24 cluster kernel: ehci_hcd 0000:00:1d.7: irq 169, io mem
0xf7df0000
Dec 19 11:51:24 cluster kernel: ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI
1.00, driver 10 Dec 2004
Dec 19 11:51:24 cluster kernel: usb usb6: configuration #1 chosen from 1
choice
Dec 19 11:51:24 cluster kernel: hub 6-0:1.0: USB hub found
Dec 19 11:51:24 cluster kernel: hub 6-0:1.0: 8 ports detected
Dec 19 11:51:24 cluster kernel: Broadcom NetXtreme II Gigabit Ethernet
Driver bnx2 v1.9.3 (March 17, 2009)
Dec 19 11:51:24 cluster kernel: ACPI: PCI Interrupt 0000:03:00.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:51:24 cluster kernel: EDAC MC0: Giving out device to i5000_edac.c
I5000: DEV 0000:00:10.0
Dec 19 11:51:24 cluster kernel: eth0: Broadcom NetXtreme II BCM5708
1000Base-T (B2) PCI-X 64-bit 133MHz found at mem f8000000, IRQ 169, node
addr 001f29e8baf8
Dec 19 11:51:24 cluster kernel: ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:51:24 cluster kernel: eth1: Broadcom NetXtreme II BCM5708
1000Base-T (B2) PCI-X 64-bit 133MHz found at mem fa000000, IRQ 177, node
addr 001f29e8baf6
Dec 19 11:51:24 cluster kernel: floppy0: no floppy controllers found
Dec 19 11:51:24 cluster kernel: Floppy drive(s): fd0 is 1.44M
Dec 19 11:51:24 cluster kernel: floppy0: no floppy controllers found
Dec 19 11:51:24 cluster kernel: lp: driver loaded but no devices found
Dec 19 11:51:24 cluster kernel: ACPI: Power Button (FF) [PWRF]
Dec 19 11:51:24 cluster kernel: md: Autodetecting RAID arrays.
Dec 19 11:51:24 cluster kernel: md: autorun ...
Dec 19 11:51:24 cluster kernel: md: ... autorun DONE.
Dec 19 11:51:24 cluster kernel: device-mapper: uevent: version 1.0.3
Dec 19 11:51:24 cluster kernel: device-mapper: ioctl: 4.11.5-ioctl
(2007-12-12) initialised: dm-devel at redhat.com
Dec 19 11:51:24 cluster kernel: device-mapper: multipath: version 1.0.5
loaded
Dec 19 11:51:24 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 11:51:24 cluster kernel: EXT3 FS on cciss/c0d0p1, internal journal
Dec 19 11:51:24 cluster kernel: EXT3-fs: recovery complete.
Dec 19 11:51:24 cluster kernel: EXT3-fs: mounted filesystem with ordered
data mode.
Dec 19 11:51:24 cluster kernel: Adding 16779884k swap on /dev/cciss/c0d0p2.
Priority:-1 extents:1 across:16779884k
Dec 19 11:51:24 cluster kernel: IA-32 Microcode Update Driver: v1.14a <
tigran at veritas.com>
Dec 19 11:51:24 cluster kernel: 802.1Q VLAN Support v1.8 Ben Greear <
greearb at candelatech.com>
Dec 19 11:51:24 cluster kernel: All bugs added by David S. Miller <
davem at redhat.com>
Dec 19 11:51:24 cluster kernel: NET: Registered protocol family 10
Dec 19 11:51:24 cluster kernel: lo: Disabled Privacy Extensions
Dec 19 11:51:24 cluster kernel: IPv6 over IPv4 tunneling driver
Dec 19 11:51:24 cluster kernel: ADDRCONF(NETDEV_UP): ib0: link is not ready
Dec 19 11:51:24 cluster kernel: ib0: enabling connected mode will cause
multicast packet drops
Dec 19 11:51:24 cluster kernel: ib0: mtu > 2044 will cause multicast packet
drops.
Dec 19 11:51:24 cluster kernel: ib0: mtu > 2044 will cause multicast packet
drops.
Dec 19 11:51:24 cluster kernel: ib1: enabling connected mode will cause
multicast packet drops
Dec 19 11:51:24 cluster kernel: ib1: mtu > 2044 will cause multicast packet
drops.
Dec 19 11:51:24 cluster kernel: ib1: mtu > 2044 will cause multicast packet
drops.
Dec 19 11:51:25 cluster kernel: ADDRCONF(NETDEV_CHANGE): ib0: link becomes
ready
Dec 19 11:51:25 cluster kernel: NET: Registered protocol family 27
Dec 19 11:51:25 cluster kernel: QLGC_VNIC: Initializing QLogic Corp. Virtual
NIC (VNIC) driver version 1.3.0.0.4
Dec 19 11:51:25 cluster kernel: Loading iSCSI transport class v2.0-871.
Dec 19 11:51:25 cluster kernel: Broadcom NetXtreme II CNIC Driver cnic
v2.0.1 (Oct 01, 2009)
Dec 19 11:51:25 cluster kernel: cnic: Added CNIC device: eth0
Dec 19 11:51:25 cluster kernel: cnic: Added CNIC device: eth1
Dec 19 11:51:25 cluster kernel: Broadcom NetXtreme II iSCSI Driver bnx2i
v2.0.1e (June 22, 2009)
Dec 19 11:51:25 cluster kernel: iscsi: registered transport (bnx2i)
Dec 19 11:51:25 cluster kernel: scsi0 : Broadcom Offload iSCSI Initiator
Dec 19 11:51:25 cluster kernel: scsi1 : Broadcom Offload iSCSI Initiator
Dec 19 11:51:25 cluster kernel: process `sysctl'' is using deprecated
sysctl
(syscall) net.ipv6.neigh.ib0.base_reachable_time; Use
net.ipv6.neigh.ib0.base_reachable_time_ms instead.
Dec 19 11:51:25 cluster kernel: bnx2: eth0: using MSI
Dec 19 11:51:25 cluster kernel: ADDRCONF(NETDEV_UP): eth0: link is not ready
Dec 19 11:51:25 cluster kernel: bnx2i: iSCSI not supported, dev=eth0
Dec 19 11:51:25 cluster kernel: bnx2: eth0 NIC Copper Link is Up, 100 Mbps
full duplex
Dec 19 11:51:25 cluster kernel: ADDRCONF(NETDEV_CHANGE): eth0: link becomes
ready
Dec 19 11:51:25 cluster kernel: bnx2i: iSCSI not supported, dev=eth0
Dec 19 11:51:25 cluster kernel: bnx2: eth1: using MSI
Dec 19 11:51:25 cluster kernel: ADDRCONF(NETDEV_UP): eth1: link is not ready
Dec 19 11:51:25 cluster kernel: bnx2i: iSCSI not supported, dev=eth1
Dec 19 11:51:25 cluster kernel: bnx2i: iSCSI not supported, dev=eth1
Dec 19 11:51:25 cluster kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps
full duplex
Dec 19 11:51:25 cluster kernel: ADDRCONF(NETDEV_CHANGE): eth1: link becomes
ready
Dec 19 11:51:25 cluster init: Switching to runlevel: 6
Dec 19 11:51:27 cluster rpc.statd[5886]: Caught signal 15, un-registering
and exiting.
Dec 19 11:51:27 cluster multipathd: --------shut down-------
Dec 19 11:51:27 cluster auditd[5740]: The audit daemon is exiting.
Dec 19 11:51:27 cluster kernel: audit(1292739687.831:17): audit_pid=0
old=5740 by auid=4294967295
Dec 19 11:51:27 cluster kernel: Kernel logging (proc) stopped.
Dec 19 11:51:27 cluster kernel: Kernel log daemon terminating.
Dec 19 11:51:29 cluster exiting on signal 15
Dec 19 11:55:48 cluster syslogd 1.4.1: restart.
Dec 19 11:55:48 cluster kernel: klogd 1.4.1, log source = /proc/kmsg
started.
Dec 19 11:55:48 cluster kernel: Linux version
2.6.18-164.11.1.el5_lustre.1.8.3 (lbuild at x86-build-1) (gcc version 4.1.2
20080704 (Red Hat 4.1.2-46)) #1 SMP Fri Apr 9 18:00:39 MDT 2010
Dec 19 11:55:48 cluster kernel: Command line: ro root=LABEL=/ rhgb quiet
crashkernel=128M at 16M
Dec 19 11:55:48 cluster kernel: BIOS-provided physical RAM map:
Dec 19 11:55:48 cluster kernel: BIOS-e820: 0000000000010000 -
000000000009ec00 (usable)
Dec 19 11:55:48 cluster kernel: BIOS-e820: 000000000009ec00 -
00000000000a0000 (reserved)
Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000000f0000 -
0000000000100000 (reserved)
Dec 19 11:55:48 cluster kernel: BIOS-e820: 0000000000100000 -
00000000bfe54000 (usable)
Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000bfe54000 -
00000000bfe5c000 (ACPI data)
Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000bfe5c000 -
00000000bfe5d000 (usable)
Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000bfe5d000 -
00000000c0000000 (reserved)
Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000e0000000 -
00000000f0000000 (reserved)
Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000fec00000 -
00000000fed00000 (reserved)
Dec 19 11:55:49 cluster kernel: BIOS-e820: 00000000fee00000 -
00000000fee10000 (reserved)
Dec 19 11:55:49 cluster kernel: BIOS-e820: 00000000ffc00000 -
0000000100000000 (reserved)
Dec 19 11:55:49 cluster kernel: BIOS-e820: 0000000100000000 -
000000043ffff000 (usable)
Dec 19 11:55:49 cluster kernel: DMI 2.4 present.
Dec 19 11:55:49 cluster rpc.statd[3888]: Version 1.0.9 Starting
Dec 19 11:55:49 cluster rpc.statd[3888]: statd running as root. chown
/var/lib/nfs/statd/sm to choose different user
Dec 19 11:55:49 cluster kernel: No NUMA configuration found
Dec 19 11:55:49 cluster kernel: Faking a node at
0000000000000000-000000043ffff000
Dec 19 11:55:49 cluster kernel: Bootmem setup node 0
0000000000000000-000000043ffff000
Dec 19 11:55:49 cluster kernel: ACPI: PM-Timer IO Port: 0x908
Dec 19 11:55:49 cluster kernel: ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00]
enabled)
Dec 19 11:55:49 cluster kernel: Processor #0 6:15 APIC version 20
Dec 19 11:55:49 cluster kernel: ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04]
disabled)
Dec 19 11:55:49 cluster kernel: ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02]
disabled)
Dec 19 11:55:49 cluster kernel: ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06]
enabled)
Dec 19 11:55:49 cluster kernel: Processor #6 6:15 APIC version 20
Dec 19 11:55:49 cluster kernel: ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01]
enabled)
Dec 19 11:55:49 cluster kernel: Processor #1 6:15 APIC version 20
Dec 19 11:55:49 cluster kernel: ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05]
disabled)
Dec 19 11:55:49 cluster kernel: ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03]
disabled)
Dec 19 11:55:49 cluster kernel: ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07]
enabled)
Dec 19 11:55:49 cluster kernel: Processor #7 6:15 APIC version 20
Dec 19 11:55:49 cluster kernel: ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl
lint[0x1])
Dec 19 11:55:49 cluster kernel: ACPI: IOAPIC (id[0x08] address[0xfec00000]
gsi_base[0])
Dec 19 11:55:49 cluster kernel: IOAPIC[0]: apic_id 8, version 32, address
0xfec00000, GSI 0-23
Dec 19 11:55:49 cluster kernel: ACPI: IOAPIC (id[0x09] address[0xfec82000]
gsi_base[24])
Dec 19 11:55:49 cluster kernel: IOAPIC[1]: apic_id 9, version 32, address
0xfec82000, GSI 24-47
Dec 19 11:55:49 cluster kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0
global_irq 2 high edge)
Dec 19 11:55:49 cluster kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9
global_irq 9 high level)
Dec 19 11:55:49 cluster kernel: Setting APIC routing to physical flat
Dec 19 11:55:49 cluster kernel: ACPI: HPET id: 0x8086a201 base: 0xfed00000
Dec 19 11:55:49 cluster kernel: Using ACPI (MADT) for SMP configuration
information
Dec 19 11:55:49 cluster kernel: Nosave address range: 000000000009e000 -
000000000009f000
Dec 19 11:55:49 cluster kernel: Nosave address range: 000000000009f000 -
00000000000a0000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000000a0000 -
00000000000f0000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000000f0000 -
0000000000100000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000bfe54000 -
00000000bfe5c000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000bfe5d000 -
00000000c0000000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000c0000000 -
00000000e0000000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000e0000000 -
00000000f0000000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000f0000000 -
00000000fec00000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000fec00000 -
00000000fed00000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000fed00000 -
00000000fee00000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000fee00000 -
00000000fee10000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000fee10000 -
00000000ffc00000
Dec 19 11:55:49 cluster kernel: Nosave address range: 00000000ffc00000 -
0000000100000000
Dec 19 11:55:49 cluster kernel: Allocating PCI resources starting at
c2000000 (gap: c0000000:20000000)
Dec 19 11:55:49 cluster kernel: SMP: Allowing 8 CPUs, 4 hotplug CPUs
Dec 19 11:55:49 cluster kernel: Built 1 zonelists. Total pages: 4131531
Dec 19 11:55:50 cluster kernel: Kernel command line: ro root=LABEL=/ rhgb
quiet crashkernel=128M at 16M
Dec 19 11:55:50 cluster kernel: Initializing CPU#0
Dec 19 11:55:50 cluster kernel: PID hash table entries: 4096 (order: 12,
32768 bytes)
Dec 19 11:55:50 cluster kernel: Console: colour VGA+ 80x25
Dec 19 11:55:50 cluster kernel: Dentry cache hash table entries: 2097152
(order: 12, 16777216 bytes)
Dec 19 11:55:50 cluster kernel: Inode-cache hash table entries: 1048576
(order: 11, 8388608 bytes)
Dec 19 11:55:50 cluster kernel: Checking aperture...
Dec 19 11:55:50 cluster kernel: ACPI: DMAR not present
Dec 19 11:55:50 cluster kernel: PCI-DMA: Using software bounce buffering for
IO (SWIOTLB)
Dec 19 11:55:50 cluster kernel: Placing software IO TLB between 0xa85e000 -
0xe85e000
Dec 19 11:55:50 cluster kernel: Memory: 16301236k/17825788k available (2553k
kernel code, 473812k reserved, 1293k data, 208k init)
Dec 19 11:55:50 cluster kernel: Calibrating delay loop (skipped), value
calculated using timer frequency.. 5999.99 BogoMIPS (lpj=2999996)
Dec 19 11:55:50 cluster kernel: Security Framework v1.0.0 initialized
Dec 19 11:55:50 cluster kernel: SELinux: Initializing.
Dec 19 11:55:50 cluster kernel: selinux_register_security: Registering
secondary module capability
Dec 19 11:55:50 cluster kernel: Capability LSM initialized as secondary
Dec 19 11:55:50 cluster kernel: Mount-cache hash table entries: 256
Dec 19 11:55:50 cluster kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Dec 19 11:55:50 cluster kernel: CPU: L2 cache: 4096K
Dec 19 11:55:50 cluster kernel: using mwait in idle threads.
Dec 19 11:55:50 cluster kernel: CPU: Physical Processor ID: 0
Dec 19 11:55:50 cluster kernel: CPU: Processor Core ID: 0
Dec 19 11:55:50 cluster kernel: CPU0: Thermal monitoring enabled (TM2)
Dec 19 11:55:50 cluster kernel: SMP alternatives: switching to UP code
Dec 19 11:55:50 cluster kernel: ACPI: Core revision 20060707
Dec 19 11:55:50 cluster kernel: Using local APIC timer interrupts.
Dec 19 11:55:50 cluster kernel: Detected 20.833 MHz APIC timer.
Dec 19 11:55:50 cluster kernel: SMP alternatives: switching to SMP code
Dec 19 11:55:50 cluster kernel: Booting processor 1/4 APIC 0x6
Dec 19 11:55:50 cluster kernel: Initializing CPU#1
Dec 19 11:55:50 cluster kernel: Calibrating delay using timer specific
routine.. 6000.23 BogoMIPS (lpj=3000118)
Dec 19 11:55:50 cluster kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Dec 19 11:55:50 cluster kernel: CPU: L2 cache: 4096K
Dec 19 11:55:50 cluster kernel: CPU: Physical Processor ID: 3
Dec 19 11:55:50 cluster kernel: CPU: Processor Core ID: 0
Dec 19 11:55:50 cluster kernel: CPU1: Thermal monitoring enabled (TM2)
Dec 19 11:55:50 cluster kernel: Intel(R) Xeon(R) CPU 5160 @
3.00GHz stepping 0b
Dec 19 11:55:50 cluster kernel: SMP alternatives: switching to SMP code
Dec 19 11:55:50 cluster kernel: Booting processor 2/4 APIC 0x1
Dec 19 11:55:50 cluster kernel: Initializing CPU#2
Dec 19 11:55:50 cluster S14daninit: started
Dec 19 11:55:50 cluster kernel: Calibrating delay using timer specific
routine.. 6000.09 BogoMIPS (lpj=3000048)
Dec 19 11:55:50 cluster kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Dec 19 11:55:50 cluster kernel: CPU: L2 cache: 4096K
Dec 19 11:55:50 cluster kernel: CPU: Physical Processor ID: 0
Dec 19 11:55:50 cluster kernel: CPU: Processor Core ID: 1
Dec 19 11:55:50 cluster kernel: CPU2: Thermal monitoring enabled (TM2)
Dec 19 11:55:50 cluster kernel: Intel(R) Xeon(R) CPU 5160 @
3.00GHz stepping 0b
Dec 19 11:55:50 cluster kernel: SMP alternatives: switching to SMP code
Dec 19 11:55:50 cluster kernel: Booting processor 3/4 APIC 0x7
Dec 19 11:55:50 cluster kernel: Initializing CPU#3
Dec 19 11:55:50 cluster kernel: Calibrating delay using timer specific
routine.. 6000.04 BogoMIPS (lpj=3000023)
Dec 19 11:55:51 cluster kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Dec 19 11:55:51 cluster kernel: CPU: L2 cache: 4096K
Dec 19 11:55:51 cluster kernel: CPU: Physical Processor ID: 3
Dec 19 11:55:51 cluster kernel: CPU: Processor Core ID: 1
Dec 19 11:55:51 cluster kernel: CPU3: Thermal monitoring enabled (TM2)
Dec 19 11:55:51 cluster kernel: Intel(R) Xeon(R) CPU 5160 @
3.00GHz stepping 0b
Dec 19 11:55:51 cluster kernel: Brought up 4 CPUs
Dec 19 11:55:51 cluster kernel: testing NMI watchdog ... OK.
Dec 19 11:55:51 cluster kernel: time.c: Using 14.318180 MHz WALL HPET GTOD
HPET/TSC timer.
Dec 19 11:55:51 cluster kernel: time.c: Detected 2999.996 MHz processor.
Dec 19 11:55:51 cluster kernel: migration_cost=17,9208
Dec 19 11:55:51 cluster kernel: checking if image is initramfs... it is
Dec 19 11:55:51 cluster kernel: Freeing initrd memory: 2596k freed
Dec 19 11:55:51 cluster kernel: NET: Registered protocol family 16
Dec 19 11:55:51 cluster kdump: kexec: loaded kdump kernel
Dec 19 11:55:51 cluster kernel: ACPI: bus type pci registered
Dec 19 11:55:51 cluster kernel: PCI: Using MMCONFIG at e0000000
Dec 19 11:55:51 cluster kernel: ACPI: Interpreter enabled
Dec 19 11:55:51 cluster kernel: ACPI: Using IOAPIC for interrupt routing
Dec 19 11:55:51 cluster kernel: ACPI: No dock devices found.
Dec 19 11:55:51 cluster kernel: ACPI: PCI Root Bridge [PCI0] (0000:00)
Dec 19 11:55:51 cluster kernel: PCI: Ignoring BAR0-3 of IDE controller
0000:00:1f.1
Dec 19 11:55:51 cluster kernel: PCI: PXH quirk detected, disabling MSI for
SHPC device
Dec 19 11:55:51 cluster kdump: started up
Dec 19 11:55:51 cluster kernel: PCI: Transparent bridge - 0000:00:1e.0
Dec 19 11:55:51 cluster hcid[4084]: Bluetooth HCI daemon
Dec 19 11:55:51 cluster hcid[4084]: Register path:/org/bluez fallback:1
Dec 19 11:55:51 cluster kernel: ACPI: PCI Interrupt Link [LNKA] (IRQs *5 7
10 11)
Dec 19 11:55:51 cluster kernel: ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *7
10 11)
Dec 19 11:55:51 cluster kernel: ACPI: PCI Interrupt Link [LNKC] (IRQs 5 7
*10 11)
Dec 19 11:55:51 cluster kernel: ACPI: PCI Interrupt Link [LNKD] (IRQs 5 7
*10 11)
Dec 19 11:55:51 cluster kernel: ACPI: PCI Interrupt Link [LNKE] (IRQs 5 7 10
11) *0, disabled.
Dec 19 11:55:51 cluster kernel: ACPI: PCI Interrupt Link [LNKF] (IRQs *5 7
10 11)
Dec 19 11:55:51 cluster kernel: ACPI: PCI Interrupt Link [LNKG] (IRQs 5 7
*10 11)
Dec 19 11:55:51 cluster sdpd[4088]: Bluetooth SDP daemon
Dec 19 11:55:51 cluster kernel: ACPI: PCI Interrupt Link [LNKH] (IRQs 5 *7
10 11)
Dec 19 11:55:51 cluster kernel: Linux Plug and Play Support v0.97 (c) Adam
Belay
Dec 19 11:55:51 cluster kernel: pnp: PnP ACPI init
Dec 19 11:55:51 cluster kernel: pnp: PnP ACPI: found 10 devices
Dec 19 11:55:51 cluster kernel: usbcore: registered new driver usbfs
Dec 19 11:55:51 cluster kernel: usbcore: registered new driver hub
Dec 19 11:55:51 cluster kernel: PCI: Using ACPI for IRQ routing
Dec 19 11:55:51 cluster kernel: PCI: If a device doesn''t work, try
"pci=routeirq". If it helps, post a report
Dec 19 11:55:51 cluster kernel: NetLabel: Initializing
Dec 19 11:55:52 cluster kernel: NetLabel: domain hash size = 128
Dec 19 11:55:52 cluster kernel: NetLabel: protocols = UNLABELED CIPSOv4
Dec 19 11:55:52 cluster kernel: NetLabel: unlabeled traffic allowed by
default
Dec 19 11:55:52 cluster kernel: hpet0: at MMIO 0xfed00000 (virtual
0xffffffffff5fe000), IRQs 2, 8, 0
Dec 19 11:55:52 cluster kernel: hpet0: 3 64-bit timers, 14318180 Hz
Dec 19 11:55:52 cluster kernel: ACPI: DMAR not present
Dec 19 11:55:52 cluster kernel: PCI-GART: No AMD northbridge found.
Dec 19 11:55:52 cluster kernel: pnp: 00:01: ioport range 0x408-0x40f has
been reserved
Dec 19 11:55:52 cluster kernel: pnp: 00:01: ioport range 0x4d0-0x4d1 has
been reserved
Dec 19 11:55:52 cluster kernel: pnp: 00:01: ioport range 0x700-0x71f has
been reserved
Dec 19 11:55:52 cluster kernel: PCI: Bridge: 0000:0a:00.0
Dec 19 11:55:52 cluster kernel: IO window: disabled.
Dec 19 11:55:52 cluster pcscd: pcscdaemon.c:507:main() pcsc-lite 1.4.4
daemon ready.
Dec 19 11:55:52 cluster kernel: MEM window: disabled.
Dec 19 11:55:52 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:52 cluster hidd[4194]: Bluetooth HID daemon
Dec 19 11:55:52 cluster kernel: PCI: Bridge: 0000:0a:01.0
Dec 19 11:55:52 cluster kernel: IO window: disabled.
Dec 19 11:55:52 cluster kernel: MEM window: disabled.
Dec 19 11:55:52 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:52 cluster pcscd:
hotplug_libusb.c:402:HPEstablishUSBNotifications() Driver ifd-egate.bundle
does not support IFD_GENERATE_HOTPLUG. Using active polling instead.
Dec 19 11:55:52 cluster kernel: PCI: Bridge: 0000:0a:02.0
Dec 19 11:55:52 cluster pcscd:
hotplug_libusb.c:411:HPEstablishUSBNotifications() Polling forced every 1
second(s)
Dec 19 11:55:52 cluster kernel: IO window: disabled.
Dec 19 11:55:52 cluster automount[4242]: lookup_read_master:
lookup(nisplus): couldn''t locate nis+ table auto.master
Dec 19 11:55:52 cluster kernel: MEM window: disabled.
Dec 19 11:55:52 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:52 cluster kernel: PCI: Bridge: 0000:09:00.0
Dec 19 11:55:52 cluster kernel: IO window: disabled.
Dec 19 11:55:52 cluster kernel: MEM window: disabled.
Dec 19 11:55:52 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:52 cluster kernel: PCI: Bridge: 0000:09:00.3
Dec 19 11:55:52 cluster kernel: IO window: disabled.
Dec 19 11:55:52 cluster kernel: MEM window: disabled.
Dec 19 11:55:52 cluster hpiod: 1.6.7 accepting connections at 2208...
Dec 19 11:55:52 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:52 cluster kernel: PCI: Bridge: 0000:00:02.0
Dec 19 11:55:52 cluster kernel: IO window: disabled.
Dec 19 11:55:52 cluster kernel: MEM window: fdc00000-fdcfffff
Dec 19 11:55:53 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:53 cluster kernel: PCI: Bridge: 0000:00:03.0
Dec 19 11:55:53 cluster kernel: IO window: 4000-4fff
Dec 19 11:55:53 cluster kernel: MEM window: fda00000-fdbfffff
Dec 19 11:55:53 cluster kernel: PREFETCH window
0x00000000c2000000-0x00000000c20fffff
Dec 19 11:55:53 cluster kernel: PCI: Bridge: 0000:12:01.0
Dec 19 11:55:53 cluster kernel: IO window: 5000-5fff
Dec 19 11:55:53 cluster kernel: MEM window: fdd00000-fddfffff
Dec 19 11:55:53 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:53 cluster kernel: PCI: Bridge: 0000:12:02.0
Dec 19 11:55:53 cluster kernel: IO window: 6000-6fff
Dec 19 11:55:53 cluster kernel: MEM window: fde00000-fdefffff
Dec 19 11:55:53 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:53 cluster kernel: PCI: Bridge: 0000:11:00.0
Dec 19 11:55:53 cluster kernel: IO window: 5000-6fff
Dec 19 11:55:53 cluster kernel: MEM window: fdd00000-fdefffff
Dec 19 11:55:53 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:53 cluster kernel: PCI: Bridge: 0000:00:04.0
Dec 19 11:55:53 cluster kernel: IO window: 5000-6fff
Dec 19 11:55:53 cluster kernel: MEM window: fdd00000-fdefffff
Dec 19 11:55:53 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:53 cluster kernel: PCI: Bridge: 0000:00:05.0
Dec 19 11:55:53 cluster kernel: IO window: disabled.
Dec 19 11:55:53 cluster kernel: MEM window: disabled.
Dec 19 11:55:53 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:53 cluster kernel: PCI: Bridge: 0000:00:06.0
Dec 19 11:55:53 cluster kernel: IO window: disabled.
Dec 19 11:55:53 cluster kernel: MEM window: fdf00000-fdffffff
Dec 19 11:55:53 cluster kernel: PREFETCH window
0x00000000d0000000-0x00000000dfffffff
Dec 19 11:55:53 cluster kernel: PCI: Bridge: 0000:00:07.0
Dec 19 11:55:53 cluster kernel: IO window: disabled.
Dec 19 11:55:53 cluster kernel: MEM window: disabled.
Dec 19 11:55:53 cluster kernel: PREFETCH window: disabled.
Dec 19 11:55:53 cluster kernel: PCI: Bridge: 0000:02:00.0
Dec 19 11:55:54 cluster kernel: IO window: disabled.
Dec 19 11:55:54 cluster kernel: MEM window: f8000000-f9ffffff
Dec 19 11:55:54 cluster kernel: PREFETCH window
0x00000000c2100000-0x00000000c21fffff
Dec 19 11:55:54 cluster kernel: PCI: Bridge: 0000:00:1c.0
Dec 19 11:55:54 cluster kernel: IO window: disabled.
Dec 19 11:55:54 cluster kernel: MEM window: f8000000-f9ffffff
Dec 19 11:55:54 cluster kernel: PREFETCH window
0x00000000c2100000-0x00000000c21fffff
Dec 19 11:55:54 cluster kernel: PCI: Bridge: 0000:04:00.0
Dec 19 11:55:54 cluster kernel: IO window: disabled.
Dec 19 11:55:54 cluster kernel: MEM window: fa000000-fbffffff
Dec 19 11:55:54 cluster kernel: PREFETCH window
0x00000000c2200000-0x00000000c22fffff
Dec 19 11:55:54 cluster kernel: PCI: Bridge: 0000:00:1c.1
Dec 19 11:55:54 cluster kernel: IO window: disabled.
Dec 19 11:55:54 cluster kernel: MEM window: fa000000-fbffffff
Dec 19 11:55:54 cluster kernel: PREFETCH window
0x00000000c2200000-0x00000000c22fffff
Dec 19 11:55:54 cluster kernel: PCI: Bridge: 0000:00:1e.0
Dec 19 11:55:54 cluster kernel: IO window: 2000-3fff
Dec 19 11:55:54 cluster snmpd[4309]: mibII/mta_sendmail.c:open_sendmailst:
could not guess version of statistics file "/var/log/mail/statistics"
Dec 19 11:55:54 cluster kernel: MEM window: f7e00000-f7ffffff
Dec 19 11:55:54 cluster kernel: PREFETCH window
0x00000000c8000000-0x00000000cfffffff
Dec 19 11:55:54 cluster kernel: GSI 16 sharing vector 0xA9 and IRQ 16
Dec 19 11:55:54 cluster ntpd[4380]: ntpd 4.2.2p1 at 1.1570-o Thu Jan 22
02:50:40 UTC 2009 (1)
Dec 19 11:55:54 cluster kernel: ACPI: PCI Interrupt 0000:0a:00.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:55:54 cluster ntpd[4381]: precision = 1.000 usec
Dec 19 11:55:54 cluster gpm[4397]: *** info [startup.c(95)]:
Dec 19 11:55:55 cluster snmpd[4309]: /etc/snmp/snmpd.conf: line 461: Error:
Blank line following %{pass_entry} token.
Dec 19 11:55:55 cluster gpm[4397]: Started gpm successfully. Entered daemon
mode.
Dec 19 11:55:55 cluster ntpd[4381]: Listening on interface wildcard,
0.0.0.0#123 Disabled
Dec 19 11:55:55 cluster kernel: GSI 17 sharing vector 0xB1 and IRQ 17
Dec 19 11:55:55 cluster ntpd[4381]: Listening on interface wildcard, ::#123
Disabled
Dec 19 11:55:55 cluster snmpd[4309]: net-snmp: 1 error(s) in config file(s)
Dec 19 11:55:55 cluster kernel: ACPI: PCI Interrupt 0000:0a:01.0[A] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:55:55 cluster ntpd[4381]: Listening on interface lo, ::1#123
Enabled
Dec 19 11:55:55 cluster ntpd[4381]: Listening on interface ib0,
fe80::21a:4bff:ff0c:663d#123 Enabled
Dec 19 11:55:55 cluster kernel: GSI 18 sharing vector 0xB9 and IRQ 18
Dec 19 11:55:55 cluster ntpd[4381]: Listening on interface eth0,
fe80::21f:29ff:fee8:baf8#123 Enabled
Dec 19 11:55:55 cluster kernel: ACPI: PCI Interrupt 0000:0a:02.0[A] -> GSI
18 (level, low) -> IRQ 185
Dec 19 11:55:55 cluster ntpd[4381]: Listening on interface eth1,
fe80::21f:29ff:fee8:baf6#123 Enabled
Dec 19 11:55:55 cluster ntpd[4381]: Listening on interface lo, 127.0.0.1#123
Enabled
Dec 19 11:55:55 cluster ntpd[4381]: Listening on interface eth0,
192.168.8.10#123 Enabled
Dec 19 11:55:55 cluster ntpd[4381]: Listening on interface eth1,
172.32.0.159#123 Enabled
Dec 19 11:55:55 cluster ntpd[4381]: Listening on interface ib0,
172.22.0.159#123 Enabled
Dec 19 11:55:56 cluster ntpd[4381]: kernel time sync status 0040
Dec 19 11:55:56 cluster snmpd[4309]: /etc/snmp/snmpd.conf: line 461: Error:
Blank line following %{pass_entry} token.
Dec 19 11:55:56 cluster snmpd[4309]: net-snmp: 1 error(s) in config file(s)
Dec 19 11:55:56 cluster ntpd[4381]: frequency initialized -11.054 PPM from
/var/lib/ntp/drift
Dec 19 11:55:56 cluster kernel: ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:55:56 cluster kernel: ACPI: PCI Interrupt 0000:00:1c.1[B] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:55:56 cluster kernel: NET: Registered protocol family 2
Dec 19 11:55:56 cluster kernel: IP route cache hash table entries: 524288
(order: 10, 4194304 bytes)
Dec 19 11:55:56 cluster kernel: TCP established hash table entries: 262144
(order: 10, 4194304 bytes)
Dec 19 11:55:56 cluster kernel: TCP bind hash table entries: 65536 (order:
8, 1048576 bytes)
Dec 19 11:55:56 cluster kernel: TCP: Hash tables configured (established
262144 bind 65536)
Dec 19 11:55:56 cluster kernel: TCP reno registered
Dec 19 11:55:56 cluster kernel: audit: initializing netlink socket
(disabled)
Dec 19 11:55:56 cluster kernel: type=2000 audit(1292759695.768:1):
initialized
Dec 19 11:55:56 cluster kernel: Total HugeTLB memory allocated, 0
Dec 19 11:55:56 cluster kernel: VFS: Disk quotas dquot_6.5.1
Dec 19 11:55:56 cluster kernel: Dquot-cache hash table entries: 512 (order
0, 4096 bytes)
Dec 19 11:55:56 cluster kernel: Initializing Cryptographic API
Dec 19 11:55:56 cluster kernel: alg: No test for crc32c (crc32c-generic)
Dec 19 11:55:56 cluster kernel: ksign: Installing public key data
Dec 19 11:55:56 cluster kernel: Loading keyring
Dec 19 11:55:56 cluster kernel: io scheduler noop registered
Dec 19 11:55:56 cluster kernel: io scheduler anticipatory registered
Dec 19 11:55:56 cluster kernel: io scheduler deadline registered (default)
Dec 19 11:55:56 cluster kernel: io scheduler cfq registered
Dec 19 11:55:56 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:55:56 cluster last message repeated 5 times
Dec 19 11:55:56 cluster kernel: ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:55:56 cluster xinetd[4362]: xinetd Version 2.3.14 started with
libwrap loadavg labeled-networking options compiled in.
Dec 19 11:55:56 cluster xinetd[4362]: Started working: 0 available services
Dec 19 11:55:56 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:55:56 cluster kernel: ACPI: PCI Interrupt 0000:00:1c.1[B] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:55:56 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:55:56 cluster kernel: ACPI: PCI Interrupt 0000:0a:00.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:55:56 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:55:56 cluster kernel: ACPI: PCI Interrupt 0000:0a:01.0[A] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:55:56 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:55:56 cluster kernel: ACPI: PCI Interrupt 0000:0a:02.0[A] -> GSI
18 (level, low) -> IRQ 185
Dec 19 11:55:56 cluster kernel: assign_interrupt_mode Found MSI capability
Dec 19 11:55:56 cluster kernel: pci_hotplug: PCI Hot Plug PCI Core version:
0.5
Dec 19 11:55:56 cluster kernel: ACPI (exconfig-0456): Dynamic SSDT Load -
OemId [HP ] OemTableId [ CPU0CST] [20060707]
Dec 19 11:55:56 cluster kernel: ACPI (exconfig-0456): Dynamic SSDT Load -
OemId [HP ] OemTableId [ CPU1CST] [20060707]
Dec 19 11:55:56 cluster kernel: ACPI (exconfig-0456): Dynamic SSDT Load -
OemId [HP ] OemTableId [ CPU6CST] [20060707]
Dec 19 11:55:56 cluster kernel: ACPI (exconfig-0456): Dynamic SSDT Load -
OemId [HP ] OemTableId [ CPU7CST] [20060707]
Dec 19 11:55:56 cluster kernel: ACPI: Thermal Zone [THM0] (8 C)
Dec 19 11:55:56 cluster kernel: Real Time Clock Driver v1.12ac
Dec 19 11:55:56 cluster kernel: Non-volatile memory driver v1.2
Dec 19 11:55:56 cluster kernel: Linux agpgart interface v0.101 (c) Dave
Jones
Dec 19 11:55:56 cluster kernel: Serial: 8250/16550 driver $Revision: 1.90 $
4 ports, IRQ sharing enabled
Dec 19 11:55:56 cluster kernel: serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is
a 16550A
Dec 19 11:55:56 cluster kernel: serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is
a 16550A
Dec 19 11:55:56 cluster kernel: 00:09: ttyS0 at I/O 0x3f8 (irq = 4) is a
16550A
Dec 19 11:55:56 cluster kernel: brd: module loaded
Dec 19 11:55:56 cluster kernel: Uniform Multi-Platform E-IDE driver
Revision: 7.00alpha2
Dec 19 11:55:56 cluster kernel: ide: Assuming 33MHz system bus speed for PIO
modes; override with idebus=xx
Dec 19 11:55:57 cluster kernel: ESB2: IDE controller at PCI slot
0000:00:1f.1
Dec 19 11:55:57 cluster kernel: ACPI: PCI Interrupt 0000:00:1f.1[A] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:55:57 cluster kernel: ESB2: chipset revision 9
Dec 19 11:55:57 cluster kernel: ESB2: not 100% native mode: will probe irqs
later
Dec 19 11:55:57 cluster kernel: ide0: BM-DMA at 0x0500-0x0507, BIOS
settings: hda:pio, hdb:pio
Dec 19 11:55:57 cluster kernel: ide1: BM-DMA at 0x0508-0x050f, BIOS
settings: hdc:pio, hdd:pio
Dec 19 11:55:57 cluster kernel: ide-floppy driver 0.99.newide
Dec 19 11:55:57 cluster kernel: usbcore: registered new driver hiddev
Dec 19 11:55:57 cluster kernel: usbcore: registered new driver usbhid
Dec 19 11:55:57 cluster kernel: drivers/usb/input/hid-core.c: v2.6:USB HID
core driver
Dec 19 11:55:57 cluster kernel: PNP: PS/2 Controller
[PNP0303:KBD,PNP0f0e:PS2M] at 0x60,0x64 irq 1,12
Dec 19 11:55:57 cluster kernel: serio: i8042 KBD port at 0x60,0x64 irq 1
Dec 19 11:55:57 cluster kernel: serio: i8042 AUX port at 0x60,0x64 irq 12
Dec 19 11:55:57 cluster kernel: mice: PS/2 mouse device common for all mice
Dec 19 11:55:57 cluster kernel: md: md driver 0.90.3 MAX_MD_DEVS=256,
MD_SB_DISKS=27
Dec 19 11:55:57 cluster kernel: md: bitmap version 4.39
Dec 19 11:55:57 cluster kernel: TCP bic registered
Dec 19 11:55:57 cluster kernel: Initializing IPsec netlink socket
Dec 19 11:55:57 cluster kernel: NET: Registered protocol family 1
Dec 19 11:55:57 cluster kernel: NET: Registered protocol family 17
Dec 19 11:55:57 cluster kernel: ACPI: (supports S0 S4 S5)
Dec 19 11:55:57 cluster kernel: Initalizing network drop monitor service
Dec 19 11:55:57 cluster kernel: Freeing unused kernel memory: 208k freed
Dec 19 11:55:57 cluster kernel: Write protecting the kernel read-only data:
498k
Dec 19 11:55:57 cluster kernel: ACPI: PCI Interrupt 0000:00:1d.7[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:55:57 cluster kernel: ehci_hcd 0000:00:1d.7: EHCI Host Controller
Dec 19 11:55:57 cluster kernel: ehci_hcd 0000:00:1d.7: new USB bus
registered, assigned bus number 1
Dec 19 11:55:57 cluster kernel: ehci_hcd 0000:00:1d.7: debug port 1
Dec 19 11:55:57 cluster kernel: ehci_hcd 0000:00:1d.7: irq 169, io mem
0xf7df0000
Dec 19 11:55:57 cluster kernel: ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI
1.00, driver 10 Dec 2004
Dec 19 11:55:57 cluster kernel: usb usb1: configuration #1 chosen from 1
choice
Dec 19 11:55:57 cluster kernel: hub 1-0:1.0: USB hub found
Dec 19 11:55:58 cluster kernel: hub 1-0:1.0: 8 ports detected
Dec 19 11:55:58 cluster kernel: USB Universal Host Controller Interface
driver v3.0
Dec 19 11:55:58 cluster kernel: ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.0: UHCI Host Controller
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.0: new USB bus
registered, assigned bus number 2
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.0: irq 169, io base
0x00001000
Dec 19 11:55:58 cluster kernel: usb usb2: configuration #1 chosen from 1
choice
Dec 19 11:55:58 cluster kernel: hub 2-0:1.0: USB hub found
Dec 19 11:55:58 cluster kernel: hub 2-0:1.0: 2 ports detected
Dec 19 11:55:58 cluster kernel: ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.1: UHCI Host Controller
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.1: new USB bus
registered, assigned bus number 3
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.1: irq 177, io base
0x00001020
Dec 19 11:55:58 cluster kernel: usb usb3: configuration #1 chosen from 1
choice
Dec 19 11:55:58 cluster kernel: hub 3-0:1.0: USB hub found
Dec 19 11:55:58 cluster kernel: hub 3-0:1.0: 2 ports detected
Dec 19 11:55:58 cluster kernel: ACPI: PCI Interrupt 0000:00:1d.2[C] -> GSI
18 (level, low) -> IRQ 185
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.2: UHCI Host Controller
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.2: new USB bus
registered, assigned bus number 4
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.2: irq 185, io base
0x00001040
Dec 19 11:55:58 cluster kernel: usb usb4: configuration #1 chosen from 1
choice
Dec 19 11:55:58 cluster kernel: hub 4-0:1.0: USB hub found
Dec 19 11:55:58 cluster kernel: hub 4-0:1.0: 2 ports detected
Dec 19 11:55:58 cluster kernel: GSI 19 sharing vector 0x62 and IRQ 19
Dec 19 11:55:58 cluster kernel: ACPI: PCI Interrupt 0000:00:1d.3[D] -> GSI
19 (level, low) -> IRQ 98
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.3: UHCI Host Controller
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.3: new USB bus
registered, assigned bus number 5
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:00:1d.3: irq 98, io base
0x00001060
Dec 19 11:55:58 cluster kernel: usb usb5: configuration #1 chosen from 1
choice
Dec 19 11:55:58 cluster kernel: hub 5-0:1.0: USB hub found
Dec 19 11:55:58 cluster kernel: hub 5-0:1.0: 2 ports detected
Dec 19 11:55:58 cluster kernel: GSI 20 sharing vector 0x6A and IRQ 20
Dec 19 11:55:58 cluster kernel: ACPI: PCI Interrupt 0000:01:04.4[B] -> GSI
22 (level, low) -> IRQ 106
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:01:04.4: UHCI Host Controller
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:01:04.4: new USB bus
registered, assigned bus number 6
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:01:04.4: port count
misdetected? forcing to 2 ports
Dec 19 11:55:58 cluster kernel: uhci_hcd 0000:01:04.4: irq 106, io base
0x00003800
Dec 19 11:55:58 cluster kernel: usb usb6: configuration #1 chosen from 1
choice
Dec 19 11:55:58 cluster kernel: hub 6-0:1.0: USB hub found
Dec 19 11:55:58 cluster kernel: hub 6-0:1.0: 2 ports detected
Dec 19 11:55:58 cluster kernel: SCSI subsystem initialized
Dec 19 11:55:59 cluster kernel: HP CISS Driver (v 3.6.20-RH3)
Dec 19 11:55:59 cluster kernel: ACPI: PCI Interrupt 0000:06:00.0[A] -> GSI
18 (level, low) -> IRQ 185
Dec 19 11:55:59 cluster kernel: cciss0: <0x3230> at PCI 0000:06:00.0 IRQ
130
using DAC
Dec 19 11:55:59 cluster kernel: blocks= 143305920 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders17562
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: blocks= 143305920 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders17562
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: cciss/c0d0: p1 p2 p3
Dec 19 11:55:59 cluster kernel: GSI 21 sharing vector 0x92 and IRQ 21
Dec 19 11:55:59 cluster kernel: ACPI: PCI Interrupt 0000:13:04.0[A] -> GSI
25 (level, low) -> IRQ 146
Dec 19 11:55:59 cluster kernel: cciss1: <0x46> at PCI 0000:13:04.0 IRQ 146
using DAC
Dec 19 11:55:59 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: cciss/c1d0: p1 p2 p3 p4
Dec 19 11:55:59 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: cciss/c1d1: unknown partition table
Dec 19 11:55:59 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: cciss/c1d2: unknown partition table
Dec 19 11:55:59 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: cciss/c1d3: unknown partition table
Dec 19 11:55:59 cluster kernel: GSI 22 sharing vector 0x9A and IRQ 22
Dec 19 11:55:59 cluster kernel: ACPI: PCI Interrupt 0000:13:05.0[A] -> GSI
24 (level, low) -> IRQ 154
Dec 19 11:55:59 cluster kernel: usb 6-1: new full speed USB device using
uhci_hcd and address 2
Dec 19 11:55:59 cluster kernel: cciss2: <0x46> at PCI 0000:13:05.0 IRQ 154
using DAC
Dec 19 11:55:59 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:55:59 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:55:59 cluster kernel:
Dec 19 11:55:59 cluster kernel: cciss/c2d0: unknown partition table
Dec 19 11:55:59 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: cciss/c2d1:<6>usb 6-1: configuration #1
chosen from 1 choice
Dec 19 11:56:00 cluster kernel: input: HP Virtual Keyboard as
/class/input/input0
Dec 19 11:56:00 cluster kernel: input: USB HID v1.01 Keyboard [HP Virtual
Keyboard] on usb-0000:01:04.4-1
Dec 19 11:56:00 cluster kernel: input: HP Virtual Keyboard as
/class/input/input1
Dec 19 11:56:00 cluster kernel: input: USB HID v1.01 Mouse [HP Virtual
Keyboard] on usb-0000:01:04.4-1
Dec 19 11:56:00 cluster kernel: unknown partition table
Dec 19 11:56:00 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: cciss/c2d2: unknown partition table
Dec 19 11:56:00 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: cciss/c2d3: unknown partition table
Dec 19 11:56:00 cluster kernel: GSI 23 sharing vector 0xA2 and IRQ 23
Dec 19 11:56:00 cluster kernel: ACPI: PCI Interrupt 0000:14:04.0[A] -> GSI
31 (level, low) -> IRQ 162
Dec 19 11:56:00 cluster kernel: cciss3: <0x46> at PCI 0000:14:04.0 IRQ 162
using DAC
Dec 19 11:56:00 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: cciss/c3d0: unknown partition table
Dec 19 11:56:00 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: cciss/c3d1: unknown partition table
Dec 19 11:56:00 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:56:00 cluster kernel:
Dec 19 11:56:00 cluster kernel: cciss/c3d2: unknown partition table
Dec 19 11:56:00 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:00 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:01 cluster kernel:
Dec 19 11:56:01 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:01 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:01 cluster kernel:
Dec 19 11:56:01 cluster kernel: cciss/c3d3: unknown partition table
Dec 19 11:56:01 cluster kernel: GSI 24 sharing vector 0xAA and IRQ 24
Dec 19 11:56:01 cluster kernel: ACPI: PCI Interrupt 0000:14:05.0[A] -> GSI
30 (level, low) -> IRQ 170
Dec 19 11:56:01 cluster kernel: cciss4: <0x46> at PCI 0000:14:05.0 IRQ 170
using DAC
Dec 19 11:56:01 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:56:01 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:56:01 cluster kernel:
Dec 19 11:56:01 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:56:01 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:56:01 cluster kernel:
Dec 19 11:56:01 cluster kernel: cciss/c4d0: unknown partition table
Dec 19 11:56:01 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:01 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:01 cluster kernel:
Dec 19 11:56:01 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:01 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:01 cluster kernel:
Dec 19 11:56:01 cluster kernel: cciss/c4d1: unknown partition table
Dec 19 11:56:01 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:56:01 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:56:01 cluster kernel:
Dec 19 11:56:01 cluster kernel: blocks= 2097120 block_size= 512
Dec 19 11:56:01 cluster kernel: heads= 255, sectors= 32, cylinders257
Dec 19 11:56:01 cluster kernel:
Dec 19 11:56:01 cluster kernel: cciss/c4d2: unknown partition table
Dec 19 11:56:01 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:01 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:01 cluster kernel:
Dec 19 11:56:01 cluster kernel: blocks= 4294967295 block_size= 512
Dec 19 11:56:01 cluster kernel: heads= 255, sectors= 32, cylinders526345
Dec 19 11:56:01 cluster kernel:
Dec 19 11:56:01 cluster kernel: cciss/c4d3: unknown partition table
Dec 19 11:56:01 cluster kernel: shpchp: Standard Hot Plug PCI Controller
Driver version: 0.4
Dec 19 11:56:01 cluster kernel: device-mapper: uevent: version 1.0.3
Dec 19 11:56:01 cluster kernel: device-mapper: ioctl: 4.11.5-ioctl
(2007-12-12) initialised: dm-devel at redhat.com
Dec 19 11:56:01 cluster kernel: device-mapper: dm-raid45: initialized
v0.2594l
Dec 19 11:56:01 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 11:56:01 cluster kernel: EXT3-fs: mounted filesystem with ordered
data mode.
Dec 19 11:56:01 cluster kernel: SELinux: Disabled at runtime.
Dec 19 11:56:01 cluster kernel: type=1404 audit(1292759720.481:2): selinux=0
auid=4294967295 ses=4294967295
Dec 19 11:56:01 cluster kernel: Broadcom NetXtreme II Gigabit Ethernet
Driver bnx2 v1.9.3 (March 17, 2009)
Dec 19 11:56:01 cluster kernel: ACPI: PCI Interrupt 0000:03:00.0[A] -> GSI
16 (level, low) -> IRQ 169
Dec 19 11:56:01 cluster kernel: EDAC MC: Ver: 2.0.1 Apr 9 2010
Dec 19 11:56:01 cluster kernel: EDAC MC0: Giving out device to i5000_edac.c
I5000: DEV 0000:00:10.0
Dec 19 11:56:01 cluster kernel: intel_rng: FWH not detected
Dec 19 11:56:01 cluster kernel: input: PC Speaker as /class/input/input2
Dec 19 11:56:01 cluster kernel: Floppy drive(s): fd0 is 1.44M
Dec 19 11:56:02 cluster kernel: eth0: Broadcom NetXtreme II BCM5708
1000Base-T (B2) PCI-X 64-bit 133MHz found at mem f8000000, IRQ 169, node
addr 001f29e8baf8
Dec 19 11:56:02 cluster kernel: ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI
17 (level, low) -> IRQ 177
Dec 19 11:56:02 cluster kernel: eth1: Broadcom NetXtreme II BCM5708
1000Base-T (B2) PCI-X 64-bit 133MHz found at mem fa000000, IRQ 177, node
addr 001f29e8baf6
Dec 19 11:56:02 cluster kernel: ib_mthca: Mellanox InfiniBand HCA driver
v1.0 (April 4, 2008)
Dec 19 11:56:02 cluster kernel: ib_mthca: Initializing 0000:17:00.0
Dec 19 11:56:02 cluster kernel: ACPI: PCI Interrupt 0000:17:00.0[A] -> GSI
19 (level, low) -> IRQ 98
Dec 19 11:56:02 cluster kernel: ACPI: PCI Interrupt 0000:01:04.2[B] -> GSI
22 (level, low) -> IRQ 106
Dec 19 11:56:02 cluster kernel: floppy0: no floppy controllers found
Dec 19 11:56:02 cluster kernel: Floppy drive(s): fd0 is 1.44M
Dec 19 11:56:02 cluster kernel: floppy0: no floppy controllers found
Dec 19 11:56:02 cluster kernel: lp: driver loaded but no devices found
Dec 19 11:56:02 cluster kernel: ACPI: Power Button (FF) [PWRF]
Dec 19 11:56:02 cluster kernel: md: Autodetecting RAID arrays.
Dec 19 11:56:02 cluster kernel: md: autorun ...
Dec 19 11:56:02 cluster kernel: md: ... autorun DONE.
Dec 19 11:56:02 cluster kernel: device-mapper: multipath: version 1.0.5
loaded
Dec 19 11:56:02 cluster kernel: EXT3 FS on cciss/c0d0p3, internal journal
Dec 19 11:56:02 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 11:56:02 cluster kernel: EXT3 FS on cciss/c0d0p1, internal journal
Dec 19 11:56:02 cluster kernel: EXT3-fs: mounted filesystem with ordered
data mode.
Dec 19 11:56:02 cluster kernel: Adding 16779884k swap on /dev/cciss/c0d0p2.
Priority:-1 extents:1 across:16779884k
Dec 19 11:56:02 cluster kernel: IA-32 Microcode Update Driver: v1.14a <
tigran at veritas.com>
Dec 19 11:56:02 cluster kernel: microcode: CPU3 updated from revision 0xb4
to 0xb7, date = 08062007
Dec 19 11:56:02 cluster kernel: microcode: CPU2 updated from revision 0xb4
to 0xb7, date = 08062007
Dec 19 11:56:02 cluster kernel: microcode: CPU1 updated from revision 0xb4
to 0xb7, date = 08062007
Dec 19 11:56:02 cluster kernel: microcode: CPU0 updated from revision 0xb4
to 0xb7, date = 08062007
Dec 19 11:56:02 cluster kernel: 802.1Q VLAN Support v1.8 Ben Greear <
greearb at candelatech.com>
Dec 19 11:56:02 cluster kernel: All bugs added by David S. Miller <
davem at redhat.com>
Dec 19 11:56:02 cluster kernel: NET: Registered protocol family 10
Dec 19 11:56:02 cluster kernel: lo: Disabled Privacy Extensions
Dec 19 11:56:02 cluster kernel: IPv6 over IPv4 tunneling driver
Dec 19 11:56:02 cluster kernel: ADDRCONF(NETDEV_UP): ib0: link is not ready
Dec 19 11:56:02 cluster kernel: ib0: enabling connected mode will cause
multicast packet drops
Dec 19 11:56:02 cluster kernel: ib0: mtu > 2044 will cause multicast packet
drops.
Dec 19 11:56:02 cluster kernel: ib0: mtu > 2044 will cause multicast packet
drops.
Dec 19 11:56:02 cluster kernel: ADDRCONF(NETDEV_CHANGE): ib0: link becomes
ready
Dec 19 11:56:02 cluster kernel: ib1: enabling connected mode will cause
multicast packet drops
Dec 19 11:56:02 cluster kernel: ib1: mtu > 2044 will cause multicast packet
drops.
Dec 19 11:56:02 cluster kernel: ib1: mtu > 2044 will cause multicast packet
drops.
Dec 19 11:56:02 cluster kernel: NET: Registered protocol family 27
Dec 19 11:56:02 cluster kernel: QLGC_VNIC: Initializing QLogic Corp. Virtual
NIC (VNIC) driver version 1.3.0.0.4
Dec 19 11:56:02 cluster kernel: Loading iSCSI transport class v2.0-871.
Dec 19 11:56:02 cluster kernel: Broadcom NetXtreme II CNIC Driver cnic
v2.0.1 (Oct 01, 2009)
Dec 19 11:56:02 cluster kernel: cnic: Added CNIC device: eth0
Dec 19 11:56:02 cluster kernel: cnic: Added CNIC device: eth1
Dec 19 11:56:02 cluster kernel: Broadcom NetXtreme II iSCSI Driver bnx2i
v2.0.1e (June 22, 2009)
Dec 19 11:56:02 cluster kernel: iscsi: registered transport (bnx2i)
Dec 19 11:56:02 cluster kernel: scsi0 : Broadcom Offload iSCSI Initiator
Dec 19 11:56:02 cluster kernel: scsi1 : Broadcom Offload iSCSI Initiator
Dec 19 11:56:02 cluster kernel: process `sysctl'' is using deprecated
sysctl
(syscall) net.ipv6.neigh.ib0.base_reachable_time; Use
net.ipv6.neigh.ib0.base_reachable_time_ms instead.
Dec 19 11:56:02 cluster kernel: bnx2: eth0: using MSI
Dec 19 11:56:02 cluster kernel: ADDRCONF(NETDEV_UP): eth0: link is not ready
Dec 19 11:56:02 cluster kernel: bnx2i: iSCSI not supported, dev=eth0
Dec 19 11:56:02 cluster kernel: bnx2: eth0 NIC Copper Link is Up, 100 Mbps
full duplex
Dec 19 11:56:02 cluster kernel: ADDRCONF(NETDEV_CHANGE): eth0: link becomes
ready
Dec 19 11:56:02 cluster kernel: bnx2i: iSCSI not supported, dev=eth0
Dec 19 11:56:03 cluster kernel: bnx2: eth1: using MSI
Dec 19 11:56:03 cluster kernel: ADDRCONF(NETDEV_UP): eth1: link is not ready
Dec 19 11:56:03 cluster kernel: bnx2i: iSCSI not supported, dev=eth1
Dec 19 11:56:03 cluster kernel: bnx2i: iSCSI not supported, dev=eth1
Dec 19 11:56:03 cluster kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps
full duplex
Dec 19 11:56:03 cluster kernel: ADDRCONF(NETDEV_CHANGE): eth1: link becomes
ready
Dec 19 11:56:03 cluster kernel: Bluetooth: Core ver 2.10
Dec 19 11:56:03 cluster kernel: NET: Registered protocol family 31
Dec 19 11:56:03 cluster kernel: Bluetooth: HCI device and connection manager
initialized
Dec 19 11:56:03 cluster kernel: Bluetooth: HCI socket layer initialized
Dec 19 11:56:03 cluster kernel: Bluetooth: L2CAP ver 2.8
Dec 19 11:56:03 cluster kernel: Bluetooth: L2CAP socket layer initialized
Dec 19 11:56:03 cluster kernel: Bluetooth: RFCOMM socket layer initialized
Dec 19 11:56:03 cluster kernel: Bluetooth: RFCOMM TTY layer initialized
Dec 19 11:56:03 cluster kernel: Bluetooth: RFCOMM ver 1.8
Dec 19 11:56:03 cluster kernel: Bluetooth: HIDP (Human Interface Emulation)
ver 1.1
Dec 19 11:56:03 cluster kernel: ipmi message handler version 39.1
Dec 19 11:56:03 cluster kernel: ipmi message handler version 39.1
Dec 19 11:56:03 cluster kernel: ipmi device interface
Dec 19 11:56:03 cluster kernel: IPMI System Interface driver.
Dec 19 11:56:03 cluster kernel: ipmi_si: Trying SMBIOS-specified kcs state
machine at i/o address 0xca2, slave address 0x20, irq 0
Dec 19 11:56:03 cluster kernel: ipmi: Found new BMC (man_id: 0x00000b,
prod_id: 0x0000, dev_id: 0x11)
Dec 19 11:56:03 cluster kernel: IPMI kcs interface initialized
Dec 19 11:56:03 cluster kernel: ipmi_si: Trying ACPI-specified kcs state
machine at i/o address 0xca2, slave address 0x0, irq 0
Dec 19 11:56:03 cluster kernel: ipmi_si: duplicate interface
Dec 19 11:56:03 cluster kernel: GSI 25 sharing vector 0xDA and IRQ 25
Dec 19 11:56:03 cluster kernel: ACPI: PCI Interrupt 0000:01:04.6[A] -> GSI
21 (level, low) -> IRQ 218
Dec 19 11:56:03 cluster kernel: ipmi_si: Trying PCI-specified kcs state
machine at mem address 0xf7ef0000, slave address 0x0, irq 218
Dec 19 11:56:03 cluster kernel: Using irq 218
Dec 19 11:56:03 cluster kernel: ipmi: interfacing existing BMC (man_id:
0x00000b, prod_id: 0x0000, dev_id: 0x11)
Dec 19 11:56:03 cluster kernel: IPMI kcs interface initialized
Dec 19 11:56:25 cluster kernel: Fusion MPT base driver 3.04.07rh
Dec 19 11:56:25 cluster kernel: Copyright (c) 1999-2008 LSI Corporation
Dec 19 11:56:25 cluster kernel: Fusion MPT misc device (ioctl) driver
3.04.07rh
Dec 19 11:56:25 cluster kernel: mptctl: Registered with Fusion MPT base
driver
Dec 19 11:56:25 cluster kernel: mptctl: /dev/mptctl @ (major,minor=10,220)
Dec 19 11:56:27 cluster avahi-daemon[6129]: Found user ''avahi''
(UID 70) and
group ''avahi'' (GID 70).
Dec 19 11:56:27 cluster avahi-daemon[6129]: Successfully dropped root
privileges.
Dec 19 11:56:27 cluster avahi-daemon[6129]: avahi-daemon 0.6.16 starting up.
Dec 19 11:56:27 cluster avahi-daemon[6129]: WARNING: No NSS support for mDNS
detected, consider installing nss-mdns!
Dec 19 11:56:27 cluster avahi-daemon[6129]: Successfully called chroot().
Dec 19 11:56:27 cluster avahi-daemon[6129]: Successfully dropped remaining
capabilities.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Loading service file
/services/sftp-ssh.service.
Dec 19 11:56:27 cluster avahi-daemon[6129]: New relevant interface ib0.IPv6
for mDNS.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Joining mDNS multicast group on
interface ib0.IPv6 with address fe80::21a:4bff:ff0c:663d.
Dec 19 11:56:27 cluster avahi-daemon[6129]: New relevant interface ib0.IPv4
for mDNS.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Joining mDNS multicast group on
interface ib0.IPv4 with address 172.22.0.159.
Dec 19 11:56:27 cluster avahi-daemon[6129]: New relevant interface eth1.IPv6
for mDNS.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Joining mDNS multicast group on
interface eth1.IPv6 with address fe80::21f:29ff:fee8:baf6.
Dec 19 11:56:27 cluster avahi-daemon[6129]: New relevant interface eth1.IPv4
for mDNS.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Joining mDNS multicast group on
interface eth1.IPv4 with address 172.32.0.159.
Dec 19 11:56:27 cluster avahi-daemon[6129]: New relevant interface eth0.IPv6
for mDNS.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Joining mDNS multicast group on
interface eth0.IPv6 with address fe80::21f:29ff:fee8:baf8.
Dec 19 11:56:27 cluster avahi-daemon[6129]: New relevant interface eth0.IPv4
for mDNS.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Joining mDNS multicast group on
interface eth0.IPv4 with address 192.168.8.10.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Network interface enumeration
completed.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Registering new address record
for fe80::21a:4bff:ff0c:663d on ib0.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Registering new address record
for 172.22.0.159 on ib0.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Registering new address record
for fe80::21f:29ff:fee8:baf6 on eth1.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Registering new address record
for 172.32.0.159 on eth1.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Registering new address record
for fe80::21f:29ff:fee8:baf8 on eth0.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Registering new address record
for 192.168.8.10 on eth0.
Dec 19 11:56:27 cluster avahi-daemon[6129]: Registering HINFO record with
values ''X86_64''/''LINUX''.
Dec 19 11:56:28 cluster avahi-daemon[6129]: Server startup complete. Host
name is cluster.local. Local service cookie is 3472620115.
Dec 19 11:56:28 cluster avahi-daemon[6129]: Service "SFTP File Transfer on
cluster" (/services/sftp-ssh.service) successfully established.
Dec 19 11:56:30 cluster smartd[6547]: smartd version 5.38
[x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Dec 19 11:56:30 cluster smartd[6547]: Home page is
http://smartmontools.sourceforge.net/
Dec 19 11:56:30 cluster smartd[6547]: Opened configuration file
/etc/smartd.conf
Dec 19 11:56:30 cluster smartd[6547]: Configuration file /etc/smartd.conf
was parsed, found DEVICESCAN, scanning devices
Dec 19 11:56:30 cluster smartd[6547]: Problem creating device name scan list
Dec 19 11:56:30 cluster smartd[6547]: Problem creating device name scan list
Dec 19 11:56:30 cluster smartd[6547]: In the system''s table of devices
NO
devices found to scan
Dec 19 11:56:30 cluster smartd[6547]: Monitoring 0 ATA and 0 SCSI devices
Dec 19 11:56:30 cluster smartd[6654]: smartd has fork()ed into background
mode. New PID=6654.
Dec 19 11:56:30 cluster vcagentd: Type: 4 - Event ID: 1073741884
Dec 19 11:56:55 cluster snmpd[4309]: netsnmp_assert 1 == new_val->high
failed int64.c:419 netsnmp_c64_check32_and_update()
Dec 19 11:57:25 cluster snmpd[4309]: c64 32 bit check failed
Dec 19 11:59:09 cluster ntpd[4381]: synchronized to 172.22.0.129, stratum 11
Dec 19 11:59:09 cluster ntpd[4381]: time reset -0.489736 s
Dec 19 11:59:09 cluster ntpd[4381]: kernel time sync enabled 0001
Dec 19 12:02:29 cluster ntpd[4381]: synchronized to LOCAL(0), stratum 10
Dec 19 12:10:25 cluster snmpd[4309]: looks like a 64bit wrap, but prev!=new
Dec 19 19:49:37 cluster kernel: Lustre: OBD class driver,
http://www.lustre.org/
Dec 19 19:49:37 cluster kernel: Lustre: Lustre Version: 1.8.3
Dec 19 19:49:37 cluster kernel: Lustre: Build Version:
1.8.3-20100409182943-PRISTINE-2.6.18-164.11.1.el5_lustre.1.8.3
Dec 19 19:49:37 cluster kernel: Lustre: Listener bound to
ib0:172.22.0.159:987:mthca0
Dec 19 19:49:37 cluster kernel: Lustre: Register global MR array, MR size:
0xffffffffffffffff, array size: 1
Dec 19 19:49:37 cluster kernel: Lustre: Added LNI 172.22.0.159 at
o2ib[8/64/0/180]
Dec 19 19:49:37 cluster kernel: Lustre: Added LNI 172.32.0.159 at
tcp[8/256/0/180]
Dec 19 19:49:37 cluster kernel: Lustre: Accept secure, port 988
Dec 19 19:49:37 cluster kernel: Lustre: Lustre Client File System;
http://www.lustre.org/
Dec 19 19:49:37 cluster kernel: init dynlocks cache
Dec 19 19:49:37 cluster kernel: ldiskfs created from ext3-2.6-rhel5
Dec 19 19:49:39 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 19:49:39 cluster kernel: LDISKFS-fs warning: maximal mount count
reached, running e2fsck is recommended
Dec 19 19:49:39 cluster kernel: LDISKFS FS on cciss/c3d1, internal journal
Dec 19 19:49:39 cluster kernel: LDISKFS-fs: recovery complete.
Dec 19 19:49:39 cluster kernel: LDISKFS-fs: mounted filesystem with ordered
data mode.
Dec 19 19:49:39 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 19:49:39 cluster kernel: LDISKFS-fs warning: maximal mount count
reached, running e2fsck is recommended
Dec 19 19:49:39 cluster kernel: LDISKFS FS on cciss/c3d1, internal journal
Dec 19 19:49:39 cluster kernel: LDISKFS-fs: mounted filesystem with ordered
data mode.
Dec 19 19:49:39 cluster kernel: LDISKFS-fs: file extents enabled
Dec 19 19:49:39 cluster kernel: LDISKFS-fs: mballoc enabled
Dec 19 19:49:39 cluster kernel: Lustre: MGC172.22.0.150 at o2ib: Reactivating
import
Dec 19 19:49:39 cluster kernel: LustreError: 137-5: UUID
''cluster-ost27_UUID'' is not available for connect (no target)
Dec 19 19:49:39 cluster kernel: LustreError:
30071:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff81041c57bc00 x1355357749683344/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292768479 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 19:49:39 cluster kernel: Lustre: Filtering OBD driver;
http://www.lustre.org/
Dec 19 19:49:40 cluster kernel: Lustre:
30338:0:(filter.c:990:filter_init_server_data()) RECOVERY: service
dan3-OST0006, 267 recoverable clients, 0 delayed clients, last_rcvd
120259807539
Dec 19 19:49:40 cluster kernel: LustreError: 137-5: UUID
''cluster-ost27_UUID'' is not available for connect (not set
up)
Dec 19 19:49:40 cluster kernel: LustreError: Skipped 3 previous similar
messages
Dec 19 19:49:40 cluster kernel: LustreError:
30072:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff81041dab7400 x1355357749628836/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292768480 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 19:49:40 cluster kernel: LustreError:
30072:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar
messages
Dec 19 19:49:40 cluster kernel: Lustre: dan3-OST0006: Now serving
dan3-OST0006 on /etc/dan/luns/lun72 with recovery enabled
Dec 19 19:49:40 cluster kernel: Lustre: dan3-OST0006: Will be in recovery
for at least 5:00, or until 267 clients reconnect
Dec 19 19:49:40 cluster kernel: Lustre: dan3-OST0006.ost: set parameter
quota_type=ug2
Dec 19 19:49:40 cluster kernel: Lustre:
30094:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 266
recoverable clients remain
Dec 19 19:49:40 cluster kernel: Lustre:
30095:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 265
recoverable clients remain
Dec 19 19:49:40 cluster kernel: LustreError: 137-5: UUID
''cluster-ost28_UUID'' is not available for connect (no target)
Dec 19 19:49:40 cluster kernel: LustreError:
30133:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff81041b42e000 x1353539538627603/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292768480 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 19:49:40 cluster kernel: LustreError:
30133:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 25 previous
similar messages
Dec 19 19:49:40 cluster kernel: Lustre:
30127:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 261
recoverable clients remain
Dec 19 19:49:40 cluster kernel: Lustre:
30127:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 3
previous similar messages
Dec 19 19:49:40 cluster kernel: LustreError: Skipped 28 previous similar
messages
Dec 19 19:49:41 cluster kernel: LustreError: 137-5: UUID
''cluster-ost28_UUID'' is not available for connect (no target)
Dec 19 19:49:41 cluster kernel: LustreError:
30165:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff81041e575050 x1353498440841407/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292768481 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 19:49:41 cluster kernel: LustreError:
30165:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 50 previous
similar messages
Dec 19 19:49:41 cluster kernel: Lustre:
30142:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 244
recoverable clients remain
Dec 19 19:49:41 cluster kernel: Lustre:
30142:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 16
previous similar messages
Dec 19 19:49:41 cluster kernel: LustreError: Skipped 50 previous similar
messages
Dec 19 19:49:43 cluster kernel: LustreError: 137-5: UUID
''cluster-ost28_UUID'' is not available for connect (no target)
Dec 19 19:49:43 cluster kernel: LustreError:
30171:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff81041fd6e050 x1355357754857631/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292768483 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 19:49:43 cluster kernel: LustreError:
30171:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 41 previous
similar messages
Dec 19 19:49:43 cluster kernel: Lustre:
30071:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 230
recoverable clients remain
Dec 19 19:49:43 cluster kernel: Lustre:
30071:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 13
previous similar messages
Dec 19 19:49:43 cluster kernel: LustreError: Skipped 41 previous similar
messages
Dec 19 19:49:48 cluster kernel: LustreError: 137-5: UUID
''cluster-ost28_UUID'' is not available for connect (no target)
Dec 19 19:49:48 cluster kernel: LustreError:
30192:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff81041b293800 x1353488992619011/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292768488 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 19:49:48 cluster kernel: LustreError:
30192:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 45 previous
similar messages
Dec 19 19:49:48 cluster kernel: Lustre:
30077:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 214
recoverable clients remain
Dec 19 19:49:48 cluster kernel: Lustre:
30077:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 15
previous similar messages
Dec 19 19:49:48 cluster kernel: LustreError: Skipped 45 previous similar
messages
Dec 19 19:49:52 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 19:49:52 cluster kernel: LDISKFS-fs warning: maximal mount count
reached, running e2fsck is recommended
Dec 19 19:49:52 cluster kernel: LDISKFS FS on cciss/c3d3, internal journal
Dec 19 19:49:52 cluster kernel: LDISKFS-fs: recovery complete.
Dec 19 19:49:52 cluster kernel: LDISKFS-fs: mounted filesystem with ordered
data mode.
Dec 19 19:49:52 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 19:49:52 cluster kernel: LDISKFS-fs warning: maximal mount count
reached, running e2fsck is recommended
Dec 19 19:49:52 cluster kernel: LDISKFS FS on cciss/c3d3, internal journal
Dec 19 19:49:52 cluster kernel: LDISKFS-fs: mounted filesystem with ordered
data mode.
Dec 19 19:49:52 cluster kernel: LDISKFS-fs: file extents enabled
Dec 19 19:49:52 cluster kernel: LDISKFS-fs: mballoc enabled
Dec 19 19:49:52 cluster kernel: Lustre:
30375:0:(filter.c:990:filter_init_server_data()) RECOVERY: service
dan3-OST0007, 267 recoverable clients, 0 delayed clients, last_rcvd
98786412422
Dec 19 19:49:52 cluster kernel: Lustre: dan3-OST0007: Now serving
dan3-OST0007 on /etc/dan/luns/lun74 with recovery enabled
Dec 19 19:49:52 cluster kernel: Lustre: dan3-OST0007: Will be in recovery
for at least 5:00, or until 267 clients reconnect
Dec 19 19:49:52 cluster kernel: Lustre: dan3-OST0007.ost: set parameter
quota_type=ug2
Dec 19 19:49:56 cluster kernel: LustreError: 137-5: UUID
''cluster-ost37_UUID'' is not available for connect (no target)
Dec 19 19:49:56 cluster kernel: LustreError:
30090:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff810417b49400 x1355357743323540/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292768496 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 19:49:56 cluster kernel: LustreError:
30090:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 141 previous
similar messages
Dec 19 19:49:56 cluster kernel: Lustre:
30160:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0007: 233
recoverable clients remain
Dec 19 19:49:56 cluster kernel: Lustre:
30160:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 91
previous similar messages
Dec 19 19:49:56 cluster kernel: LustreError: Skipped 140 previous similar
messages
Dec 19 19:49:59 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 19:49:59 cluster kernel: LDISKFS-fs warning: maximal mount count
reached, running e2fsck is recommended
Dec 19 19:49:59 cluster kernel: LDISKFS FS on cciss/c4d1, internal journal
Dec 19 19:49:59 cluster kernel: LDISKFS-fs: recovery complete.
Dec 19 19:49:59 cluster kernel: LDISKFS-fs: mounted filesystem with ordered
data mode.
Dec 19 19:49:59 cluster kernel: LustreError:
30046:0:(filter_log.c:135:filter_cancel_cookies_cb()) error cancelling log
cookies: rc = -19
Dec 19 19:49:59 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 19:49:59 cluster kernel: LDISKFS-fs warning: maximal mount count
reached, running e2fsck is recommended
Dec 19 19:49:59 cluster kernel: LDISKFS FS on cciss/c4d1, internal journal
Dec 19 19:49:59 cluster kernel: LDISKFS-fs: mounted filesystem with ordered
data mode.
Dec 19 19:49:59 cluster kernel: LDISKFS-fs: file extents enabled
Dec 19 19:49:59 cluster kernel: LDISKFS-fs: mballoc enabled
Dec 19 19:49:59 cluster kernel: Lustre:
30549:0:(filter.c:990:filter_init_server_data()) RECOVERY: service
dan4-OST0006, 260 recoverable clients, 0 delayed clients, last_rcvd
120259121660
Dec 19 19:49:59 cluster kernel: Lustre: dan4-OST0006: Now serving
dan4-OST0006 on /etc/dan/luns/lun76 with recovery enabled
Dec 19 19:49:59 cluster kernel: Lustre: dan4-OST0006: Will be in recovery
for at least 5:00, or until 260 clients reconnect
Dec 19 19:49:59 cluster kernel: Lustre: dan4-OST0006.ost: set parameter
quota_type=ug2
Dec 19 19:50:11 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 19:50:11 cluster kernel: LDISKFS-fs warning: maximal mount count
reached, running e2fsck is recommended
Dec 19 19:50:11 cluster kernel: LDISKFS FS on cciss/c4d3, internal journal
Dec 19 19:50:11 cluster kernel: LDISKFS-fs: recovery complete.
Dec 19 19:50:11 cluster kernel: LDISKFS-fs: mounted filesystem with ordered
data mode.
Dec 19 19:50:12 cluster kernel: LustreError: 137-5: UUID
''cluster-ost38_UUID'' is not available for connect (no target)
Dec 19 19:50:12 cluster kernel: LustreError: Skipped 163 previous similar
messages
Dec 19 19:50:12 cluster kernel: Lustre:
30438:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan4-OST0006: 184
recoverable clients remain
Dec 19 19:50:12 cluster kernel: Lustre:
30438:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 266
previous similar messages
Dec 19 19:50:12 cluster kernel: LustreError:
30192:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-19) req at ffff81041999b400 x1353744256940738/t0
o8-><?>@<?>:0/0 lens 368/0
e 0 to 0 dl 1292768512 ref 1 fl Interpret:/0/0 rc -19/0
Dec 19 19:50:12 cluster kernel: LustreError:
30192:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 164 previous
similar messages
Dec 19 19:50:12 cluster kernel: kjournald starting. Commit interval 5
seconds
Dec 19 19:50:12 cluster kernel: LDISKFS-fs warning: maximal mount count
reached, running e2fsck is recommended
Dec 19 19:50:12 cluster kernel: LDISKFS FS on cciss/c4d3, internal journal
Dec 19 19:50:12 cluster kernel: LDISKFS-fs: mounted filesystem with ordered
data mode.
Dec 19 19:50:12 cluster kernel: LDISKFS-fs: file extents enabled
Dec 19 19:50:12 cluster kernel: LDISKFS-fs: mballoc enabled
Dec 19 19:50:12 cluster kernel: Lustre:
30887:0:(filter.c:990:filter_init_server_data()) RECOVERY: service
dan4-OST0007, 260 recoverable clients, 0 delayed clients, last_rcvd
98784319061
Dec 19 19:50:12 cluster kernel: Lustre: dan4-OST0007: Now serving
dan4-OST0007 on /etc/dan/luns/lun78 with recovery enabled
Dec 19 19:50:12 cluster kernel: Lustre: dan4-OST0007: Will be in recovery
for at least 5:00, or until 260 clients reconnect
Dec 19 19:50:12 cluster kernel: Lustre: dan4-OST0007.ost: set parameter
quota_type=ug2
Dec 19 19:50:45 cluster kernel: Lustre:
30157:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 35
recoverable clients remain
Dec 19 19:50:45 cluster kernel: Lustre:
30157:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 507
previous similar messages
Dec 19 19:54:23 cluster kernel: Lustre:
30391:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
50459cf1-b8b1-640b-55b7-33ab3dd1aef9 reconnecting
Dec 19 19:54:23 cluster kernel: Lustre:
30391:0:(ldlm_lib.c:875:target_handle_connect()) dan3-OST0006: refuse
reconnection from 50459cf1-b8b1-640b-55b7-33ab3dd1aef9 at 172.22.0.58@o2ib to
0xffff81041d3c4c00; still busy with 1 active RPCs
Dec 19 19:54:23 cluster kernel: LustreError:
30391:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error
(-16) req at ffff810408887400 x1355357749617338/t0
o8->50459cf1-b8b1-640b-55b7-33ab3dd1aef9 at NET_0x50000ac16003a_UUID:0/0 lens
368/264 e 0 to 0 dl 1292768763 ref 1 fl Interpret:/0/0 rc -16/0
Dec 19 19:54:23 cluster kernel: LustreError:
30391:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 1 previous similar
message
Dec 19 19:54:23 cluster kernel: Lustre:
30924:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
3db07881-c6e4-0cc9-a297-6b9e7eb56ce3 reconnecting
Dec 19 19:54:23 cluster kernel: Lustre:
30924:0:(ldlm_lib.c:875:target_handle_connect()) dan3-OST0006: refuse
reconnection from 3db07881-c6e4-0cc9-a297-6b9e7eb56ce3 at 172.22.0.38@o2ib to
0xffff81041b911400; still busy with 1 active RPCs
Dec 19 19:54:23 cluster kernel: Lustre:
30162:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
1ce3ef8f-6a6d-192a-969e-f72160e11710 reconnecting
Dec 19 19:54:23 cluster kernel: Lustre:
30162:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 19:54:23 cluster kernel: Lustre:
30162:0:(ldlm_lib.c:875:target_handle_connect()) dan3-OST0006: refuse
reconnection from 1ce3ef8f-6a6d-192a-969e-f72160e11710 at 172.22.0.72@o2ib to
0xffff81041d3c4200; still busy with 1 active RPCs
Dec 19 19:54:23 cluster kernel: Lustre:
30162:0:(ldlm_lib.c:875:target_handle_connect()) Skipped 2 previous similar
messages
Dec 19 19:54:40 cluster kernel: Lustre: dan3-OST0006: Recovery period over
after 5:00, of 267 clients 266 recovered and 1 was evicted.
Dec 19 19:54:40 cluster kernel: Lustre: dan3-OST0006: sending delayed
replies to recovered clients
Dec 19 19:54:40 cluster kernel: Lustre: dan3-OST0006: received MDS
connection from 172.22.0.151 at o2ib
Dec 19 19:54:48 cluster kernel: Lustre:
30161:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006:
50459cf1-b8b1-640b-55b7-33ab3dd1aef9 reconnecting
Dec 19 19:54:48 cluster kernel: Lustre:
30161:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous
similar messages
Dec 19 19:54:52 cluster kernel: Lustre: dan3-OST0007: Recovery period over
after 5:00, of 267 clients 266 recovered and 1 was evicted.
Dec 19 19:54:52 cluster kernel: Lustre: dan3-OST0007: sending delayed
replies to recovered clients
Dec 19 19:54:52 cluster kernel: Lustre: dan3-OST0007: received MDS
connection from 172.22.0.151 at o2ib
Dec 19 19:55:00 cluster kernel: Lustre: dan4-OST0006: Recovery period over
after 5:00, of 260 clients 259 recovered and 1 was evicted.
Dec 19 19:55:00 cluster kernel: Lustre: dan4-OST0006: sending delayed
replies to recovered clients
Dec 19 19:55:00 cluster kernel: Lustre: dan4-OST0006: received MDS
connection from 172.22.0.150 at o2ib
Dec 19 19:55:12 cluster kernel: Lustre: dan4-OST0007: Recovery period over
after 5:00, of 260 clients 259 recovered and 1 was evicted.
Dec 19 19:55:12 cluster kernel: Lustre: dan4-OST0007: sending delayed
replies to recovered clients
Dec 19 19:55:12 cluster kernel: Lustre: dan4-OST0007: received MDS
connection from 172.22.0.150 at o2ib
Regards,
Daniel A
On Tue, Dec 21, 2010 at 12:30 PM, Jeff Johnson <
jeff.johnson at aeoncomputing.com> wrote:
> Daniel,
>
> Check the health and stability of your raid-6 volume. Make sure the raid is
> healthy and online. Use whatever monitor utility came with your raid card
or
> check /proc/mdstat if it''s a Linux mdraid. Check /var/log/messages
for error
> messages from your raid or other hardware.
>
> --Jeff
>
> ---mobile signature---
> Jeff Johnson - Aeon Computing
> jeff.johnson at aeoncomputing.com
>
> On Dec 20, 2010, at 22:27, Daniel Raj <danielraj2006 at gmail.com>
wrote:
>
> Hi Jeff,
>
>
> Thanks for your reply
>
> *Storage information *:
>
>
> DL380G5 == OSS + 16GB Ram
> OS == SFS G3.2-2 + centos 5.3 + lustre 1.8.3
> MSA60 box == OST
> RAID 6
>
>
> Regards,
>
> Daniel A
>
> On Tue, Dec 21, 2010 at 11:45 AM, Jeff Johnson <<jeff.johnson at
aeoncomputing.com>
> jeff.johnson at aeoncomputing.com> wrote:
>
>> Daniel,
>>
>> It looks like your OST backend storage device may be having an issue. I
>> would check the health and stability of the backend storage device or
raid
>> you are using for an OST device. It wouldn''t likely cause a
system reboot of
>> your OSS system. There may be more problems, hardware and/or OS related
that
>> are causing the system to reboot in addition to Lustre complaining that
it
>> can''t find the OST storage device.
>>
>> Others here on the list will likely give you a more detailed answer.
The
>> storage device is the place i would look first.
>>
>> --Jeff
>>
>> --
>> ------------------------------
>> Jeff Johnson
>> Manager
>> Aeon Computing
>>
>> <jeff.johnson at aeoncomputing.com>jeff.johnson at
aeoncomputing.com
>> <http://www.aeoncomputing.com/>www.aeoncomputing.com
>> t: 858-412-3810 x101 f: 858-412-3845
>> m: 619-204-9061
>>
>> 4905 Morena Boulevard, Suite 1313 - San Diego, CA 92117
>>
>>
>> On Mon, Dec 20, 2010 at 9:43 PM, Daniel Raj < <danielraj2006 at
gmail.com>
>> danielraj2006 at gmail.com> wrote:
>>
>>>
>>>
>>>
>>> Hi Genius,
>>>
>>>
>>> Good Day !!!!!!!!!!!!!!!!!!
>>>
>>>
>>> I am Daniel. My OSS getting automatically rebooted again and again
.
>>> kindly help to me
>>>
>>> Its showing the below error messages
>>>
>>>
>>> *kernel: LustreError:
23351:0:(ldlm_lib.c:1892:target_send_reply_msg())
>>> @@@ processing error (-19) req at ffff810400e24400
x1353488904620274/t0
>>> o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292738958
ref 1 fl Interpret:/0/0 rc
>>> -19/0
>>> kernel: LustreError: 137-5: UUID
''south-ost7_UUID'' is not available for
>>> connect (no target)
>>> kernel: LustreError:
23284:0:(ldlm_lib.c:1892:target_send_reply_msg())
>>> @@@ processing error (-19) req at ffff8101124c7c00
x1353488904620359/t0
>>> o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292739025
ref 1 fl Interpret:/0/0 rc
>>> -19/0
>>> *
>>>
>>> Regards,
>>>
>>> Daniel A
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101221/ba471994/attachment-0001.html
Jeff Johnson
2010-Dec-21 11:51 UTC
[Lustre-discuss] Fwd: Reg /// OSS rebooted automatically
Daniel, In the future you might want to consider posting some entries or pieces of a log rather than the entire log file. =) Was this from the OSS that you say was rebooting or from your MDS node? I would look at the log file of the OSS node(s) that contain OST0006 and OST0007 and see if there are any RAID errors. It might be a network problem as well. Morning is coming and one of the developers will likely respond to this with more suggestions. --Jeff ---mobile signature--- Jeff Johnson - Aeon Computing jeff.johnson at aeoncomputing.com m: 619-204-9061 On Dec 20, 2010, at 23:13, Daniel Raj <danielraj2006 at gmail.com> wrote:> Dec 19 04:19:49 cluster kernel: Lustre: 23300:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006: d957783f-e60b-07b0-2c86-ecfbc7eb57b6 reconnecting > Dec 19 04:19:49 cluster kernel: Lustre: 23300:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 4 previous similar messages > Dec 19 04:30:05 cluster kernel: Lustre: 23308:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006: d957783f-e60b-07b0-2c86-ecfbc7eb57b6 reconnecting > Dec 19 04:30:05 cluster kernel: LustreError: 137-5: UUID ''cluster-ost7_UUID'' is not available for connect (no target) > Dec 19 04:30:05 cluster kernel: LustreError: 23290:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error (-19) req at ffff8103fd722c00 x1355442914715019/t0 o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292713305 ref 1 fl Interpret:/0/0 rc -19/0-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101221/bf1dc5f4/attachment.html
Brian J. Murrell
2010-Dec-21 12:50 UTC
[Lustre-discuss] Fwd: Reg /// OSS rebooted automatically
On Tue, 2010-12-21 at 11:13 +0530, Daniel Raj wrote:> > I am Daniel. My OSS getting automatically rebooted again and againIf you mean a full reboot and not a panic, this is very likely not a Lustre problem.> *kernel: LustreError: 23351:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ > processing error (-19) req at ffff810400e24400 x1353488904620274/t0 > o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292738958 ref 1 fl Interpret:/0/0 rc > -19/0 > kernel: LustreError: 137-5: UUID ''south-ost7_UUID'' is not available for > connect (no target) > kernel: LustreError: 23284:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ > processing error (-19) req at ffff8101124c7c00 x1353488904620359/t0 > o8-><?>@<?>:0/0 lens 368/0 e 0 to 0 dl 1292739025 ref 1 fl Interpret:/0/0 rc > -19/0 > *None of these messages indicates a reboot or any condition that would cause a reboot. But then again, you have provided only a very small amount of the log from which no conclusions can be drawn. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101221/36fa418b/attachment.bin
On Tue, 2010-12-21 at 10:00 +0530, Daniel Raj wrote:> Hi Genius, > > > Good Day !!!!!!!!!!!!!!!!!! > > > I am Daniel. My OSS getting automatically rebooted again and again . kindly > help to meDaniel, This is inappropriate content for this list. This list is for discussing issues about the development of Lustre. You already posted this same message to lustre-discuss and I responded there telling you that you have not provided enough information and that it''s entirely possible (probable perhaps) that your problem is not even related to Lustre since what you are getting are full reboots. Please direct any followups to lustre-discuss, not this list. Thanx, b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-devel/attachments/20101221/fa80c230/attachment.bin
Brian J. Murrell
2010-Dec-21 14:47 UTC
[Lustre-discuss] Fwd: Reg /// OSS rebooted automatically
On Tue, 2010-12-21 at 12:43 +0530, Daniel Raj wrote:> Dec 19 11:42:26 cluster kernel: LustreError: > 23330:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff810049702400 x1353488904620861/t0 o8-><?>@<?>:0/0 lens 368/0 > e 0 to 0 dl 1292739246 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 11:42:26 cluster kernel: LustreError: > 23330:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar > messages > Dec 19 11:42:26 cluster kernel: LustreError: Skipped 3 previous similar > messages > Dec 19 11:44:05 cluster kernel: LustreError: 137-5: UUID ''cluster-ost8_UUID'' > is not available for connect (no target) > Dec 19 11:44:05 cluster kernel: LustreError: > 23292:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff810283dee000 x1353488904620989/t0 o8-><?>@<?>:0/0 lens 368/0 > e 0 to 0 dl 1292739345 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 11:44:05 cluster kernel: LustreError: > 23292:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar > messages > Dec 19 11:44:05 cluster kernel: LustreError: Skipped 3 previous similar > messagesBelow is a reboot. If this was a"spontaneous" (i.e. not induced by a system administrator) reboot it was not caused by Lustre. Do you have some sort of heartbeat/failover software installed and running? Could it be that which is causing your reboots?> Dec 19 11:48:11 cluster syslogd 1.4.1: restart. > Dec 19 11:48:11 cluster kernel: klogd 1.4.1, log source = /proc/kmsg > started. > Dec 19 11:48:11 cluster kernel: Linux version > 2.6.18-164.11.1.el5_lustre.1.8.3 (lbuild at x86-build-1) (gcc version 4.1.2 > 20080704 (Red Hat 4.1.2-46)) #1 SMP Fri Apr 9 18:00:39 MDT 2010 > Dec 19 11:48:11 cluster kernel: Command line: ro root=LABEL=/ rhgb quiet > irqpoll maxcpus=1 reset_devices memmap=exactmap > memmap=640K at 0Kmemmap=5276K at 16384Kmemmap=125140K at 22300Kelfcorehdr=147440K > memmap=32K#3144016K > Dec 19 11:48:11 cluster kernel: BIOS-provided physical RAM map: > Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000000010000 - > 000000000009ec00 (usable) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 000000000009ec00 - > 00000000000a0000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000000100000 - > 00000000bfe54000 (usable) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe54000 - > 00000000bfe5c000 (ACPI data) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe5c000 - > 00000000bfe5d000 (usable) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe5d000 - > 00000000c0000000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000e0000000 - > 00000000f0000000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000fec00000 - > 00000000fed00000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000fee00000 - > 00000000fee10000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000ffc00000 - > 0000000100000000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000100000000 - > 000000043ffff000 (usable) > Dec 19 11:48:11 cluster kernel: user-defined physical RAM map: > Dec 19 11:48:11 cluster kernel: user: 0000000000000000 - 00000000000a0000 > (usable) > Dec 19 11:48:12 cluster kernel: user: 0000000001000000 - 0000000001527000 > (usable) > Dec 19 11:48:12 cluster kernel: user: 00000000015c7000 - 0000000008ffc000 > (usable) > Dec 19 11:48:12 cluster kernel: user: 00000000bfe54000 - 00000000bfe5c000 > (ACPI data) > ... [reboot continues] > Dec 19 11:51:25 cluster init: Switching to runlevel: 6The above is a manual shutdown. Somebody told the system to shut down at this point.> Dec 19 11:51:27 cluster rpc.statd[5886]: Caught signal 15, un-registering > and exiting. > Dec 19 11:51:27 cluster multipathd: --------shut down------- > Dec 19 11:51:27 cluster auditd[5740]: The audit daemon is exiting. > Dec 19 11:51:27 cluster kernel: audit(1292739687.831:17): audit_pid=0 > old=5740 by auid=4294967295 > Dec 19 11:51:27 cluster kernel: Kernel logging (proc) stopped. > Dec 19 11:51:27 cluster kernel: Kernel log daemon terminating. > Dec 19 11:51:29 cluster exiting on signal 15Below is a new system restart as a result of the above manual shutdown.> Dec 19 11:55:48 cluster syslogd 1.4.1: restart. > Dec 19 11:55:48 cluster kernel: klogd 1.4.1, log source = /proc/kmsg > started. > Dec 19 11:55:48 cluster kernel: Linux version > 2.6.18-164.11.1.el5_lustre.1.8.3 (lbuild at x86-build-1) (gcc version 4.1.2 > 20080704 (Red Hat 4.1.2-46)) #1 SMP Fri Apr 9 18:00:39 MDT 2010 > Dec 19 11:55:48 cluster kernel: Command line: ro root=LABEL=/ rhgb quiet > crashkernel=128M at 16M > Dec 19 11:55:48 cluster kernel: BIOS-provided physical RAM map: > Dec 19 11:55:48 cluster kernel: BIOS-e820: 0000000000010000 - > 000000000009ec00 (usable) > Dec 19 11:55:48 cluster kernel: BIOS-e820: 000000000009ec00 - > 00000000000a0000 (reserved) > Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000000f0000 - > 0000000000100000 (reserved) > Dec 19 11:55:48 cluster kernel: BIOS-e820: 0000000000100000 - > 00000000bfe54000 (usable) > Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000bfe54000 - > 00000000bfe5c000 (ACPI data) > Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000bfe5c000 - > 00000000bfe5d000 (usable) > Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000bfe5d000 - > 00000000c0000000 (reserved) > Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000e0000000 - > 00000000f0000000 (reserved) > Dec 19 11:55:48 cluster kernel: BIOS-e820: 00000000fec00000 - > 00000000fed00000 (reserved) > Dec 19 11:55:49 cluster kernel: BIOS-e820: 00000000fee00000 - > 00000000fee10000 (reserved) > Dec 19 11:55:49 cluster kernel: BIOS-e820: 00000000ffc00000 - > 0000000100000000 (reserved) > Dec 19 11:55:49 cluster kernel: BIOS-e820: 0000000100000000 - > 000000043ffff000 (usable) > ... [continued system boot]Below is the start of Lustre being started...> Dec 19 19:49:37 cluster kernel: ldiskfs created from ext3-2.6-rhel5 > Dec 19 19:49:39 cluster kernel: kjournald starting. Commit interval 5 > seconds > Dec 19 19:49:39 cluster kernel: LDISKFS-fs warning: maximal mount count > reached, running e2fsck is recommended > Dec 19 19:49:39 cluster kernel: LDISKFS FS on cciss/c3d1, internal journal > Dec 19 19:49:39 cluster kernel: LDISKFS-fs: recovery complete. > Dec 19 19:49:39 cluster kernel: LDISKFS-fs: mounted filesystem with ordered > data mode. > Dec 19 19:49:39 cluster kernel: kjournald starting. Commit interval 5 > seconds > Dec 19 19:49:39 cluster kernel: LDISKFS-fs warning: maximal mount count > reached, running e2fsck is recommended > Dec 19 19:49:39 cluster kernel: LDISKFS FS on cciss/c3d1, internal journal > Dec 19 19:49:39 cluster kernel: LDISKFS-fs: mounted filesystem with ordered > data mode. > Dec 19 19:49:39 cluster kernel: LDISKFS-fs: file extents enabled > Dec 19 19:49:39 cluster kernel: LDISKFS-fs: mballoc enabled > Dec 19 19:49:39 cluster kernel: Lustre: MGC172.22.0.150 at o2ib: Reactivating > import > Dec 19 19:49:39 cluster kernel: LustreError: 137-5: UUID > ''cluster-ost27_UUID'' is not available for connect (no target) > Dec 19 19:49:39 cluster kernel: LustreError: > 30071:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff81041c57bc00 x1355357749683344/t0 o8-><?>@<?>:0/0 lens 368/0 > e 0 to 0 dl 1292768479 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 19:49:39 cluster kernel: Lustre: Filtering OBD driver; > http://www.lustre.org/ > Dec 19 19:49:40 cluster kernel: Lustre: > 30338:0:(filter.c:990:filter_init_server_data()) RECOVERY: service > dan3-OST0006, 267 recoverable clients, 0 delayed clients, last_rcvd > 120259807539 > Dec 19 19:49:40 cluster kernel: LustreError: 137-5: UUID > ''cluster-ost27_UUID'' is not available for connect (not set up) > Dec 19 19:49:40 cluster kernel: LustreError: Skipped 3 previous similar > messages > Dec 19 19:49:40 cluster kernel: LustreError: > 30072:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff81041dab7400 x1355357749628836/t0 o8-><?>@<?>:0/0 lens 368/0 > e 0 to 0 dl 1292768480 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 19:49:40 cluster kernel: LustreError: > 30072:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous similar > messages > Dec 19 19:49:40 cluster kernel: Lustre: dan3-OST0006: Now serving > dan3-OST0006 on /etc/dan/luns/lun72 with recovery enabled > Dec 19 19:49:40 cluster kernel: Lustre: dan3-OST0006: Will be in recovery > for at least 5:00, or until 267 clients reconnect > Dec 19 19:49:40 cluster kernel: Lustre: dan3-OST0006.ost: set parameter > quota_type=ug2 > Dec 19 19:49:40 cluster kernel: Lustre: > 30094:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 266 > recoverable clients remain > Dec 19 19:49:40 cluster kernel: Lustre: > 30095:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 265 > recoverable clients remain > Dec 19 19:49:40 cluster kernel: LustreError: 137-5: UUID > ''cluster-ost28_UUID'' is not available for connect (no target) > Dec 19 19:49:40 cluster kernel: LustreError: > 30133:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff81041b42e000 x1353539538627603/t0 o8-><?>@<?>:0/0 lens 368/0 > e 0 to 0 dl 1292768480 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 19:49:40 cluster kernel: LustreError: > 30133:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 25 previous > similar messages > Dec 19 19:49:40 cluster kernel: Lustre: > 30127:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 261 > recoverable clients remain > Dec 19 19:49:40 cluster kernel: Lustre: > 30127:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 3 > previous similar messages > Dec 19 19:49:40 cluster kernel: LustreError: Skipped 28 previous similar > messages > Dec 19 19:49:41 cluster kernel: LustreError: 137-5: UUID > ''cluster-ost28_UUID'' is not available for connect (no target) > Dec 19 19:49:41 cluster kernel: LustreError: > 30165:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff81041e575050 x1353498440841407/t0 o8-><?>@<?>:0/0 lens 368/0 > e 0 to 0 dl 1292768481 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 19:49:41 cluster kernel: LustreError: > 30165:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 50 previous > similar messages > Dec 19 19:49:41 cluster kernel: Lustre: > 30142:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 244 > recoverable clients remain > Dec 19 19:49:41 cluster kernel: Lustre: > 30142:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 16 > previous similar messages > Dec 19 19:49:41 cluster kernel: LustreError: Skipped 50 previous similar > messages > Dec 19 19:49:43 cluster kernel: LustreError: 137-5: UUID > ''cluster-ost28_UUID'' is not available for connect (no target) > Dec 19 19:49:43 cluster kernel: LustreError: > 30171:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff81041fd6e050 x1355357754857631/t0 o8-><?>@<?>:0/0 lens 368/0 > e 0 to 0 dl 1292768483 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 19:49:43 cluster kernel: LustreError: > 30171:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 41 previous > similar messages > Dec 19 19:49:43 cluster kernel: Lustre: > 30071:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 230 > recoverable clients remain > Dec 19 19:49:43 cluster kernel: Lustre: > 30071:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 13 > previous similar messages > Dec 19 19:49:43 cluster kernel: LustreError: Skipped 41 previous similar > messages > Dec 19 19:49:48 cluster kernel: LustreError: 137-5: UUID > ''cluster-ost28_UUID'' is not available for connect (no target) > Dec 19 19:49:48 cluster kernel: LustreError: > 30192:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff81041b293800 x1353488992619011/t0 o8-><?>@<?>:0/0 lens 368/0 > e 0 to 0 dl 1292768488 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 19:49:48 cluster kernel: LustreError: > 30192:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 45 previous > similar messages > Dec 19 19:49:48 cluster kernel: Lustre: > 30077:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 214 > recoverable clients remain > Dec 19 19:49:48 cluster kernel: Lustre: > 30077:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 15 > previous similar messages > Dec 19 19:49:48 cluster kernel: LustreError: Skipped 45 previous similar > messages > Dec 19 19:49:52 cluster kernel: kjournald starting. Commit interval 5 > seconds > Dec 19 19:49:52 cluster kernel: LDISKFS-fs warning: maximal mount count > reached, running e2fsck is recommended > Dec 19 19:49:52 cluster kernel: LDISKFS FS on cciss/c3d3, internal journal > Dec 19 19:49:52 cluster kernel: LDISKFS-fs: recovery complete. > Dec 19 19:49:52 cluster kernel: LDISKFS-fs: mounted filesystem with ordered > data mode. > Dec 19 19:49:52 cluster kernel: kjournald starting. Commit interval 5 > seconds > Dec 19 19:49:52 cluster kernel: LDISKFS-fs warning: maximal mount count > reached, running e2fsck is recommended > Dec 19 19:49:52 cluster kernel: LDISKFS FS on cciss/c3d3, internal journal > Dec 19 19:49:52 cluster kernel: LDISKFS-fs: mounted filesystem with ordered > data mode. > Dec 19 19:49:52 cluster kernel: LDISKFS-fs: file extents enabled > Dec 19 19:49:52 cluster kernel: LDISKFS-fs: mballoc enabled > Dec 19 19:49:52 cluster kernel: Lustre: > 30375:0:(filter.c:990:filter_init_server_data()) RECOVERY: service > dan3-OST0007, 267 recoverable clients, 0 delayed clients, last_rcvd > 98786412422 > Dec 19 19:49:52 cluster kernel: Lustre: dan3-OST0007: Now serving > dan3-OST0007 on /etc/dan/luns/lun74 with recovery enabled > Dec 19 19:49:52 cluster kernel: Lustre: dan3-OST0007: Will be in recovery > for at least 5:00, or until 267 clients reconnect > Dec 19 19:49:52 cluster kernel: Lustre: dan3-OST0007.ost: set parameter > quota_type=ug2 > Dec 19 19:49:56 cluster kernel: LustreError: 137-5: UUID > ''cluster-ost37_UUID'' is not available for connect (no target) > Dec 19 19:49:56 cluster kernel: LustreError: > 30090:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff810417b49400 x1355357743323540/t0 o8-><?>@<?>:0/0 lens 368/0 > e 0 to 0 dl 1292768496 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 19:49:56 cluster kernel: LustreError: > 30090:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 141 previous > similar messages > Dec 19 19:49:56 cluster kernel: Lustre: > 30160:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0007: 233 > recoverable clients remain > Dec 19 19:49:56 cluster kernel: Lustre: > 30160:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 91 > previous similar messages > Dec 19 19:49:56 cluster kernel: LustreError: Skipped 140 previous similar > messages > Dec 19 19:49:59 cluster kernel: kjournald starting. Commit interval 5 > seconds > Dec 19 19:49:59 cluster kernel: LDISKFS-fs warning: maximal mount count > reached, running e2fsck is recommended > Dec 19 19:49:59 cluster kernel: LDISKFS FS on cciss/c4d1, internal journal > Dec 19 19:49:59 cluster kernel: LDISKFS-fs: recovery complete. > Dec 19 19:49:59 cluster kernel: LDISKFS-fs: mounted filesystem with ordered > data mode. > Dec 19 19:49:59 cluster kernel: LustreError: > 30046:0:(filter_log.c:135:filter_cancel_cookies_cb()) error cancelling log > cookies: rc = -19 > Dec 19 19:49:59 cluster kernel: kjournald starting. Commit interval 5 > seconds > Dec 19 19:49:59 cluster kernel: LDISKFS-fs warning: maximal mount count > reached, running e2fsck is recommended > Dec 19 19:49:59 cluster kernel: LDISKFS FS on cciss/c4d1, internal journal > Dec 19 19:49:59 cluster kernel: LDISKFS-fs: mounted filesystem with ordered > data mode. > Dec 19 19:49:59 cluster kernel: LDISKFS-fs: file extents enabled > Dec 19 19:49:59 cluster kernel: LDISKFS-fs: mballoc enabled > Dec 19 19:49:59 cluster kernel: Lustre: > 30549:0:(filter.c:990:filter_init_server_data()) RECOVERY: service > dan4-OST0006, 260 recoverable clients, 0 delayed clients, last_rcvd > 120259121660 > Dec 19 19:49:59 cluster kernel: Lustre: dan4-OST0006: Now serving > dan4-OST0006 on /etc/dan/luns/lun76 with recovery enabled > Dec 19 19:49:59 cluster kernel: Lustre: dan4-OST0006: Will be in recovery > for at least 5:00, or until 260 clients reconnect > Dec 19 19:49:59 cluster kernel: Lustre: dan4-OST0006.ost: set parameter > quota_type=ug2 > Dec 19 19:50:11 cluster kernel: kjournald starting. Commit interval 5 > seconds > Dec 19 19:50:11 cluster kernel: LDISKFS-fs warning: maximal mount count > reached, running e2fsck is recommended > Dec 19 19:50:11 cluster kernel: LDISKFS FS on cciss/c4d3, internal journal > Dec 19 19:50:11 cluster kernel: LDISKFS-fs: recovery complete. > Dec 19 19:50:11 cluster kernel: LDISKFS-fs: mounted filesystem with ordered > data mode. > Dec 19 19:50:12 cluster kernel: LustreError: 137-5: UUID > ''cluster-ost38_UUID'' is not available for connect (no target) > Dec 19 19:50:12 cluster kernel: LustreError: Skipped 163 previous similar > messages > Dec 19 19:50:12 cluster kernel: Lustre: > 30438:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan4-OST0006: 184 > recoverable clients remain > Dec 19 19:50:12 cluster kernel: Lustre: > 30438:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 266 > previous similar messages > Dec 19 19:50:12 cluster kernel: LustreError: > 30192:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff81041999b400 x1353744256940738/t0 o8-><?>@<?>:0/0 lens 368/0 > e 0 to 0 dl 1292768512 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 19:50:12 cluster kernel: LustreError: > 30192:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 164 previous > similar messages > Dec 19 19:50:12 cluster kernel: kjournald starting. Commit interval 5 > seconds > Dec 19 19:50:12 cluster kernel: LDISKFS-fs warning: maximal mount count > reached, running e2fsck is recommended > Dec 19 19:50:12 cluster kernel: LDISKFS FS on cciss/c4d3, internal journal > Dec 19 19:50:12 cluster kernel: LDISKFS-fs: mounted filesystem with ordered > data mode. > Dec 19 19:50:12 cluster kernel: LDISKFS-fs: file extents enabled > Dec 19 19:50:12 cluster kernel: LDISKFS-fs: mballoc enabled > Dec 19 19:50:12 cluster kernel: Lustre: > 30887:0:(filter.c:990:filter_init_server_data()) RECOVERY: service > dan4-OST0007, 260 recoverable clients, 0 delayed clients, last_rcvd > 98784319061 > Dec 19 19:50:12 cluster kernel: Lustre: dan4-OST0007: Now serving > dan4-OST0007 on /etc/dan/luns/lun78 with recovery enabled > Dec 19 19:50:12 cluster kernel: Lustre: dan4-OST0007: Will be in recovery > for at least 5:00, or until 260 clients reconnect > Dec 19 19:50:12 cluster kernel: Lustre: dan4-OST0007.ost: set parameter > quota_type=ug2 > Dec 19 19:50:45 cluster kernel: Lustre: > 30157:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) dan3-OST0006: 35 > recoverable clients remain > Dec 19 19:50:45 cluster kernel: Lustre: > 30157:0:(ldlm_lib.c:1788:target_queue_last_replay_reply()) Skipped 507 > previous similar messages > Dec 19 19:54:23 cluster kernel: Lustre: > 30391:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006: > 50459cf1-b8b1-640b-55b7-33ab3dd1aef9 reconnecting > Dec 19 19:54:23 cluster kernel: Lustre: > 30391:0:(ldlm_lib.c:875:target_handle_connect()) dan3-OST0006: refuse > reconnection from 50459cf1-b8b1-640b-55b7-33ab3dd1aef9 at 172.22.0.58@o2ib to > 0xffff81041d3c4c00; still busy with 1 active RPCs > Dec 19 19:54:23 cluster kernel: LustreError: > 30391:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-16) req at ffff810408887400 x1355357749617338/t0 > o8->50459cf1-b8b1-640b-55b7-33ab3dd1aef9 at NET_0x50000ac16003a_UUID:0/0 lens > 368/264 e 0 to 0 dl 1292768763 ref 1 fl Interpret:/0/0 rc -16/0 > Dec 19 19:54:23 cluster kernel: LustreError: > 30391:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 1 previous similar > message > Dec 19 19:54:23 cluster kernel: Lustre: > 30924:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006: > 3db07881-c6e4-0cc9-a297-6b9e7eb56ce3 reconnecting > Dec 19 19:54:23 cluster kernel: Lustre: > 30924:0:(ldlm_lib.c:875:target_handle_connect()) dan3-OST0006: refuse > reconnection from 3db07881-c6e4-0cc9-a297-6b9e7eb56ce3 at 172.22.0.38@o2ib to > 0xffff81041b911400; still busy with 1 active RPCs > Dec 19 19:54:23 cluster kernel: Lustre: > 30162:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006: > 1ce3ef8f-6a6d-192a-969e-f72160e11710 reconnecting > Dec 19 19:54:23 cluster kernel: Lustre: > 30162:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous > similar messages > Dec 19 19:54:23 cluster kernel: Lustre: > 30162:0:(ldlm_lib.c:875:target_handle_connect()) dan3-OST0006: refuse > reconnection from 1ce3ef8f-6a6d-192a-969e-f72160e11710 at 172.22.0.72@o2ib to > 0xffff81041d3c4200; still busy with 1 active RPCs > Dec 19 19:54:23 cluster kernel: Lustre: > 30162:0:(ldlm_lib.c:875:target_handle_connect()) Skipped 2 previous similar > messages > Dec 19 19:54:40 cluster kernel: Lustre: dan3-OST0006: Recovery period over > after 5:00, of 267 clients 266 recovered and 1 was evicted. > Dec 19 19:54:40 cluster kernel: Lustre: dan3-OST0006: sending delayed > replies to recovered clients > Dec 19 19:54:40 cluster kernel: Lustre: dan3-OST0006: received MDS > connection from 172.22.0.151 at o2ib > Dec 19 19:54:48 cluster kernel: Lustre: > 30161:0:(ldlm_lib.c:575:target_handle_reconnect()) dan3-OST0006: > 50459cf1-b8b1-640b-55b7-33ab3dd1aef9 reconnecting > Dec 19 19:54:48 cluster kernel: Lustre: > 30161:0:(ldlm_lib.c:575:target_handle_reconnect()) Skipped 2 previous > similar messages > Dec 19 19:54:52 cluster kernel: Lustre: dan3-OST0007: Recovery period over > after 5:00, of 267 clients 266 recovered and 1 was evicted. > Dec 19 19:54:52 cluster kernel: Lustre: dan3-OST0007: sending delayed > replies to recovered clients > Dec 19 19:54:52 cluster kernel: Lustre: dan3-OST0007: received MDS > connection from 172.22.0.151 at o2ib > Dec 19 19:55:00 cluster kernel: Lustre: dan4-OST0006: Recovery period over > after 5:00, of 260 clients 259 recovered and 1 was evicted. > Dec 19 19:55:00 cluster kernel: Lustre: dan4-OST0006: sending delayed > replies to recovered clients > Dec 19 19:55:00 cluster kernel: Lustre: dan4-OST0006: received MDS > connection from 172.22.0.150 at o2ib > Dec 19 19:55:12 cluster kernel: Lustre: dan4-OST0007: Recovery period over > after 5:00, of 260 clients 259 recovered and 1 was evicted. > Dec 19 19:55:12 cluster kernel: Lustre: dan4-OST0007: sending delayed > replies to recovered clients > Dec 19 19:55:12 cluster kernel: Lustre: dan4-OST0007: received MDS > connection from 172.22.0.150 at o2ibAnd lustre continues to start up. So I saw a total of two reboots. One was initiated by an operator and one looked spontaneous, like some kind of power outage event, or a "STONITH" by some heartbeat/failover software, not lustre. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101221/acbb31fb/attachment.bin
Kevin Van Maren
2010-Dec-21 16:32 UTC
[Lustre-discuss] Fwd: Reg /// OSS rebooted automatically
Daniel Raj wrote:> Hi Jeff, > > > *messages :*> Dec 19 11:42:26 cluster kernel: LustreError: > 23330:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous > similar messages > Dec 19 11:42:26 cluster kernel: LustreError: Skipped 3 previous > similar messages > Dec 19 11:44:05 cluster kernel: LustreError: 137-5: UUID > ''cluster-ost8_UUID'' is not available for connect (no target) > Dec 19 11:44:05 cluster kernel: LustreError: > 23292:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) req at ffff810283dee000 x1353488904620989/t0 o8-><?>@<?>:0/0 lens > 368/0 e 0 to 0 dl 1292739345 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 11:44:05 cluster kernel: LustreError: > 23292:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous > similar messages > Dec 19 11:44:05 cluster kernel: LustreError: Skipped 3 previous > similar messagesLooks like here the system rebooted on its own. Kernel crashes are not always able to be sent to syslog, so you would need the serial console output. You might also want to check the IPMI SEL (system event log) to see if there were any hardware errors recorded for your node.> Dec 19 11:48:11 cluster syslogd 1.4.1: restart. > Dec 19 11:48:11 cluster kernel: klogd 1.4.1, log source = /proc/kmsg > started. > Dec 19 11:48:11 cluster kernel: Linux version > 2.6.18-164.11.1.el5_lustre.1.8.3 (lbuild at x86-build-1) (gcc version > 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Fri Apr 9 18:00:39 MDT 2010 > Dec 19 11:48:11 cluster kernel: Command line: ro root=LABEL=/ rhgb > quiet irqpoll maxcpus=1 reset_devices memmap=exactmap memmap=640K at 0K > memmap=5276K at 16384K memmap=125140K at 22300K elfcorehdr=147440K > memmap=32K#3144016K > Dec 19 11:48:11 cluster kernel: BIOS-provided physical RAM map: > Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000000010000 - > 000000000009ec00 (usable) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 000000000009ec00 - > 00000000000a0000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000000100000 - > 00000000bfe54000 (usable) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe54000 - > 00000000bfe5c000 (ACPI data) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe5c000 - > 00000000bfe5d000 (usable) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe5d000 - > 00000000c0000000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000e0000000 - > 00000000f0000000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000fec00000 - > 00000000fed00000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000fee00000 - > 00000000fee10000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000ffc00000 - > 0000000100000000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000100000000 - > 000000043ffff000 (usable)Here someone initiated a reboot:> Dec 19 11:51:25 cluster init: Switching to runlevel: 6 > Dec 19 11:51:27 cluster rpc.statd[5886]: Caught signal 15, > un-registering and exiting. > Dec 19 11:51:27 cluster multipathd: --------shut down------- > Dec 19 11:51:27 cluster auditd[5740]: The audit daemon is exiting. > Dec 19 11:51:27 cluster kernel: audit(1292739687.831:17): audit_pid=0 > old=5740 by auid=4294967295 > Dec 19 11:51:27 cluster kernel: Kernel logging (proc) stopped. > Dec 19 11:51:27 cluster kernel: Kernel log daemon terminating. > Dec 19 11:51:29 cluster exiting on signal 15 > Dec 19 11:55:48 cluster syslogd 1.4.1: restart. > Dec 19 11:55:48 cluster kernel: klogd 1.4.1, log source = /proc/kmsg > started. > Dec 19 11:55:48 cluster kernel: Linux version > 2.6.18-164.11.1.el5_lustre.1.8.3 (lbuild at x86-build-1) (gcc version > 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Fri Apr 9 18:00:39 MDT 2010 > Dec 19 11:55:48 cluster kernel: Command line: ro root=LABEL=/ rhgb > quiet crashkernel=128M at 16M > Dec 19 11:55:48 cluster kernel: BIOS-provided physical RAM map: