Wojciech Turek
2007-Nov-01 12:35 UTC
[Lustre-discuss] no handle for file close and tlrpc_import_delay_req
Dear All, I am seeing following errors on MDS: Nov 1 12:14:13 mds01 kernel: LustreError: 17076:0:(mds_open.c: 1474:mds_close()) Skipped 139 previous similar messages Nov 1 12:14:27 mds01 kernel: LustreError: 17088:0:(mds_open.c: 1474:mds_close()) @@@ no handle for file close ino 26997837: cookie 0x47a2a9d95b67cfb6 req at 0000010094367c00 x32950/t0 o35->451db1a1-8c58- f825-eaab-a1dd24586e93 at NET_0x200000a8f092e_UUID:-1 lens 296/560 ref 0 fl Interpret:/0/0 rc 0/0 Nov 1 12:14:32 mds01 kernel: LustreError: 17089:0:(mds_open.c: 1474:mds_close()) @@@ no handle for file close ino 28697676: cookie 0x47a2a9d964ee6f0e req at 000001009181d400 x28502/t0 o35->e838fcbc-4b8c- f448-a5d2-5e472e474229 at NET_0x200000a8f060b_UUID:-1 lens 296/560 ref 0 fl Interpret:/0/0 rc 0/0 Nov 1 12:14:32 mds01 kernel: LustreError: 17089:0:(mds_open.c: 1474:mds_close()) Skipped 4 previous similar messages Nov 1 12:14:47 mds01 kernel: LustreError: 17061:0:(mds_open.c: 1474:mds_close()) @@@ no handle for file close ino 28697676: cookie 0x47a2a9d964ee7ade req at 00000100cd26ec00 x113640/t0 o35->d774ea81- c2ee-01a7-0d28-87054e92c858 at NET_0x200000a8f0510_UUID:-1 lens 296/560 ref 0 fl Interpret:/0/0 rc 0/0 Nov 1 12:14:47 mds01 kernel: LustreError: 17061:0:(mds_open.c: 1474:mds_close()) Skipped 2 previous similar messages Nov 1 12:15:47 mds01 kernel: Lustre: ddn-home-MDT0000: haven''t heard from client 9e6c2d9a-1649-3c61-0fda-b5052af0e09f (at 10.143.5.11 at tcp) in 227 seconds. I think it''s dead, and I am evicting it. Nov 1 12:15:47 mds01 kernel: Lustre: Skipped 33 previous similar messagesNov 1 12:15:55 mds01 kernel: LustreError: 17076:0: (mds_open.c:1474:mds_close()) @@@ no handle for file close ino 25301776: cookie 0x47a2a9d95b4c6238 req at 00000100cd217c00 x211014/t0 o35->31eec1e1-1f7d-a43b-ed8c-9841a694da28 at NET_0x200000a8f0421_UUID:-1 lens 296/560 ref 0 fl Interpret:/0/0 rc 0/0 Nov 1 12:15:55 mds01 kernel: LustreError: 17076:0:(mds_open.c: 1474:mds_close()) Skipped 24 previous similar messagesNov 1 12:15:55 mds01 kernel: LustreError: 17076:0:(ldlm_lib.c: 1437:target_send_reply_msg()) @@@ processing error (-116) req at 00000100cd217c00 x211014/t0 o35->31eec1e1-1f7d-a43b- ed8c-9841a694da28 at NET_0x200000a8f0421_UUID:-1 lens 296/560 ref 0 fl Interpret:/0/0 rc -116/0 Nov 1 12:15:55 mds01 kernel: LustreError: 17076:0:(ldlm_lib.c: 1437:target_send_reply_msg()) Skipped 39 previous similar messagesNov 1 12:17:00 mds01 kernel: LustreError: 16649:0: (mds_open.c:1474:mds_close()) @@@ no handle for file close ino 28968880: cookie 0x47a2a9d95bcda4b2 req at 00000100c26b4800 x58156/t0 o35->1ed2c692-aea1-c9b8-ee90-6e8d4269cda8 at NET_0x200000a8f0503_UUID:-1 lens 296/560 ref 0 fl Interpret:/0/0 rc 0/0 Nov 1 12:17:00 mds01 kernel: LustreError: 16649:0:(mds_open.c: 1474:mds_close()) Skipped 2 previous similar messages Nov 1 12:18:00 mds01 kernel: LustreError: 17066:0:(mds_open.c: 1474:mds_close()) @@@ no handle for file close ino 28968880: cookie 0x47a2a9d95bcd8dd6 req at 00000100ccf76a00 x42691/t0 o35->0a5afd52- cffe-0bff-9c69-e6027f201f5f at NET_0x200000a8f0424_UUID:-1 lens 296/560 ref 0 fl Interpret:/0/0 rc 0/0 Nov 1 12:18:00 mds01 kernel: LustreError: 17066:0:(mds_open.c: 1474:mds_close()) Skipped 1 previous similar message and OSS are showing following errors: Nov 1 12:21:35 storage10.beowulf.cluster kernel: LustreError: 23337:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID req at 00000100b0211000 x434527/t0 o101->MGS at MGC10.143.245.201@tcp_0:26 lens 232/240 ref 1 fl Rpc:/0/0 rc 0/0 Nov 1 12:21:35 storage10.beowulf.cluster kernel: LustreError: 23337:0:(client.c:519:ptlrpc_import_delay_req()) Skipped 119 previous similar messages Nov 1 12:21:59 storage08.beowulf.cluster kernel: LustreError: 22609:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID req at 00000100af90a800 x755145/t0 o101->MGS at MGC10.143.245.201@tcp_0:26 lens 232/240 ref 1 fl Rpc:/0/0 rc 0/0 Nov 1 12:21:59 storage08.beowulf.cluster kernel: LustreError: 22609:0:(client.c:519:ptlrpc_import_delay_req()) Skipped 119 previous similar messages Nov 1 12:23:31 storage09.beowulf.cluster kernel: LustreError: 23045:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID req at 00000100c31fc600 x511984/t0 o101->MGS at MGC10.143.245.201@tcp_0:26 lens 232/240 ref 1 fl Rpc:/0/0 rc 0/0 Nov 1 12:23:31 storage09.beowulf.cluster kernel: LustreError: 23045:0:(client.c:519:ptlrpc_import_delay_req()) Skipped 119 previous similar messages Nov 1 12:24:32 storage07.beowulf.cluster kernel: LustreError: 22220:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID req at 0000010119e39400 x1064767/t0 o101->MGS at MGC10.143.245.201@tcp_0:26 lens 232/240 ref 1 fl Rpc:/0/0 rc 0/0 Nov 1 12:24:32 storage07.beowulf.cluster kernel: LustreError: 22220:0:(client.c:519:ptlrpc_import_delay_req()) Skipped 119 previous similar messages Does anybody has an idea what can be the reason of this errors? My system consist of 4 OSS, 24 OST, 1 MDS, 585 clients Lustre version is 1.6.3 Kernel version on the whole cluster is 2.6.9-55.0.9.EL_lustre.1.6.3smp Thanks for you help! Mr Wojciech Turek Assistant System Manager University of Cambridge High Performance Computing service email: wjt27 at cam.ac.uk tel. +441223763517 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20071101/029fdd89/attachment-0002.html