David Noriega
2011-Aug-29 15:26 UTC
[Lustre-discuss] Client unable to connect after reboot: Unable to process log 108
I''ve begun to notice this behavor in my clients. Not sure whats going on, but when a client reboots, its unable to mount lustre. I have to use ''lctrl ping'' to ping any of the lustre nodes before I''m able to mount the lustre filesystem. Any ideas? Lustre: OBD class driver, http://www.lustre.org/ Lustre: Lustre Version: 1.8.4 Lustre: Build Version: 1.8.4-20100726215630-PRISTINE-2.6.18-194.3.1.el5_lustre.1.8.4 Lustre: Added LNI 192.168.1.2 at tcp [8/256/0/180] Lustre: Accept secure, port 988 Lustre: Lustre Client File System; http://www.lustre.org/ Lustre: 3977:0:(client.c:1476:ptlrpc_expire_one_request()) @@@ Request x1378464080855041 sent from MGC192.168.5.104 at tcp to NID 192.168.5.104 at tcp 5s ago has timed out (5s prior to deadline). req at ffff81032d28dc00 x1378464080855041/t0 o250->MGS at MGC192.168.5.104@tcp_0:26/25 lens 368/584 e 0 to 1 dl 1314605796 ref 1 fl Rpc:N/0/0 rc 0/0 Lustre: 3977:0:(client.c:1476:ptlrpc_expire_one_request()) @@@ Request x1378464080855043 sent from MGC192.168.5.104 at tcp to NID 192.168.5.105 at tcp 5s ago has timed out (5s prior to deadline). req at ffff81033f410c00 x1378464080855043/t0 o250->MGS at MGC192.168.5.104@tcp_1:26/25 lens 368/584 e 0 to 1 dl 1314605821 ref 1 fl Rpc:N/0/0 rc 0/0 LustreError: 3839:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req at ffff81032d28d800 x1378464080855044/t0 o501->MGS at MGC192.168.5.104@tcp_1:26/25 lens 264/432 e 0 to 1 dl 0 ref 1 fl Rpc:/0/0 rc 0/0 LustreError: 15c-8: MGC192.168.5.104 at tcp: The configuration from log ''lustre-client'' failed (-108). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information. LustreError: 3839:0:(llite_lib.c:1086:ll_fill_super()) Unable to process log: -108 Lustre: client ffff81033887dc00 umount complete LustreError: 3839:0:(obd_mount.c:2050:lustre_fill_super()) Unable to mount (-108) Installing knfsd (copyright (C) 1996 okir at monad.swb.de). NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory NFSD: starting 90-second grace period FS-Cache: Loaded Lustre: 3977:0:(client.c:1476:ptlrpc_expire_one_request()) @@@ Request x1378464080855045 sent from MGC192.168.5.104 at tcp to NID 192.168.5.104 at tcp 0s ago has failed due to network error (5s prior to deadline). req at ffff810324d67400 x1378464080855045/t0 o250->MGS at MGC192.168.5.104@tcp_0:26/25 lens 368/584 e 0 to 1 dl 1314605832 ref 1 fl Rpc:N/0/0 rc 0/0 Lustre: 3977:0:(client.c:1476:ptlrpc_expire_one_request()) @@@ Request x1378464080855047 sent from MGC192.168.5.104 at tcp to NID 192.168.5.105 at tcp 0s ago has failed due to network error (5s prior to deadline). req at ffff810330d9c800 x1378464080855047/t0 o250->MGS at MGC192.168.5.104@tcp_1:26/25 lens 368/584 e 0 to 1 dl 1314605857 ref 1 fl Rpc:N/0/0 rc 0/0 LustreError: 5178:0:(client.c:858:ptlrpc_import_delay_req()) @@@ IMP_INVALID req at ffff810324d67000 x1378464080855048/t0 o501->MGS at MGC192.168.5.104@tcp_1:26/25 lens 264/432 e 0 to 1 dl 0 ref 1 fl Rpc:/0/0 rc 0/0 LustreError: 15c-8: MGC192.168.5.104 at tcp: The configuration from log ''lustre-client'' failed (-108). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information. LustreError: 5178:0:(llite_lib.c:1086:ll_fill_super()) Unable to process log: -108 Lustre: client ffff81032f4a3400 umount complete LustreError: 5178:0:(obd_mount.c:2050:lustre_fill_super()) Unable to mount (-108) -- Personally, I liked the university. They gave us money and facilities, we didn''t have to produce anything! You''ve never been out of college! You don''t know what it''s like out there! I''ve worked in the private sector. They expect results. -Ray Ghostbusters
David Noriega
2011-Sep-01 02:40 UTC
[Lustre-discuss] Client unable to connect after reboot: Unable to process log 108
I think I''ll add the lctl ping to a start up script as a workaround, but any ideas why this is happening? On Mon, Aug 29, 2011 at 10:26 AM, David Noriega <tsk133 at my.utsa.edu> wrote:> I''ve begun to notice this behavor in my clients. Not sure whats going > on, but when a client reboots, its unable to mount lustre. I have to > use ''lctrl ping'' to ping any of the lustre nodes before I''m able to > mount the lustre filesystem. Any ideas? > > Lustre: OBD class driver, http://www.lustre.org/ > Lustre: ? ? Lustre Version: 1.8.4 > Lustre: ? ? Build Version: > 1.8.4-20100726215630-PRISTINE-2.6.18-194.3.1.el5_lustre.1.8.4 > Lustre: Added LNI 192.168.1.2 at tcp [8/256/0/180] > Lustre: Accept secure, port 988 > Lustre: Lustre Client File System; http://www.lustre.org/ > Lustre: 3977:0:(client.c:1476:ptlrpc_expire_one_request()) @@@ Request > x1378464080855041 sent from MGC192.168.5.104 at tcp to NID > 192.168.5.104 at tcp 5s ago has timed out (5s prior to deadline). > ?req at ffff81032d28dc00 x1378464080855041/t0 > o250->MGS at MGC192.168.5.104@tcp_0:26/25 lens 368/584 e 0 to 1 dl > 1314605796 ref 1 fl Rpc:N/0/0 rc 0/0 > Lustre: 3977:0:(client.c:1476:ptlrpc_expire_one_request()) @@@ Request > x1378464080855043 sent from MGC192.168.5.104 at tcp to NID > 192.168.5.105 at tcp 5s ago has timed out (5s prior to deadline). > ?req at ffff81033f410c00 x1378464080855043/t0 > o250->MGS at MGC192.168.5.104@tcp_1:26/25 lens 368/584 e 0 to 1 dl > 1314605821 ref 1 fl Rpc:N/0/0 rc 0/0 > LustreError: 3839:0:(client.c:858:ptlrpc_import_delay_req()) @@@ > IMP_INVALID ?req at ffff81032d28d800 x1378464080855044/t0 > o501->MGS at MGC192.168.5.104@tcp_1:26/25 lens 264/432 e 0 to 1 dl 0 ref > 1 fl Rpc:/0/0 rc 0/0 > LustreError: 15c-8: MGC192.168.5.104 at tcp: The configuration from log > ''lustre-client'' failed (-108). This may be the result of communication > errors between this node and the MGS, a bad configuration, or other > errors. See the syslog for more information. > LustreError: 3839:0:(llite_lib.c:1086:ll_fill_super()) Unable to > process log: -108 > Lustre: client ffff81033887dc00 umount complete > LustreError: 3839:0:(obd_mount.c:2050:lustre_fill_super()) Unable to > mount ?(-108) > Installing knfsd (copyright (C) 1996 okir at monad.swb.de). > NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory > NFSD: starting 90-second grace period > FS-Cache: Loaded > Lustre: 3977:0:(client.c:1476:ptlrpc_expire_one_request()) @@@ Request > x1378464080855045 sent from MGC192.168.5.104 at tcp to NID > 192.168.5.104 at tcp 0s ago has failed due to network error (5s prior to > deadline). > ?req at ffff810324d67400 x1378464080855045/t0 > o250->MGS at MGC192.168.5.104@tcp_0:26/25 lens 368/584 e 0 to 1 dl > 1314605832 ref 1 fl Rpc:N/0/0 rc 0/0 > Lustre: 3977:0:(client.c:1476:ptlrpc_expire_one_request()) @@@ Request > x1378464080855047 sent from MGC192.168.5.104 at tcp to NID > 192.168.5.105 at tcp 0s ago has failed due to network error (5s prior to > deadline). > ?req at ffff810330d9c800 x1378464080855047/t0 > o250->MGS at MGC192.168.5.104@tcp_1:26/25 lens 368/584 e 0 to 1 dl > 1314605857 ref 1 fl Rpc:N/0/0 rc 0/0 > LustreError: 5178:0:(client.c:858:ptlrpc_import_delay_req()) @@@ > IMP_INVALID ?req at ffff810324d67000 x1378464080855048/t0 > o501->MGS at MGC192.168.5.104@tcp_1:26/25 lens 264/432 e 0 to 1 dl 0 ref > 1 fl Rpc:/0/0 rc 0/0 > LustreError: 15c-8: MGC192.168.5.104 at tcp: The configuration from log > ''lustre-client'' failed (-108). This may be the result of communication > errors between this node and the MGS, a bad configuration, or other > errors. See the syslog for more information. > LustreError: 5178:0:(llite_lib.c:1086:ll_fill_super()) Unable to > process log: -108 > Lustre: client ffff81032f4a3400 umount complete > LustreError: 5178:0:(obd_mount.c:2050:lustre_fill_super()) Unable to > mount ?(-108) > > -- > Personally, I liked the university. They gave us money and facilities, > we didn''t have to produce anything! You''ve never been out of college! > You don''t know what it''s like out there! I''ve worked in the private > sector. They expect results. -Ray Ghostbusters >-- Personally, I liked the university. They gave us money and facilities, we didn''t have to produce anything! You''ve never been out of college! You don''t know what it''s like out there! I''ve worked in the private sector. They expect results. -Ray Ghostbusters