I''ve setup a new lustre system using 2.2.0-RC2 on Rhel6 with infiniband (distro version). Everything seems to be working okay. However, i tried to mount my first machine via TCP only (no ib card present). I can lctl ping the MGS/MDT/OST nodes, but can''t mount here''s a snipped version of the error log from the client (no output comes out on the mds) can''t add initial connection setup data1-mdt0000-mdc-ffff88013991e00 failed (-2) Err -2 on cfg command: lustre cmd=cf003 0:data1-mdt0000-dc 1:data1-mdt0000_uuid 2:192.168.1.28 at o2ib mgc192.168.1.28:tcp: the configuration from log ''data1-client'' failed (-2) unable to process logs: -2 device 3 not setup got rc -108 from cancel rpc: canceling anyway i see in the third line that it references o2ib, but i don''t see why it should, the client has no IB card and when i created the MGS/MDT filesystems, i specified the NIDS of both the TCP and O2IB connections.
Hi Michael, Check if http://jira.whamcloud.com/browse/LU-1308 is what you are looking for. Cheers, Marek On 06.06.2012 20:01, Michael Di Domenico wrote:> I''ve setup a new lustre system using 2.2.0-RC2 on Rhel6 with > infiniband (distro version). Everything seems to be working okay. > However, i tried to mount my first machine via TCP only (no ib card > present). I can lctl ping the MGS/MDT/OST nodes, but can''t mount > > here''s a snipped version of the error log from the client (no output > comes out on the mds) > > can''t add initial connection > setup data1-mdt0000-mdc-ffff88013991e00 failed (-2) > Err -2 on cfg command: lustre cmd=cf003 0:data1-mdt0000-dc > 1:data1-mdt0000_uuid 2:192.168.1.28 at o2ib > mgc192.168.1.28:tcp: the configuration from log ''data1-client'' failed (-2) > unable to process logs: -2 > device 3 not setup > got rc -108 from cancel rpc: canceling anyway > > i see in the third line that it references o2ib, but i don''t see why > it should, the client has no IB card and when i created the MGS/MDT > filesystems, i specified the NIDS of both the TCP and O2IB > connections. > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > >-- Marek Magrys Systems Administrator ACC Cyfronet AGH-UST
W dniu 2012-06-08 12:24, Marek Magry? pisze:> Hi Michael, > > Check if http://jira.whamcloud.com/browse/LU-1308 is what you are > looking for. > > Cheers, > Marek > > On 06.06.2012 20:01, Michael Di Domenico wrote: >> I''ve setup a new lustre system using 2.2.0-RC2 on Rhel6 with >> infiniband (distro version). Everything seems to be working okay. >> However, i tried to mount my first machine via TCP only (no ib card >> present). I can lctl ping the MGS/MDT/OST nodes, but can''t mount >> >> here''s a snipped version of the error log from the client (no output >> comes out on the mds) >> >> can''t add initial connection >> setup data1-mdt0000-mdc-ffff88013991e00 failed (-2) >> Err -2 on cfg command: lustre cmd=cf003 0:data1-mdt0000-dc >> 1:data1-mdt0000_uuid 2:192.168.1.28 at o2ib >> mgc192.168.1.28:tcp: the configuration from log ''data1-client'' failed (-2) >> unable to process logs: -2 >> device 3 not setup >> got rc -108 from cancel rpc: canceling anyway >> >> i see in the third line that it references o2ib, but i don''t see why >> it should, the client has no IB card and when i created the MGS/MDT >> filesystems, i specified the NIDS of both the TCP and O2IB >> connections. >> >Please check your /etc/modprobe.conf. There should be line simillar to this: options lnet networks=tcp(ib0),o2ib(ib0) You may also check the NID''s. There should be simmillar to this: root at mds1 ~ # lctl list_nids 192.168.16.11 at o2ib 192.168.16.11 at tcp root at mds1 ~ # Best regads Jacek Herold