Hi there, does anyone have a complete lnet routing example that he/she wants to share that contains a network diagram and all modprobe.conf options for clients, servers and the routers? I found only one mail in the mailing list and the interesting parts have gone through a filter and now a lot of the configuration options are ''[EMAIL PROTECTED]''. Thanks a lot in advance, Michael -- Michael Kluge, M.Sc. Technische Universit?t Dresden Center for Information Services and High Performance Computing (ZIH) D-01062 Dresden Germany Contact: Willersbau, Room WIL A 208 Phone: (+49) 351 463-34217 Fax: (+49) 351 463-37773 e-mail: michael.kluge at tu-dresden.de WWW: http://www.tu-dresden.de/zih -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100624/46240cf3/attachment.html
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Michael, This is what I''m using on my test systems: I have the servers set up on 192.168.1.0/24 and clients set up on 192.168.2.0/24, with no network routing between them and a lustre router bridging the two networks with ip addresses of 192.168.1.31 and 192.168.2.31. I''ve a attached a quick diagram. modprobe.conf for MDS and OSS servers: options lnet networks="tcp0(eth2)" routes="tcp1 192.168.1.31 at tcp0" modprobe.conf for router: options lnet networks="tcp0(eth2), tcp1(eth3)" forwarding="enabled" modprobe.conf for clients: options lnet networks="tcp1(eth2)" routes="tcp0 192.168.2.31 at tcp1" What I have is pretty minimal, but it gets the job done. - -Josh On 06/24/2010 06:15 AM, Michael Kluge wrote:> Hi there, > > does anyone have a complete lnet routing example that he/she wants to > share that contains a network diagram and all modprobe.conf options for > clients, servers and the routers? I found only one mail in the mailing > list and the interesting parts have gone through a filter and now a lot > of the configuration options are ''[EMAIL PROTECTED]''. > > > Thanks a lot in advance, > Michael > > -- > > Michael Kluge, M.Sc. > > Technische Universit?t Dresden > Center for Information Services and > High Performance Computing (ZIH) > D-01062 Dresden > Germany > > Contact: > Willersbau, Room WIL A 208 > Phone: (+49) 351 463-34217 > Fax: (+49) 351 463-37773 > e-mail: michael.kluge at tu-dresden.de <mailto:michael.kluge at tu-dresden.de> > WWW: http://www.tu-dresden.de/zih > > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkwjYEIACgkQcqyJPuRTYp9tTACeIGttWBu44dc4SKB/0IIjHhF9 i3QAn17sBD38/3MdsYuiGcUOruZVS8j/ =SLQp -----END PGP SIGNATURE----- -------------- next part -------------- A non-text attachment was scrubbed... Name: lustre_routing.png Type: image/png Size: 14305 bytes Desc: not available Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100624/7023c829/attachment-0001.png -------------- next part -------------- A non-text attachment was scrubbed... Name: lustre_routing.png.sig Type: application/octet-stream Size: 72 bytes Desc: not available Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100624/7023c829/attachment-0001.obj
Hi Josh, thanks a lot! Michael Am 24.06.2010 um 15:40 schrieb Joshua Walgenbach:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Michael, > > This is what I''m using on my test systems: > > I have the servers set up on 192.168.1.0/24 and clients set up on > 192.168.2.0/24, with no network routing between them and a lustre router > bridging the two networks with ip addresses of 192.168.1.31 and > 192.168.2.31. I''ve a attached a quick diagram. > > modprobe.conf for MDS and OSS servers: > > options lnet networks="tcp0(eth2)" routes="tcp1 192.168.1.31 at tcp0" > > modprobe.conf for router: > > options lnet networks="tcp0(eth2), tcp1(eth3)" forwarding="enabled" > > modprobe.conf for clients: > > options lnet networks="tcp1(eth2)" routes="tcp0 192.168.2.31 at tcp1" > > What I have is pretty minimal, but it gets the job done. > > - -Josh > > On 06/24/2010 06:15 AM, Michael Kluge wrote: >> Hi there, >> >> does anyone have a complete lnet routing example that he/she wants to >> share that contains a network diagram and all modprobe.conf options for >> clients, servers and the routers? I found only one mail in the mailing >> list and the interesting parts have gone through a filter and now a lot >> of the configuration options are ''[EMAIL PROTECTED]''. >> >> >> Thanks a lot in advance, >> Michael >> >> -- >> >> Michael Kluge, M.Sc. >> >> Technische Universit?t Dresden >> Center for Information Services and >> High Performance Computing (ZIH) >> D-01062 Dresden >> Germany >> >> Contact: >> Willersbau, Room WIL A 208 >> Phone: (+49) 351 463-34217 >> Fax: (+49) 351 463-37773 >> e-mail: michael.kluge at tu-dresden.de <mailto:michael.kluge at tu-dresden.de> >> WWW: http://www.tu-dresden.de/zih >> >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAkwjYEIACgkQcqyJPuRTYp9tTACeIGttWBu44dc4SKB/0IIjHhF9 > i3QAn17sBD38/3MdsYuiGcUOruZVS8j/ > =SLQp > -----END PGP SIGNATURE----- > <lustre_routing.png><lustre_routing.png.sig>_______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss-- Michael Kluge, M.Sc. Technische Universit?t Dresden Center for Information Services and High Performance Computing (ZIH) D-01062 Dresden Germany Contact: Willersbau, Room WIL A 208 Phone: (+49) 351 463-34217 Fax: (+49) 351 463-37773 e-mail: michael.kluge at tu-dresden.de WWW: http://www.tu-dresden.de/zih -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100624/9270811f/attachment.html
Michael, Joshua, you should also investigate the ip2nets option. This allows using the same modprobe.conf options on both the clients and servers, since it uses the IP addresses to determine the LNET networks rather than having to specify the interface names directly. Cheers, Andreas On 2010-06-24, at 7:40 AM, Joshua Walgenbach <jjw at indiana.edu> wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Michael, > > This is what I''m using on my test systems: > > I have the servers set up on 192.168.1.0/24 and clients set up on > 192.168.2.0/24, with no network routing between them and a lustre router > bridging the two networks with ip addresses of 192.168.1.31 and > 192.168.2.31. I''ve a attached a quick diagram. > > modprobe.conf for MDS and OSS servers: > > options lnet networks="tcp0(eth2)" routes="tcp1 192.168.1.31 at tcp0" > > modprobe.conf for router: > > options lnet networks="tcp0(eth2), tcp1(eth3)" forwarding="enabled" > > modprobe.conf for clients: > > options lnet networks="tcp1(eth2)" routes="tcp0 192.168.2.31 at tcp1" > > What I have is pretty minimal, but it gets the job done. > > - -Josh > > On 06/24/2010 06:15 AM, Michael Kluge wrote: >> Hi there, >> >> does anyone have a complete lnet routing example that he/she wants to >> share that contains a network diagram and all modprobe.conf options for >> clients, servers and the routers? I found only one mail in the mailing >> list and the interesting parts have gone through a filter and now a lot >> of the configuration options are ''[EMAIL PROTECTED]''. >> >> >> Thanks a lot in advance, >> Michael >> >> -- >> >> Michael Kluge, M.Sc. >> >> Technische Universit?t Dresden >> Center for Information Services and >> High Performance Computing (ZIH) >> D-01062 Dresden >> Germany >> >> Contact: >> Willersbau, Room WIL A 208 >> Phone: (+49) 351 463-34217 >> Fax: (+49) 351 463-37773 >> e-mail: michael.kluge at tu-dresden.de <mailto:michael.kluge at tu-dresden.de> >> WWW: http://www.tu-dresden.de/zih >> >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAkwjYEIACgkQcqyJPuRTYp9tTACeIGttWBu44dc4SKB/0IIjHhF9 > i3QAn17sBD38/3MdsYuiGcUOruZVS8j/ > =SLQp > -----END PGP SIGNATURE----- > <lustre_routing.png> > <lustre_routing.png.sig> > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss