Prajith Lal
2006-Jul-09 21:16 UTC
[Lustre-discuss] Lustre Configuration with OSTs on different servers
Hi, I am trying to setup Lustre FS with one MDS (host1), 4 OSTs (0,2 on host2 and 1,3 on host3) and one client (host4). My XML is config.xml and I ran the XML file in host2 and host3 and it create the OSTs successfully. But when I try to create the MDS it displays the following error message. MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no MDS mount options: errors=remount-ro MDS failed to start. Check the syslog for details. (May need to run lconf --write-conf) when I check the dmesg it shows LustreError: 6458:0:(handler.c:2015:mds_postsetup()) No profile found: mds1 LustreError: 6486:0:(obd_config.c:323:class_cleanup()) Device 1 not setup Can you tell me why this happens? Thanks, Prajith * * -- Lals -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20060707/ed804786/attachment.html
wangdi
2006-Jul-09 21:37 UTC
[Lustre-discuss] Lustre Configuration with OSTs on different servers
Hi, Could you add -v when you start MDS with lconf and post the result here. Then we will know more. thanks wangdi Prajith Lal wrote:> > Hi, > I am trying to setup Lustre FS with one MDS (host1), 4 OSTs (0,2 on > host2 and 1,3 on host3) and one client (host4). > My XML is config.xml and I ran the XML file in host2 and host3 and it > create the OSTs successfully. But when I try to create the MDS it > displays the following error message. > > MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no > MDS mount options: errors=remount-ro > MDS failed to start. Check the syslog for details. (May need to run > lconf --write-conf) > > when I check the dmesg it shows > LustreError: 6458:0:(handler.c:2015:mds_postsetup()) No profile found: > mds1 > LustreError: 6486:0:(obd_config.c:323:class_cleanup()) Device 1 not setup > > Can you tell me why this happens? > > Thanks, > Prajith > / > / > > > -- > Lals > ------------------------------------------------------------------------ > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss >
wangdi
2006-Jul-11 20:29 UTC
[Lustre-discuss] Lustre Configuration with OSTs on different servers
Hi, It seems MDS can not reach OST. Could you check your firewall config and /etc/hosts? thanks wangdi Prajith Lal wrote:> Hi > When I try with -v the result is > > configuring for host: [''host1.test.com <http://host1.test.com>''] > Checking XML modification time > + debugfs -c -R ''stat /LOGS'' /home/lustre/mds1 2>&1 | grep mtime > setting /proc/sys/net/core/rmem_max to at least 16777216 > setting /proc/sys/net/core/wmem_max to at least 16777216 > Service: network NET_host1.test.com_tcp .test.com_tcp_UUID > loading module: libcfs srcdir None devdir libcfs > + /sbin/modprobe libcfs > loading module: lnet srcdir None devdir lnet > + /sbin/modprobe lnet > + /sbin/modprobe lnet > loading module: ksocklnd srcdir None devdir klnds/socklnd > + /sbin/modprobe ksocklnd > Service: ldlm ldlm ldlm_UUID > loading module: lvfs srcdir None devdir lvfs > + /sbin/modprobe lvfs > loading module: obdclass srcdir None devdir obdclass > + /sbin/modprobe obdclass > loading module: ptlrpc srcdir None devdir ptlrpc > + /sbin/modprobe ptlrpc > Service: mdsdev MDD_mds1_host1.test.com m1.test.com_UUID_5 > original inode_size 0 > stripe_count 1 inode_size 512 > loading module: mdc srcdir None devdir mdc > + /sbin/modprobe mdc > loading module: osc srcdir None devdir osc > + /sbin/modprobe osc > loading module: lov srcdir None devdir lov > + /sbin/modprobe lov > loading module: mds srcdir None devdir mds > + /sbin/modprobe mds > loading module: ldiskfs srcdir None devdir ldiskfs > + /sbin/modprobe ldiskfs > loading module: fsfilt_ldiskfs srcdir None devdir lvfs > + /sbin/modprobe fsfilt_ldiskfs > + sysctl lnet/debug_path /tmp/lustre-log-host1.test.com > <http://log-host1.test.com> > + /usr/sbin/lctl modules > /tmp/ogdb-host1.test.com > <http://host1.test.com> > Service: network NET_host1.test.com_tcp .test.com_tcp_UUID > NETWORK: NET_host1.test.com_tcp .test.com_tcp_UUID tcp host1.test.com > <http://host1.test.com> > Service: ldlm ldlm ldlm_UUID > Service: mdsdev MDD_mds1_host1.test.com m1.test.com_UUID_5 > original inode_size 0 > stripe_count 1 inode_size 512 > MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no > + losetup /dev/loop0 > + losetup /dev/loop1 > + losetup /dev/loop2 > + losetup /dev/loop3 > + losetup /dev/loop4 > + losetup /dev/loop5 > + losetup /dev/loop6 > + losetup /dev/loop7 > + losetup /dev/loop0 > + losetup /dev/loop0 /home/lustre/mds1 > + /usr/sbin/lctl > attach mdt MDT MDT_UUID > quit > + /usr/sbin/lctl > cfg_device MDT > setup > quit > + dumpe2fs -f -h /dev/loop0 > no external journal found for /dev/loop0 > MDS mount options: errors=remount-ro > + /usr/sbin/lctl > attach mds mds1 mds1_UUID > quit > + /usr/sbin/lctl > cfg_device mds1 > setup /dev/loop0 ldiskfs mds1 errors=remount-ro > quit > > > On 7/10/06, *wangdi* <wangdi@clusterfs.com > <mailto:wangdi@clusterfs.com>> wrote: > > Hi, > Could you add -v when you start MDS with lconf and post the result > here. > Then we will know more. > > thanks > wangdi > > Prajith Lal wrote: > > > > Hi, > > I am trying to setup Lustre FS with one MDS (host1), 4 OSTs (0,2 on > > host2 and 1,3 on host3) and one client (host4). > > My XML is config.xml and I ran the XML file in host2 and host3 > and it > > create the OSTs successfully. But when I try to create the MDS it > > displays the following error message. > > > > MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no > > MDS mount options: errors=remount-ro > > MDS failed to start. Check the syslog for details. (May need to run > > lconf --write-conf) > > > > when I check the dmesg it shows > > LustreError: 6458:0:(handler.c:2015:mds_postsetup()) No profile > found: > > mds1 > > LustreError: 6486:0:(obd_config.c:323:class_cleanup()) Device 1 > not setup > > > > Can you tell me why this happens? > > > > Thanks, > > Prajith > > / > > / > > > > > > -- > > Lals > > > ------------------------------------------------------------------------ > > > > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss@clusterfs.com <mailto:Lustre-discuss@clusterfs.com> > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > > > > > > > -- > Lals
wangdi
2006-Jul-12 02:41 UTC
[Lustre-discuss] Lustre Configuration with OSTs on different servers
Hi, Could you send me your config.sh? What is your lconf args when starting lustre? thanks wangdi Prajith Lal wrote:> Hi, > After stopping the firewall(iptables) I ran the script. So I think > there is no firewall problem. The /etc/hosts file is mapped with the > other IPs. When I tried host1 as MDS and OST and host2, host3 as > clients it worked. > Any idea? > > Thanks, > Lal > > On 7/12/06, *wangdi* <wangdi@clusterfs.com > <mailto:wangdi@clusterfs.com>> wrote: > > Hi, > > It seems MDS can not reach OST. > Could you check your firewall config and /etc/hosts? > > thanks > wangdi > > Prajith Lal wrote: > > Hi > > When I try with -v the result is > > > > configuring for host: ['' host1.test.com <http://host1.test.com> > <http://host1.test.com>''] > > Checking XML modification time > > + debugfs -c -R ''stat /LOGS'' /home/lustre/mds1 2>&1 | grep mtime > > setting /proc/sys/net/core/rmem_max to at least 16777216 > > setting /proc/sys/net/core/wmem_max to at least 16777216 > > Service: network NET_host1.test.com_tcp .test.com_tcp_UUID > > loading module: libcfs srcdir None devdir libcfs > > + /sbin/modprobe libcfs > > loading module: lnet srcdir None devdir lnet > > + /sbin/modprobe lnet > > + /sbin/modprobe lnet > > loading module: ksocklnd srcdir None devdir klnds/socklnd > > + /sbin/modprobe ksocklnd > > Service: ldlm ldlm ldlm_UUID > > loading module: lvfs srcdir None devdir lvfs > > + /sbin/modprobe lvfs > > loading module: obdclass srcdir None devdir obdclass > > + /sbin/modprobe obdclass > > loading module: ptlrpc srcdir None devdir ptlrpc > > + /sbin/modprobe ptlrpc > > Service: mdsdev MDD_mds1_host1.test.com m1.test.com_UUID_5 > > original inode_size 0 > > stripe_count 1 inode_size 512 > > loading module: mdc srcdir None devdir mdc > > + /sbin/modprobe mdc > > loading module: osc srcdir None devdir osc > > + /sbin/modprobe osc > > loading module: lov srcdir None devdir lov > > + /sbin/modprobe lov > > loading module: mds srcdir None devdir mds > > + /sbin/modprobe mds > > loading module: ldiskfs srcdir None devdir ldiskfs > > + /sbin/modprobe ldiskfs > > loading module: fsfilt_ldiskfs srcdir None devdir lvfs > > + /sbin/modprobe fsfilt_ldiskfs > > + sysctl lnet/debug_path /tmp/lustre-log-host1.test.com > <http://log-host1.test.com> > > <http://log-host1.test.com> > > + /usr/sbin/lctl modules > /tmp/ogdb- host1.test.com > <http://host1.test.com> > > <http://host1.test.com> > > Service: network NET_host1.test.com_tcp .test.com_tcp_UUID > > NETWORK: NET_host1.test.com_tcp .test.com_tcp_UUID tcp > host1.test.com <http://host1.test.com> > > <http://host1.test.com> > > Service: ldlm ldlm ldlm_UUID > > Service: mdsdev MDD_mds1_host1.test.com m1.test.com_UUID_5 > > original inode_size 0 > > stripe_count 1 inode_size 512 > > MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no > > + losetup /dev/loop0 > > + losetup /dev/loop1 > > + losetup /dev/loop2 > > + losetup /dev/loop3 > > + losetup /dev/loop4 > > + losetup /dev/loop5 > > + losetup /dev/loop6 > > + losetup /dev/loop7 > > + losetup /dev/loop0 > > + losetup /dev/loop0 /home/lustre/mds1 > > + /usr/sbin/lctl > > attach mdt MDT MDT_UUID > > quit > > + /usr/sbin/lctl > > cfg_device MDT > > setup > > quit > > + dumpe2fs -f -h /dev/loop0 > > no external journal found for /dev/loop0 > > MDS mount options: errors=remount-ro > > + /usr/sbin/lctl > > attach mds mds1 mds1_UUID > > quit > > + /usr/sbin/lctl > > cfg_device mds1 > > setup /dev/loop0 ldiskfs mds1 errors=remount-ro > > quit > > > > > > On 7/10/06, *wangdi* <wangdi@clusterfs.com > <mailto:wangdi@clusterfs.com> > > <mailto:wangdi@clusterfs.com <mailto:wangdi@clusterfs.com>>> wrote: > > > > Hi, > > Could you add -v when you start MDS with lconf and post the > result > > here. > > Then we will know more. > > > > thanks > > wangdi > > > > Prajith Lal wrote: > > > > > > Hi, > > > I am trying to setup Lustre FS with one MDS (host1), 4 > OSTs (0,2 on > > > host2 and 1,3 on host3) and one client (host4). > > > My XML is config.xml and I ran the XML file in host2 and host3 > > and it > > > create the OSTs successfully. But when I try to create the > MDS it > > > displays the following error message. > > > > > > MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no > > > MDS mount options: errors=remount-ro > > > MDS failed to start. Check the syslog for details. (May > need to run > > > lconf --write-conf) > > > > > > when I check the dmesg it shows > > > LustreError: 6458:0:(handler.c:2015:mds_postsetup()) No > profile > > found: > > > mds1 > > > LustreError: 6486:0:(obd_config.c:323:class_cleanup()) > Device 1 > > not setup > > > > > > Can you tell me why this happens? > > > > > > Thanks, > > > Prajith > > > / > > > / > > > > > > > > > -- > > > Lals > > > > > > ------------------------------------------------------------------------ > > > > > > > > _______________________________________________ > > > Lustre-discuss mailing list > > > Lustre-discuss@clusterfs.com > <mailto:Lustre-discuss@clusterfs.com> > <mailto:Lustre-discuss@clusterfs.com > <mailto:Lustre-discuss@clusterfs.com>> > > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > > > > > > > > > > > > > -- > > Lals > > > > > -- > Lals
Prajith Lal
2006-Jul-12 03:12 UTC
[Lustre-discuss] Lustre Configuration with OSTs on different servers
Hi When I try with -v the result is configuring for host: [''host1.test.com''] Checking XML modification time + debugfs -c -R ''stat /LOGS'' /home/lustre/mds1 2>&1 | grep mtime setting /proc/sys/net/core/rmem_max to at least 16777216 setting /proc/sys/net/core/wmem_max to at least 16777216 Service: network NET_host1.test.com_tcp .test.com_tcp_UUID loading module: libcfs srcdir None devdir libcfs + /sbin/modprobe libcfs loading module: lnet srcdir None devdir lnet + /sbin/modprobe lnet + /sbin/modprobe lnet loading module: ksocklnd srcdir None devdir klnds/socklnd + /sbin/modprobe ksocklnd Service: ldlm ldlm ldlm_UUID loading module: lvfs srcdir None devdir lvfs + /sbin/modprobe lvfs loading module: obdclass srcdir None devdir obdclass + /sbin/modprobe obdclass loading module: ptlrpc srcdir None devdir ptlrpc + /sbin/modprobe ptlrpc Service: mdsdev MDD_mds1_host1.test.com m1.test.com_UUID_5 original inode_size 0 stripe_count 1 inode_size 512 loading module: mdc srcdir None devdir mdc + /sbin/modprobe mdc loading module: osc srcdir None devdir osc + /sbin/modprobe osc loading module: lov srcdir None devdir lov + /sbin/modprobe lov loading module: mds srcdir None devdir mds + /sbin/modprobe mds loading module: ldiskfs srcdir None devdir ldiskfs + /sbin/modprobe ldiskfs loading module: fsfilt_ldiskfs srcdir None devdir lvfs + /sbin/modprobe fsfilt_ldiskfs + sysctl lnet/debug_path /tmp/lustre-log-host1.test.com + /usr/sbin/lctl modules > /tmp/ogdb-host1.test.com Service: network NET_host1.test.com_tcp .test.com_tcp_UUID NETWORK: NET_host1.test.com_tcp .test.com_tcp_UUID tcp host1.test.com Service: ldlm ldlm ldlm_UUID Service: mdsdev MDD_mds1_host1.test.com m1.test.com_UUID_5 original inode_size 0 stripe_count 1 inode_size 512 MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no + losetup /dev/loop0 + losetup /dev/loop1 + losetup /dev/loop2 + losetup /dev/loop3 + losetup /dev/loop4 + losetup /dev/loop5 + losetup /dev/loop6 + losetup /dev/loop7 + losetup /dev/loop0 + losetup /dev/loop0 /home/lustre/mds1 + /usr/sbin/lctl attach mdt MDT MDT_UUID quit + /usr/sbin/lctl cfg_device MDT setup quit + dumpe2fs -f -h /dev/loop0 no external journal found for /dev/loop0 MDS mount options: errors=remount-ro + /usr/sbin/lctl attach mds mds1 mds1_UUID quit + /usr/sbin/lctl cfg_device mds1 setup /dev/loop0 ldiskfs mds1 errors=remount-ro quit On 7/10/06, wangdi <wangdi@clusterfs.com> wrote:> > Hi, > Could you add -v when you start MDS with lconf and post the result here. > Then we will know more. > > thanks > wangdi > > Prajith Lal wrote: > > > > Hi, > > I am trying to setup Lustre FS with one MDS (host1), 4 OSTs (0,2 on > > host2 and 1,3 on host3) and one client (host4). > > My XML is config.xml and I ran the XML file in host2 and host3 and it > > create the OSTs successfully. But when I try to create the MDS it > > displays the following error message. > > > > MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no > > MDS mount options: errors=remount-ro > > MDS failed to start. Check the syslog for details. (May need to run > > lconf --write-conf) > > > > when I check the dmesg it shows > > LustreError: 6458:0:(handler.c:2015:mds_postsetup()) No profile found: > > mds1 > > LustreError: 6486:0:(obd_config.c:323:class_cleanup()) Device 1 not > setup > > > > Can you tell me why this happens? > > > > Thanks, > > Prajith > > / > > / > > > > > > -- > > Lals > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss@clusterfs.com > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > > > >-- Lals -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20060711/d394167b/attachment-0001.html
Prajith Lal
2006-Jul-12 03:12 UTC
[Lustre-discuss] Lustre Configuration with OSTs on different servers
Hi, You are right it was the firewall problem. Actually I stopped the iptables but forget to stop SELinux, when I change the SELinux status disabled it worked. Is there any commands for starting Lustre with SELinux enabled This is my config script lmc -m /etc/lustre/config.xml --add net --node host1.test.com --nid host1.test.com --nettype tcp lmc -m /etc/lustre/config.xml --add net --node host2.test.com --nid host2.test.com --nettype tcp lmc -m /etc/lustre/config.xml --add net --node host3.test.com --nid host3.test.com --nettype tcp lmc -m /etc/lustre/config.xml --add net --node host4.test.com --nid host4.test.com --nettype tcp ## Create MDS lmc -m /etc/lustre/config.xml --add mds --node host1.test.com --mds mds1 --dev /home/lustre/mds1 --size 10485760 --failover ##Creating lov lmc -m /etc/lustre/config.xml --add lov --lov lov1 --mds mds1 --stripe_sz 1048576 --stripe_cnt 0 --stripe_pattern 0 ##Creating OST lmc -m /etc/lustre/config.xml --add ost --node host2.test.com --ost ost0 --lov lov1 --dev /home/lustre/ost0 --size 10485760 --failover lmc -m /etc/lustre/config.xml --add ost --node host3.test.com --ost ost1 --lov lov1 --dev /home/lustre/ost1 --size 10485760 --failover lmc -m /etc/lustre/config.xml --add ost --node host2.test.com --ost ost2 --lov lov1 --dev /home/lustre/ost2 --size 10485760 --failover lmc -m /etc/lustre/config.xml --add ost --node host3.test.com --ost ost3 --lov lov1 --dev /home/lustre/ost3 --size 10485760 --failover ##Create clients lmc -m /etc/lustre/config.xml --add net --node client --nid ''*'' --nettype tcp lmc -m /etc/lustre/config.xml --add mtpt --node client --path /mnt/lustre --mds mds1 --lov lov1 On 7/12/06, wangdi <wangdi@clusterfs.com> wrote:> > > Hi, > > Could you send me your config.sh? > What is your lconf args when starting lustre? > > thanks > wangdi > > > Prajith Lal wrote: > > Hi, > > After stopping the firewall(iptables) I ran the script. So I think > > there is no firewall problem. The /etc/hosts file is mapped with the > > other IPs. When I tried host1 as MDS and OST and host2, host3 as > > clients it worked. > > Any idea? > > > > Thanks, > > Lal > > > > On 7/12/06, *wangdi* <wangdi@clusterfs.com > > <mailto:wangdi@clusterfs.com>> wrote: > > > > Hi, > > > > It seems MDS can not reach OST. > > Could you check your firewall config and /etc/hosts? > > > > thanks > > wangdi > > > > Prajith Lal wrote: > > > Hi > > > When I try with -v the result is > > > > > > configuring for host: ['' host1.test.com <http://host1.test.com> > > <http://host1.test.com>''] > > > Checking XML modification time > > > + debugfs -c -R ''stat /LOGS'' /home/lustre/mds1 2>&1 | grep mtime > > > setting /proc/sys/net/core/rmem_max to at least 16777216 > > > setting /proc/sys/net/core/wmem_max to at least 16777216 > > > Service: network NET_host1.test.com_tcp .test.com_tcp_UUID > > > loading module: libcfs srcdir None devdir libcfs > > > + /sbin/modprobe libcfs > > > loading module: lnet srcdir None devdir lnet > > > + /sbin/modprobe lnet > > > + /sbin/modprobe lnet > > > loading module: ksocklnd srcdir None devdir klnds/socklnd > > > + /sbin/modprobe ksocklnd > > > Service: ldlm ldlm ldlm_UUID > > > loading module: lvfs srcdir None devdir lvfs > > > + /sbin/modprobe lvfs > > > loading module: obdclass srcdir None devdir obdclass > > > + /sbin/modprobe obdclass > > > loading module: ptlrpc srcdir None devdir ptlrpc > > > + /sbin/modprobe ptlrpc > > > Service: mdsdev MDD_mds1_host1.test.com m1.test.com_UUID_5 > > > original inode_size 0 > > > stripe_count 1 inode_size 512 > > > loading module: mdc srcdir None devdir mdc > > > + /sbin/modprobe mdc > > > loading module: osc srcdir None devdir osc > > > + /sbin/modprobe osc > > > loading module: lov srcdir None devdir lov > > > + /sbin/modprobe lov > > > loading module: mds srcdir None devdir mds > > > + /sbin/modprobe mds > > > loading module: ldiskfs srcdir None devdir ldiskfs > > > + /sbin/modprobe ldiskfs > > > loading module: fsfilt_ldiskfs srcdir None devdir lvfs > > > + /sbin/modprobe fsfilt_ldiskfs > > > + sysctl lnet/debug_path /tmp/lustre-log-host1.test.com > > <http://log-host1.test.com> > > > <http://log-host1.test.com> > > > + /usr/sbin/lctl modules > /tmp/ogdb- host1.test.com > > <http://host1.test.com> > > > <http://host1.test.com> > > > Service: network NET_host1.test.com_tcp .test.com_tcp_UUID > > > NETWORK: NET_host1.test.com_tcp .test.com_tcp_UUID tcp > > host1.test.com <http://host1.test.com> > > > <http://host1.test.com> > > > Service: ldlm ldlm ldlm_UUID > > > Service: mdsdev MDD_mds1_host1.test.com m1.test.com_UUID_5 > > > original inode_size 0 > > > stripe_count 1 inode_size 512 > > > MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no > > > + losetup /dev/loop0 > > > + losetup /dev/loop1 > > > + losetup /dev/loop2 > > > + losetup /dev/loop3 > > > + losetup /dev/loop4 > > > + losetup /dev/loop5 > > > + losetup /dev/loop6 > > > + losetup /dev/loop7 > > > + losetup /dev/loop0 > > > + losetup /dev/loop0 /home/lustre/mds1 > > > + /usr/sbin/lctl > > > attach mdt MDT MDT_UUID > > > quit > > > + /usr/sbin/lctl > > > cfg_device MDT > > > setup > > > quit > > > + dumpe2fs -f -h /dev/loop0 > > > no external journal found for /dev/loop0 > > > MDS mount options: errors=remount-ro > > > + /usr/sbin/lctl > > > attach mds mds1 mds1_UUID > > > quit > > > + /usr/sbin/lctl > > > cfg_device mds1 > > > setup /dev/loop0 ldiskfs mds1 errors=remount-ro > > > quit > > > > > > > > > On 7/10/06, *wangdi* <wangdi@clusterfs.com > > <mailto:wangdi@clusterfs.com> > > > <mailto:wangdi@clusterfs.com <mailto:wangdi@clusterfs.com>>> > wrote: > > > > > > Hi, > > > Could you add -v when you start MDS with lconf and post the > > result > > > here. > > > Then we will know more. > > > > > > thanks > > > wangdi > > > > > > Prajith Lal wrote: > > > > > > > > Hi, > > > > I am trying to setup Lustre FS with one MDS (host1), 4 > > OSTs (0,2 on > > > > host2 and 1,3 on host3) and one client (host4). > > > > My XML is config.xml and I ran the XML file in host2 and > host3 > > > and it > > > > create the OSTs successfully. But when I try to create the > > MDS it > > > > displays the following error message. > > > > > > > > MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no > > > > MDS mount options: errors=remount-ro > > > > MDS failed to start. Check the syslog for details. (May > > need to run > > > > lconf --write-conf) > > > > > > > > when I check the dmesg it shows > > > > LustreError: 6458:0:(handler.c:2015:mds_postsetup()) No > > profile > > > found: > > > > mds1 > > > > LustreError: 6486:0:(obd_config.c:323:class_cleanup()) > > Device 1 > > > not setup > > > > > > > > Can you tell me why this happens? > > > > > > > > Thanks, > > > > Prajith > > > > / > > > > / > > > > > > > > > > > > -- > > > > Lals > > > > > > > > > > ------------------------------------------------------------------------ > > > > > > > > > > > _______________________________________________ > > > > Lustre-discuss mailing list > > > > Lustre-discuss@clusterfs.com > > <mailto:Lustre-discuss@clusterfs.com> > > <mailto:Lustre-discuss@clusterfs.com > > <mailto:Lustre-discuss@clusterfs.com>> > > > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > > > > > > > > > > > > > > > > > > > -- > > > Lals > > > > > > > > > > -- > > Lals > >-- Lals -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20060712/f4f3d574/attachment-0001.html
Prajith Lal
2006-Jul-12 03:12 UTC
[Lustre-discuss] Lustre Configuration with OSTs on different servers
Hi, After stopping the firewall(iptables) I ran the script. So I think there is no firewall problem. The /etc/hosts file is mapped with the other IPs. When I tried host1 as MDS and OST and host2, host3 as clients it worked. Any idea? Thanks, Lal On 7/12/06, wangdi <wangdi@clusterfs.com> wrote:> > Hi, > > It seems MDS can not reach OST. > Could you check your firewall config and /etc/hosts? > > thanks > wangdi > > Prajith Lal wrote: > > Hi > > When I try with -v the result is > > > > configuring for host: [''host1.test.com <http://host1.test.com>''] > > Checking XML modification time > > + debugfs -c -R ''stat /LOGS'' /home/lustre/mds1 2>&1 | grep mtime > > setting /proc/sys/net/core/rmem_max to at least 16777216 > > setting /proc/sys/net/core/wmem_max to at least 16777216 > > Service: network NET_host1.test.com_tcp .test.com_tcp_UUID > > loading module: libcfs srcdir None devdir libcfs > > + /sbin/modprobe libcfs > > loading module: lnet srcdir None devdir lnet > > + /sbin/modprobe lnet > > + /sbin/modprobe lnet > > loading module: ksocklnd srcdir None devdir klnds/socklnd > > + /sbin/modprobe ksocklnd > > Service: ldlm ldlm ldlm_UUID > > loading module: lvfs srcdir None devdir lvfs > > + /sbin/modprobe lvfs > > loading module: obdclass srcdir None devdir obdclass > > + /sbin/modprobe obdclass > > loading module: ptlrpc srcdir None devdir ptlrpc > > + /sbin/modprobe ptlrpc > > Service: mdsdev MDD_mds1_host1.test.com m1.test.com_UUID_5 > > original inode_size 0 > > stripe_count 1 inode_size 512 > > loading module: mdc srcdir None devdir mdc > > + /sbin/modprobe mdc > > loading module: osc srcdir None devdir osc > > + /sbin/modprobe osc > > loading module: lov srcdir None devdir lov > > + /sbin/modprobe lov > > loading module: mds srcdir None devdir mds > > + /sbin/modprobe mds > > loading module: ldiskfs srcdir None devdir ldiskfs > > + /sbin/modprobe ldiskfs > > loading module: fsfilt_ldiskfs srcdir None devdir lvfs > > + /sbin/modprobe fsfilt_ldiskfs > > + sysctl lnet/debug_path /tmp/lustre-log-host1.test.com > > <http://log-host1.test.com> > > + /usr/sbin/lctl modules > /tmp/ogdb-host1.test.com > > <http://host1.test.com> > > Service: network NET_host1.test.com_tcp .test.com_tcp_UUID > > NETWORK: NET_host1.test.com_tcp .test.com_tcp_UUID tcp host1.test.com > > <http://host1.test.com> > > Service: ldlm ldlm ldlm_UUID > > Service: mdsdev MDD_mds1_host1.test.com m1.test.com_UUID_5 > > original inode_size 0 > > stripe_count 1 inode_size 512 > > MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no > > + losetup /dev/loop0 > > + losetup /dev/loop1 > > + losetup /dev/loop2 > > + losetup /dev/loop3 > > + losetup /dev/loop4 > > + losetup /dev/loop5 > > + losetup /dev/loop6 > > + losetup /dev/loop7 > > + losetup /dev/loop0 > > + losetup /dev/loop0 /home/lustre/mds1 > > + /usr/sbin/lctl > > attach mdt MDT MDT_UUID > > quit > > + /usr/sbin/lctl > > cfg_device MDT > > setup > > quit > > + dumpe2fs -f -h /dev/loop0 > > no external journal found for /dev/loop0 > > MDS mount options: errors=remount-ro > > + /usr/sbin/lctl > > attach mds mds1 mds1_UUID > > quit > > + /usr/sbin/lctl > > cfg_device mds1 > > setup /dev/loop0 ldiskfs mds1 errors=remount-ro > > quit > > > > > > On 7/10/06, *wangdi* <wangdi@clusterfs.com > > <mailto:wangdi@clusterfs.com>> wrote: > > > > Hi, > > Could you add -v when you start MDS with lconf and post the result > > here. > > Then we will know more. > > > > thanks > > wangdi > > > > Prajith Lal wrote: > > > > > > Hi, > > > I am trying to setup Lustre FS with one MDS (host1), 4 OSTs (0,2 > on > > > host2 and 1,3 on host3) and one client (host4). > > > My XML is config.xml and I ran the XML file in host2 and host3 > > and it > > > create the OSTs successfully. But when I try to create the MDS it > > > displays the following error message. > > > > > > MDSDEV: mds1 mds1_UUID /home/lustre/mds1 ldiskfs 10485760 no > > > MDS mount options: errors=remount-ro > > > MDS failed to start. Check the syslog for details. (May need to > run > > > lconf --write-conf) > > > > > > when I check the dmesg it shows > > > LustreError: 6458:0:(handler.c:2015:mds_postsetup()) No profile > > found: > > > mds1 > > > LustreError: 6486:0:(obd_config.c:323:class_cleanup()) Device 1 > > not setup > > > > > > Can you tell me why this happens? > > > > > > Thanks, > > > Prajith > > > / > > > / > > > > > > > > > -- > > > Lals > > > > > > ------------------------------------------------------------------------ > > > > > > > > _______________________________________________ > > > Lustre-discuss mailing list > > > Lustre-discuss@clusterfs.com <mailto:Lustre-discuss@clusterfs.com> > > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > > > > > > > > > > > > > -- > > Lals > >-- Lals -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20060711/743c9d18/attachment-0001.html