hi All, # grep 10.1.1.11 messages Nov 12 10:34:03 meta1 kernel: LustreError: 3369:0:(ldlm_lockd.c:1556:ldlm_cancel_handler()) operation 103 from 12345-10.1.1.11 at tcp with bad export cookie 7036822694260320730 Nov 13 08:29:47 meta1 kernel: Lustre: 3305:0:(router.c:167:lnet_notify()) Ignoring prediction from 10.1.1.1 at tcp of 10.1.1.11 at tcp down 828593430613 seconds in the future I tried to mount a new node1 to the cluster (mount -t lustre /dev/sdb1 /mnt) and on the node1 seems everything well, but on the meta server I see this on the logs. This is an already working cluster with 5 more nodes. Lustre is 1.6.4.3 on CentOS 5.1. What does it want to say except ''connection aborted''? Thank you, tamas
Hi all, i''m using lustre 1.8.1.1 with 4 OSS each of which mounts 10 ost , i''m trying to use 2 more OSS, and i''d like to move some ost already in use in the new oss. Nobody knows if is it possible, and if yes, nobody knows what i''ve to do? thanks in advance Antonio Concas CRS4 Italy
On Mon, 2009-11-02 at 21:29 +0100, Antonio Concas wrote:> Hi all,Hi,> i''m using lustre 1.8.1.1 with 4 OSS each of which mounts 10 ost ,Wow. 10 OSTs. These OSSes have lots of memory, right? With 10 OSTs, you have 4G of memory being used just for journals.> i''m trying to use 2 more OSS, and i''d like to move some ost already in > use in the new oss.Are you actually relocating or just expanding? (i.e. will the OSTs on the existing OSSes remain?) Or is your hope to reduce the per OSS OST count by adding more OSSes and keeping the OST count the same and just point LUNS at new OSSes? If you are expanding, I''d just create the new OSTs and start doing some "poor mans rebalanceing" to move files on to the new OSTs. If you are doing the latter and just creating more OSSes for existing OSTs, then you will need to follow the procedure in the manual for changing a NID. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091102/88f698ee/attachment.bin
Hi Brian, Each oss has 32 Gb of RAM, and my hope is to reduce the per OSS OST. so, the only thing that i''ve to do is to "Use the writeconf command to erase the configuration logs for the file system. On the MDT" and after that mount ost on new oss? is''it correct? thanks Antonio Brian J. Murrell wrote:> On Mon, 2009-11-02 at 21:29 +0100, Antonio Concas wrote: > >> Hi all, >> > > Hi, > > >> i''m using lustre 1.8.1.1 with 4 OSS each of which mounts 10 ost , >> > > Wow. 10 OSTs. These OSSes have lots of memory, right? With 10 OSTs, > you have 4G of memory being used just for journals. > > >> i''m trying to use 2 more OSS, and i''d like to move some ost already in >> use in the new oss. >> > > Are you actually relocating or just expanding? (i.e. will the OSTs on > the existing OSSes remain?) Or is your hope to reduce the per OSS OST > count by adding more OSSes and keeping the OST count the same and just > point LUNS at new OSSes? > > If you are expanding, I''d just create the new OSTs and start doing some > "poor mans rebalanceing" to move files on to the new OSTs. > > If you are doing the latter and just creating more OSSes for existing > OSTs, then you will need to follow the procedure in the manual for > changing a NID. > > b. > > > ------------------------------------------------------------------------ > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
On Mon, 2009-11-02 at 22:04 +0100, Antonio Concas wrote:> Hi Brian,Hi,> Each oss has 32 Gb of RAM,Ahhh. Good.> and my hope is to reduce the per OSS OST.OK.> so, the only thing that i''ve to do is to > "Use the writeconf command to erase the configuration logs for the file > system. > On the MDT" > > and after that mount ost on new oss?Before you erase configuration logs for a filesystem, you must first unmount all targets (i.e. bring down the filesystem) and then bring them all back up after the writeconf. The details are in the manual I believe. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091102/a3c1ca20/attachment.bin
Brian J. Murrell wrote:> On Mon, 2009-11-02 at 22:04 +0100, Antonio Concas wrote: > >> Hi Brian, >> > > Hi, > > >> Each oss has 32 Gb of RAM, >> > > Ahhh. Good. > > >> and my hope is to reduce the per OSS OST. >> > > OK. > > >> so, the only thing that i''ve to do is to >> "Use the writeconf command to erase the configuration logs for the file >> system. >> On the MDT" >> >> and after that mount ost on new oss? >> > > Before you erase configuration logs for a filesystem, you must first > unmount all targets (i.e. bring down the filesystem) and then bring them > all back up after the writeconf. The details are in the manual I > believe. > >I''ve already umounted all targets, and looking in the manual i found only this: Changing a Server NID To change a server NID: 1. Update the LNET configuration in the /etc/modprobe.conf file so the list of server NIDs (lctl list_nids) is correct. 2. Use the writeconf command to erase the configuration logs for the file system. On the MDT, run: $ mdt> tunefs.lustre --writeconf <mount point> After the writeconf command is run, the configuration logs are re-generated as servers restart, and the current server NIDs are used. 3. If the MGS?s NID was changed, communicate the new MGS location to each server. Run: tunefs.lustre --erase-param --mgsnode=<new_nid(s)> --writeconf /dev/.. only this, so simple?> b. >Antonio