Hello , Im trying to implement the perfect 2 nodes cluster(for testing perposes at the moment) and basicly run into 2 problems wich related to redundency with drbd on the OSS/OST What im trying to do is configure the 2 nodes in a way that the 2 backup each other in case 1 crashes The configuration is as follow : Node 1 is MGS/MDT on volume drbd0 Node 2 is the 1 and only OSS/OST for this cluster and mounts drbd1 At this point its all working , tested clients mounts from the 2 nodes and it mounts ok To test some failure situations i tested MGS/MDT failure and OST failures , the heerbeat V2 is doing its part ok and the other node take over the resources of the other node e.g mounts the MGS/MDT or the OST up to now it all works but in such cases that the other node take over the resource of the other node its not possible to mount a client anymore , sometimes the client simply cant mount(manualy after changing the mgs id on the mount command) and sometimes it mounts but cant work with the volume . After watching the system logs for a while iv notices a very strange thing , when the OST fail to the other node it register itslef as new OST with new ID e.g the original OST id was OST0000 then it mounts as OST0003 and so on so the client can mount but look for OST0000 insted of OST0003 cant find it thefor see a cluster with 0 OST''s then the mount is ok but he cant work with empty storage . This behaviour is very strange seens when you format the OST on drbd volume it replicates its structure to its pair so when its pair take over the system see it as the same drive much like a shared storage . Iv also tried to format the OST with --failnode option with no luck , after a node swich over its messing up the all cluster and the client can or cant mount in ither case it take very long to mount a client even with the -o abort_recov swich on the client mount command My guess is that it related to lustre cache schema of storing cluster information upon connection or something with local configuration on the OOS or MGS/MDT that doesnt get replicated to the other node i looked for information on lustre doumentations with not much luck . Is my confuguration even possible ? does anybody run into same issues ? Thanks , ---------------------------------------------------------- PH(2001) Internet Services . -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080126/9a5754da/attachment-0002.html
On Sat, 2008-01-26 at 14:12 +0200, Mailer PH wrote:> After watching the system logs for a while iv notices a very strange > thing , when the OST fail to the other node > it register itslef as new OST with new ID e.g the original OST id was > OST0000 then it mounts as OST0003This is wrong. An OST''s UUID should not change in response to a failover event. It sounds like you don''t really have a (drbd) mirror of the OST on both nodes, or that you are using the wrong drbd device in your configurations.> This behaviour is very strange seens when you format the OST on drbd > volume it replicates its structure to its pair so when its pair take > over > the system see it as the same drive much like a shared storage .That''s how drbd is supposed to work and is why when the failover node takes over the device, the UUID *should* be the same. That it''s not is the problem you need to get to the root of.> Is my confuguration even possible ? does anybody run into same > issues ?What you are trying to accomplish is indeed possible. It sounds like you are just not there yet. I don''t really know much about the particular implementation of drbd so you are going to have to do some reading or get help from the drbd users lists, but I''d suggest you debug drbd first, and confirm absolutely that when you fail a node, the drbd volume on the other node is indeed the same as the failed node. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080128/c2c277c5/attachment-0002.bin