Hi, I''m interested to find out what possible solutions there are for upgrading storage hardware within a cluster over time, either following a failure or just through nodes coming to the end of their normal working life. We would expect a cluster to exist for many years whereas the individual nodes may only last a few years each. Ideally it should be possible to migrate data off an OST as required but there doesn''t appear to be anything in the manual which covers this use case specifically. The closest thing seems to be in section 4.3.11 of the manual "Removing and Restoring OSTs" (http://manual.lustre.org/manual/LustreManual18_HTML/ConfiguringLustre.h tml#50532400_57420): "OSTs can be removed from and restored to a Lustre file system. Currently in Lustre, removing an OST really means that the OST is ''deactivated'' in the file system, not permanently removed. A removed OST still appears in the file system; do not create a new OST with the same name." Thus one route to migration to new hardware could be to remove an OST (making sure the name is not reused) then use step 2.5 in section 4.3.11.1 to copy to the _new_ hardware, rather than recovering to the same hardware. Does anyone have experience with this type of use case or knowledge of alternative ways of handling this which they could describe for me? Many thanks, Daniel. ******************************************************* British Atmospheric Data Centre STFC Rutherford Appleton Laboratory Chilton, Didcot, Oxfordshire, OX11 0QX ******************************************************* -- Scanned by iCritical. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091116/c24f4cfb/attachment.html
Hi Daniel, I have done this on a live file system by deactivating the OSTs, migrating (see section 26.2 of the manual for a sample script) the data off the OSTs in question, replacing the hardware and migrating it back. _________________________________________ Ron Jerome Programmer/Analyst National Research Council Canada 1200 Montreal Road, Ottawa, Ontario K1A 0R6 Government of Canada From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of daniel.hagon at stfc.ac.uk Sent: November 16, 2009 9:18 AM To: lustre-discuss at lists.lustre.org Subject: [Lustre-discuss] Hardware upgrade routes Hi, I''m interested to find out what possible solutions there are for upgrading storage hardware within a cluster over time, either following a failure or just through nodes coming to the end of their normal working life. We would expect a cluster to exist for many years whereas the individual nodes may only last a few years each. Ideally it should be possible to migrate data off an OST as required but there doesn''t appear to be anything in the manual which covers this use case specifically. The closest thing seems to be in section 4.3.11 of the manual "Removing and Restoring OSTs" (http://manual.lustre.org/manual/LustreManual18_HTML/ConfiguringLustre.h tml#50532400_57420): "OSTs can be removed from and restored to a Lustre file system. Currently in Lustre, removing an OST really means that the OST is ''deactivated'' in the file system, not permanently removed. A removed OST still appears in the file system; do not create a new OST with the same name." Thus one route to migration to new hardware could be to remove an OST (making sure the name is not reused) then use step 2.5 in section 4.3.11.1 to copy to the _new_ hardware, rather than recovering to the same hardware. Does anyone have experience with this type of use case or knowledge of alternative ways of handling this which they could describe for me? Many thanks, Daniel. ******************************************************* British Atmospheric Data Centre STFC Rutherford Appleton Laboratory Chilton, Didcot, Oxfordshire, OX11 0QX ******************************************************* -- Scanned by iCritical. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091116/02bacf48/attachment-0001.html
Hi Jerome, Thanks for pointing out that script - sometimes the obvious answer is there staring you in the face J Were there any major differences in the configurations of the old and new hardware, i.e. did you need to make them appear as similar as possible or was this not necessary? Daniel. From: Jerome, Ron [mailto:Ron.Jerome at nrc-cnrc.gc.ca] Sent: 16 November 2009 14:31 To: Hagon, Daniel (STFC,RAL,SSTD); lustre-discuss at lists.lustre.org Subject: RE: [Lustre-discuss] Hardware upgrade routes Hi Daniel, I have done this on a live file system by deactivating the OSTs, migrating (see section 26.2 of the manual for a sample script) the data off the OSTs in question, replacing the hardware and migrating it back. _________________________________________ Ron Jerome Programmer/Analyst National Research Council Canada 1200 Montreal Road, Ottawa, Ontario K1A 0R6 Government of Canada From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of daniel.hagon at stfc.ac.uk Sent: November 16, 2009 9:18 AM To: lustre-discuss at lists.lustre.org Subject: [Lustre-discuss] Hardware upgrade routes Hi, I''m interested to find out what possible solutions there are for upgrading storage hardware within a cluster over time, either following a failure or just through nodes coming to the end of their normal working life. We would expect a cluster to exist for many years whereas the individual nodes may only last a few years each. Ideally it should be possible to migrate data off an OST as required but there doesn''t appear to be anything in the manual which covers this use case specifically. The closest thing seems to be in section 4.3.11 of the manual "Removing and Restoring OSTs" (http://manual.lustre.org/manual/LustreManual18_HTML/ConfiguringLustre.h tml#50532400_57420): "OSTs can be removed from and restored to a Lustre file system. Currently in Lustre, removing an OST really means that the OST is ''deactivated'' in the file system, not permanently removed. A removed OST still appears in the file system; do not create a new OST with the same name." Thus one route to migration to new hardware could be to remove an OST (making sure the name is not reused) then use step 2.5 in section 4.3.11.1 to copy to the _new_ hardware, rather than recovering to the same hardware. Does anyone have experience with this type of use case or knowledge of alternative ways of handling this which they could describe for me? Many thanks, Daniel. ******************************************************* British Atmospheric Data Centre STFC Rutherford Appleton Laboratory Chilton, Didcot, Oxfordshire, OX11 0QX ******************************************************* -- Scanned by iCritical. -- Scanned by iCritical. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091116/392b14cd/attachment.html
On Mon, 2009-11-16 at 14:17 +0000, daniel.hagon at stfc.ac.uk wrote:> Thus one route to migration to new hardware could be to remove an OST > (making sure the name is not reused) then use step 2.5 in section > 4.3.11.1 to copy to the _new_ hardware, rather than recovering to the > same hardware.You have the right idea there. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091116/b870b610/attachment.bin