Lustre discuss - Oct 2007 - lustre interoperability

Dear Clusterfs,
I''ve seriously trekked through the lustre documentation and
haven''t
found an answer regarding this. Is there an official policy regarding
interoperability among different versions of various lustre
components?

For example (and I''m sure other groups are in the same boat here),
it''s relatively painless to perform a rolling upgrade to the lustre
clients, but upgrading the OSS or MDS takes more convincing. Is it OK
for me to run a patched but 1.6.0 based OSS with a 1.6.3 client? In
this case all of the lustre components (kernel, lustre, ldiskfs) are
the same version for each host. Similarly, is it OK to run a lustre
kernel version out of sync with the userland tools? For example a
1.6.0 kernel with a 1.6.3 lustre build on the same host?

I understand that many of these combinations do in fact work, I''m more
interested if they''re likely to lead to data corruption or client
evictions. I''m not sure how often incompatibilities arise, but if
it''s
relatively rare, it would be useful if that was announced on the
change log. Of course if there''s a serious "Do at your own
risk"
policy that would also be useful to know.

Thank You,
                     jeff

-- 
Jeff Blasius / jeff.blasius at yale.edu
Phone: (203)432-9940  51 Prospect Rm. 011
High Performance Computing (HPC)
Linux Systems Design & Support (LSDS)
Yale University Information Technology Services (ITS)

Hi Jeff,

On Fri, Oct 12, 2007 at 12:01:04PM -0400, Jeff Blasius wrote:
> I''ve seriously trekked through the lustre documentation and
haven''t
> found an answer regarding this. Is there an official policy regarding
> interoperability among different versions of various lustre
> components?
By coincidence, I just sent information about this to our documentation
team.  It should eventually reach the manual.  Here it is:

---
Our supported upgrades are from one minor version to another, for
example 1.4.10 to 1.4.11 or 1.6.2 to 1.6.3, and also from the latest
1.4.x version to the latest 1.6.x version, so 1.4.11 to 1.6.3 is
supported.

We also support downgrades within the same ranges.  For example, if you
upgrade from 1.6.2 to 1.6.3, you can also downgrade to 1.6.2 (but a
fresh install of 1.6.3 is _not_ guaranteed to be downgradeable.)

Note that other combinations will work and we support them in specific
cases when requested by customers, but the ranges above will always
work.
---
> For example (and I''m sure other groups are in the same boat here),
> it''s relatively painless to perform a rolling upgrade to the
lustre
> clients, but upgrading the OSS or MDS takes more convincing. Is it OK
> for me to run a patched but 1.6.0 based OSS with a 1.6.3 client? In
> this case all of the lustre components (kernel, lustre, ldiskfs) are
> the same version for each host. Similarly, is it OK to run a lustre
> kernel version out of sync with the userland tools? For example a
> 1.6.0 kernel with a 1.6.3 lustre build on the same host?
Not necessarily.  You should do a rolling upgrade to 1.6.1, then 1.6.2,
then 1.6.3.  Upgrading will be easier if you stay more current - 1.6.0
is fairly old at this point.

Having said that, I believe 1.6.0 and 1.6.3 is actually something that
will work, but I''m not 100% certain of this, so I''ll allow
others to
correct me.

Cheers,
Jody
> I understand that many of these combinations do in fact work, I''m
more
> interested if they''re likely to lead to data corruption or client
> evictions. I''m not sure how often incompatibilities arise, but if
it''s
> relatively rare, it would be useful if that was announced on the
> change log. Of course if there''s a serious "Do at your own
risk"
> policy that would also be useful to know.
> 
> Thank You,
>                      jeff
> 
> -- 
> Jeff Blasius / jeff.blasius at yale.edu
> Phone: (203)432-9940  51 Prospect Rm. 011
> High Performance Computing (HPC)
> Linux Systems Design & Support (LSDS)
> Yale University Information Technology Services (ITS)
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> 
--

Hello,

I am working on incorporating failover and high availability functionality
with Lustre into one of our new research clusters.

I am also a bit of a lazy sysadmin, and prefer to not have to modify much on
a day-to-day basis. I am using the ROCKS cluster framework in conjunction
with Lustre, so a good deal of the work is handled in an automated fashion.

I have a question, though ...

I have redundant MDS/MDT servers, and when generating their /etc/fstab
files, I have them both list the MDT volume under their mounted drives (so
that they both mount the metadata volume). Being a bit of a neophyte, I?m
not sure if this is the right thing to do. Is this going to cause a problem
in future? What are the possible consequences of both hosts mounting the
same metadata volume simultaneously? How do the OSTs handle this situation?

thanks,
Klaus
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20071012/f19a6c6b/attachment-0002.html

Hi Klaus,

On Friday 12 October 2007 10:18:49 am Klaus Steden
wrote:
> I have redundant MDS/MDT servers, and when generating their
> /etc/fstab files, I have them both list the MDT volume under their
> mounted drives (so that they both mount the metadata volume). Being a
> bit of a neophyte, I?m not sure if this is the right thing to do. Is
> this going to cause a problem in future? What are the possible
> consequences of both hosts mounting the same metadata volume
> simultaneously? How do the OSTs handle this situation?
I''m afraid mounting the MDT device on both hosts is a very bad idea.
Not
only for Lustre, but in a general fashion, unless you''re using some 
specific filesystem which has been designed in that goal (like OCFS2 or 
GFS). A ''standard'' filesystem like ext3 can''t handle
simultaneous
writes from different hosts at the same time. It''s likely to cause 
heavy data corruption in the MDT filesystem, and the OSTs are not 
supposed to deal with anything like that.

You can still reference the MDT device in both MDS'' fstab, but be sure 
you add the "noauto" option so the volume won''t be mounted at
boot. It
will be the high-availability provider''s job (likely heartbeat) to deal
with mounting the MDT device on the active MDS.

See 
http://manual.lustre.org/manual/LustreManual16_HTML/DynamicHTML-08-1.html#wp1035482
for more details.

Cheers,
-- 
Kilian

Heheheh, I figured.

What about mounting OST targets from multiple hosts, i.e. OST0000 and
OST0001 on both oss-0-0 and oss-0-1? Just as bad?

Klaus

On 10/12/07 10:38 AM, "Kilian CAVALOTTI" <kilian at
stanford.edu>did etch on
stone tablets:
> Hi Klaus,
> 
> On Friday 12 October 2007 10:18:49 am Klaus Steden wrote:
>> > I have redundant MDS/MDT servers, and when generating their
>> > /etc/fstab files, I have them both list the MDT volume under their
>> > mounted drives (so that they both mount the metadata volume).
Being a
>> > bit of a neophyte, I?m not sure if this is the right thing to do.
Is
>> > this going to cause a problem in future? What are the possible
>> > consequences of both hosts mounting the same metadata volume
>> > simultaneously? How do the OSTs handle this situation?
> 
> I''m afraid mounting the MDT device on both hosts is a very bad
idea. Not
> only for Lustre, but in a general fashion, unless you''re using
some
> specific filesystem which has been designed in that goal (like OCFS2 or
> GFS). A ''standard'' filesystem like ext3 can''t
handle simultaneous
> writes from different hosts at the same time. It''s likely to cause
> heavy data corruption in the MDT filesystem, and the OSTs are not
> supposed to deal with anything like that.
> 
> You can still reference the MDT device in both MDS'' fstab, but be
sure
> you add the "noauto" option so the volume won''t be
mounted at boot. It
> will be the high-availability provider''s job (likely heartbeat) to
deal
> with mounting the MDT device on the active MDS.
> 
> See
>
http://manual.lustre.org/manual/LustreManual16_HTML/DynamicHTML-08-1.html#wp10
> 35482
> for more details.
> 
> Cheers,
> --
> Kilian
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20071012/4bc9660f/attachment-0002.html

On Friday 12 October 2007 11:15:16 am Klaus Steden
wrote:
> Heheheh, I figured.
>
> What about mounting OST targets from multiple hosts, i.e. OST0000 and
> OST0001 on both oss-0-0 and oss-0-1? Just as bad?
Yep, for the same reasons.

If you want redundancy for your OSTs, you can have a kind of 
active/active setup where oss-0-0 would be active for OST0000 and 
passive for OST0001, and oss-0-1 would be passive for OST0000 and 
active for OST0001.

Cheers,
-- 
Kilian

Jody -

Not a problem to add a section in the Lustre manual re: supported upgrades.
However, as we do not currently sync Lustre releases with releases of the
Lustre manual, this wouldn''t help us to address the issue that Jeff
poses.

For future Lustre releases, could we add a supported upgrade/interop
statement to the change log? I''ll also add this content to the manual
(which
will have more value once Lustre s/w and manual releases are synchronized).

Sheila

On 10/12/07, Jody McIntyre <scjody at clusterfs.com>
wrote:
>
> Hi Jeff,
>
> On Fri, Oct 12, 2007 at 12:01:04PM -0400, Jeff Blasius wrote:
>
> > I''ve seriously trekked through the lustre documentation and
haven''t
> > found an answer regarding this. Is there an official policy regarding
> > interoperability among different versions of various lustre
> > components?
>
> By coincidence, I just sent information about this to our documentation
> team.  It should eventually reach the manual.  Here it is:
>
> ---
> Our supported upgrades are from one minor version to another, for
> example 1.4.10 to 1.4.11 or 1.6.2 to 1.6.3, and also from the latest
> 1.4.x version to the latest 1.6.x version, so 1.4.11 to 1.6.3 is
> supported.
>
> We also support downgrades within the same ranges.  For example, if you
> upgrade from 1.6.2 to 1.6.3, you can also downgrade to 1.6.2 (but a
> fresh install of 1.6.3 is _not_ guaranteed to be downgradeable.)
>
> Note that other combinations will work and we support them in specific
> cases when requested by customers, but the ranges above will always
> work.
> ---
>
> > For example (and I''m sure other groups are in the same boat
here),
> > it''s relatively painless to perform a rolling upgrade to the
lustre
> > clients, but upgrading the OSS or MDS takes more convincing. Is it OK
> > for me to run a patched but 1.6.0 based OSS with a 1.6.3 client? In
> > this case all of the lustre components (kernel, lustre, ldiskfs) are
> > the same version for each host. Similarly, is it OK to run a lustre
> > kernel version out of sync with the userland tools? For example a
> > 1.6.0 kernel with a 1.6.3 lustre build on the same host?
>
> Not necessarily.  You should do a rolling upgrade to 1.6.1, then 1.6.2,
> then 1.6.3.  Upgrading will be easier if you stay more current - 1.6.0
> is fairly old at this point.
>
> Having said that, I believe 1.6.0 and 1.6.3 is actually something that
> will work, but I''m not 100% certain of this, so I''ll
allow others to
> correct me.
>
> Cheers,
> Jody
>
> > I understand that many of these combinations do in fact work,
I''m more
> > interested if they''re likely to lead to data corruption or
client
> > evictions. I''m not sure how often incompatibilities arise,
but if it''s
> > relatively rare, it would be useful if that was announced on the
> > change log. Of course if there''s a serious "Do at your
own risk"
> > policy that would also be useful to know.
> >
> > Thank You,
> >                      jeff
> >
> > --
> > Jeff Blasius / jeff.blasius at yale.edu
> > Phone: (203)432-9940  51 Prospect Rm. 011
> > High Performance Computing (HPC)
> > Linux Systems Design & Support (LSDS)
> > Yale University Information Technology Services (ITS)
> >
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at clusterfs.com
> > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> >
>
> --
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20071012/9ec04866/attachment-0002.html

Neither of these is something that can be easily accomplished from
/etc/fstab, I would have to implement the failover functionality described
in the link you sent, correct?

Klaus


On 10/12/07 11:21 AM, "Kilian CAVALOTTI" <kilian at
stanford.edu>did etch on
stone tablets:
> On Friday 12 October 2007 11:15:16 am Klaus Steden wrote:
>> > Heheheh, I figured.
>> >
>> > What about mounting OST targets from multiple hosts, i.e. OST0000
and
>> > OST0001 on both oss-0-0 and oss-0-1? Just as bad?
> 
> Yep, for the same reasons.
> 
> If you want redundancy for your OSTs, you can have a kind of
> active/active setup where oss-0-0 would be active for OST0000 and
> passive for OST0001, and oss-0-1 would be passive for OST0000 and
> active for OST0001.
> 
> Cheers,
> --
> Kilian
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20071012/6edb1b79/attachment-0002.html

Hello again,

Another (hopefully) quick question:

When designing a cluster with high availability using Lustre ... is there a
recommended location for the MGS? Are there advantages to housing the MGS
information on the same volume as metadata?

thanks,
Klaus
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20071012/c133d7fa/attachment-0002.html

On Oct 12, 2007  11:50 -0700, Klaus Steden wrote:
> Neither of these is something that can be easily accomplished from
> /etc/fstab, I would have to implement the failover functionality described
> in the link you sent, correct?
Correct.  If you are using failover then you also need to have the HA
software manage the filesystem mounting.  With 1.6 this is relatively
easy, as filesystem failover is a fairly standard configuration (ala
NFS server failover also).
> On 10/12/07 11:21 AM, "Kilian CAVALOTTI" <kilian at
stanford.edu>did etch on
> stone tablets:
> > On Friday 12 October 2007 11:15:16 am Klaus Steden wrote:
> >> > Heheheh, I figured.
> >> >
> >> > What about mounting OST targets from multiple hosts, i.e.
OST0000 and
> >> > OST0001 on both oss-0-0 and oss-0-1? Just as bad?
> > 
> > Yep, for the same reasons.
> > 
> > If you want redundancy for your OSTs, you can have a kind of
> > active/active setup where oss-0-0 would be active for OST0000 and
> > passive for OST0001, and oss-0-1 would be passive for OST0000 and
> > active for OST0001.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Hi Jody,
Thank you for this information. Can or someone else comment on what it
means to upgrade? I assume it''s not enough to simply build the
software or update via rpm. After updating, you should go through the
process of having all of the OSS''s join the MGS/MDS and verify a
successful client connection?

Just for my curiosity, is there a standard process the server
components undergo during the upgrade?
Thank You,
                         jeff

On 10/12/07, Jody McIntyre <scjody at clusterfs.com>
wrote:
> Hi Jeff,
>
> On Fri, Oct 12, 2007 at 12:01:04PM -0400, Jeff Blasius wrote:
>
> > I''ve seriously trekked through the lustre documentation and
haven''t
> > found an answer regarding this. Is there an official policy regarding
> > interoperability among different versions of various lustre
> > components?
>
> By coincidence, I just sent information about this to our documentation
> team.  It should eventually reach the manual.  Here it is:
>
> ---
> Our supported upgrades are from one minor version to another, for
> example 1.4.10 to 1.4.11 or 1.6.2 to 1.6.3, and also from the latest
> 1.4.x version to the latest 1.6.x version, so 1.4.11 to 1.6.3 is
> supported.
>
> We also support downgrades within the same ranges.  For example, if you
> upgrade from 1.6.2 to 1.6.3, you can also downgrade to 1.6.2 (but a
> fresh install of 1.6.3 is _not_ guaranteed to be downgradeable.)
>
> Note that other combinations will work and we support them in specific
> cases when requested by customers, but the ranges above will always
> work.
> ---
>
> > For example (and I''m sure other groups are in the same boat
here),
> > it''s relatively painless to perform a rolling upgrade to the
lustre
> > clients, but upgrading the OSS or MDS takes more convincing. Is it OK
> > for me to run a patched but 1.6.0 based OSS with a 1.6.3 client? In
> > this case all of the lustre components (kernel, lustre, ldiskfs) are
> > the same version for each host. Similarly, is it OK to run a lustre
> > kernel version out of sync with the userland tools? For example a
> > 1.6.0 kernel with a 1.6.3 lustre build on the same host?
>
> Not necessarily.  You should do a rolling upgrade to 1.6.1, then 1.6.2,
> then 1.6.3.  Upgrading will be easier if you stay more current - 1.6.0
> is fairly old at this point.
>
> Having said that, I believe 1.6.0 and 1.6.3 is actually something that
> will work, but I''m not 100% certain of this, so I''ll
allow others to
> correct me.
>
> Cheers,
> Jody
>
> > I understand that many of these combinations do in fact work,
I''m more
> > interested if they''re likely to lead to data corruption or
client
> > evictions. I''m not sure how often incompatibilities arise,
but if it''s
> > relatively rare, it would be useful if that was announced on the
> > change log. Of course if there''s a serious "Do at your
own risk"
> > policy that would also be useful to know.
> >
> > Thank You,
> >                      jeff
> >
> > --
> > Jeff Blasius / jeff.blasius at yale.edu
> > Phone: (203)432-9940  51 Prospect Rm. 011
> > High Performance Computing (HPC)
> > Linux Systems Design & Support (LSDS)
> > Yale University Information Technology Services (ITS)
> >
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at clusterfs.com
> > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> >
>
> --
>

-- 
Jeff Blasius / jeff.blasius at yale.edu
Phone: (203)432-9940  51 Prospect Rm. 011
High Performance Computing (HPC)
UNIX Systems Administrator, Linux Systems Design & Support (LSDS)
Yale University Information Technology Services (ITS)

Hi Sheila,

On Fri, Oct 12, 2007 at 12:27:23PM -0600, Sheila Barthel wrote:
> Not a problem to add a section in the Lustre manual re: supported upgrades.
> However, as we do not currently sync Lustre releases with releases of the
> Lustre manual, this wouldn''t help us to address the issue that
Jeff poses.
I don''t think we need to list specific versions or anything.  Just
include the general policies I have outlined below as well as some
examples - that way the information won''t need to be revised regularly.

Cheers,
Jody
> For future Lustre releases, could we add a supported upgrade/interop
> statement to the change log? I''ll also add this content to the
manual (which
> will have more value once Lustre s/w and manual releases are synchronized).
> 
> Sheila
> 
> On 10/12/07, Jody McIntyre <scjody at clusterfs.com> wrote:
> >
> > Hi Jeff,
> >
> > On Fri, Oct 12, 2007 at 12:01:04PM -0400, Jeff Blasius wrote:
> >
> > > I''ve seriously trekked through the lustre documentation
and haven''t
> > > found an answer regarding this. Is there an official policy
regarding
> > > interoperability among different versions of various lustre
> > > components?
> >
> > By coincidence, I just sent information about this to our
documentation
> > team.  It should eventually reach the manual.  Here it is:
> >
> > ---
> > Our supported upgrades are from one minor version to another, for
> > example 1.4.10 to 1.4.11 or 1.6.2 to 1.6.3, and also from the latest
> > 1.4.x version to the latest 1.6.x version, so 1.4.11 to 1.6.3 is
> > supported.
> >
> > We also support downgrades within the same ranges.  For example, if
you
> > upgrade from 1.6.2 to 1.6.3, you can also downgrade to 1.6.2 (but a
> > fresh install of 1.6.3 is _not_ guaranteed to be downgradeable.)
> >
> > Note that other combinations will work and we support them in specific
> > cases when requested by customers, but the ranges above will always
> > work.
> > ---
> >
> > > For example (and I''m sure other groups are in the same
boat here),
> > > it''s relatively painless to perform a rolling upgrade to
the lustre
> > > clients, but upgrading the OSS or MDS takes more convincing. Is
it OK
> > > for me to run a patched but 1.6.0 based OSS with a 1.6.3 client?
In
> > > this case all of the lustre components (kernel, lustre, ldiskfs)
are
> > > the same version for each host. Similarly, is it OK to run a
lustre
> > > kernel version out of sync with the userland tools? For example a
> > > 1.6.0 kernel with a 1.6.3 lustre build on the same host?
> >
> > Not necessarily.  You should do a rolling upgrade to 1.6.1, then
1.6.2,
> > then 1.6.3.  Upgrading will be easier if you stay more current - 1.6.0
> > is fairly old at this point.
> >
> > Having said that, I believe 1.6.0 and 1.6.3 is actually something that
> > will work, but I''m not 100% certain of this, so I''ll
allow others to
> > correct me.
> >
> > Cheers,
> > Jody
> >
> > > I understand that many of these combinations do in fact work,
I''m more
> > > interested if they''re likely to lead to data corruption
or client
> > > evictions. I''m not sure how often incompatibilities
arise, but if it''s
> > > relatively rare, it would be useful if that was announced on the
> > > change log. Of course if there''s a serious "Do at
your own risk"
> > > policy that would also be useful to know.
> > >
> > > Thank You,
> > >                      jeff
> > >
> > > --
> > > Jeff Blasius / jeff.blasius at yale.edu
> > > Phone: (203)432-9940  51 Prospect Rm. 011
> > > High Performance Computing (HPC)
> > > Linux Systems Design & Support (LSDS)
> > > Yale University Information Technology Services (ITS)
> > >
> > > _______________________________________________
> > > Lustre-discuss mailing list
> > > Lustre-discuss at clusterfs.com
> > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> > >
> >
> > --
> >
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at clusterfs.com
> > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> >
> >
--

Hi Jeff,

On Sun, Oct 14, 2007 at 11:31:45PM -0400, Jeff Blasius wrote:
> Thank you for this information. Can or someone else comment on what it
> means to upgrade? I assume it''s not enough to simply build the
> software or update via rpm. After updating, you should go through the
> process of having all of the OSS''s join the MGS/MDS and verify a
> successful client connection?
This part is in the manual :)  See:
http://manual.lustre.org/manual/LustreManual16_HTML/DynamicHTML-13-1.html
> Just for my curiosity, is there a standard process the server
> components undergo during the upgrade?
I don''t understand what you mean here.  Can you explain your question?

Cheers,
Jody
> Thank You,
>                          jeff
> 
> On 10/12/07, Jody McIntyre <scjody at clusterfs.com> wrote:
> > Hi Jeff,
> >
> > On Fri, Oct 12, 2007 at 12:01:04PM -0400, Jeff Blasius wrote:
> >
> > > I''ve seriously trekked through the lustre documentation
and haven''t
> > > found an answer regarding this. Is there an official policy
regarding
> > > interoperability among different versions of various lustre
> > > components?
> >
> > By coincidence, I just sent information about this to our
documentation
> > team.  It should eventually reach the manual.  Here it is:
> >
> > ---
> > Our supported upgrades are from one minor version to another, for
> > example 1.4.10 to 1.4.11 or 1.6.2 to 1.6.3, and also from the latest
> > 1.4.x version to the latest 1.6.x version, so 1.4.11 to 1.6.3 is
> > supported.
> >
> > We also support downgrades within the same ranges.  For example, if
you
> > upgrade from 1.6.2 to 1.6.3, you can also downgrade to 1.6.2 (but a
> > fresh install of 1.6.3 is _not_ guaranteed to be downgradeable.)
> >
> > Note that other combinations will work and we support them in specific
> > cases when requested by customers, but the ranges above will always
> > work.
> > ---
> >
> > > For example (and I''m sure other groups are in the same
boat here),
> > > it''s relatively painless to perform a rolling upgrade to
the lustre
> > > clients, but upgrading the OSS or MDS takes more convincing. Is
it OK
> > > for me to run a patched but 1.6.0 based OSS with a 1.6.3 client?
In
> > > this case all of the lustre components (kernel, lustre, ldiskfs)
are
> > > the same version for each host. Similarly, is it OK to run a
lustre
> > > kernel version out of sync with the userland tools? For example a
> > > 1.6.0 kernel with a 1.6.3 lustre build on the same host?
> >
> > Not necessarily.  You should do a rolling upgrade to 1.6.1, then
1.6.2,
> > then 1.6.3.  Upgrading will be easier if you stay more current - 1.6.0
> > is fairly old at this point.
> >
> > Having said that, I believe 1.6.0 and 1.6.3 is actually something that
> > will work, but I''m not 100% certain of this, so I''ll
allow others to
> > correct me.
> >
> > Cheers,
> > Jody
> >
> > > I understand that many of these combinations do in fact work,
I''m more
> > > interested if they''re likely to lead to data corruption
or client
> > > evictions. I''m not sure how often incompatibilities
arise, but if it''s
> > > relatively rare, it would be useful if that was announced on the
> > > change log. Of course if there''s a serious "Do at
your own risk"
> > > policy that would also be useful to know.
> > >
> > > Thank You,
> > >                      jeff
> > >
> > > --
> > > Jeff Blasius / jeff.blasius at yale.edu
> > > Phone: (203)432-9940  51 Prospect Rm. 011
> > > High Performance Computing (HPC)
> > > Linux Systems Design & Support (LSDS)
> > > Yale University Information Technology Services (ITS)
> > >
> > > _______________________________________________
> > > Lustre-discuss mailing list
> > > Lustre-discuss at clusterfs.com
> > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> > >
> >
> > --
> >
> 
> 
> -- 
> Jeff Blasius / jeff.blasius at yale.edu
> Phone: (203)432-9940  51 Prospect Rm. 011
> High Performance Computing (HPC)
> UNIX Systems Administrator, Linux Systems Design & Support (LSDS)
> Yale University Information Technology Services (ITS)
> 
--

Could somebody (Andreas maybe :) give a definitive answer on this...
> > Having said that, I believe 1.6.0 and 1.6.3 is actually something
that
> > will work, but I''m not 100% certain of this, so I''ll
allow others to
> > correct me.
... as I would like to do just this, go from 1.6.0.1 to 1.6.3 and would
rather not have to go through the intermediate versions if I don''t have
to.

Thanks in advance. 

Ron Jerome
National Research Council Canada

> -----Original Message-----
> From: lustre-discuss-bounces at clusterfs.com [mailto:lustre-discuss-
> bounces at clusterfs.com] On Behalf Of Jody McIntyre
> Sent: October 15, 2007 11:58 PM
> To: Jeff Blasius
> Cc: lustre-discuss at clusterfs.com
> Subject: Re: [Lustre-discuss] lustre interoperability
> 
> Hi Jeff,
> 
> On Sun, Oct 14, 2007 at 11:31:45PM -0400, Jeff Blasius wrote:
> 
> > Thank you for this information. Can or someone else comment on what
> it
> > means to upgrade? I assume it''s not enough to simply build
the
> > software or update via rpm. After updating, you should go through
the
> > process of having all of the OSS''s join the MGS/MDS and
verify a
> > successful client connection?
> 
> This part is in the manual :)  See:
> http://manual.lustre.org/manual/LustreManual16_HTML/DynamicHTML-13-
> 1.html
> 
> > Just for my curiosity, is there a standard process the server
> > components undergo during the upgrade?
> 
> I don''t understand what you mean here.  Can you explain your
question?
> 
> Cheers,
> Jody
> 
> > Thank You,
> >                          jeff
> >
> > On 10/12/07, Jody McIntyre <scjody at clusterfs.com> wrote:
> > > Hi Jeff,
> > >
> > > On Fri, Oct 12, 2007 at 12:01:04PM -0400, Jeff Blasius wrote:
> > >
> > > > I''ve seriously trekked through the lustre
documentation and
> haven''t
> > > > found an answer regarding this. Is there an official policy
> regarding
> > > > interoperability among different versions of various lustre
> > > > components?
> > >
> > > By coincidence, I just sent information about this to our
> documentation
> > > team.  It should eventually reach the manual.  Here it is:
> > >
> > > ---
> > > Our supported upgrades are from one minor version to another, for
> > > example 1.4.10 to 1.4.11 or 1.6.2 to 1.6.3, and also from the
> latest
> > > 1.4.x version to the latest 1.6.x version, so 1.4.11 to 1.6.3 is
> > > supported.
> > >
> > > We also support downgrades within the same ranges.  For example,
if
> you
> > > upgrade from 1.6.2 to 1.6.3, you can also downgrade to 1.6.2 (but
a
> > > fresh install of 1.6.3 is _not_ guaranteed to be downgradeable.)
> > >
> > > Note that other combinations will work and we support them in
> specific
> > > cases when requested by customers, but the ranges above will
always
> > > work.
> > > ---
> > >
> > > > For example (and I''m sure other groups are in the
same boat
> here),
> > > > it''s relatively painless to perform a rolling
upgrade to the
> lustre
> > > > clients, but upgrading the OSS or MDS takes more convincing.
Is
> it OK
> > > > for me to run a patched but 1.6.0 based OSS with a 1.6.3
client?
> In
> > > > this case all of the lustre components (kernel, lustre,
ldiskfs)
> are
> > > > the same version for each host. Similarly, is it OK to run a
> lustre
> > > > kernel version out of sync with the userland tools? For
example
a
> > > > 1.6.0 kernel with a 1.6.3 lustre build on the same host?
> > >
> > > Not necessarily.  You should do a rolling upgrade to 1.6.1, then
> 1.6.2,
> > > then 1.6.3.  Upgrading will be easier if you stay more current -
> 1.6.0
> > > is fairly old at this point.
> > >
> > > Having said that, I believe 1.6.0 and 1.6.3 is actually something
> that
> > > will work, but I''m not 100% certain of this, so
I''ll allow others
> to
> > > correct me.
> > >
> > > Cheers,
> > > Jody
> > >
> > > > I understand that many of these combinations do in fact
work,
I''m
> more
> > > > interested if they''re likely to lead to data
corruption or
client
> > > > evictions. I''m not sure how often incompatibilities
arise, but
if
> it''s
> > > > relatively rare, it would be useful if that was announced on
the
> > > > change log. Of course if there''s a serious "Do
at your own risk"
> > > > policy that would also be useful to know.
> > > >
> > > > Thank You,
> > > >                      jeff
> > > >
> > > > --
> > > > Jeff Blasius / jeff.blasius at yale.edu
> > > > Phone: (203)432-9940  51 Prospect Rm. 011
> > > > High Performance Computing (HPC)
> > > > Linux Systems Design & Support (LSDS)
> > > > Yale University Information Technology Services (ITS)
> > > >
> > > > _______________________________________________
> > > > Lustre-discuss mailing list
> > > > Lustre-discuss at clusterfs.com
> > > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> > > >
> > >
> > > --
> > >
> >
> >
> > --
> > Jeff Blasius / jeff.blasius at yale.edu
> > Phone: (203)432-9940  51 Prospect Rm. 011
> > High Performance Computing (HPC)
> > UNIX Systems Administrator, Linux Systems Design & Support (LSDS)
> > Yale University Information Technology Services (ITS)
> >
> 
> --
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss

On Oct 18, 2007  10:49 -0400, Jerome, Ron wrote:
> Could somebody (Andreas maybe :) give a definitive answer on this...
> 
> > > Having said that, I believe 1.6.0 and 1.6.3 is actually something
> that
> > > will work, but I''m not 100% certain of this, so
I''ll allow others to
> > > correct me.
> 
> ... as I would like to do just this, go from 1.6.0.1 to 1.6.3 and would
> rather not have to go through the intermediate versions if I don''t
have
> to.
I''m not aware of any incompatibilities between 1.6.0 and 1.6.3, and we
have a good mechanism to handle protocol negotiation when we add features
to Lustre (since 1.4.2).  That said, we can''t possibly do an all-to-all
test matrix, so we only test against the previous minor version (1.6.3
against 1.6.2) and the upgrade from the most recent major version (1.4.12).

There have been no changes to the on-disk format since 1.4 days (excluding
the config logs) that I''m aware of.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Thanks Andreas,
 
So I guess that means I can give it a try and hope for the best :-) or would I
be wiser to go step by step given this is a production system serving 32 TB?
 
Ron.

________________________________

From: lustre-discuss-bounces at clusterfs.com on behalf of Andreas Dilger
Sent: Thu 10/18/2007 5:56 PM
To: Jerome, Ron
Cc: lustre-discuss at clusterfs.com
Subject: Re: [Lustre-discuss] lustre interoperability



On Oct 18, 2007  10:49 -0400, Jerome, Ron wrote:
> Could somebody (Andreas maybe :) give a definitive answer on this...
>
> > > Having said that, I believe 1.6.0 and 1.6.3 is actually something
> that
> > > will work, but I''m not 100% certain of this, so
I''ll allow others to
> > > correct me.
>
> ... as I would like to do just this, go from 1.6.0.1 to 1.6.3 and would
> rather not have to go through the intermediate versions if I don''t
have
> to.
I''m not aware of any incompatibilities between 1.6.0 and 1.6.3, and we
have a good mechanism to handle protocol negotiation when we add features
to Lustre (since 1.4.2).  That said, we can''t possibly do an all-to-all
test matrix, so we only test against the previous minor version (1.6.3
against 1.6.2) and the upgrade from the most recent major version (1.4.12).

There have been no changes to the on-disk format since 1.4 days (excluding
the config logs) that I''m aware of.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at clusterfs.com
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20071018/44a43dd6/attachment-0002.html

Jerome, Ron wrote:
> Thanks Andreas,
>  
> So I guess that means I can give it a try and hope for the best :-) or
would I be wiser to go step by step given this is a production system serving 32
TB?
>  
> Ron.
> 
You should be able to jump from 1.6.0 to 1.6.3 without issues.
As Andreas said, we do not test more than one minor version, but there
are no changes in any protocol, configuration, or on-disk data.

We try very hard to make great big announcements/release notes when we 
_do make major changes.

cliffw
> ________________________________
> 
> From: lustre-discuss-bounces at clusterfs.com on behalf of Andreas Dilger
> Sent: Thu 10/18/2007 5:56 PM
> To: Jerome, Ron
> Cc: lustre-discuss at clusterfs.com
> Subject: Re: [Lustre-discuss] lustre interoperability
> 
> 
> 
> On Oct 18, 2007  10:49 -0400, Jerome, Ron wrote:
>> Could somebody (Andreas maybe :) give a definitive answer on this...
>>
>>>> Having said that, I believe 1.6.0 and 1.6.3 is actually
something
>> that
>>>> will work, but I''m not 100% certain of this, so
I''ll allow others to
>>>> correct me.
>> ... as I would like to do just this, go from 1.6.0.1 to 1.6.3 and would
>> rather not have to go through the intermediate versions if I
don''t have
>> to.
> 
> I''m not aware of any incompatibilities between 1.6.0 and 1.6.3,
and we
> have a good mechanism to handle protocol negotiation when we add features
> to Lustre (since 1.4.2).  That said, we can''t possibly do an
all-to-all
> test matrix, so we only test against the previous minor version (1.6.3
> against 1.6.2) and the upgrade from the most recent major version (1.4.12).
> 
> There have been no changes to the on-disk format since 1.4 days (excluding
> the config logs) that I''m aware of.
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Principal Software Engineer
> Cluster File Systems, Inc.
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> 
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss

Just for the record...
 
I successfully upgraded from 1.6.0.1 to 1.6.3 in a single step without issue.  
 
Thanks to all who provided input on this.
 
Ron.

________________________________

From: lustre-discuss-bounces at clusterfs.com on behalf of Jerome, Ron
Sent: Thu 10/18/2007 9:33 PM
To: lustre-discuss at clusterfs.com
Subject: Re: [Lustre-discuss] lustre interoperability


Thanks Andreas,
 
So I guess that means I can give it a try and hope for the best :-) or would I
be wiser to go step by step given this is a production system serving 32 TB?
 
Ron.

________________________________

From: lustre-discuss-bounces at clusterfs.com on behalf of Andreas Dilger
Sent: Thu 10/18/2007 5:56 PM
To: Jerome, Ron
Cc: lustre-discuss at clusterfs.com
Subject: Re: [Lustre-discuss] lustre interoperability



On Oct 18, 2007  10:49 -0400, Jerome, Ron wrote:
> Could somebody (Andreas maybe :) give a definitive answer on this...
>
> > > Having said that, I believe 1.6.0 and 1.6.3 is actually something
> that
> > > will work, but I''m not 100% certain of this, so
I''ll allow others to
> > > correct me.
>
> ... as I would like to do just this, go from 1.6.0.1 to 1.6.3 and would
> rather not have to go through the intermediate versions if I don''t
have
> to.
I''m not aware of any incompatibilities between 1.6.0 and 1.6.3, and we
have a good mechanism to handle protocol negotiation when we add features
to Lustre (since 1.4.2).  That said, we can''t possibly do an all-to-all
test matrix, so we only test against the previous minor version (1.6.3
against 1.6.2) and the upgrade from the most recent major version (1.4.12).

There have been no changes to the on-disk format since 1.4 days (excluding
the config logs) that I''m aware of.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at clusterfs.com
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20071022/19d1d10e/attachment-0002.html

Yes, that will work fine.
Well, let me rephrase: there is no reason I can think of why it
wouldn''t
work.  But we don''t generally test interoperability -- the only regular
testing we do is with 1.4.x to the latest release.  Let me also say that 
we almost never change on-disk formats, so if you''re upgrading your 
entire cluster from 1.6.0.1 to 1.6.3 then there is in fact no 
"interoperability" issue at all; it''s all 1.6.3.  Indeed, it
is just
updating the RPMs.  You don''t have to --writeconf or anything like
that.


Jerome, Ron wrote:
> Could somebody (Andreas maybe :) give a definitive answer on this...
>
>   
>>> Having said that, I believe 1.6.0 and 1.6.3 is actually something
>>>       
> that
>   
>>> will work, but I''m not 100% certain of this, so
I''ll allow others to
>>> correct me.
>>>       
>
> ... as I would like to do just this, go from 1.6.0.1 to 1.6.3 and would
> rather not have to go through the intermediate versions if I don''t
have
> to.
>
> Thanks in advance. 
>
> Ron Jerome
> National Research Council Canada
>
>
>   
>> -----Original Message-----
>> From: lustre-discuss-bounces at clusterfs.com [mailto:lustre-discuss-
>> bounces at clusterfs.com] On Behalf Of Jody McIntyre
>> Sent: October 15, 2007 11:58 PM
>> To: Jeff Blasius
>> Cc: lustre-discuss at clusterfs.com
>> Subject: Re: [Lustre-discuss] lustre interoperability
>>
>> Hi Jeff,
>>
>> On Sun, Oct 14, 2007 at 11:31:45PM -0400, Jeff Blasius wrote:
>>
>>     
>>> Thank you for this information. Can or someone else comment on what
>>>       
>> it
>>     
>>> means to upgrade? I assume it''s not enough to simply build
the
>>> software or update via rpm. After updating, you should go through
>>>       
> the
>   
>>> process of having all of the OSS''s join the MGS/MDS and
verify a
>>> successful client connection?
>>>       
>> This part is in the manual :)  See:
>> http://manual.lustre.org/manual/LustreManual16_HTML/DynamicHTML-13-
>> 1.html
>>
>>     
>>> Just for my curiosity, is there a standard process the server
>>> components undergo during the upgrade?
>>>       
>> I don''t understand what you mean here.  Can you explain your
question?
>>
>> Cheers,
>> Jody
>>
>>     
>>> Thank You,
>>>                          jeff
>>>
>>> On 10/12/07, Jody McIntyre <scjody at clusterfs.com> wrote:
>>>       
>>>> Hi Jeff,
>>>>
>>>> On Fri, Oct 12, 2007 at 12:01:04PM -0400, Jeff Blasius wrote:
>>>>
>>>>         
>>>>> I''ve seriously trekked through the lustre
documentation and
>>>>>           
>> haven''t
>>     
>>>>> found an answer regarding this. Is there an official policy
>>>>>           
>> regarding
>>     
>>>>> interoperability among different versions of various lustre
>>>>> components?
>>>>>           
>>>> By coincidence, I just sent information about this to our
>>>>         
>> documentation
>>     
>>>> team.  It should eventually reach the manual.  Here it is:
>>>>
>>>> ---
>>>> Our supported upgrades are from one minor version to another,
for
>>>> example 1.4.10 to 1.4.11 or 1.6.2 to 1.6.3, and also from the
>>>>         
>> latest
>>     
>>>> 1.4.x version to the latest 1.6.x version, so 1.4.11 to 1.6.3
is
>>>> supported.
>>>>
>>>> We also support downgrades within the same ranges.  For
example,
>>>>         
> if
>   
>> you
>>     
>>>> upgrade from 1.6.2 to 1.6.3, you can also downgrade to 1.6.2
(but
>>>>         
> a
>   
>>>> fresh install of 1.6.3 is _not_ guaranteed to be
downgradeable.)
>>>>
>>>> Note that other combinations will work and we support them in
>>>>         
>> specific
>>     
>>>> cases when requested by customers, but the ranges above will
>>>>         
> always
>   
>>>> work.
>>>> ---
>>>>
>>>>         
>>>>> For example (and I''m sure other groups are in the
same boat
>>>>>           
>> here),
>>     
>>>>> it''s relatively painless to perform a rolling
upgrade to the
>>>>>           
>> lustre
>>     
>>>>> clients, but upgrading the OSS or MDS takes more
convincing. Is
>>>>>           
>> it OK
>>     
>>>>> for me to run a patched but 1.6.0 based OSS with a 1.6.3
client?
>>>>>           
>> In
>>     
>>>>> this case all of the lustre components (kernel, lustre,
ldiskfs)
>>>>>           
>> are
>>     
>>>>> the same version for each host. Similarly, is it OK to run
a
>>>>>           
>> lustre
>>     
>>>>> kernel version out of sync with the userland tools? For
example
>>>>>           
> a
>   
>>>>> 1.6.0 kernel with a 1.6.3 lustre build on the same host?
>>>>>           
>>>> Not necessarily.  You should do a rolling upgrade to 1.6.1,
then
>>>>         
>> 1.6.2,
>>     
>>>> then 1.6.3.  Upgrading will be easier if you stay more current
-
>>>>         
>> 1.6.0
>>     
>>>> is fairly old at this point.
>>>>
>>>> Having said that, I believe 1.6.0 and 1.6.3 is actually
something
>>>>         
>> that
>>     
>>>> will work, but I''m not 100% certain of this, so
I''ll allow others
>>>>         
>> to
>>     
>>>> correct me.
>>>>
>>>> Cheers,
>>>> Jody
>>>>
>>>>         
>>>>> I understand that many of these combinations do in fact
work,
>>>>>           
> I''m
>   
>> more
>>     
>>>>> interested if they''re likely to lead to data
corruption or
>>>>>           
> client
>   
>>>>> evictions. I''m not sure how often
incompatibilities arise, but
>>>>>           
> if
>   
>> it''s
>>     
>>>>> relatively rare, it would be useful if that was announced
on the
>>>>> change log. Of course if there''s a serious
"Do at your own risk"
>>>>> policy that would also be useful to know.
>>>>>
>>>>> Thank You,
>>>>>                      jeff
>>>>>
>>>>> --
>>>>> Jeff Blasius / jeff.blasius at yale.edu
>>>>> Phone: (203)432-9940  51 Prospect Rm. 011
>>>>> High Performance Computing (HPC)
>>>>> Linux Systems Design & Support (LSDS)
>>>>> Yale University Information Technology Services (ITS)
>>>>>
>>>>> _______________________________________________
>>>>> Lustre-discuss mailing list
>>>>> Lustre-discuss at clusterfs.com
>>>>> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>>>>>
>>>>>           
>>>> --
>>>>
>>>>         
>>> --
>>> Jeff Blasius / jeff.blasius at yale.edu
>>> Phone: (203)432-9940  51 Prospect Rm. 011
>>> High Performance Computing (HPC)
>>> UNIX Systems Administrator, Linux Systems Design & Support
(LSDS)
>>> Yale University Information Technology Services (ITS)
>>>
>>>       
>> --
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at clusterfs.com
>> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>>     
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>