I notice that Lustre 1.6.5 brings with it the md layer RAID5 patches for RHEL5 kernels. thanks! :-) are all the RHEL4 optimisations there, so we should get the same performance if we now move our OSS''s to RHEL5? cheers, robin
On Jul 18, 2008 08:39 -0400, Robin Humble wrote:> I notice that Lustre 1.6.5 brings with it the md layer RAID5 patches > for RHEL5 kernels. thanks! :-) > are all the RHEL4 optimisations there, so we should get the same > performance if we now move our OSS''s to RHEL5?That is my understanding, yes. Note that there are also similar performance improvements for RAID-6. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
On Sun, Jul 20, 2008 at 11:08:41PM -0600, Andreas Dilger wrote:>On Jul 18, 2008 08:39 -0400, Robin Humble wrote: >> I notice that Lustre 1.6.5 brings with it the md layer RAID5 patches >> for RHEL5 kernels. thanks! :-) >> are all the RHEL4 optimisations there, so we should get the same >> performance if we now move our OSS''s to RHEL5? >That is my understanding, yes.done, and yes, performance is largely the same as RHEL4. cool! Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP rhel4 oss 16G:256k 84624 99 842138 92 310044 91 77675 99 491239 96 285.8 10 rhel5 oss 16G:256k 86085 99 827731 95 327007 97 79639 100 495487 98 456.2 18 streaming writes are down marginally on rhel5, but seeks/s are up 50%. BTW - the above is with 1.6.4.3 clients. 1.6.5.1 client still perform badly for us. eg. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP 16G:256k 77216 99 462659 100 296050 96 68100 81 648350 93 422.2 13 which shows better streaming writes, but ~1/2 the streaming read speed :-(> Note that there are also similar >performance improvements for RAID-6.I can''t see the RAID6 patches in the tree for RHEL5... am I missing something? cheers, robin
On Aug 01, 2008 09:38 -0400, Robin Humble wrote:> done, and yes, performance is largely the same as RHEL4. cool! > > Version 1.03 ------Sequential Output------ --Sequential Input- --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > rhel4 oss 16G:256k 84624 99 842138 92 310044 91 77675 99 491239 96 285.8 10 > rhel5 oss 16G:256k 86085 99 827731 95 327007 97 79639 100 495487 98 456.2 18 > > streaming writes are down marginally on rhel5, but seeks/s are up 50%.Good to know, thanks.> BTW - the above is with 1.6.4.3 clients.Is this with 1.6.5 servers or 1.6.4.3 servers?> 1.6.5.1 client still perform badly for us. eg.Have you tried disabling the checksums? lctl set_param osc.*.checksums=0 Note that 1.6.5 clients -> 1.6.5 servers with checksums enabled will perform better than mixed client/server because 1.6.5 has a more efficient checksum algorithm.> Version 1.03 ------Sequential Output------ --Sequential Input- --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > 16G:256k 77216 99 462659 100 296050 96 68100 81 648350 93 422.2 13 > > which shows better streaming writes, but ~1/2 the streaming read speed :-(You are getting that backward... 55% of the previous write speed, 90% of the previous overwrite speed, and 130% of the previous read speed.> > Note that there are also similar > >performance improvements for RAID-6. > > I can''t see the RAID6 patches in the tree for RHEL5... am I missing > something?Sigh, RAID6 patches were ported to RHEL4, but not RHEL5... I''ve filed bug 16587 about that, but have no idea when it will be completed. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
On Fri, Aug 01, 2008 at 01:51:36PM -0600, Andreas Dilger wrote:>On Aug 01, 2008 09:38 -0400, Robin Humble wrote: >> done, and yes, performance is largely the same as RHEL4. cool! >> >> Version 1.03 ------Sequential Output------ --Sequential Input- --Random- >> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- >> Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP >> rhel4 oss 16G:256k 84624 99 842138 92 310044 91 77675 99 491239 96 285.8 10 >> rhel5 oss 16G:256k 86085 99 827731 95 327007 97 79639 100 495487 98 456.2 18 >> >> streaming writes are down marginally on rhel5, but seeks/s are up 50%. >Good to know, thanks. > >> BTW - the above is with 1.6.4.3 clients. >Is this with 1.6.5 servers or 1.6.4.3 servers?that''s with 1.6.5.1 RHEL5 servers.>> 1.6.5.1 client still perform badly for us. eg. >Have you tried disabling the checksums? > lctl set_param osc.*.checksums=0yes, checksums were disabled.>Note that 1.6.5 clients -> 1.6.5 servers with checksums enabled will perform >better than mixed client/server because 1.6.5 has a more efficient checksum >algorithm.>> Version 1.03 ------Sequential Output------ --Sequential Input- --Random- >> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- >> Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP >> 16G:256k 77216 99 462659 100 296050 96 68100 81 648350 93 422.2 13 >> >> which shows better streaming writes, but ~1/2 the streaming read speed :-( >You are getting that backward... 55% of the previous write speed, >90% of the previous overwrite speed, and 130% of the previous read speed.doh! yes, backwards... that was patchless 2.6.23 clients BTW.>> > Note that there are also similar >> >performance improvements for RAID-6. >> I can''t see the RAID6 patches in the tree for RHEL5... am I missing >> something? >Sigh, RAID6 patches were ported to RHEL4, but not RHEL5... I''ve filed >bug 16587 about that, but have no idea when it will be completed.cool - thanks! cheers, robin