I had very similar experience, with both OCFS2 and GFS. I was running 'iozone -s 262144 -r 1k' and when I had both nodes mount the partition and run the benchmark, the wallclock time was 3 or 4 times that of a single node (which was comparable to ext3 performance). Each node had 256MB of memory since I wanted to minimize (but not completely eliminate) effects of buffer cache. I am attaching the numbers, if anyone is interested. -boris Date: Wed, 8 Feb 2006 12:51:31 -0300> From: "Marcio Costa" <marciopc at interativainternet.com.br> > Subject: [Ocfs2-users] RHEL4 x86_64 OCFS2 10gR2 performance > To: "ocfs2-users at oss.oracle.com" <ocfs2-users at oss.oracle.com> > Cc: marciopc at interativainternet.com.br, > josue at interativainternet.com.br > Message-ID: <20060208153800.M95583 at interativainternet.com.br> > Content-Type: text/plain; charset=iso-8859-1 > > Hi, > > I have 2 IBM x346 eServers, both of them with 2 Qlogic HBAs (qla2340). > They > are connected rendundantly to a IBM DS4300 (fiber storage). I have Oracle > 10gR2 RAC installed. RHEL4 U2 x86_64. My problem is the following: > > Each server works very well, if alone,ie, if the other one has not mounted > the ocfs2 partition. But when both of them are working, the performance > goes > to hell. > For example: there is a job that, when it runs on one server alone, takes > 4 > seconds, but when I put both servers on line, it takes 3 minutes. > > What could it be ? Is there any conf that I could check. I was thinking > about change the ocfs2 for other fs (gfs or other)... could it be the case > ? > > Thanks in advance. > > Marcio Costa. > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20060322/5716c561/attachment.html -------------- next part -------------- random random bkwd record stride write rewrite read reread read write read rewrite read fwrite frewrite fread freread ext3 12.5min 113718 8335 91962 186143 4345 515 9612 258904 6002 112859 7230 76225 139576 gfs 13.5min 27217 8337 50117 62312 1611 604 8233 81180 5749 33633 7958 53301 40331 ocfs2 14.5min 42102 9345 65887 92481 1210 566 8136 155370 5605 41571 8699 78925 71724 gfs(n1) 46min 21467 5159 29705 35512 348 172 808 81188 4970 32680 8039 35667 58961 gfs(n2) 48min 40046 3493 29565 25093 504 327 906 81390 456 30035 4085 24953 22493 ocfs2 (n1) 38min 26813 4375 27406 27408 367 251 892 156194 5038 49998 8882 80288 111914 ocfs2 (n2) 35.5min 22756 5330 36728 29607 673 400 907 153949 953 45964 5117 34055 40158
Stephan A. Rickauer
2006-Apr-04 09:22 UTC
[Ocfs2-users] RHEL4 x86_64 OCFS2 10gR2 performance
> I have 2 IBM x346 eServers, both of them with 2 Qlogic HBAs (qla2340). They > are connected rendundantly to a IBM DS4300 (fiber storage). I have Oracle > 10gR2 RAC installed. RHEL4 U2 x86_64. My problem is the following: > > Each server works very well, if alone,ie, if the other one has not mounted > the ocfs2 partition. But when both of them are working, the performance goes > to hell. > For example: there is a job that, when it runs on one server alone, takes 4 > seconds, but when I put both servers on line, it takes 3 minutes.Have you been able to fix this problem? It looks like I experience the very same effect on RHEL4, iscsi, two Opterons. However, I am only interested in native file system performance which I test by using bonnie++. With one node only I see around 80MB/s read and rw peformance. As soon as two nodes access one ocfs2 fs, performance is reduced to ~25MB/s write and 15MB/s read (!) performance. Has anyone been able to conform this being an x86_64 only issue? Thanks, -- Stephan A. Rickauer ----------------------------------------------------------- Institut f?r Neuroinformatik Tel: +41 44 635 30 50 Universit?t / ETH Z?rich Sek: +41 44 635 30 52 Winterthurerstrasse 190 Fax: +41 44 635 30 53 CH-8057 Z?rich Web: www.ini.ethz.ch RSA public key: https://www.ini.ethz.ch/~stephan/pubkey.asc ----------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 890 bytes Desc: OpenPGP digital signature Url : http://oss.oracle.com/pipermail/ocfs2-users/attachments/20060404/dcc9e218/signature.bin