Andrew Robert Nicols
2010-Apr-19 09:06 UTC
[Ocfs2-users] Recommended settings for mkfs.ocfs2
I'm planning to deploy a new data store using a pair of servers running Debian Lenny and using DRBD to replicate data between them. My intention is to use ocfs2 as the file system on these so that we can operate in a dual primary mode. The raid device I'm using gives 15TB of usable space and, having had a brief look through the ocfs2-users archive, I see that in January an issue with space left on device was fixed, but this isn't available in the stock Lenny kernel yet (2.6.26). From the notes on the bug, I see that altering the number of slots on the file system can help to alleviate the issue, but what are the recommendations on what mkfs.ocfs2 options work best. I've already created the file system with 2 node slots, but I see from comment #13 that Sunil recommends against adding slots. I am free to re-create the file system if need be. To summarise: * 2 Node cluster * 15TB storage * ocfs2-tools version 1.4.1 * 2.6.26 kernel Any advice would be gratefully received, Andrew Nicols -- Systems Developer e: andrew.nicols at luns.net.uk im: a.nicols at jabber.lancs.ac.uk t: +44 (0)1524 5 10147 Lancaster University Network Services is a limited company registered in England and Wales. Registered number: 04311892. Registered office: University House, Lancaster University, Lancaster, LA1 4YW -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Digital signature Url : http://oss.oracle.com/pipermail/ocfs2-users/attachments/20100419/b50fb540/attachment.bin
lenny-backports has a 2.6.32 based kernel that might already have the free space fix in it. I haven't checked yet. Also you don't really explain what you're trying to use the data store for (eg: lots of small files, video files, heavy writes, heavy reads, random, sequential, etc.). It may impact the options you want to give to mkfs. Brian Andrew Robert Nicols <andrew.nicols at luns.net.uk> 2010-04-19 10:06:> I'm planning to deploy a new data store using a pair of servers running > Debian Lenny and using DRBD to replicate data between them. > > My intention is to use ocfs2 as the file system on these so that we can > operate in a dual primary mode. > > The raid device I'm using gives 15TB of usable space and, having had a > brief look through the ocfs2-users archive, I see that in January an issue > with space left on device was fixed, but this isn't available in the stock > Lenny kernel yet (2.6.26). > > From the notes on the bug, I see that altering the number of slots on the > file system can help to alleviate the issue, but what are the > recommendations on what mkfs.ocfs2 options work best. > > I've already created the file system with 2 node slots, but I see from > comment #13 that Sunil recommends against adding slots. I am free to > re-create the file system if need be. > > To summarise: > * 2 Node cluster > * 15TB storage > * ocfs2-tools version 1.4.1 > * 2.6.26 kernel > > Any advice would be gratefully received, > > Andrew Nicols > > -- > Systems Developer > > e: andrew.nicols at luns.net.uk > im: a.nicols at jabber.lancs.ac.uk > t: +44 (0)1524 5 10147 > > Lancaster University Network Services is a limited company registered in > England and Wales. Registered number: 04311892. Registered office: > University House, Lancaster University, Lancaster, LA1 4YW> _______________________________________________ > Ocfs2-users mailing list > Ocfs2-users at oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Digital signature Url : http://oss.oracle.com/pipermail/ocfs2-users/attachments/20100419/c15dab97/attachment.bin
Andrew, I would make sure you use say large cluster and block sizes if possible with the inband FS option enabled (you will loose some space but ive noticed it tends to run a bit better). As for the bug, its one ive been fighting with. However using only 2 nodes will make it take a long time to occur (took 2 yrs on my cluster) with 6 nodes. David -----Original Message----- From: ocfs2-users-bounces at oss.oracle.com [mailto:ocfs2-users-bounces at oss.oracle.com] On Behalf Of Andrew Robert Nicols Sent: Monday, April 19, 2010 9:26 AM To: ocfs2-users at oss.oracle.com Subject: Re: [Ocfs2-users] Recommended settings for mkfs.ocfs2 Hi Brian, Thank you for taking the time to reply. On Mon, Apr 19, 2010 at 08:53:47AM -0500, Brian Kroth wrote:> lenny-backports has a 2.6.32 based kernel that might already have the > free space fix in it. I haven't checked yet.>From what I can tell, the ENOSPC issue isn't fixed until 2.6.33(http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189#c25) so the version in backports (or even Squeeze) isn't much help yet I'm afraid.> Also you don't really explain what you're trying to use the data store > for (eg: lots of small files, video files, heavy writes, heavy reads, > random, sequential, etc.). It may impact the options you want to give > to mkfs.Sorry - that didn't occur to me. This is going to be a file store for a variety of user submitted data for a web application (Moodle). At present we have a variety of: * videos * audio * images * database backups (gunzipped tar) * large files (primarily zip) * small files The activity is primarily read with writes too but I'm not sure on the exact characteristics at present. I'd guess fairly random rather than sequential and there are periods with heavy writes. Files are served to 6 frontend web servers over NFS for serving with Apache2. We've currently got 2.2TB of space used. Thank you for your input - if there's anything else which would be useful, I'll see if I can provide it. Andrew -- Systems Developer e: andrew.nicols at luns.net.uk im: a.nicols at jabber.lancs.ac.uk t: +44 (0)1524 5 10147 Lancaster University Network Services is a limited company registered in England and Wales. Registered number: 04311892. Registered office: University House, Lancaster University, Lancaster, LA1 4YW