Hi everybody. After burning way, way too much project time trying to use another cluster filesystem I have a few questions I would appreciate getting answered. I am building a mail platform that's supposed to scale up to 200k accounts. Currently around 100k accounts on different servers shall be migrated, yesterday. Setup: A SAN, ISCSI A bunch of servers doing their best to kill it with POP, some IMAP on Maildir, and postfix writing. Some auxiliary servers, some on xen doing maintenance and other minor functions. So, my questions: * Is ocfs suitable? * Will locking issues be a major concern? * Which version should I use, I will compile myself, and I need to have a xen domU kernel working, based on 2.6.18? I have found the patches at http://www.kernel.org/pub/linux/kernel/people/mfasheh/ocfs2/backports/2.6.18/ * Would 2.6.18 benefit from other patches, performance and stability wise? mmap would be nice. Thank you, and have a nice day. //Peter Hellman
OCFS2 included in 2.6.18 does not have the shared writeable mmap nor inline data nor sparse files. The first two should be useful in your use-case. We are currently backporting all those features to el5 (2.6.18). Hopefully, we'll have something for general testing in the next month. Sunil Peter Hellman wrote:> Hi everybody. > > After burning way, way too much project time trying to use another > cluster filesystem > I have a few questions I would appreciate getting answered. > > I am building a mail platform that's supposed to scale up to 200k > accounts. Currently > around 100k accounts on different servers shall be migrated, yesterday. > > Setup: > > A SAN, ISCSI > A bunch of servers doing their best to kill it with POP, some IMAP on > Maildir, and postfix writing. > Some auxiliary servers, some on xen doing maintenance and other minor > functions. > > > So, my questions: > > * Is ocfs suitable? > > * Will locking issues be a major concern? > > * Which version should I use, I will compile myself, and I need to have > a xen domU kernel working, > based on 2.6.18? > > I have found the patches at > http://www.kernel.org/pub/linux/kernel/people/mfasheh/ocfs2/backports/2.6.18/ > * Would 2.6.18 benefit from other patches, performance and stability > wise? mmap would be nice. > > > Thank you, and have a nice day. > //Peter Hellman > > > > _______________________________________________ > Ocfs2-users mailing list > Ocfs2-users@oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users >
Dear Peter, We run our apache servers with the files stored on ocfs2. For us, sometimes, locking can be an issue. If you have a way to verify that when a user comes in, they're only checking the account through one server, and not multiple servers, you will improve the situation by far. How many servers are you planning to implement? Thanks, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20071218/9f7971ff/attachment.html
In the cluster.conf file (/etc/ocfs2/cluster.conf)
The nodes are listed with IP and name:
node:
ip_port = 7777
ip_address = 192.168.1.101
number = 0
name = appsdb1
cluster = lab
On these nodes, there are 2 NICs, one for public and one private VLAN
(192.168 address)
In the hosts file, the public IP is
172.30.200.101 appsdb1.domain.com appsdb1
192.168.1.101 priv1.domain.com priv1
Does the cluster.conf need to be configured with the public IP and name
or private IP and private name?
The information contained in this E-mail may be confidential and/or proprietary
to Inter-Tel and/or its affiliates. The information transmitted herewith is
intended only for use by the individual or entity to which it is addressed. If
the reader of this message is not the intended recipient, you are hereby
notified that any review, retransmission, dissemination, distribution, copying
or other use of, or taking of any action in reliance upon this information is
strictly prohibited. If you have received this communication in error, please
contact the sender and delete the material from your computer.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://oss.oracle.com/pipermail/ocfs2-users/attachments/20071219/f5bc8011/attachment.html
I have a 4 node RAC cluster that had all 4 nodes crash. The systems locked up. Everytime I tried to bring up a box, it hung. So I commented out all the ocfs2 volumes from the fstab file and the system booted fine. I have 3 volumes that were in use at the time and 3 volumes that have were unmounted. After booting, I see even though no volumes are mounted (commented out of the fstab), I can see that it THINKS they are! #mounted.ocfs2 -f Device FS Nodes /dev/sde1 ocfs2 appsdb1 /dev/sdf1 ocfs2 appsdb3 /dev/sdg1 ocfs2 appsdb3 /dev/sdh1 ocfs2 Not mounted /dev/sdi1 ocfs2 Not mounted /dev/sdj1 ocfs2 Not mounted I set o2cb and ocfs2 NOT to start on boot up on all 4 nodes and rebooted. After the reboot it looks like the volumes think they are still in use by node 0 and 2. #mounted.ocfs2 -f Device FS Nodes /dev/sde1 ocfs2 0 /dev/sdf1 ocfs2 2 /dev/sdg1 ocfs2 2 /dev/sdh1 ocfs2 Not mounted /dev/sdi1 ocfs2 Not mounted /dev/sdj1 ocfs2 Not mounted I started o2cb and fsck.ocfs2 one of the volumes #fsck.ocfs2 /dev/sde1 Checking OCFS2 filesystem in /dev/sde1: label: st_dbf1 uuid: 08 dd 84 44 da 6e 4e 7e 80 a0 82 2f fd 80 de c0 number of blocks: 131070208 bytes per block: 4096 number of clusters: 511993 bytes per cluster: 1048576 max slots: 4 /dev/sde1 is clean. It will be checked after 20 additional mounts. Where should I be looking here? TIA The information contained in this E-mail may be confidential and/or proprietary to Inter-Tel and/or its affiliates. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20071219/4f5abc53/attachment.html
Check the top reported issues on ocfs2 home page. More info is in the faq. Short answer: Public. Kendall, Kim wrote:> > In the cluster.conf file (/etc/ocfs2/cluster.conf) > > > > The nodes are listed with IP and name: > > > > node: > > ip_port = 7777 > > ip_address = 192.168.1.101 > > number = 0 > > name = appsdb1 > > cluster = lab > > > > > > On these nodes, there are 2 NICs, one for public and one private VLAN > (192.168 address) > > > > In the hosts file, the public IP is > > > > 172.30.200.101 appsdb1.domain.com appsdb1 > > 192.168.1.101 priv1.domain.com priv1 > > > > > > Does the cluster.conf need to be configured with the public IP and > name or private IP and private name? > > > > > > > > > > The information contained in this E-mail may be confidential and/or > proprietary to Inter-Tel and/or its affiliates. The information > transmitted herewith is intended only for use by the individual or > entity to which it is addressed. If the reader of this message is not > the intended recipient, you are hereby notified that any review, > retransmission, dissemination, distribution, copying or other use of, > or taking of any action in reliance upon this information is strictly > prohibited. If you have received this communication in error, please > contact the sender and delete the material from your computer. > > ------------------------------------------------------------------------ > > _______________________________________________ > Ocfs2-users mailing list > Ocfs2-users@oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users
mounted.ocfs2 -f dirty reads slot_map to show the information. The only time this information is incorrect if the last node that had this volume mounted, crashed. slot_map information is recovered only when that volume is mounted again. Do: $ watch -d -n2 "debugfs.ocfs2 -R \"hb\" /dev/sde1" This will show if any node is still heartbeating. There are instances when a node crashes but is still heartbeating. See if that is the case. If so, reset that box. Lastly, to run fsck, do fsck.ocfs2 -f /dev/sde1. Kendall, Kim wrote:> > I have a 4 node RAC cluster that had all 4 nodes crash. The systems > locked up. > > > > Everytime I tried to bring up a box, it hung. So I commented out all > the ocfs2 volumes from the fstab file and the system booted fine. > > > > I have 3 volumes that were in use at the time and 3 volumes that have > were unmounted. After booting, I see even though no volumes are > mounted (commented out of the fstab), I can see that it THINKS they are! > > > > #mounted.ocfs2 -f > > Device FS Nodes > > /dev/sde1 ocfs2 appsdb1 > > /dev/sdf1 ocfs2 appsdb3 > > /dev/sdg1 ocfs2 appsdb3 > > /dev/sdh1 ocfs2 Not mounted > > /dev/sdi1 ocfs2 Not mounted > > /dev/sdj1 ocfs2 Not mounted > > > > I set o2cb and ocfs2 NOT to start on boot up on all 4 nodes and > rebooted. After the reboot it looks like the volumes think they are > still in use by node 0 and 2. > > > > #mounted.ocfs2 -f > > Device FS Nodes > > /dev/sde1 ocfs2 0 > > /dev/sdf1 ocfs2 2 > > /dev/sdg1 ocfs2 2 > > /dev/sdh1 ocfs2 Not mounted > > /dev/sdi1 ocfs2 Not mounted > > /dev/sdj1 ocfs2 Not mounted > > > > I started o2cb and fsck.ocfs2 one of the volumes > > > > #fsck.ocfs2 /dev/sde1 > > Checking OCFS2 filesystem in /dev/sde1: > > label: st_dbf1 > > uuid: 08 dd 84 44 da 6e 4e 7e 80 a0 82 2f fd 80 de c0 > > number of blocks: 131070208 > > bytes per block: 4096 > > number of clusters: 511993 > > bytes per cluster: 1048576 > > max slots: 4 > > > > /dev/sde1 is clean. It will be checked after 20 additional mounts. > > > > > > Where should I be looking here? > > > > TIA > > > > > > The information contained in this E-mail may be confidential and/or > proprietary to Inter-Tel and/or its affiliates. The information > transmitted herewith is intended only for use by the individual or > entity to which it is addressed. If the reader of this message is not > the intended recipient, you are hereby notified that any review, > retransmission, dissemination, distribution, copying or other use of, > or taking of any action in reliance upon this information is strictly > prohibited. If you have received this communication in error, please > contact the sender and delete the material from your computer. > > ------------------------------------------------------------------------ > > _______________________________________________ > Ocfs2-users mailing list > Ocfs2-users@oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users