Tomasz Chmielewski
2010-Aug-11 09:29 UTC
[Gluster-users] glusterfs on 32 bit - experiences?
I was wondering about general stability of glusterfs on 32 bit x86 Linux. I have it running without problems on some lightly used 32 bit systems, but this scares me a bit if I decided to use it in production[1]: While the 3.x versions of Gluster will compile on 32bit systems we do not QA or test on 32-bit systems. We strongly suggest you do NOT run Gluster in a 32-bit environment. I was wondering why this suggestion is so strong? Is it because: 1) no QA was ever done, 2) there are some fundamental problems with glusterfs design on 32 bit systems, 3) it is because 32 bit glusterfs will crash with filesystems bigger than 16 TB, more than 1024 clients or such. I intended to run glusterfs on Amazon EC2 to provide some more persistence to data stored on the instances (in Amazon EC2, if an instance dies, it "evaporates" with all data in it, so you have to do various workarounds). Unfortunately, Amazon EC2 pricing for 64 bit is pretty heavy (and all my instances run 32 bit anyway). I wanted to run 32 bit glusterfs clients and servers on Amazon EC2 (probably no more than 6 servers, 20-30 clients, up to 1 TB data). What are your experiences with running glusterfs on 32 bit? [1] http://www.gluster.com/community/documentation/index.php/Storage_Server_Installation_and_Configuration -- Tomasz Chmielewski http://wpkg.org
> I wanted to run 32 bit glusterfs clients and servers on Amazon EC2 (probably no more than 6 servers, 20-30 clients, up to 1 TB data).I'm also running GlusterFS on Amazon EC2. Two Servers and two Clients, the Servers are m1.small (32 Bit) and the Clients c1.medium (64 Bit) It works without any problem. I had only the problem, that when the "io-threads" translator was activated on the Servers, I got some strange timeouts. Maybe this came from 32 Bit. But without the io-threads it works good. Greez Michael -- Michael Schmid Chief Technology Officer Amazee Ltd. -- www.amazee.com Technoparkstr. 1 8005 Zurich Switzerland mail: michael.schmid at amazee.com mobile: +41 79 888 13 85
Hi Tomasz, 32 bit will work properly from 3.1.x. With upto 3.0.x versions there were problem with inode number sizes and fitting it properly into 32 bit structures. We changed our internal data structure that was passed around to fix the problem. This was a core data structure so its not possible to port this change back to 3.0.x. For those who have not seen a problem, unless there are a very huge amount of files, you would not need more than 32 bit inode numbers, however some backend filesystem may generate random large 64 bit numbers for each next inode created. That can cause a problem. We have not seen much problem with ext3 based systems that don't have very huge number of files. These above are the reason for the disclaimer. 3.1 has this fixed. We are also introducing XDR/RPC in 3.1 so backward compatibility can be achieved, which we were not able to do previously - we have a rapidly developing/changing code, so the protocol would keep changing, but with XDR/RPC, this can be easily managed going forward. Regards, Tejas. ----- Original Message ----- From: "Tomasz Chmielewski" <tch at wpkg.org> To: gluster-users at gluster.org Sent: Wednesday, August 11, 2010 2:59:54 PM Subject: [Gluster-users] glusterfs on 32 bit - experiences? I was wondering about general stability of glusterfs on 32 bit x86 Linux. I have it running without problems on some lightly used 32 bit systems, but this scares me a bit if I decided to use it in production[1]: While the 3.x versions of Gluster will compile on 32bit systems we do not QA or test on 32-bit systems. We strongly suggest you do NOT run Gluster in a 32-bit environment. I was wondering why this suggestion is so strong? Is it because: 1) no QA was ever done, 2) there are some fundamental problems with glusterfs design on 32 bit systems, 3) it is because 32 bit glusterfs will crash with filesystems bigger than 16 TB, more than 1024 clients or such. I intended to run glusterfs on Amazon EC2 to provide some more persistence to data stored on the instances (in Amazon EC2, if an instance dies, it "evaporates" with all data in it, so you have to do various workarounds). Unfortunately, Amazon EC2 pricing for 64 bit is pretty heavy (and all my instances run 32 bit anyway). I wanted to run 32 bit glusterfs clients and servers on Amazon EC2 (probably no more than 6 servers, 20-30 clients, up to 1 TB data). What are your experiences with running glusterfs on 32 bit? [1] http://www.gluster.com/community/documentation/index.php/Storage_Server_Installation_and_Configuration -- Tomasz Chmielewski http://wpkg.org _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users