We are now in a process of creating a Glusterfs based NAS on the Amazon EC2 cloud using EBS volume. I did like to hear opinions/advices/experiences of how others have done this. Our scenario is simple. We need to store large number of static web files like images (size less than 2 MB per file), etc and we dont need any redundancy for this. But we need very high throughput on a high concurrency usage I know from past experience that to maximize the I/O on Amazon EBS one needs to stripe 4 EBS disk per instances with (RAID0) and to use XFS (hmm ..havent benchmarked with ext4 or btrfs yet). Now I am at a point to find a ideal setup. The first question that comes up is, whether having more servers in a glusterfs with less disks per server is better than having few servers with more disks. Note again this question is context based as we are not looking at any redundancy, so only distribute and stripped are needed. Now assume for the the tests we take 2 cc1.4xlarge servers each with 4 EBS disks, then which would be better 1. Create RAID0 for the 4 disks on each server and then use the 2 raid0 devices in distribute 2. Create Glusterfs stripe volume with the 4 disks on each server and then the 2 servers in distribute mode. 3. Create 4 stripe volumes using one disk from one server and one disk from the other, then use distribute on the 4 stripe volume. Any inputs will be highly appreciated. regards -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120120/33a04268/attachment.html>
On 20 Jan 2012, at 14:08, RDP wrote:> Any inputs will be highly appreciated.My experience with gluster is that it's dismally slow for applications like this. Would you not be better off spending the same money serving directly from S3 or using CloudFront, or distribute data (slowly) using gluster, but mirror it in ElastiCache and serve from that? Marcus -- Marcus Bointon Synchromedia Limited: Creators of http://www.smartmessages.net/ UK info at hand CRM solutions marcus at synchromedia.co.uk | http://www.synchromedia.co.uk/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120120/fa010469/attachment.html>
Hi, ----- Original Message -----> We are now in a process of creating a Glusterfs based NAS on the > Amazon EC2 cloud using EBS volume. I did like to hear > opinions/advices/experiences of how others have done this.> Our scenario is simple. We need to store large number of static web > files like images (size less than 2 MB per file), etc and we dont > need any redundancy for this. But we need very high throughput on a > high concurrency usage...> Now assume for the the tests we take 2 cc1.4xlarge servers each with > 4 EBS disks, then which would be better> 1. Create RAID0 for the 4 disks on each server and then use the 2 > raid0 devices in distribute > 2. Create Glusterfs stripe volume with the 4 disks on each server and > then the 2 servers in distribute mode. > 3. Create 4 stripe volumes using one disk from one server and one > disk from the other, then use distribute on the 4 stripe volume.> Any inputs will be highly appreciated.So a couple of things: 1. I'm not sure what striping gets you in this scenario, over and above the more common distributed setup. 2. a replica 2 setup of glusterfs will give you better consistency for EBS performance, because it will essentially average out the latency of the EBS bricks. However, you're still limited by EBS latency. If no one else offers you help here, I would hop over to #gluster on irc.freenode.net in the mornings (PST and EST). Also, if you're interested, Red Hat will be offering a Red Hat Storage AMI in the very near future. Depending on your needs, that might help.