Haris Zukanovic
2012-Mar-22 14:14 UTC
[Gluster-users] Separating reads and writes for performance benefit
Hi everyone, I decided to try the Guster FS as distributed filesystem for my Apache/PHP servers. My concrete application of the Gluster FS is for the following: I have 3 bricks on 3 different servers and 1 volume created in "replica 3" is mounted on each of the servers. Intention is to use Gluster for replication/failover purposes. 1. To see uploaded files through PHP application on all PHP servers so that I can do seamless failover with my HTTP loadbalancer 2. To update the application code on all servers at the same time 3. To have a safe replica in case of 1 brick failure Do you think Gluster FS is suitable for all of these? Anyhow, everything seems to work fine for now but the read performance is a problem. I have noticed that the read performance in the brick is very good, equal to "normal" file system... so I wonder: Is it possible to separate reads and writes for best performance? Apache/PHP would read directly from the brick and write to Gluster FS through the mount. In that case I need to protect somehow the brick by making it truly read-only to avoid accidental writes to it. How can I do that? thank you in advance -- -- Haris Zukanovic
Berend de Boer
2012-Mar-22 18:56 UTC
[Gluster-users] Separating reads and writes for performance benefit
>>>>> "Haris" == Haris Zukanovic <haris.zukanovic74 at gmail.com> writes:Haris> Anyhow, everything seems to work fine for now but the read Haris> performance is a problem. Try mounting the volume with nfs and see if that improves things. -- All the best, Berend de Boer ------------------------------------------------------ Awesome Drupal hosting: https://www.xplainhosting.com/
Marcus Bointon
2012-Mar-22 19:23 UTC
[Gluster-users] Separating reads and writes for performance benefit
On 22 Mar 2012, at 15:14, Haris Zukanovic wrote:> Intention is to use Gluster for replication/failover purposes. > 1. To see uploaded files through PHP application on all PHP servers so that I can do seamless failover with my HTTP loadbalancer > 2. To update the application code on all servers at the same time > 3. To have a safe replica in case of 1 brick failure > > Do you think Gluster FS is suitable for all of these?The first thing people usually try for this is bidirectional rsync triggered from a 1-minute cron. That works ok until you realise that you can't delete, because you can't distinguish a new file on one node from a delete on the other. So you look at csync2 and find that doesn't work! Then you run into gluster, which solves all your problems, but has small-file read performance problems. I've been thinking that a mixture of gluster and rsync could yield better performance with some tradeoffs. Use gluster to handle the fully-synced file system, but don't read from it directly: maintain a local copy of its content via rsync using --delete. This does mean that you end up storing everything twice, but it fixes both gluster's speed problem and rsync's delete problem. The downside is that it's not synchronous, so you get a delay between a write on one node and it being available on other nodes. You could work around that by writing to all locations from your app - a typical web app doesn't do many writes so that may be acceptable. If you are using ip or cookie-based stickiness in your balancer, the client will be reading from where they just wrote, and everyone else will see it shortly afterwards. Overall it's a bit like using mysql with replicated slaves. Just a thought... Marcus -- Marcus Bointon Synchromedia Limited: Creators of http://www.smartmessages.net/ UK info at hand CRM solutions marcus at synchromedia.co.uk | http://www.synchromedia.co.uk/
Haris Zukanovic
2012-Mar-23 08:12 UTC
[Gluster-users] Separating reads and writes for performance benefit
Hi Francis, Thanx for the suggestion.. I tried this. I set 644 owner root on the brick directory. Apache is not able to read the PHP files from under the brick folder. 2012/03/23 09:04:32 [crit] 15752#0: *137813 stat() "/var/www/gluster-application-read-only/www/staging/failover-test/web/server-health.php" failed (13: Permission denied), client: *.*.107.216, server: *.failover.myserver.com, request: "GET /server-health.php HTTP/1.1", host: "test.failover.myserver.com" Apache seems to be unable to change into brick dir since it has no x permission. If I set the x for other users (apache) on the brick then since apache user is owner of all the files it will be able to overwrite it also using the path directly through the brick. I maybe missing something? On 23/03/12 08.20, Francsics Bal?zs wrote:> Gluster runs on the gluster user, right? If you set the brick > permission to 644, and you read the files with a different user, you > cannot write to the dir. > > -- > Francsics Bal?zs > Sent with Sparrow <http://www.sparrowmailapp.com/?sig> > > On 2012. March 22., Thursday at 22:18, Haris Zukanovic wrote: > >> Great to hear that... >> All my tests so far have also confirmed that there are no problems >> when you only read from the brick dir. >> However, I would like somehow to prevent accidentals writes if >> possible. Do you have any suggestions on how to accomplish that? >> In fact, I am unable to allow writes to a file through the Gluster >> mount and allow to read but not write on the brick directory directly. >> >> >> >> On 22/03/12 20.47, winyaa at gmail.com <mailto:winyaa at gmail.com> wrote: >>> >>> If you are on the same machine, which the gluster runs from, and you >>> are using replicated setup, you can read from the brick directly, >>> but avoid to write into the brick dir, becouse it leads to data >>> inconsistency. >>> >>> 2012.03.22. 20:06, "Haris Zukanovic" <haris.zukanovic74 at gmail.com >>> <mailto:haris.zukanovic74 at gmail.com>> ezt ?rta: >>>> Thanx for your input... >>>> Do you mean that I should mount the brick through NFS to make the >>>> brick read-only? I would like to avioid that. I am on the same >>>> physical machine where the brick is. >>>> >>>> >>>> On 22/03/12 19.56, Berend de Boer wrote: >>>>> >>>>>> "Haris" == Haris >>>>>> Zukanovic<haris.zukanovic74 at gmail.com >>>>>> <mailto:haris.zukanovic74 at gmail.com>> writes: >>>>> >>>>> Haris> Anyhow, everything seems to work fine for now but the read >>>>> Haris> performance is a problem. >>>>> >>>>> Try mounting the volume with nfs and see if that improves things. >>>>> >>>> >>>> -- >>>> -- >>>> Haris Zukanovic >>>> >>>> _______________________________________________ >>>> Gluster-users mailing list >>>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> >>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >> >> -- >> -- >> Haris Zukanovic >-- -- Haris Zukanovic -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120323/9463d275/attachment.html>