Dear gluster-users, I am having issues using gluster as filesystem for some of our webservers. For machines with a moderate load, the system works fine and response times of the webservers is ok. But one site has a bit more load (ca. 7000 hits/sec) and when using gluserfs as webroot on this machine, the server will spike to a load of > 100 (Server has 8GB Ram, 4 Cores Xeon X5460 @3.16GHz) and eventually becomes unresponsive. The webserver runs CentOS-6.6 x86_64, apache-2.4 and php-5.6. We used a NetApp NFS mount as webroot before without problems. We are running glusterfs-server-3.5.1 on a zfs filesystem. The cluster consists of two HP DL 160G5 machine with eight Xeon E5462 @2.80GHz and 24GB RAM. What we can see on the gluster servers is that after some time the kswapd process will spike up to 100% CPU usage and will than stay at a high percentage. A restart via /etc/init.d/glusterd will solve this for some time after which it will eventually spike again. Has anybody here some experience with using glusterfs on zfs as fileserver for webroots? If so, has anyone an idea as to why we see the kswapd spiking? Any help/input would be greatly appreciated ;) We tweaked zfs a bit by adding: options zfs zfs_arc_min=8589934592 zfs_arc_max=19327352832 zfs_prefetch_disable=1 zfs_txg_timeout=5 to /etc/modprobe.d/zfs.conf The config of the zfs tank is: NAME PROPERTY VALUE SOURCE tank/customer/webroot type filesystem - tank/customer/webroot creation Fri Sep 12 9:34 2014 - tank/customer/webroot used 70.1G - tank/customer/webroot available 29.9G - tank/customer/webroot referenced 65.7G - tank/customer/webroot compressratio 1.13x - tank/customer/webroot mounted yes - tank/customer/webroot quota 100G local tank/customer/webroot reservation none default tank/customer/webroot recordsize 128K default tank/customer/webroot mountpoint /tank/buhv/SmartSuite default tank/customer/webroot sharenfs off default tank/customer/webroot checksum on default tank/customer/webroot compression on local tank/customer/webroot atime off local tank/customer/webroot devices on default tank/customer/webroot exec on default tank/customer/webroot setuid on default tank/customer/webroot readonly off default tank/customer/webroot zoned off default tank/customer/webroot snapdir hidden default tank/customer/webroot aclinherit restricted default tank/customer/webroot canmount on default tank/customer/webroot xattr sa inherited from tank tank/customer/webroot copies 1 default tank/customer/webroot version 5 - tank/customer/webroot utf8only off - tank/customer/webroot normalization none - tank/customer/webroot casesensitivity sensitive - tank/customer/webroot vscan off default tank/customer/webroot nbmand off default tank/customer/webroot sharesmb off default tank/customer/webroot refquota none default tank/customer/webroot refreservation none default tank/customer/webroot primarycache all default tank/customer/webroot secondarycache all default tank/customer/webroot usedbysnapshots 4.37G - tank/customer/webroot usedbydataset 65.7G - tank/customer/webroot usedbychildren 0 - tank/customer/webroot usedbyrefreservation 0 - tank/customer/webroot logbias latency default tank/customer/webroot dedup off default tank/customer/webroot mlslabel none default tank/customer/webroot sync disabled inherited from tank tank/customer/webroot refcompressratio 1.11x - tank/customer/webroot written 76.1M - tank/customer/webroot logicalused 72.8G - tank/customer/webroot logicalreferenced 68.7G - tank/customer/webroot snapdev hidden default tank/customer/webroot acltype off default tank/customer/webroot context none default tank/customer/webroot fscontext none default tank/customer/webroot defcontext none default tank/customer/webroot rootcontext none default tank/customer/webroot relatime off default The config of the gluster volume: Volume Name: webroot Type: Replicate Volume ID: bb7e80cd-a945-4c22-a3ec-5840cf442cd6 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: gluster-01:/tank/customer/webroot/brick Brick2: gluster-02:/tank/customer/webroot/brick Options Reconfigured: nfs.rpc-auth-allow: xx.xx.xx.xx auth.allow: xx.xx.xx.xx nfs.enable-ino32: on Thanks and take care, Bj?rn -- dbap GmbH - one-stop e-business phone +49 251 609979-0 / fax +49 251 609979-99 Martin-Luther-King Weg 42-44 - 48155 Muenster, Germany http://www.dbap.de dbap GmbH, Sitz: M?nster HRB 5891, Amtsgericht M?nster Gesch?ftsf?hrer: Bj?rn Dolkemeier, Stefan Goebel