Hi,
I am trying to attach a cache tier to normal distributed volume. I am
seeing write failures when the cache brick becomes full. following are the
steps
*>> 1. create volume using hdd brick*
*root at host:~/gluster/glusterfs# gluster volume create vol
host:/data/brick1/hdd/*
*volume create: vol: success: please start the volume to access data*
*root at host:~/gluster/glusterfs# gluster volume start vol*
*volume start: vol: success*
*>> 2. mount and write one file of size 1G*
*root at host:~/gluster/glusterfs# mount -t glusterfs host:/vol /mnt*
*root at host:~/gluster/glusterfs# dd if=/dev/zero of=/mnt/file1 bs=1G count=1*
*1+0 records in*
*1+0 records out*
*1073741824 bytes (1.1 GB) copied, 1.50069 s, 715 MB/s*
*root at host:~/gluster/glusterfs# du -sh /data/brick**
*1.1G /data/brick1*
*60K /data/brick2*
*>> 3. attach ssd brick as tier*
*root at host:~/gluster/glusterfs# gluster volume attach-tier vol
host:/data/brick2/ssd/*
*Attach tier is recommended only for testing purposes in this release. Do
you want to continue? (y/n) y*
*volume attach-tier: success*
*volume rebalance: vol: success: Rebalance on vol has been started
successfully. Use rebalance status command to check status of the rebalance
process.*
*ID: dea8d1b7-f0f4-4c17-94f5-ba0e263bc561*
*root at host:~/gluster/glusterfs# gluster volume rebalance vol tier status*
*Node Promoted files Demoted files Status*
*--------- --------- --------- ---------*
*localhost 0 0 in progress*
*volume rebalance: vol: success*
*>> 4. write data to fill up cache tier*
*root at host:~/gluster/glusterfs# dd if=/dev/zero of=/mnt/file2 bs=1G count=9
oflag=direct*
*9+0 records in*
*9+0 records out*
*9663676416 bytes (9.7 GB) copied, 36.793 s, 263 MB/s*
*root at host:~/gluster/glusterfs# du -sh /data/brick**
*1.1G /data/brick1*
*9.1G /data/brick2*
*root at host:~/gluster/glusterfs# gluster volume rebalance vol tier status*
*Node Promoted files Demoted files Status*
*--------- --------- --------- ---------*
*localhost 0 0 in progress*
*volume rebalance: vol: success*
*root at host:~/gluster/glusterfs# gluster volume rebalance vol status*
* Node Rebalanced-files size
scanned failures skipped status run time in
secs*
* --------- ----------- -----------
----------- ----------- ----------- ------------
--------------*
* localhost 0 0Bytes
0 0 0 in progress
112.00*
*volume rebalance: vol: success*
*root at host:~/gluster/glusterfs# dd if=/dev/zero of=/mnt/file3 bs=1G count=5
oflag=direct*
*dd: error writing ?/mnt/file3?: No space left on device*
*dd: closing output file ?/mnt/file3?: No space left on device*
*root at host:~/gluster/glusterfs# du -sh /data/brick**
*1.1G /data/brick1*
*9.3G /data/brick2*
*>>>> there is lot of space free in cold brick but writes are
failing...*
*root at vsan18:~/gluster/glusterfs# df -h*
*. <cut>*
*.*
*/dev/sdb3 231G 1.1G 230G 1% /data/brick1*
*/dev/ssd 9.4G 9.4G 104K 100% /data/brick2*
*host:/vol 241G 11G 230G 5% /mnt*
Please let me know if I am missing something.
Is this behavior expected. shouldn't the files be re-balanced?
Thanks,
Ameet
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20151001/60bbccde/attachment.html>