I am trying to reclaim the default 5% of reserved disk space on Lustre filesystem following procedure describe in the Lustre manual (chapter 23.3.10). File system is in use and I would like to avoid any downtime. [root at oss17 ~]# tune2fs -m 1 /dev/sdg tune2fs 1.41.12.2.ora1 (14-Aug-2010) tune2fs: MMP: device currently active while trying to open /dev/sdg I understand it is the MMP flag which prevent tune2fs to access the device, is there a way to work around this or is my only option to umount OSTs (eek, downtime) Best regards, Wojciech -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101125/b94d5891/attachment.html
I forgot to ask, is setting reserved % to 0 is safe on Lustre filesysytem? On 25 November 2010 19:15, Wojciech Turek <wjt27 at cam.ac.uk> wrote:> I am trying to reclaim the default 5% of reserved disk space on Lustre > filesystem following procedure describe in the Lustre manual (chapter > 23.3.10). File system is in use and I would like to avoid any downtime. > > [root at oss17 ~]# tune2fs -m 1 /dev/sdg > tune2fs 1.41.12.2.ora1 (14-Aug-2010) > tune2fs: MMP: device currently active while trying to open /dev/sdg > > I understand it is the MMP flag which prevent tune2fs to access the device, > is there a way to work around this or is my only option to umount OSTs (eek, > downtime) > > Best regards, > > Wojciech >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101125/3d6f5f11/attachment.html
On 2010-11-25, at 13:12, Wojciech Turek wrote:> I forgot to ask, is setting reserved % to 0 is safe on Lustre filesysytem?It is safe, but can lead to serious (and permanent) fragmentation of the filesystem, which can lead to permanent performance loss. Consider reducing it to some smaller fraction than 5%, but leaving some reserve.> On 25 November 2010 19:15, Wojciech Turek <wjt27 at cam.ac.uk> wrote: >> I am trying to reclaim the default 5% of reserved disk space on Lustre filesystem following procedure describe in the Lustre manual (chapter 23.3.10). File system is in use and I would like to avoid any downtime. >> >> [root at oss17 ~]# tune2fs -m 1 /dev/sdg >> tune2fs 1.41.12.2.ora1 (14-Aug-2010) >> tune2fs: MMP: device currently active while trying to open /dev/sdg >> >> I understand it is the MMP flag which prevent tune2fs to access the device, is there a way to work around this or is my only option to umount OSTs (eek, downtime)It should be possible to do this with a modification to tune2fs. The issue is that this operation is only safe on a mounted filesystem if it is a local mount, and not a remote mount. tune2fs probably needs to skip the MMP checking in this case, but it needs to check that the filesystem is mounted locally. Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc.
Thanks for your reply Andreas. I have one more question regarding the size of the reserved space. Our average file size is 1MB, OST size is 7.2TB and I am planning to set reserved space to 0.15% which will give us 12GB per OST. Would you agree that this is enough to keep fragmentation low? Best regards, Wojciech On 25 November 2010 23:57, Andreas Dilger <andreas.dilger at oracle.com> wrote:> On 2010-11-25, at 13:12, Wojciech Turek wrote: > > I forgot to ask, is setting reserved % to 0 is safe on Lustre > filesysytem? > > It is safe, but can lead to serious (and permanent) fragmentation of the > filesystem, which can lead to permanent performance loss. Consider reducing > it to some smaller fraction than 5%, but leaving some reserve. > > > On 25 November 2010 19:15, Wojciech Turek <wjt27 at cam.ac.uk> wrote: > >> I am trying to reclaim the default 5% of reserved disk space on Lustre > filesystem following procedure describe in the Lustre manual (chapter > 23.3.10). File system is in use and I would like to avoid any downtime. > >> > >> [root at oss17 ~]# tune2fs -m 1 /dev/sdg > >> tune2fs 1.41.12.2.ora1 (14-Aug-2010) > >> tune2fs: MMP: device currently active while trying to open /dev/sdg > >> > >> I understand it is the MMP flag which prevent tune2fs to access the > device, is there a way to work around this or is my only option to umount > OSTs (eek, downtime) > > It should be possible to do this with a modification to tune2fs. > > The issue is that this operation is only safe on a mounted filesystem if it > is a local mount, and not a remote mount. tune2fs probably needs to skip > the MMP checking in this case, but it needs to check that the filesystem is > mounted locally. > > Cheers, Andreas > -- > Andreas Dilger > Lustre Technical Lead > Oracle Corporation Canada Inc. > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20101126/1387f2db/attachment.html
On 2010-11-26, at 05:51, Wojciech Turek wrote:> Thanks for your reply Andreas. I have one more question regarding the size of the reserved space. Our average file size is 1MB, OST size is 7.2TB and I am planning to set reserved space to 0.15% which will give us 12GB per OST. Would you agree that this is enough to keep fragmentation low?I can''t really say for sure, since this is highly dependent upon your usage pattern. In any case, keeping some space in reserve is much better than no reserved space at all. We have observed significant performance degradation as filesystems approach 99% full, so I''m not confident that 0.15% reserved will avoid this problem. 12GB over 7.2TB is only about 218kB (54 blocks) per block group of unused space, so it means you have to be packing files very densely to get this kind of utilization. Even with expensive storage, 5% reserved is only about $200 per OST, so you have to balance, say, 25% performance loss from 95% - 99% full vs the $200 saved.> On 25 November 2010 23:57, Andreas Dilger <andreas.dilger at oracle.com> wrote: > On 2010-11-25, at 13:12, Wojciech Turek wrote: > > I forgot to ask, is setting reserved % to 0 is safe on Lustre filesysytem? > > It is safe, but can lead to serious (and permanent) fragmentation of the filesystem, which can lead to permanent performance loss. Consider reducing it to some smaller fraction than 5%, but leaving some reserve. > > > On 25 November 2010 19:15, Wojciech Turek <wjt27 at cam.ac.uk> wrote: > >> I am trying to reclaim the default 5% of reserved disk space on Lustre filesystem following procedure describe in the Lustre manual (chapter 23.3.10). File system is in use and I would like to avoid any downtime. > >> > >> [root at oss17 ~]# tune2fs -m 1 /dev/sdg > >> tune2fs 1.41.12.2.ora1 (14-Aug-2010) > >> tune2fs: MMP: device currently active while trying to open /dev/sdg > >> > >> I understand it is the MMP flag which prevent tune2fs to access the device, is there a way to work around this or is my only option to umount OSTs (eek, downtime) > > It should be possible to do this with a modification to tune2fs. > > The issue is that this operation is only safe on a mounted filesystem if it is a local mount, and not a remote mount. tune2fs probably needs to skip the MMP checking in this case, but it needs to check that the filesystem is mounted locally. > > Cheers, Andreas > -- > Andreas Dilger > Lustre Technical Lead > Oracle Corporation Canada Inc. > > >Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc.