Andrus, Brian Contractor
2011-Jul-16 16:30 UTC
[Lustre-discuss] lfs_migrate for rebalancing
All,
Am I incorrect in the way I read that lfs_migrate can be used to rebalance a
filesystem after adding OSTs?
Here is some output on a 22M file that was all on one OST and we now have 9:
[root at compute-3-1 ps2]# lfs getstripe PS01-071.xml
PS01-071.xml
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_stripe_offset: 6
obdidx objid objid group
6 4411098 0x434eda 0
[root at compute-3-1 ps2]# lfs_migrate -y PS01-071.xml
PS01-071.xml: done
[root at compute-3-1 ps2]# lfs getstripe PS01-071.xml
PS01-071.xml
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_stripe_offset: 4
obdidx objid objid group
4 50856625 0x30802b1 0
[root at compute-3-1 ps2]# cp PS01-071.xml PS01-071.xml.tmp
[root at compute-3-1 ps2]# lfs getstripe PS01-071.xml.tmp
PS01-071.xml.tmp
lmm_stripe_count: 9
lmm_stripe_size: 1048576
lmm_stripe_offset: 2
obdidx objid objid group
2 73524719 0x461e5ef 0
5 4389362 0x42f9f2 0
3 53566843 0x3315d7b 0
0 47139754 0x2cf4baa 0
6 4411100 0x434edc 0
1 56746451 0x361e1d3 0
8 4416089 0x436259 0
7 4411127 0x434ef7 0
4 50856626 0x30802b2 0
[root at compute-3-1 ps2]#
So it moved the file to another OST, but left the strip count as 1.
But when I cp''ed the file, it properly striped it across all the OSTs.
This doesn''t seem all that useful for rebalancing.
Brian Andrus
ITACS/Research Computing
Naval Postgraduate School
Monterey, California
voice: 831-656-6238
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110716/f2b8863c/attachment.html
Hi,
lfs_migrate will rebalance space usage across the OSTs of your filesystem by
copying files and deleting the original ones on the most empty OSTs. It will not
change stripe count of files. You have to migrate lots of files to see the new
balance. This is why you don''t see the usefulness with only one file.
lfs_migrate also accepts options to specify which OSTs you want to redistribute.
Sebastien.
________________________________
From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss-bounces
at lists.lustre.org] On Behalf Of Andrus, Brian Contractor
Sent: samedi 16 juillet 2011 18:31
To: lustre-discuss at lists.lustre.org
Subject: [Lustre-discuss] lfs_migrate for rebalancing
All,
Am I incorrect in the way I read that lfs_migrate can be used to rebalance a
filesystem after adding OSTs?
Here is some output on a 22M file that was all on one OST and we now have 9:
[root at compute-3-1 ps2]# lfs getstripe PS01-071.xml
PS01-071.xml
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_stripe_offset: 6
obdidx objid objid group
6 4411098 0x434eda 0
[root at compute-3-1 ps2]# lfs_migrate -y PS01-071.xml
PS01-071.xml: done
[root at compute-3-1 ps2]# lfs getstripe PS01-071.xml
PS01-071.xml
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_stripe_offset: 4
obdidx objid objid group
4 50856625 0x30802b1 0
[root at compute-3-1 ps2]# cp PS01-071.xml PS01-071.xml.tmp
[root at compute-3-1 ps2]# lfs getstripe PS01-071.xml.tmp
PS01-071.xml.tmp
lmm_stripe_count: 9
lmm_stripe_size: 1048576
lmm_stripe_offset: 2
obdidx objid objid group
2 73524719 0x461e5ef 0
5 4389362 0x42f9f2 0
3 53566843 0x3315d7b 0
0 47139754 0x2cf4baa 0
6 4411100 0x434edc 0
1 56746451 0x361e1d3 0
8 4416089 0x436259 0
7 4411127 0x434ef7 0
4 50856626 0x30802b2 0
[root at compute-3-1 ps2]#
So it moved the file to another OST, but left the strip count as 1.
But when I cp''ed the file, it properly striped it across all the OSTs.
This doesn''t seem all that useful for rebalancing.
Brian Andrus
ITACS/Research Computing
Naval Postgraduate School
Monterey, California
voice: 831-656-6238
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110718/f7cf1294/attachment-0001.html