huangql
2011-Aug-02 10:15 UTC
[Lustre-discuss] error creating fid on OSTs when there is free space of the pool
Dear all, These days I found the error messages on MDS to show error creating fid on some OSTs (96% usage)when there is still free space of the pool. The OSTs usage of the pool is not balance due to they don''t come online at the same time. I think the new created files should be distributed to the OSTs (51% usage)with much more free space just as described in manual. However, the MDS also stripe files to the OSTs(96% usage), so we get many error messages in MDS and client got errror ''No free space left''. Aug 2 14:58:05 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 15 previous similar messages Aug 2 15:01:37 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x210f1ce sub-object on OST idx 45/14: rc = -107 Aug 2 15:01:37 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 24 previous similar messages Aug 2 15:01:39 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x4c1f7cf sub-object on OST idx 45/14: rc = -107 Aug 2 15:01:39 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 19 previous similar messages Aug 2 15:02:27 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x63fac3a sub-object on OST idx 45/14: rc = -107 Aug 2 15:02:27 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 3 previous similar messages Aug 2 15:03:17 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x5e40ece sub-object on OST idx 45/14: rc = -107 Aug 2 15:03:17 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 25 previous similar messages Aug 2 15:03:39 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x253adce sub-object on OST idx 45/14: rc = -107 Aug 2 15:03:39 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 25 previous similar messages Aug 2 15:04:07 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x5e40ed4 sub-object on OST idx 45/14: rc = -107 Aug 2 15:07:27 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x5e41669 sub-object on OST idx 55/14: rc = -107 Aug 2 15:07:27 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 25 previous similar messages Aug 2 15:08:36 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x253ade2 sub-object on OST idx 55/14: rc = -107 Aug 2 15:08:36 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 51 previous similar messages Aug 2 15:17:39 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x510e70b sub-object on OST idx 57/14: rc = -107 Aug 2 15:17:39 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 52 previous similar messages Aug 2 15:32:35 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x5e49673 sub-object on OST idx 43/14: rc = -107 Aug 2 15:32:35 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 19 previous similar messages Aug 2 15:33:26 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x5e49678 sub-object on OST idx 43/14: rc = -107 Aug 2 15:33:26 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) Skipped 21 previous similar messages Aug 2 15:34:31 commds kernel: LustreError: 5174:0:(lov_request.c:692:lov_update_create_set()) error creating fid 0x4bfefad sub-object on OST idx 43/14: rc = -107>From the manual, it tells us to config via the proc file. But for us, the parameter value is 90% in default which means the files is likely to write to the OSTs with much more free space rather than the OSTs with more than 90% usage, I think.[root at commds ~]# cat /proc/fs/lustre/lov/publicfs-mdtlov/qos_prio_free 90% Could anyone give me some explaination for this and how to config it to adjust the OST space balance? Thank you very much in advance! Cheers, Qiulan Huang Computer Center, IHEP 2011-08-02 huangql -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110802/4feb1e26/attachment.html