Btrfs devel - Jun 2011 - [btrfs-delalloc-]

Hello all.

What we have:
SL6 - kernel 2.6.32-131.2.1.el6.x86_64
btrfs on mdadm RAID5 with 8 HDD - 27T partition.

I see this at top:
1182 root      20   0     0    0    0 R 100.0  0.0  16:39.73 
[btrfs-delalloc-]

And LA is grow. What is this and how can I fix it?

-- 
Best regards,
Proskurin Kirill
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Monday 27 of June 2011 11:04:06 Proskurin Kirill
wrote:
> Hello all.
> 
> What we have:
> SL6 - kernel 2.6.32-131.2.1.el6.x86_64
> btrfs on mdadm RAID5 with 8 HDD - 27T partition.
> 
> I see this at top:
> 1182 root      20   0     0    0    0 R 100.0  0.0  16:39.73
> [btrfs-delalloc-]
> 
> And LA is grow. What is this and how can I fix it?
delalloc is a delayed allocation kernel thread -- it probably means something 
is writing large amounts of data to the file system

2.6.32 is *old* as far as btrfs is concerned, there have been many bugs fixed 
and performance improvements since

Regards,
-- 
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawerów 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 06/27/2011 05:21 PM, Hubert Kario wrote:
> On Monday 27 of June 2011 11:04:06 Proskurin Kirill wrote:
>> Hello all.
>>
>> What we have:
>> SL6 - kernel 2.6.32-131.2.1.el6.x86_64
>> btrfs on mdadm RAID5 with 8 HDD - 27T partition.
>>
>> I see this at top:
>> 1182 root      20   0     0    0    0 R 100.0  0.0  16:39.73
>> [btrfs-delalloc-]
>>
>> And LA is grow. What is this and how can I fix it?
>
> delalloc is a delayed allocation kernel thread -- it probably means
something
> is writing large amounts of data to the file system
>
> 2.6.32 is *old* as far as btrfs is concerned, there have been many bugs
fixed
> and performance improvements since
Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse.
Now I see [btrfs-transacti] & btrfs-endio-wri] 80-100% all the time and 
io performance looks like lower then before.

Our scribe daemon in state D most of the time with half of a normal 
load. Only kernel was changed.

Any performance tune recommendation?

-- 
Best regards,
Proskurin Kirill
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 06/29/2011 11:37 AM, Proskurin Kirill wrote:
> On 06/27/2011 05:21 PM, Hubert Kario wrote:
>> On Monday 27 of June 2011 11:04:06 Proskurin Kirill wrote:
>>> Hello all.
>>>
>>> What we have:
>>> SL6 - kernel 2.6.32-131.2.1.el6.x86_64
>>> btrfs on mdadm RAID5 with 8 HDD - 27T partition.
>>>
>>> I see this at top:
>>> 1182 root      20   0     0    0    0 R 100.0  0.0  16:39.73
>>> [btrfs-delalloc-]
>>>
>>> And LA is grow. What is this and how can I fix it?
>>
>> delalloc is a delayed allocation kernel thread -- it probably means
>> something
>> is writing large amounts of data to the file system
>>
>> 2.6.32 is *old* as far as btrfs is concerned, there have been many
>> bugs fixed
>> and performance improvements since
> 
> Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse.
> Now I see [btrfs-transacti] & btrfs-endio-wri] 80-100% all the time and
> io performance looks like lower then before.
> 
> Our scribe daemon in state D most of the time with half of a normal
> load. Only kernel was changed.
> 
> Any performance tune recommendation?
> 
Can you get sysrq+w while this problem is happening so we can see whats
going on?  Thanks,

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 06/29/2011 08:14 PM, Josef Bacik wrote:
>> Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse.
>> Now I see [btrfs-transacti]&  btrfs-endio-wri] 80-100% all the time
and
>> io performance looks like lower then before.
>>
>> Our scribe daemon in state D most of the time with half of a normal
>> load. Only kernel was changed.
>>
>> Any performance tune recommendation?
>>
>
> Can you get sysrq+w while this problem is happening so we can see whats
> going on?  Thanks,
I attached sysrq+w. Hope it helps.

-- 
Best regards,
Proskurin Kirill

On 06/30/2011 10:12 AM, Proskurin Kirill wrote:
> On 06/29/2011 08:14 PM, Josef Bacik wrote:
>>> Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse.
>>> Now I see [btrfs-transacti]&  btrfs-endio-wri] 80-100% all the
time and
>>> io performance looks like lower then before.
>>>
>>> Our scribe daemon in state D most of the time with half of a normal
>>> load. Only kernel was changed.
>>>
>>> Any performance tune recommendation?
>>>
>>
>> Can you get sysrq+w while this problem is happening so we can see whats
>> going on?  Thanks,
> 
> I attached sysrq+w. Hope it helps.
> 
Heh so it just looks like you are writing a bunch of stuff, that''s not
particularly helpful.  What does this program do generically?  How full
is your fs?  Do you have snapshots/subvolumes?  If so how many?  Thanks,

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 06/30/2011 09:13 PM, Josef Bacik wrote:
> On 06/30/2011 10:12 AM, Proskurin Kirill wrote:
>> On 06/29/2011 08:14 PM, Josef Bacik wrote:
>>>> Ok - I upgrade to 2.6.39-2 but it is seems to all things get
worse.
>>>> Now I see [btrfs-transacti]&   btrfs-endio-wri] 80-100% all
the time and
>>>> io performance looks like lower then before.
>>>>
>>>> Our scribe daemon in state D most of the time with half of a
normal
>>>> load. Only kernel was changed.
>>>>
>>>> Any performance tune recommendation?
>>>>
>>>
>>> Can you get sysrq+w while this problem is happening so we can see
whats
>>> going on?  Thanks,
>>
>> I attached sysrq+w. Hope it helps.
>>
>
> Heh so it just looks like you are writing a bunch of stuff, that''s
not
> particularly helpful.  What does this program do generically?  How full
> is your fs?  Do you have snapshots/subvolumes?  If so how many?  Thanks,
Scribe is taling logs by network. ~300 hosts send it to it and scribe 
write them to fs. I use zlib compression on btrfs partition.

Mount options is noatime,noacl,compress-force=zlib

iostat -x 1 tells me what util of btrfs partition is from 50% to ~90% 
all the time.

Btrfs partition is near full:
Size  Used Avail Use%
28T   26T  2.1T  93%

I don`t use snapshots/subvolumes at this time so none.

-- 
Best regards,
Proskurin Kirill
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 06/30/2011 09:13 PM, Josef Bacik wrote:
> On 06/30/2011 10:12 AM, Proskurin Kirill wrote:
>> On 06/29/2011 08:14 PM, Josef Bacik wrote:
>>>> Ok - I upgrade to 2.6.39-2 but it is seems to all things get
worse.
>>>> Now I see [btrfs-transacti]&   btrfs-endio-wri] 80-100% all
the time and
>>>> io performance looks like lower then before.
>>>>
>>>> Our scribe daemon in state D most of the time with half of a
normal
>>>> load. Only kernel was changed.
>>>>
>>>> Any performance tune recommendation?
>>>>
>>>
>>> Can you get sysrq+w while this problem is happening so we can see
whats
>>> going on?  Thanks,
>>
>> I attached sysrq+w. Hope it helps.
>>
>
> Heh so it just looks like you are writing a bunch of stuff, that''s
not
> particularly helpful.  What does this program do generically?  How full
> is your fs?  Do you have snapshots/subvolumes?  If so how many?  Thanks,
I found what my btrfs partition now is really slow. Something wrong 
happend. I try on 2.6.32 and 2.6.39 - same result. It is because 
partition is almost full?

I run a simple cycle to clean some old files:
for i in `cat /tmp/delit`; do rm -f $i ; done

And it is takes about 5-10 second per file to delete.
I get sysrq+w while rm is work - it is attached.

-- 
Best regards,
Proskurin Kirill