Pkg xen devel - Feb 2019 - [Xen-devel] [admin] [BUG] task jbd2/xvda4-8:174 blocked for more than 120 seconds.

Hello,

Hans van Kranenburg, le sam. 09 févr. 2019 17:01:55 +0100, a
ecrit:
> > I have forwarded the original mail: all VM I/O get stuck, and thus the
> > VM becomes unusable.
> 
> These are in many cases the symptoms of running out of "grant
frames".
Oh!  That could be it indeed.  I'm wondering what could be monopolizing
them, though, and why +deb9u11 is affected while +deb9u10 is not.  I'm
afraid increasing the gnttab max size to 32 might just defer filling it
up.
>   -# ./xen-diag  gnttab_query_size 5
>   domid=5: nr_frames=11, max_nr_frames=32
The current value is 31 over max 32 indeed.
> With Xen 4.8, you can add gnttab_max_frames=64 (or another number, but
> higher than the default 32) to the xen hypervisor command line and reboot.
admin@: I made the modification in the grub config. We can probably try
to reboot with the newer hypervisor, and monitor that value.

Samuel

On 2/10/19 12:35 AM, Samuel Thibault wrote:
> Hello,
> 
> Hans van Kranenburg, le sam. 09 févr. 2019 17:01:55 +0100, a ecrit:
>>> I have forwarded the original mail: all VM I/O get stuck, and thus
the
>>> VM becomes unusable.
>>
>> These are in many cases the symptoms of running out of "grant
frames".
> 
> Oh!  That could be it indeed.  I'm wondering what could be monopolizing
> them, though, and why +deb9u11 is affected while +deb9u10 is not.  I'm
> afraid increasing the gnttab max size to 32 might just defer filling it
> up.
> 
>>   -# ./xen-diag  gnttab_query_size 5
>>   domid=5: nr_frames=11, max_nr_frames=32
> 
> The current value is 31 over max 32 indeed.
Assuming this is grant v1, there are still 4096/8=512 grant references available
(32-31=1 frame available). I do not think the I/O hang can be affected by the
lack of grant entry.

If to increase the max frame to 64 takes effect, it is weird why the I/O would
hang when there are still 512 entries available.

Dongli Zhang
> 
>> With Xen 4.8, you can add gnttab_max_frames=64 (or another number, but
>> higher than the default 32) to the xen hypervisor command line and
reboot.
> 
> admin@: I made the modification in the grub config. We can probably try
> to reboot with the newer hypervisor, and monitor that value.
> 
> Samuel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel at lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel
>

Hello,

Dongli Zhang, le lun. 11 févr. 2019 09:37:43 +0800, a
ecrit:
> On 2/10/19 12:35 AM, Samuel Thibault wrote:
> > Hello,
> > 
> > Hans van Kranenburg, le sam. 09 févr. 2019 17:01:55 +0100, a ecrit:
> >>> I have forwarded the original mail: all VM I/O get stuck, and
thus the
> >>> VM becomes unusable.
> >>
> >> These are in many cases the symptoms of running out of "grant
frames".
> > 
> > Oh!  That could be it indeed.  I'm wondering what could be
monopolizing
> > them, though, and why +deb9u11 is affected while +deb9u10 is not. 
I'm
> > afraid increasing the gnttab max size to 32 might just defer filling
it
> > up.
> > 
> >>   -# ./xen-diag  gnttab_query_size 5
> >>   domid=5: nr_frames=11, max_nr_frames=32
> > 
> > The current value is 31 over max 32 indeed.
> 
> Assuming this is grant v1, there are still 4096/8=512 grant references
available
> (32-31=1 frame available). I do not think the I/O hang can be affected by
the
> lack of grant entry.
> 
> If to increase the max frame to 64 takes effect, it is weird why the I/O
would
> hang when there are still 512 entries available.
After reboot with gnttab_max_frames=256, I could let the VM run for some
time, nr_frames is now 32.

Samuel

On 2/11/19 2:37 AM, Dongli Zhang wrote:
> 
> On 2/10/19 12:35 AM, Samuel Thibault wrote:
>>
>> Hans van Kranenburg, le sam. 09 févr. 2019 17:01:55 +0100, a ecrit:
>>>> I have forwarded the original mail: all VM I/O get stuck, and
thus the
>>>> VM becomes unusable.
>>>
>>> These are in many cases the symptoms of running out of "grant
frames".
>>
>> Oh!  That could be it indeed.  I'm wondering what could be
monopolizing
>> them, though, and why +deb9u11 is affected while +deb9u10 is not. 
I'm
>> afraid increasing the gnttab max size to 32 might just defer filling it
>> up.
>>
>>>   -# ./xen-diag  gnttab_query_size 5
>>>   domid=5: nr_frames=11, max_nr_frames=32
>>
>> The current value is 31 over max 32 indeed.
> 
> Assuming this is grant v1, there are still 4096/8=512 grant references
available
> (32-31=1 frame available). I do not think the I/O hang can be affected by
the
> lack of grant entry.
I suspect that 31 measurement was taken when the domU was not hanging yet.
> If to increase the max frame to 64 takes effect, it is weird why the I/O
would
> hang when there are still 512 entries available.
> 
>>> With Xen 4.8, you can add gnttab_max_frames=64 (or another number,
but
>>> higher than the default 32) to the xen hypervisor command line and
reboot.
>>
>> admin@: I made the modification in the grub config. We can probably try
>> to reboot with the newer hypervisor, and monitor that value.
K