thr3ads.net - Xen devel - [Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown [Nov 2007]

If this information is useful, please help other people find it:
Share via:

David Edmondson

2007-Nov-27 09:26 UTC

[Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown

In testing our implementation of the hypervisor copy based backend- 
 >frontend networking changes, we see what I believe are spurious  
warning messages during the shutdown of the frontend domain:

(XEN) /export/build/schuster/xvm-gate/xen.hg/xen/include/asm/mm.h: 
189:d0 Error pfn 30e290: rd=ffff830000fcf100, od=ffff830000fcf100,  
caf=00000000, taf=0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff83000010f240>] get_page+0x107/0x1b4
(XEN)    [<ffff83000010f10a>] get_page_and_type+0x21/0x50
(XEN)    [<ffff8300001116c4>] __gnttab_copy+0x3f5/0x5b4
(XEN)    [<ffff830000111971>] gnttab_copy+0xee/0x1c4
(XEN)    [<ffff830000111dbd>] do_grant_table_op+0x376/0x3bc
(XEN)    [<ffff8300001b83e2>] syscall_enter+0xa2/0xfc
(XEN)
(XEN) Guest stack trace from rbp=ffff830000ff3cf8:
(XEN)  ???????????????? <G><2>grant_table.c:990:d0
do_grant_table_op:
domain 0, cmd 5, count 1
(XEN) /export/build/schuster/xvm-gate/xen.hg/xen/include/asm/mm.h: 
189:d0 Error pfn 30fc2a: rd=ffff830000fcf100, od=0000000000000000,  
caf=00000000, taf=0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff83000010f240>] get_page+0x107/0x1b4
(XEN)    [<ffff83000010f10a>] get_page_and_type+0x21/0x50
(XEN)    [<ffff8300001116c4>] __gnttab_copy+0x3f5/0x5b4
(XEN)    [<ffff830000111971>] gnttab_copy+0xee/0x1c4
(XEN)    [<ffff830000111dbd>] do_grant_table_op+0x376/0x3bc
(XEN)    [<ffff8300001b83e2>] syscall_enter+0xa2/0xfc
(XEN)

What we think is happening is that the frontend dies and most of its''  
pages are freed (those that are not referenced by another domain).  
The backend doesn''t know that the frontend died yet, so it''s
still
trying to pass packets along to it. It has the rx ring mapped  
(meaning that it can''t be freed) and reads previously advertised  
grant references from it. Those grants now refer to pages that are no  
longer valid, so get_page() complains (the pages are no longer valid  
as only the frontend had references to them and they were freed).

__gnttab_copy() itself seems prepared for this situation, as failures  
to grab the target page due to a dying domain are correctly handled:

     if ( !get_page_and_type(mfn_to_page(d_frame), dd,  
PGT_writable_page) )
     {
         if ( !test_bit(_DOMF_dying, &dd->domain_flags) )
             gdprintk(XENLOG_WARNING, "Could not get dst frame %lx 
\n", d_frame);
         rc = GNTST_general_error;
         goto error_out;
     }

In our testing we believe that we''re following this path (_DOMF_dying  
is set and rc == GNTST_general_error) and that we handle the failure  
correctly.

The corresponding failure mode in the page flip code path doesn''t  
result in any INFO warnings. Should they exist in this case?

dme.
-- 
David Edmondson, Solaris Engineering, http://dme.org



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2007-Nov-27 09:41 UTC

head link

Re: [Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown

I don''t think you should get a backtrace. Did you add that?

 -- Keir

On 27/11/07 09:26, "David Edmondson" <dme@sun.com> wrote:
> In testing our implementation of the hypervisor copy based backend-
>> frontend networking changes, we see what I believe are spurious
> warning messages during the shutdown of the frontend domain:
> 
> (XEN) /export/build/schuster/xvm-gate/xen.hg/xen/include/asm/mm.h:
> 189:d0 Error pfn 30e290: rd=ffff830000fcf100, od=ffff830000fcf100,
> caf=00000000, taf=0000000000000000
> (XEN) Xen call trace:
> (XEN)    [<ffff83000010f240>] get_page+0x107/0x1b4
> (XEN)    [<ffff83000010f10a>] get_page_and_type+0x21/0x50
> (XEN)    [<ffff8300001116c4>] __gnttab_copy+0x3f5/0x5b4
> (XEN)    [<ffff830000111971>] gnttab_copy+0xee/0x1c4
> (XEN)    [<ffff830000111dbd>] do_grant_table_op+0x376/0x3bc
> (XEN)    [<ffff8300001b83e2>] syscall_enter+0xa2/0xfc
> (XEN)
> (XEN) Guest stack trace from rbp=ffff830000ff3cf8:
> (XEN)  ???????????????? <G><2>grant_table.c:990:d0
do_grant_table_op:
> domain 0, cmd 5, count 1
> (XEN) /export/build/schuster/xvm-gate/xen.hg/xen/include/asm/mm.h:
> 189:d0 Error pfn 30fc2a: rd=ffff830000fcf100, od=0000000000000000,
> caf=00000000, taf=0000000000000000
> (XEN) Xen call trace:
> (XEN)    [<ffff83000010f240>] get_page+0x107/0x1b4
> (XEN)    [<ffff83000010f10a>] get_page_and_type+0x21/0x50
> (XEN)    [<ffff8300001116c4>] __gnttab_copy+0x3f5/0x5b4
> (XEN)    [<ffff830000111971>] gnttab_copy+0xee/0x1c4
> (XEN)    [<ffff830000111dbd>] do_grant_table_op+0x376/0x3bc
> (XEN)    [<ffff8300001b83e2>] syscall_enter+0xa2/0xfc
> (XEN)
> 
> What we think is happening is that the frontend dies and most of
its''
> pages are freed (those that are not referenced by another domain).
> The backend doesn''t know that the frontend died yet, so
it''s still
> trying to pass packets along to it. It has the rx ring mapped
> (meaning that it can''t be freed) and reads previously advertised
> grant references from it. Those grants now refer to pages that are no
> longer valid, so get_page() complains (the pages are no longer valid
> as only the frontend had references to them and they were freed).
> 
> __gnttab_copy() itself seems prepared for this situation, as failures
> to grab the target page due to a dying domain are correctly handled:
> 
>      if ( !get_page_and_type(mfn_to_page(d_frame), dd,
> PGT_writable_page) )
>      {
>          if ( !test_bit(_DOMF_dying, &dd->domain_flags) )
>              gdprintk(XENLOG_WARNING, "Could not get dst frame %lx
> \n", d_frame);
>          rc = GNTST_general_error;
>          goto error_out;
>      }
> 
> In our testing we believe that we''re following this path
(_DOMF_dying
> is set and rc == GNTST_general_error) and that we handle the failure
> correctly.
> 
> The corresponding failure mode in the page flip code path doesn''t
> result in any INFO warnings. Should they exist in this case?
> 
> dme.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

David Edmondson

2007-Nov-27 09:42 UTC

head link

Re: [Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown

On 27 Nov 2007, at 9:41am, Keir Fraser wrote:
> I don''t think you should get a backtrace. Did you add that?
Yes, to aid in debugging. Sorry for the confusion.

dme.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

David Edmondson

2007-Nov-27 09:53 UTC

head link

Re: [Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown

On 27 Nov 2007, at 9:42am, David Edmondson wrote:>> I don''t think you should get a backtrace. Did you add that?
>
> Yes, to aid in debugging. Sorry for the confusion.
>
This part:
> (XEN) /export/build/schuster/xvm-gate/xen.hg/xen/include/asm/mm.h:
> 189:d0 Error pfn 30e290: rd=ffff830000fcf100, od=ffff830000fcf100,
> caf=00000000, taf=0000000000000000
is what I''m concerned about. It still seems spurious.

dme.
-- 
David Edmondson, Solaris Engineering, http://dme.org



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2007-Nov-27 12:29 UTC

head link

Re: [Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown

Okay, as for the issue of that line coming out in the first place at
XENLOG_INFO level, I''ll take a look and maybe downgrade it to
XENLOG_DEBUG.

 -- Keir


On 27/11/07 09:42, "David Edmondson" <dme@sun.com> wrote:
> 
> On 27 Nov 2007, at 9:41am, Keir Fraser wrote:
> 
>> I don''t think you should get a backtrace. Did you add that?
> 
> Yes, to aid in debugging. Sorry for the confusion.
> 
> dme.
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Maybe Matching Threads

Search for more apparently analagous threads

Xen devel - Nov 2007 - spurious warnings from get_page() via gnttab_copy() during frontend shutdown

[Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown

Re: [Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown

Re: [Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown

Re: [Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown

Re: [Xen-devel] spurious warnings from get_page() via gnttab_copy() during frontend shutdown

Maybe Matching Threads