Xen devel - Sep 2007 - [PATCH][RFC] making "xm dump-core" paralell

Hi,

xm has a subcommand "dump-core", which dumps the core image of
the domainU to the file.
This subcommand is useful, but it has the problem that xend
can''t deal with other xm commands while dealing with
"dump-core".

The attached patch fixes this problem. ( for cset#15880:a00cc97b392a )

Details of the attached patch are as follows:
- Add xc_dumpcore program. This program only calls xc_domain_dump()
  in libxc to dump the core image of a domainU.
- Change xend to make a xc_dumpcore process when it deal with
"dump-core".
  the xend''s thread which make the process waits for the process to
  finish dumping the core image.
- Make XendDumpLock to avoid that xend dumps the same domainU twice or more
  at the same time. XendDumpLock read/write lock informaition of
"dump-core"
  from/to xenstore.
- dump-core has --live and --crash options. These options send requests to
  pause/unpause, destroy the domainU. I move the code into dumpcore thread
  because xm cannot use the options well at the sencond dump-core request.


TODO:
- Xend should be fixed not to destory the domainU that it is dumping by
  "xm destoroy."  There are some other xm commands needed this kind of
  the special care but I don''t implement them at this moment yet.
  So, for example, if you run "xm dump-core <domid>" and run
"xm destroy <same domid>"
  immediately, I don''t know what happen (system may crash).

I would like you to give me comments.
Thank you.

Signed-off-by: Rikiya Ayukawa <ayukawa.rikiya@np.css.fujitsu.com>

Rikiya Ayukawa



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Sep 27, 2007 at 11:12:20AM +0900, Rikiya Ayukawa wrote:
> - Add xc_dumpcore program. This program only calls xc_domain_dump()
>   in libxc to dump the core image of a domainU.
Why?

regards
john

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 27/9/07 03:12, "Rikiya Ayukawa"
<ayukawa.rikiya@np.css.fujitsu.com>
wrote:
> Hi,
> 
> xm has a subcommand "dump-core", which dumps the core image of
> the domainU to the file.
> This subcommand is useful, but it has the problem that xend
> can''t deal with other xm commands while dealing with
"dump-core".
> 
> The attached patch fixes this problem. ( for cset#15880:a00cc97b392a )
It seems rather a lot of code to do something which should be quite simple?
You seem to have two reasons for doing a dump (xm dump-core, crash), and the
locking protocol seems to know about those somehow -- what''s that
about?

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi,

# Sorry, I mistook the function name on e-mail.
# xc_domain_dump() -> xc_domain_dumpcore()
> - Add xc_dumpcore program. This program only calls xc_domain_dump()
> >   in libxc to dump the core image of a domainU.
>   
>
> Why?
To make xend call indirectly xc_domain_dumpcore() written by C.
I think this is similar to xc_save and xc_restore programs.

xend (cset#15880:a00cc97b392a) calls xc_domain_dumpcore() directly.
It takes xend a lot of time to finish this C function.

Until the xend''s thread finishes xc_domain_dumpcore(), any other
xend''s
thread
don''t run because of GIL (global interpreter lock) in CPython
specification.

http://docs.python.org/api/threads.html

# If xc_domain_dumpcore() release GIL sometimes, other xend''s thread
can
run.
# But I guess not.

By changing xend to create a xc_dumpcore process and
to wait for the process to finish, xend don''t begin to call C function
directly,
so that other xend''s thread can run while dealing with dump-core.

Thank you,

Rikiya Ayukawa


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Sep 27, 2007 at 05:36:23PM +0900, Rikiya Ayukawa wrote:
> # Sorry, I mistook the function name on e-mail.
> # xc_domain_dump() -> xc_domain_dumpcore()
> 
> > - Add xc_dumpcore program. This program only calls xc_domain_dump()
> > >   in libxc to dump the core image of a domainU.
> >   
> >
> > Why?
> 
> To make xend call indirectly xc_domain_dumpcore() written by C.
> I think this is similar to xc_save and xc_restore programs.
> 
> xend (cset#15880:a00cc97b392a) calls xc_domain_dumpcore() directly.
> It takes xend a lot of time to finish this C function.
> 
> Until the xend''s thread finishes xc_domain_dumpcore(), any other
xend''s
> thread
> don''t run because of GIL (global interpreter lock) in CPython
specification.
> 
> http://docs.python.org/api/threads.html
But if you''re forking, why can''t you do it in Python?

regards
john

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi, John
>On Thu, Sep 27, 2007 at 05:36:23PM +0900, Rikiya Ayukawa wrote:
>
>> # Sorry, I mistook the function name on e-mail.
>> # xc_domain_dump() -> xc_domain_dumpcore()
>> 
>> > - Add xc_dumpcore program. This program only calls
xc_domain_dump()
>> > >   in libxc to dump the core image of a domainU.
>> >   
>> >
>> > Why?
>> 
>> To make xend call indirectly xc_domain_dumpcore() written by C.
>> I think this is similar to xc_save and xc_restore programs.
>> 
>> xend (cset#15880:a00cc97b392a) calls xc_domain_dumpcore() directly.
>> It takes xend a lot of time to finish this C function.
>> 
>> Until the xend''s thread finishes xc_domain_dumpcore(), any
other xend''s
>> thread
>> don''t run because of GIL (global interpreter lock) in CPython
specification.
>> 
>> http://docs.python.org/api/threads.html
>
>But if you''re forking, why can''t you do it in Python?
The child process of xend can call xc_domain_dumpcore,
but it is worse than execing the new dump command
because we must be careful about some more cases.

For example if we do not use exec(), at least, the child of xend
must unregister the xenwatch to dump a domU at the domU crashing time
because the child process of xend has the same functions of parent.
I think there are many other case that we must be careful
about multi process of xend.

xm save/restore also fork+exec. I think it''s the same reason.
Is that right, Keir?

Best Regards,

Akio Takebe


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Sep 28, 2007 at 08:02:18PM +0900, Akio Takebe wrote:
> xm save/restore also fork+exec. I think it''s the same reason.
> Is that right, Keir?
My understanding is that the only reason we have separate binaries for
that is because in the past save/restore was unstable, and had a
tendency to crash xend.

john

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > xm save/restore also fork+exec. I think it''s the same reason.
> > Is that right, Keir?
> 
> My understanding is that the only reason we have separate binaries for
> that is because in the past save/restore was unstable, and had a
> tendency to crash xend.
I can''t see much downside to having a separate process. It makes it
easier to ionice the IO etc.

Ian

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Sep 28, 2007 at 08:53:01PM +0100, Ian Pratt wrote:
> > > xm save/restore also fork+exec. I think it''s the same
reason.
> > > Is that right, Keir?
> > 
> > My understanding is that the only reason we have separate binaries for
> > that is because in the past save/restore was unstable, and had a
> > tendency to crash xend.
> 
> I can''t see much downside to having a separate process. It makes
it
> easier to ionice the IO etc.
A separate process isn''t the sticking point, it''s whether we
really need
a separate C implementation.

john

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Sep 28, 2007 at 08:53:01PM +0100, Ian Pratt
wrote:
> > > xm save/restore also fork+exec. I think it''s the same
reason.
> > > Is that right, Keir?
> > 
> > My understanding is that the only reason we have separate binaries for
> > that is because in the past save/restore was unstable, and had a
> > tendency to crash xend.
> 
> I can''t see much downside to having a separate process. It makes
it
> easier to ionice the IO etc.
Well the fact that XenD has to parse the STDOUT from these processes applying
a regex of ''^ERROR:'' in an attempt to try and catch errors is
one fairly
large downside.  Combine that with the incredibly complex interactions
between XenD and QEMU, again with tricky error reporting and the result is
a pretty hard thing to debug.

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel