Ian Pratt
2005-Oct-30 21:36 UTC
RE: [Xen-devel] [PATCH] Changes to allow dynamic enabling/disabling oftrace buffers
Nice work. Has this been well tested on both 32b and 64b? I''m still keen that we do the virq plumbing to enable the dom0 reader to select on an fd properly. It would be good to dispence with the sequence number too. Thanks, Ian> This is a set of changes which allow the tracebuffer > functionality to be enabled and disabled dynamically while > the system is running. The existing DOM0_TBUFCONTROL > hypercall is used, and three new operations have been added: > enable, disable, and set_size. This is to address concerns > that use of the trace buffers impacts system performance. > > The initial value of opt_tbuf_size is set to zero, so that > trace buffers remain disabled by default. They can be enabled > as they used to be, by adding tbuf_size= to the xen command > line at boot time. They can also be turned on and off any > time when the system is running. The size of the trace buffer > allocation can also be changed dynamically while the system > is running. The trace buffers may be completely deallocated > by setting the size to zero. No change in buffer size can be > done while tracing is enabled; > > Other Changes made: > - Update the constants in tools/xentrace/formats to match > those in public/trace.h > - libxc functions for enable/disable, get/set size Still left to do: > - remove all ifdef''s for trace buffers and the ''trace'' build option > - xm command to do enable/disable trace buffers and get/set size > > > Signed-off-by: Rob Gardner <rob.gardner@hp.com> > > > # HG changeset patch > # User rob.gardner@hp.com > # Node ID 63d05000cfa355c0696aa7efea67668c8cae19f0 > # Parent e70ea9465b310e1cba9678ac4e9ad534bb8b670a > Changes to allow the tracebuffer functionality to be enabled and > disabled dynamically while the system is running. > > diff -r e70ea9465b31 -r 63d05000cfa3 tools/libxc/Makefile > --- a/tools/libxc/Makefile Thu Oct 27 09:53:02 2005 > +++ b/tools/libxc/Makefile Sun Oct 30 20:59:45 2005 > @@ -23,6 +23,7 @@ > SRCS += xc_physdev.c > SRCS += xc_private.c > SRCS += xc_sedf.c > +SRCS += xc_tbuf.c > BUILD_SRCS += xc_linux_build.c > BUILD_SRCS += xc_load_bin.c > BUILD_SRCS += xc_load_elf.c > diff -r e70ea9465b31 -r 63d05000cfa3 tools/libxc/xenctrl.h > --- a/tools/libxc/xenctrl.h Thu Oct 27 09:53:02 2005 > +++ b/tools/libxc/xenctrl.h Sun Oct 30 20:59:45 2005 > @@ -149,7 +149,7 @@ > const char *corename); > > /* > - * This function sets the maximum number vcpus that a domian > may create. > + * This function sets the maximum number of vcpus that a > domain may create. > * > * @parm xc_handle a handle to an open hypervisor interface. > * @parm domid the domain id in which vcpus are to be created. > @@ -483,6 +483,47 @@ > > /* Get current total pages allocated to a domain. */ > long xc_get_tot_pages(int xc_handle, uint32_t domid); > + > + > +/* > + * Trace Buffer Operations > + */ > + > +/** > + * This function enables or disables tracing. Trace buffer > memory must > + * be already allocated by setting the size to a non-zero > value, otherwise > + * tracing cannot be enabled. > + * > + * @parm xc_handle a handle to an open hypervisor interface > + * @parm enable the desired action, 1 for enable, 0 for disable > + * @return 0 on success, -1 on failure. > + */ > +int xc_tbuf_enable(int xc_handle, int enable); > + > +/** > + * This function sets the size of the trace buffers. Setting > it to zero > + * deallocates the memory used for trace buffers, and setting it to a > + * non-zero value specifies the number of pages per cpu to allocate. > + * To change the size of an existing allocation, you must > first deallocate > + * it then reallocate it. No change in size is allowed when > tracing is > + * enabled; A disable call must be made first. > + * > + * @parm xc_handle a handle to an open hypervisor interface > + * @parm size the size in pages per cpu for the trace buffers > + * @return 0 on success, -1 on failure. > + */ > +int xc_tbuf_set_size(int xc_handle, uint32_t size); > + > +/** > + * This function retrieves the current size of the trace buffers. > + * Note that the size returned is in terms of bytes, not pages. > + > + * @parm xc_handle a handle to an open hypervisor interface > + * @parm size will contain the size in bytes for the trace buffers > + * @return 0 on success, -1 on failure. > + */ > +int xc_tbuf_get_size(int xc_handle, uint32_t *size); > + > > /* Execute a privileged dom0 operation. */ > int xc_dom0_op(int xc_handle, dom0_op_t *op); > diff -r e70ea9465b31 -r 63d05000cfa3 tools/xentrace/Makefile > --- a/tools/xentrace/Makefile Thu Oct 27 09:53:02 2005 > +++ b/tools/xentrace/Makefile Sun Oct 30 20:59:45 2005 > @@ -14,7 +14,7 @@ > HDRS = $(wildcard *.h) > OBJS = $(patsubst %.c,%.o,$(wildcard *.c)) > > -BIN = xentrace > +BIN = xentrace tbctl setsize > SCRIPTS = xentrace_format > MAN1 = $(wildcard *.1) > MAN8 = $(wildcard *.8) > diff -r e70ea9465b31 -r 63d05000cfa3 tools/xentrace/formats > --- a/tools/xentrace/formats Thu Oct 27 09:53:02 2005 > +++ b/tools/xentrace/formats Sun Oct 30 20:59:45 2005 > @@ -1,16 +1,16 @@ > -0x00020001 CPU%(cpu)d %(tsc)d sched_add_domain > [ domid > = 0x%(1)08x, edomid = 0x%(2)08x ] > -0x00020002 CPU%(cpu)d %(tsc)d sched_rem_domain > [ domid > = 0x%(1)08x, edomid = 0x%(2)08x ] > -0x00020003 CPU%(cpu)d %(tsc)d domain_sleep > [ domid > = 0x%(1)08x, edomid = 0x%(2)08x ] > -0x00020004 CPU%(cpu)d %(tsc)d domain_wake > [ domid = > 0x%(1)08x, edomid = 0x%(2)08x ] > -0x00020005 CPU%(cpu)d %(tsc)d do_yield [ domid = > 0x%(1)08x, edomid = 0x%(2)08x ] > -0x00020006 CPU%(cpu)d %(tsc)d do_block [ domid = > 0x%(1)08x, edomid = 0x%(2)08x ] > -0x00020007 CPU%(cpu)d %(tsc)d domain_shutdown [ > domid = 0x%(1)08x, edomid = 0x%(2)08x, reason = 0x%(3)08x ] > -0x00020008 CPU%(cpu)d %(tsc)d sched_ctl > -0x00020009 CPU%(cpu)d %(tsc)d sched_adjdom > [ domid > = 0x%(1)08x ] > -0x0002000a CPU%(cpu)d %(tsc)d __enter_scheduler [ > prev<domid:edomid> = 0x%(1)08x : 0x%(2)08x, next<domid:edomid> = > 0x%(3)08x : 0x%(4)08x ] > -0x0002000B CPU%(cpu)d %(tsc)d s_timer_fn > -0x0002000c CPU%(cpu)d %(tsc)d t_timer_fn > -0x0002000d CPU%(cpu)d %(tsc)d dom_timer_fn > +0x0002f001 CPU%(cpu)d %(tsc)d sched_add_domain > [ domid > = 0x%(1)08x, edomid = 0x%(2)08x ] > +0x0002f002 CPU%(cpu)d %(tsc)d sched_rem_domain > [ domid > = 0x%(1)08x, edomid = 0x%(2)08x ] > +0x0002f003 CPU%(cpu)d %(tsc)d domain_sleep > [ domid > = 0x%(1)08x, edomid = 0x%(2)08x ] > +0x0002f004 CPU%(cpu)d %(tsc)d domain_wake > [ domid = > 0x%(1)08x, edomid = 0x%(2)08x ] > +0x0002f005 CPU%(cpu)d %(tsc)d do_yield [ domid = > 0x%(1)08x, edomid = 0x%(2)08x ] > +0x0002f006 CPU%(cpu)d %(tsc)d do_block [ domid = > 0x%(1)08x, edomid = 0x%(2)08x ] > +0x0002f007 CPU%(cpu)d %(tsc)d domain_shutdown [ > domid = 0x%(1)08x, edomid = 0x%(2)08x, reason = 0x%(3)08x ] > +0x0002f008 CPU%(cpu)d %(tsc)d sched_ctl > +0x0002f009 CPU%(cpu)d %(tsc)d sched_adjdom > [ domid > = 0x%(1)08x ] > +0x0002f00a CPU%(cpu)d %(tsc)d __enter_scheduler [ > prev<domid:edomid> = 0x%(1)08x : 0x%(2)08x, next<domid:edomid> = > 0x%(3)08x : 0x%(4)08x ] > +0x0002f00B CPU%(cpu)d %(tsc)d s_timer_fn > +0x0002f00c CPU%(cpu)d %(tsc)d t_timer_fn > +0x0002f00d CPU%(cpu)d %(tsc)d dom_timer_fn > > 0x00080001 CPU%(cpu)d %(tsc)d VMX_VMEXIT > [ domid > = 0x%(1)08x, eip = 0x%(2)08x, reason = 0x%(3)08x ] > 0x00080002 CPU%(cpu)d %(tsc)d VMX_VECTOR > [ domid > = 0x%(1)08x, eip = 0x%(2)08x, vector = 0x%(3)08x ] > diff -r e70ea9465b31 -r 63d05000cfa3 xen/Rules.mk > --- a/xen/Rules.mk Thu Oct 27 09:53:02 2005 > +++ b/xen/Rules.mk Sun Oct 30 20:59:45 2005 > @@ -6,7 +6,7 @@ > debug ?= n > perfc ?= n > perfc_arrays?= n > -trace ?= n > +trace ?= y > domu_debug ?= n > crash_debug ?= n > > diff -r e70ea9465b31 -r 63d05000cfa3 xen/common/trace.c > --- a/xen/common/trace.c Thu Oct 27 09:53:02 2005 > +++ b/xen/common/trace.c Sun Oct 30 20:59:45 2005 > @@ -5,8 +5,9 @@ > * > * Copyright (C) 2004 by Intel Research Cambridge > * > - * Author: Mark Williamson, mark.a.williamson@intel.com > - * Date: January 2004 > + * Authors: Mark Williamson, mark.a.williamson@intel.com > + * Rob Gardner, rob.gardner@hp.com > + * Date: October 2005 > * > * Copyright (C) 2005 Bin Ren > * > @@ -31,13 +32,14 @@ > #include <public/dom0_ops.h> > > /* opt_tbuf_size: trace buffer size (in pages) */ > -static unsigned int opt_tbuf_size = 10; > +static unsigned int opt_tbuf_size = 0; > integer_param("tbuf_size", opt_tbuf_size); > > /* Pointers to the meta-data objects for all system trace buffers */ > struct t_buf *t_bufs[NR_CPUS]; > > -/* a flag recording whether initialisation has been done */ > +/* a flag recording whether initialization has been done */ > +/* or more properly, if the tbuf subsystem is enabled right now */ > int tb_init_done = 0; > > /* which CPUs tracing is enabled on */ > @@ -45,25 +47,50 @@ > > /* which tracing events are enabled */ > u32 tb_event_mask = TRC_ALL; > -/** > - * init_trace_bufs - performs initialisation of the per-cpu > trace buffers. > - * > - * This function is called at start of day in order to > initialise the > per-cpu > + > +/** > + * init_trace_bufs - performs initialization of the per-cpu > trace buffers. > + * > + * This function is called at start of day in order to > initialize the > per-cpu > * trace buffers. The trace buffers are then available for > debugging > use, via > * the %TRACE_xD macros exported in <xen/trace.h>. > */ > void init_trace_bufs(void) > +{ > + extern int alloc_trace_bufs(void); > + > + if ( opt_tbuf_size == 0 ) > + { > + printk("Xen trace buffers: disabled\n"); > + return; > + } > + > + if (alloc_trace_bufs() == 0) { > + printk("Xen trace buffers: initialised\n"); > + wmb(); /* above must be visible before tb_init_done > flag set */ > + tb_init_done = 1; > + } > +} > + > +/** > + * alloc_trace_bufs - performs initialization of the per-cpu > trace buffers. > + * > + * This function is called at start of day in order to > initialize the > per-cpu > + * trace buffers. The trace buffers are then available for > debugging > use, via > + * the %TRACE_xD macros exported in <xen/trace.h>. > + * > + * This function may also be called later when enabling trace buffers > + * via the SET_SIZE hypercall. > + */ > +int alloc_trace_bufs(void) > { > int i, order; > unsigned long nr_pages; > char *rawbuf; > struct t_buf *buf; > - > + > if ( opt_tbuf_size == 0 ) > - { > - printk("Xen trace buffers: disabled\n"); > - return; > - } > + return -EINVAL; > > nr_pages = num_online_cpus() * opt_tbuf_size; > order = get_order_from_pages(nr_pages); > @@ -71,7 +98,7 @@ > if ( (rawbuf = alloc_xenheap_pages(order)) == NULL ) > { > printk("Xen trace buffers: memory allocation failed\n"); > - return; > + return -EINVAL; > } > > /* Share pages so that xentrace can map them. */ > @@ -88,13 +115,50 @@ > buf->rec = (struct t_rec *)(buf + 1); > buf->rec_addr = __pa(buf->rec); > } > - > - printk("Xen trace buffers: initialised\n"); > - > - wmb(); /* above must be visible before tb_init_done flag set */ > - > - tb_init_done = 1; > -} > + return 0; > +} > + > + > +/** > + * tb_set_size - handle the logic involved with dynamically > + * allocating and deallocating tbufs > + * > + * This function is called when the SET_SIZE hypercall is done. > + */ > +int tb_set_size(int size) > +{ > + // There are three cases to handle: > + // 1. Changing from 0 to non-zero ==> simple allocate > + // 2. Changing from non-zero to 0 ==> simple deallocate > + // 3. Changing size ==> deallocate and reallocate? Or disallow? > + // User can just do a change to 0, then a change to > the new size. > + // > + // Tracing must be disabled (tb_init_done==0) before calling this > + > + if (opt_tbuf_size == 0 && size > 0) { > + // What if size is too big? alloc_xenheap will complain. > + opt_tbuf_size = size; > + if (alloc_trace_bufs() != 0) > + return -EINVAL; > + wmb(); > + printk("Xen trace buffers: initialized\n"); > + return 0; > + } > + else if (opt_tbuf_size > 0 && size == 0) { > + int order = get_order_from_pages(num_online_cpus() * > opt_tbuf_size); > + // is there a way to undo SHARE_PFN_WITH_DOMAIN? > + free_xenheap_pages(t_bufs[0], order); > + opt_tbuf_size = 0; > + printk("Xen trace buffers: uninitialized\n"); > + return 0; > + } > + else { > + printk("tb_set_size from %d to %d not implemented\n", > opt_tbuf_size, size); > + printk("change size from %d to 0, and then to %d\n", > opt_tbuf_size, size); > + return -EINVAL; > + } > +} > + > > /** > * tb_control - DOM0 operations on trace buffers. > @@ -105,8 +169,10 @@ > static spinlock_t lock = SPIN_LOCK_UNLOCKED; > int rc = 0; > > - if ( !tb_init_done ) > - return -EINVAL; > + // Commenting this out since we have to allow some of > these operations > + // in order to enable dynamic control of the trace buffers. > + // if ( !tb_init_done ) > + // return -EINVAL; > > spin_lock(&lock); > > @@ -123,6 +189,30 @@ > break; > case DOM0_TBUF_SET_EVT_MASK: > tb_event_mask = tbc->evt_mask; > + break; > + case DOM0_TBUF_SET_SIZE: > + // Change trace buffer allocation. > + // Trace buffers must be disabled to do this. > + if (tb_init_done) { > + printk("attempt to change size with tbufs enabled\n"); > + rc = -EINVAL; > + } > + else > + rc = tb_set_size(tbc->size); > + break; > + case DOM0_TBUF_ENABLE: > + // Enable trace buffers. Size must be non-zero, ie, buffers > + // must already be allocated. > + if (opt_tbuf_size == 0) > + rc = -EINVAL; > + else > + tb_init_done = 1; > + break; > + case DOM0_TBUF_DISABLE: > + // Disable trace buffers. Just stops new records from being > written, > + // does not deallocate any memory. > + tb_init_done = 0; > + printk("Xen trace buffers: disabled\n"); > break; > default: > rc = -EINVAL; > diff -r e70ea9465b31 -r 63d05000cfa3 xen/include/public/dom0_ops.h > --- a/xen/include/public/dom0_ops.h Thu Oct 27 09:53:02 2005 > +++ b/xen/include/public/dom0_ops.h Sun Oct 30 20:59:45 2005 > @@ -193,6 +193,9 @@ > #define DOM0_TBUF_GET_INFO 0 > #define DOM0_TBUF_SET_CPU_MASK 1 > #define DOM0_TBUF_SET_EVT_MASK 2 > +#define DOM0_TBUF_SET_SIZE 3 > +#define DOM0_TBUF_ENABLE 4 > +#define DOM0_TBUF_DISABLE 5 > uint8_t op; > /* IN/OUT variables */ > unsigned long cpu_mask; > diff -r e70ea9465b31 -r 63d05000cfa3 tools/libxc/xc_tbuf.c > --- /dev/null Thu Oct 27 09:53:02 2005 > +++ b/tools/libxc/xc_tbuf.c Sun Oct 30 20:59:45 2005 > @@ -0,0 +1,51 @@ > +/************************************************************ > ****************** > + * xc_tbuf.c > + * > + * API for manipulating and accessing trace buffer parameters > + * > + * Copyright (c) 2005, Rob Gardner > + */ > + > +#include "xc_private.h" > + > +int xc_tbuf_enable(int xc_handle, int enable) > +{ > + dom0_op_t op; > + > + op.cmd = DOM0_TBUFCONTROL; > + op.interface_version = DOM0_INTERFACE_VERSION; > + if (enable) > + op.u.tbufcontrol.op = DOM0_TBUF_ENABLE; > + else > + op.u.tbufcontrol.op = DOM0_TBUF_DISABLE; > + > + return xc_dom0_op(xc_handle, &op); > +} > + > +int xc_tbuf_set_size(int xc_handle, uint32_t size) > +{ > + dom0_op_t op; > + > + op.cmd = DOM0_TBUFCONTROL; > + op.interface_version = DOM0_INTERFACE_VERSION; > + op.u.tbufcontrol.op = DOM0_TBUF_SET_SIZE; > + op.u.tbufcontrol.size = size; > + > + return xc_dom0_op(xc_handle, &op); > +} > + > +int xc_tbuf_get_size(int xc_handle, uint32_t *size) > +{ > + int rc; > + dom0_op_t op; > + > + op.cmd = DOM0_TBUFCONTROL; > + op.interface_version = DOM0_INTERFACE_VERSION; > + op.u.tbufcontrol.op = DOM0_TBUF_GET_INFO; > + > + rc = xc_dom0_op(xc_handle, &op); > + if (rc == 0) > + *size = op.u.tbufcontrol.size; > + return rc; > +} > + > diff -r e70ea9465b31 -r 63d05000cfa3 tools/xentrace/setsize.c > --- /dev/null Thu Oct 27 09:53:02 2005 > +++ b/tools/xentrace/setsize.c Sun Oct 30 20:59:45 2005 > @@ -0,0 +1,36 @@ > +#include <stdlib.h> > +#include <stdio.h> > +#include <errno.h> > +#include <xenctrl.h> > + > +int main(int argc, char * argv[]) > +{ > + unsigned int size; > + int xc_handle = xc_interface_open(); > + > + if (xc_tbuf_get_size(xc_handle, &size) != 0) { > + perror("Failure to get tbuf info from Xen. Guess size is 0."); > + printf("This may mean that tracing is not compiled into xen.\n"); > + exit(1); > + } > + else > + printf("Current tbuf size: 0x%x\n", size); > + > + if (argc < 2) > + exit(0); > + > + size = atoi(argv[1]); > + > + if (xc_tbuf_set_size(xc_handle, size) != 0) { > + perror("set_size Hypercall failure"); > + exit(1); > + } > + > + if (xc_tbuf_get_size(xc_handle, &size) != 0) > + perror("Failure to get tbuf info from Xen. Guess size is 0."); > + else > + printf("New tbuf size: 0x%x\n", size); > + > + xc_interface_close(xc_handle); > + return 0; > +} > diff -r e70ea9465b31 -r 63d05000cfa3 tools/xentrace/tbctl.c > --- /dev/null Thu Oct 27 09:53:02 2005 > +++ b/tools/xentrace/tbctl.c Sun Oct 30 20:59:45 2005 > @@ -0,0 +1,26 @@ > +#include <stdlib.h> > +#include <stdio.h> > +#include <errno.h> > +#include <xenctrl.h> > + > +int main(int argc, char * argv[]) > +{ > + int enable; > + int xc_handle = xc_interface_open(); > + > + if (argc < 2) { > + printf("usage: %s [0|1]\n", argv[0]); > + exit(1); > + } > + enable = atoi(argv[1]); > + > + if (xc_tbuf_enable(xc_handle, enable) != 0) { > + perror("Enable/Disable Hypercall failure"); > + exit(1); > + } > + else > + printf("Tracing now %s\n", (enable ? "enabled" : "disabled")); > + > + xc_interface_close(xc_handle); > + return 0; > +} > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Rob Gardner
2005-Oct-30 22:39 UTC
Re: [Xen-devel] [PATCH] Changes to allow dynamic enabling/disabling oftrace buffers
Ian Pratt wrote:>Nice work. Has this been well tested on both 32b and 64b? > >Sorry, only had 32b handy. Perhaps somebody can help out and test it on 64b? Rob _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel