thr3ads.net - Xen devel - [Xen-devel] Announcement: xenalyze trace analysis tool released [Aug 2009]

If this information is useful, please help other people find it:
Share via:

George Dunlap

2009-Aug-13 15:51 UTC

[Xen-devel] Announcement: xenalyze trace analysis tool released

I am pleased to announce the public availability of xenalyze, a tool
I''ve developed over the last 2+ years to analyze the output of
xentrace.

Mercurial repository is available at
http://xenbits.xensource.com/ext/xenalyze.hg

Xenalyze is a tool I''ve developed while doing performance analysis for
XenSource and Citrix.  Features include:
* Orders trace records across physical cpus by tsc.  Attempts to
automatically detect and adjust for tsc skew.
* Tracks domains and vcpus across pcpus, to collect information about
VMs, not processors
* Statistical information about particular events or states generally includes:
 + Total time across the run
 + Percentage of time spent
 + Average number of cycles for an event
 + 5th, 50th, and 95th percentiles of a sample of events
* Statistical data collection includes:
 + Scheduler runstates: running, runnable, blocked, offline
 + Time spent in VMEXITs (i.e., from VMEXIT to VMENTER)
  - Includes time spent doing emulation for specific
  - Optionally includes time spent for particular MMIO / IO addresses
 + Time spent running on physical processors before being migrated
 + Optional tracking of guest cr3 values (generally corresponding to
guest processes)
* "Dump" mode, which will dump a human-readable format of the trace in
the order the events were generated
* Optional "symbol file" which will symbolically interpret guest EIPs
* "Interval" mode, which allows you to graph data over time
" Varous "scatterplot" modes, which allow you to visualize data
such
as guest EIP values
* Deals gracefully with lost records.
* Graphical progress bar and completion pop-up for long traces

There is some minimal documentation in a file called xenalyze.html,
and back-patches to work with earlier versions of Xen in the directory
called back-patches/.

Questions, comments and patches welcome.  The tool is doubtless rather
quirky, as I''m both the main developer and user.

Enjoy,
 -George Dunlap

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Akio Takebe

2009-Aug-14 01:26 UTC

head link

Re: [Xen-devel] Announcement: xenalyze trace analysis tool released

Hi, George

It''s interesting tool.
I got many error on x86_64 dom0 while compiling. The patch fixes these.

gcc -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
-Wno-unused-value -Wdeclaration-after-statement -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -mno-tls-direct-seg-refs -Werror -o xenalyze xenalyze.c
cc1: warnings being treated as errors
xenalyze.c: In function ''init_hvm_data'':
xenalyze.c:1321: warning: format ''%d'' expects type
''int'', but argument 3 has type ''size_t''
xenalyze.c: In function ''hvm_msr_write_process'':
xenalyze.c:4055: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c: In function ''hvm_msr_read_process'':
xenalyze.c:4104: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c: In function ''shadow_emulate_process'':
xenalyze.c:5128: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c:5142: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c:5156: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c: In function ''shadow_parse_other'':
xenalyze.c:5228: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c:5244: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c:5256: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c: In function ''shadow_fixup_process'':
xenalyze.c:5445: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c:5458: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c:5471: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c: In function ''shadow_mmio_process'':
xenalyze.c:5559: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c:5570: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c: In function ''vcpu_create'':
xenalyze.c:6080: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c: In function ''domain_create'':
xenalyze.c:6122: warning: format ''%d'' expects type
''int'', but argument 4 has type ''long unsigned
int''
xenalyze.c: In function ''scan_for_new_pcpu'':
xenalyze.c:7119: warning: format ''%lld'' expects type
''long long int'', but argument 5 has type
''loff_t''
xenalyze.c: In function ''process_cpu_change'':
xenalyze.c:7194: warning: format ''%llx'' expects type
''long long unsigned int'', but argument 4 has type
''loff_t''
xenalyze.c:7194: warning: format ''%llx'' expects type
''long long unsigned int'', but argument 5 has type
''loff_t''
xenalyze.c:7220: warning: format ''%lld'' expects type
''long long int'', but argument 5 has type
''loff_t''
xenalyze.c:7245: warning: format ''%llx'' expects type
''long long unsigned int'', but argument 5 has type
''loff_t''
xenalyze.c: In function ''__read_record'':
xenalyze.c:7538: warning: format ''%d'' expects type
''int'', but argument 4 has type ''ssize_t''
xenalyze.c:7547: warning: format ''%d'' expects type
''int'', but argument 4 has type ''ssize_t''
xenalyze.c:7547: warning: format ''%d'' expects type
''int'', but argument 5 has type ''ssize_t''
xenalyze.c: In function ''__fill_in_record_info'':
xenalyze.c:7594: warning: format ''%016llx'' expects type
''long long unsigned int'', but argument 3 has type
''loff_t''
make: *** [xenalyze] Error 1

Signed-off-by: Akio Takebe <takebe_akio@jp.fujitsu.com>

Best Regards,

Akio Takebe


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

George Dunlap

2009-Aug-14 10:19 UTC

head link

Re: [Xen-devel] Announcement: xenalyze trace analysis tool released

Hmm, that won''t do; with this patch it won''t compile on
x86-32. :-)

I''ll take a look and see what the best solution is.  Thanks for the
patch!

 -George

On Fri, Aug 14, 2009 at 2:26 AM, Akio Takebe<takebe_akio@jp.fujitsu.com>
wrote:> Hi, George
>
> It''s interesting tool.
> I got many error on x86_64 dom0 while compiling. The patch fixes these.
>
> gcc -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
> -Wno-unused-value -Wdeclaration-after-statement -D_LARGEFILE_SOURCE
> -D_LARGEFILE64_SOURCE -mno-tls-direct-seg-refs -Werror -o xenalyze
> xenalyze.c
> cc1: warnings being treated as errors
> xenalyze.c: In function ''init_hvm_data'':
> xenalyze.c:1321: warning: format ''%d'' expects type
''int'', but argument 3 has
> type ''size_t''
> xenalyze.c: In function ''hvm_msr_write_process'':
> xenalyze.c:4055: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c: In function ''hvm_msr_read_process'':
> xenalyze.c:4104: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c: In function ''shadow_emulate_process'':
> xenalyze.c:5128: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c:5142: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c:5156: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c: In function ''shadow_parse_other'':
> xenalyze.c:5228: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c:5244: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c:5256: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c: In function ''shadow_fixup_process'':
> xenalyze.c:5445: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c:5458: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c:5471: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c: In function ''shadow_mmio_process'':
> xenalyze.c:5559: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c:5570: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c: In function ''vcpu_create'':
> xenalyze.c:6080: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c: In function ''domain_create'':
> xenalyze.c:6122: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''long unsigned int''
> xenalyze.c: In function ''scan_for_new_pcpu'':
> xenalyze.c:7119: warning: format ''%lld'' expects type
''long long int'', but
> argument 5 has type ''loff_t''
> xenalyze.c: In function ''process_cpu_change'':
> xenalyze.c:7194: warning: format ''%llx'' expects type
''long long unsigned
> int'', but argument 4 has type ''loff_t''
> xenalyze.c:7194: warning: format ''%llx'' expects type
''long long unsigned
> int'', but argument 5 has type ''loff_t''
> xenalyze.c:7220: warning: format ''%lld'' expects type
''long long int'', but
> argument 5 has type ''loff_t''
> xenalyze.c:7245: warning: format ''%llx'' expects type
''long long unsigned
> int'', but argument 5 has type ''loff_t''
> xenalyze.c: In function ''__read_record'':
> xenalyze.c:7538: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''ssize_t''
> xenalyze.c:7547: warning: format ''%d'' expects type
''int'', but argument 4 has
> type ''ssize_t''
> xenalyze.c:7547: warning: format ''%d'' expects type
''int'', but argument 5 has
> type ''ssize_t''
> xenalyze.c: In function ''__fill_in_record_info'':
> xenalyze.c:7594: warning: format ''%016llx'' expects type
''long long unsigned
> int'', but argument 3 has type ''loff_t''
> make: *** [xenalyze] Error 1
>
> Signed-off-by: Akio Takebe <takebe_akio@jp.fujitsu.com>
>
> Best Regards,
>
> Akio Takebe
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2010-Apr-30 22:02 UTC

head link

Re: [Xen-devel] Announcement: xenalyze trace analysis tool released

On 08/14/2009 03:19 AM, George Dunlap wrote:> Hmm, that won''t do; with this patch it won''t compile on
x86-32. :-)
>
> I''ll take a look and see what the best solution is.  Thanks for
the patch!
>   
This should fix it properly.  The key is to use the "z" format
modifier
meaning "size_t".

    J

Fix 64-bit compilation

This fixes 64-bit compilation by using the ''z'' printf format
modifier
to tell the compiler we''re using size_t-typed arguments, and explicitly
casting loff_t types to unsigned long long to match the format.

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

diff -r 6466afa95122 -r 4efef3092ce7 dump-raw.c
--- a/dump-raw.c	Mon Nov 30 12:43:09 2009 -0600
+++ b/dump-raw.c	Fri Apr 30 12:56:57 2010 -0700
@@ -94,7 +94,7 @@
         return 0;
     } else if(r < sizeof(uint32_t)) {
         /* Full header not read */
-        fprintf(stderr, "%s: short read (%d bytes)\n",
+        fprintf(stderr, "%s: short read (%zd bytes)\n",
                 __func__, r);
         exit(1);
     }
@@ -103,7 +103,7 @@
 
     if(r < rsize) {
         /* Full record not read */
-        fprintf(stderr, "%s: short read (%d, expected %d)\n",
+        fprintf(stderr, "%s: short read (%zd, expected %zd)\n",
                 __func__, r, rsize);
         return 0;
     }
@@ -146,7 +146,8 @@
     /* File sanity check */
     if(p->file_offset != p->next_cpu_change_offset) {
         printf("Strange, pcpu %d expected offet %llx, actual
%llx!\n",
-                p->pid, p->next_cpu_change_offset, p->file_offset);
+	       p->pid, (unsigned long long)p->next_cpu_change_offset,
+	       (unsigned long long)p->file_offset);
     }
 
     p->next_cpu_change_offset = p->file_offset + ri->size +
r->window_size;
@@ -159,7 +160,7 @@
     int i;
     
     printf("R p%2d o%016llx %8lx %d ",
-           p->pid, p->file_offset,
+           p->pid, (unsigned long long)p->file_offset,
            (unsigned long)ri->rec.event, ri->rec.extra_words);
 
     if(ri->rec.cycle_flag)
diff -r 6466afa95122 -r 4efef3092ce7 xenalyze.c
--- a/xenalyze.c	Mon Nov 30 12:43:09 2009 -0600
+++ b/xenalyze.c	Fri Apr 30 12:56:57 2010 -0700
@@ -1328,7 +1328,7 @@
         if(h->summary.extint_histogram)
             bzero(h->summary.extint_histogram, size);
         else {
-            fprintf(stderr, "FATAL: Could not allocate %d bytes for
interrupt histogram!\n",
+            fprintf(stderr, "FATAL: Could not allocate %zd bytes for
interrupt histogram!\n",
                     size);
             exit(1);
         }
@@ -4062,7 +4062,7 @@
 
     if(ri->extra_words != (sizeof(*r)/sizeof(unsigned long) + 1))
     {
-        fprintf(warn, "FATAL: msr_write extra_words %d, expected
%d!\n",
+        fprintf(warn, "FATAL: msr_write extra_words %d, expected
%zd!\n",
                 ri->extra_words, sizeof(*r)/sizeof(unsigned long));
         dump_unexpected_and_exit(ri);
     } 
@@ -4111,7 +4111,7 @@
 
     if(ri->extra_words != (sizeof(*r)/sizeof(unsigned long) + 1))
     {
-        fprintf(warn, "FATAL: msr_read extra_words %d, expected
%d!\n",
+        fprintf(warn, "FATAL: msr_read extra_words %d, expected
%zd!\n",
                 ri->extra_words, sizeof(*r)/sizeof(unsigned long));
         dump_unexpected_and_exit(ri);
     } 
@@ -5152,7 +5152,7 @@
     case 2:
         if(sizeof(r->gpl2) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl2),
h->v->guest_paging_levels,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -5166,7 +5166,7 @@
     case 3:
         if(sizeof(r->gpl3) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl3),
h->v->guest_paging_levels,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -5180,7 +5180,7 @@
     case 4:
         if(sizeof(r->gpl4) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl4),
h->v->guest_paging_levels,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -5252,7 +5252,7 @@
     case 2:
         if(sizeof(r->gpl2) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl2), rec_gpl,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -5268,7 +5268,7 @@
     case 3:
         if(sizeof(r->gpl3) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl3), rec_gpl,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -5280,7 +5280,7 @@
     case 4:
         if(sizeof(r->gpl4) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl4), rec_gpl,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -5469,7 +5469,7 @@
     case 2:
         if(sizeof(r->gpl2) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl2),
h->v->guest_paging_levels,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -5482,7 +5482,7 @@
     case 3:
         if(sizeof(r->gpl3) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl3),
h->v->guest_paging_levels,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -5495,7 +5495,7 @@
     case 4:
         if(sizeof(r->gpl4) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl4),
h->v->guest_paging_levels,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -5583,7 +5583,7 @@
     case 3:
         if(sizeof(r->gpl2) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl2),
h->v->guest_paging_levels,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -5594,7 +5594,7 @@
     case 4:
         if(sizeof(r->gpl4) != ri->extra_words * 4)
         {
-            fprintf(warn, "%s: expected %d bytes for %d-level guest, got
%d!\n",
+            fprintf(warn, "%s: expected %zd bytes for %d-level guest, got
%d!\n",
                     __func__, sizeof(r->gpl4),
h->v->guest_paging_levels,
                     ri->extra_words * 4);
             dump_unexpected_and_exit(ri);
@@ -6106,7 +6106,7 @@
 
     if((v=malloc(sizeof(*v)))==NULL)
     {
-        fprintf(stderr, "%s: malloc %d failed!\n", __func__,
sizeof(*d));
+        fprintf(stderr, "%s: malloc %zd failed!\n", __func__,
sizeof(*d));
         exit(1);
     }
 
@@ -6148,7 +6148,7 @@
 
     if((d=malloc(sizeof(*d)))==NULL)
     {
-        fprintf(stderr, "%s: malloc %d failed!\n", __func__,
sizeof(*d));
+        fprintf(stderr, "%s: malloc %zd failed!\n", __func__,
sizeof(*d));
         exit(1);
     }
 
@@ -7145,7 +7145,7 @@
         struct pcpu_info *p = P.pcpu + cd->cpu;
 
         fprintf(warn, "%s: Activating pcpu %d at offset %lld\n",
-                __func__, cd->cpu, offset);
+                __func__, cd->cpu, (unsigned long long)offset);
 
         p->active = 1;
         /* Process this cpu_change record first */
@@ -7220,7 +7220,8 @@
     /* File sanity check */
     if(p->file_offset != p->next_cpu_change_offset) {
         fprintf(warn, "Strange, pcpu %d expected offet %llx, actual
%llx!\n",
-                p->pid, p->next_cpu_change_offset, p->file_offset);
+                p->pid, (unsigned long long)p->next_cpu_change_offset,
+                (unsigned long long)p->file_offset);
     }
 
     if(r->cpu > MAX_CPUS)
@@ -7246,7 +7247,7 @@
             P.max_active_pcpu = r->cpu;
 
         fprintf(warn, "%s: Activating pcpu %d at offset %lld\n",
-               __func__, r->cpu, p->file_offset);
+                __func__, r->cpu, (unsigned long long)p->file_offset);
         
         sched_default_vcpu_activate(p2);
 
@@ -7271,7 +7272,7 @@
             activate_early_eof();
         } else if(P.early_eof && p->file_offset >
P.last_epoch_offset) {
             fprintf(warn, "%s: early_eof activated, pcpu %d past
last_epoch_offset %llx, deactivating.\n",
-                    __func__, p->pid, P.last_epoch_offset);
+                    __func__, p->pid, (unsigned long
long)P.last_epoch_offset);
             deactivate_pcpu(p);
         }
     } 
@@ -7571,7 +7572,7 @@
         return 0;
     } else if(r < sizeof(uint32_t)) {
         /* Full header not read */
-        fprintf(stderr, "%s: short read (%d bytes)\n",
+        fprintf(stderr, "%s: short read (%zd bytes)\n",
                 __func__, r);
         exit(1);
     }
@@ -7580,7 +7581,7 @@
 
     if(r < rsize) {
         /* Full record not read */
-        fprintf(stderr, "%s: short read (%d, expected %d)\n",
+        fprintf(stderr, "%s: short read (%zd, expected %zd)\n",
                 __func__, r, rsize);
         return 0;
     }
@@ -7627,7 +7628,7 @@
     if ( opt.dump_raw_reads ) {
         int i;
         printf("R p%2d o%016llx %8lx %d ",
-               p->pid, p->file_offset,
+               p->pid, (unsigned long long)p->file_offset,
                (unsigned long)ri->rec.event, ri->rec.extra_words);
 
         if(ri->rec.cycle_flag)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Aug 2009 - Announcement: xenalyze trace analysis tool released

[Xen-devel] Announcement: xenalyze trace analysis tool released

Re: [Xen-devel] Announcement: xenalyze trace analysis tool released

Re: [Xen-devel] Announcement: xenalyze trace analysis tool released

Re: [Xen-devel] Announcement: xenalyze trace analysis tool released