well, well. Plan 9 is up and running and mounting a root file system over 9p to a linux machine running a 9p server (Russ cox''s u9fs). I''ve still got a glitch in there, but boy this is fun. It''s a memory corruption, doubtless some other mistake on my part :-) I''m off on other stuff this week but hope to nail it next week. ron ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> well, well. Plan 9 is up and running and mounting a root file system over > 9p to a linux machine running a 9p server (Russ cox''s u9fs).Cool! I for one can''t wait to try out Plan 9 on my hardware. I''m curious as to how your source for this port will be distributed... In fact, I don''t know how the p9 source is managed - is it possible for you to get the port checked in to the main tree, or will it be available as a separate patch?> I''m off on other stuff this week but hope to nail it next week.Cool! We''re really close to Xen 2.0 release and it''s doubly nice to have new OS ports working as well :-) Cheers, Mark ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Wed, 13 Oct 2004, Mark A. Williamson wrote:> I''m curious as to how your source for this port will be distributed... > In fact, I don''t know how the p9 source is managed - is it possible for > you to get the port checked in to the main tree, or will it be available > as a separate patch?I have been told I can get it in. There will be issues as the Xen stuff is -- I guess -- GPL and Plna 9 is not, so we''re going to get into that damned mess. So I don''t know how it will go. ron ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > I have been told I can get it in. There will be issues as the Xen stuff is > -- I guess -- GPL and Plna 9 is not, so we''re going to get into that > damned mess.So long as the source is not intermingled it shouldn''t be an issue. -Kip ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > > On Wed, 13 Oct 2004, Mark A. Williamson wrote: > > > I''m curious as to how your source for this port will be distributed... > > In fact, I don''t know how the p9 source is managed - is it possible for > > you to get the port checked in to the main tree, or will it be available > > as a separate patch? > > I have been told I can get it in. There will be issues as the Xen stuff is > -- I guess -- GPL and Plna 9 is not, so we''re going to get into that > damned mess. > > So I don''t know how it will go.Personally, I''m happy to relicense our ''unprivileged guest'' drivers under a BSD-style license. This would include at least the following files that contain no Linux code: linux-2.6.8.1-xen-sparse: drivers/xen/netfront/* drivers/xen/blkfront/* drivers/xen/evtchn/* drivers/console/* arch/xen/i386/mm/hypervisor.c arch/xen/kernel/ctrl_if.c arch/xen/kernel/evtchn.c include/asm-xen/ctrl_if.h include/asm-xen/evtchn.h include/asm-xen/hypervisor.h include/asm-xen/multicall.h include/asm-xen/suspend.h I consider this the least we can do given the lack of documentation that would allow OS porters to roll their own. :-) There is the issue that there are bits and pieces of other people''s code in there as well --- e.g., Christian Limpach, Mike Wray, Mark Williamson. But I don''t foresee any problems. -- Keir ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
I would''ve thought a (legally) efficient way of doing that would be to have a library-set available under the LGPL, that call the GPL code and are called by the non-GPLed guest OSes. I''m not sure about the actual efficiency in terms of processor cycles, of such an approach, but if it collocates the lawyers in the dogbox, which is where they should be, instead of breathing down your neck, that should be an acceptable trade. Just my $0.02 (and it''s very highly inflated of course!) Wesley Parish On Thu, 14 Oct 2004 21:10, Keir Fraser wrote:> > On Wed, 13 Oct 2004, Mark A. Williamson wrote: > > > I''m curious as to how your source for this port will be distributed... > > > In fact, I don''t know how the p9 source is managed - is it possible for > > > you to get the port checked in to the main tree, or will it be > > > available as a separate patch? > > > > I have been told I can get it in. There will be issues as the Xen stuff > > is -- I guess -- GPL and Plna 9 is not, so we''re going to get into that > > damned mess. > > > > So I don''t know how it will go. > > Personally, I''m happy to relicense our ''unprivileged guest'' drivers > under a BSD-style license. This would include at least the following > files that contain no Linux code: > > linux-2.6.8.1-xen-sparse: > drivers/xen/netfront/* > drivers/xen/blkfront/* > drivers/xen/evtchn/* > drivers/console/* > arch/xen/i386/mm/hypervisor.c > arch/xen/kernel/ctrl_if.c > arch/xen/kernel/evtchn.c > include/asm-xen/ctrl_if.h > include/asm-xen/evtchn.h > include/asm-xen/hypervisor.h > include/asm-xen/multicall.h > include/asm-xen/suspend.h > > I consider this the least we can do given the lack of documentation > that would allow OS porters to roll their own. :-) > > There is the issue that there are bits and pieces of other people''s > code in there as well --- e.g., Christian Limpach, Mike Wray, Mark > Williamson. But I don''t foresee any problems. > > -- Keir > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel-- Wesley Parish * * * Clinersterton beademung - in all of love. RIP James Blish * * * Mau e ki, "He aha te mea nui?" You ask, "What is the most important thing?" Maku e ki, "He tangata, he tangata, he tangata." I reply, "It is people, it is people, it is people." ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
No need for LGPL trickery as we are talking about separately licensing files that contain only our code. We can do that as long as we take no GPL code that is copyrighted by someone else. The method you describe below for circumventing the GPL by adding an ''interface barrier'' of LGPL code is certainly popular (I could cite a number of examples). However, I don''t believe it is permitted by the GPL. :-) Linux permits it for binary-only drivers only because Linus is prepared to suffer it. -- Keir> I would''ve thought a (legally) efficient way of doing that would be to have a > library-set available under the LGPL, that call the GPL code and are called > by the non-GPLed guest OSes. > > I''m not sure about the actual efficiency in terms of processor cycles, of such > an approach, but if it collocates the lawyers in the dogbox, which is where > they should be, instead of breathing down your neck, that should be an > acceptable trade. > > Just my $0.02 (and it''s very highly inflated of course!) > > Wesley Parish------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Thu, 14 Oct 2004, Keir Fraser wrote:> Personally, I''m happy to relicense our ''unprivileged guest'' drivers > under a BSD-style license. This would include at least the following > files that contain no Linux code: > > linux-2.6.8.1-xen-sparse: > drivers/xen/netfront/* > drivers/xen/blkfront/* > drivers/xen/evtchn/* > drivers/console/* > arch/xen/i386/mm/hypervisor.c > arch/xen/kernel/ctrl_if.c > arch/xen/kernel/evtchn.c > include/asm-xen/ctrl_if.h > include/asm-xen/evtchn.h > include/asm-xen/hypervisor.h > include/asm-xen/multicall.h > include/asm-xen/suspend.hThat would help. I think it''s a good idea. thanks ron ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> There is the issue that there are bits and pieces of other people''s > code in there as well --- e.g., Christian Limpach, Mike Wray, Mark > Williamson. But I don''t foresee any problems.Fine by me. The code I wrote really belongs to Intel but they BSD license stuff anyhow, so it shouldn''t be a problem. Mark ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
I''ll look into cleaning up / adding licensing notices to the repository next week. -- Keir> > > On Thu, 14 Oct 2004, Keir Fraser wrote: > > > Personally, I''m happy to relicense our ''unprivileged guest'' drivers > > under a BSD-style license. This would include at least the following > > files that contain no Linux code: > > > > linux-2.6.8.1-xen-sparse: > > drivers/xen/netfront/* > > drivers/xen/blkfront/* > > drivers/xen/evtchn/* > > drivers/console/* > > arch/xen/i386/mm/hypervisor.c > > arch/xen/kernel/ctrl_if.c > > arch/xen/kernel/evtchn.c > > include/asm-xen/ctrl_if.h > > include/asm-xen/evtchn.h > > include/asm-xen/hypervisor.h > > include/asm-xen/multicall.h > > include/asm-xen/suspend.h > > That would help. > > I think it''s a good idea. > > thanks > > ron------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Hi, we are benchmarking Xen in a cluster, and got some bad results. We might do something wrong, and wonder if anyone have similar problems. When we benchmark troughput from native Linux to native Linux (two physical nodes in the cluster) we get 786.034 MByte/s When we benchmark from a virtual domain (running on Xen on a physical node) to an another virtual domain (on another physical node) we get 56.480 MByte/s (1:16) The difference is huge, and we wonder if the bottleneck could be the fact that we are using software routing (We use this in order to route from the physical node to the virtual OSs), or if this is just a downside of Xen? I would guess it IS the SW routing, so is there any good alternatives to make virtual domains communicate on a cluster without sw routing? cheers, Rune J.A
> Hi, we are benchmarking Xen in a cluster, and got some bad results. We > might do something wrong, and wonder if anyone > have similar problems. > > When we benchmark troughput from native Linux to native Linux (two > physical nodes in the cluster) we get 786.034 MByte/s > When we benchmark from a virtual domain (running on Xen on a physical > node) to an another virtual domain (on another physical node) we get > 56.480 MByte/s (1:16)(Presumably you mean MBits rather than Mbytes) The numbers you''re getting are terrible compared to what we see. Running between virtual domains on a cluster we measure throughput as high as 897Mb/s (same as Linux native). Our results were recorded with dual 2.4GHz Xeons with tg3 NICs and a 128KB socket buffer, measured using ttcp. With the virtual domain running on the other physical CPU from domain 0 we get 897Mb/s. We get similar results running the virtual domain on the other hyperthread of the same physical CPU. We observe a performance reduction if we run the virtual domain on the same (logical) CPU as domain 0, down to 660Mb/s [843Mb/s on a dual 3GHz machine, so we appear to be CPU limited in this case].> The difference is huge, and we wonder if the bottleneck could be the > fact that we are using software routing (We use this in order to route > from the physical node to the virtual OSs), or if this is just a > downside of Xen?Our results were recorded using the dom0 linux bridge code rather than using routing. One thing to check is that you have don''t have CONFIG_IP_NF_CONNTRACK set to ''y'' -- this slays performance. Also, if you''re running multiple domains on the same CPU you may be running into CPU scheduling issues. Some tweaks to scheduler parameters may fix this.> I would guess it IS the SW routing, so is there any good alternatives > to make virtual domains communicate on a cluster without sw routing?The Xen 2.0 architecture is not as slick as the monolithic-hypervisor approach of Xen 1.2, but we get better hardware support and a lot more flexibility. However, we do burn more CPU to achieve the same IO rate. We just have to wait for Moore''s law to catch up ;-) Ian ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark A. Williamson
2004-Oct-14 23:24 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems!
> When we benchmark from a virtual domain (running on Xen on a physical > node) to an another virtual domain (on another physical node) we get > 56.480 MByte/s (1:16)Ouch. How are you benchmarking this? (what tool, what parameters, etc.). It''ll help me reproduce this on our test systems. Then we''ll know if it''s your config or if there''s something to track down. We did see some weird performance for small packets at one stage and I''m not sure if that was ever resolved. If it''s the same problem, I can do a binary chop search of changesets in order to locate it. Cheers, Mark ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Rune Johan Andresen
2004-Oct-15 13:48 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems!
Hi, we benchmark with Scali MPI. We are in the first stage with an easy ping program which first send 1 byte to measure the latency, then we tested with 10^5, 10^7 and 10^8 size packages. We also get the same results when sending from domain0 to domain0 in the cluster. We are now testing if the routing table is the bottleneck, I let you know the results. Thank you :) Cheers, Rune J.A On Oct 15, 2004, at 1:24 AM, Mark A. Williamson wrote:>> When we benchmark from a virtual domain (running on Xen on a physical >> node) to an another virtual domain (on another physical node) we get >> 56.480 MByte/s (1:16) > > Ouch. How are you benchmarking this? (what tool, what parameters, > etc.). > It''ll help me reproduce this on our test systems. Then we''ll know if > it''s > your config or if there''s something to track down. > > We did see some weird performance for small packets at one stage and > I''m not > sure if that was ever resolved. If it''s the same problem, I can do a > binary > chop search of changesets in order to locate it. > > Cheers, > Mark------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
I have now amended the licensing terms for the files below, as noted at the top of each file concerned. I also added a clarifying COPYING file to the root of the repository. Any files that you have derived solely from any version of the files below are also subject to the new license (this also includes versions you may have taken from within a different version of Linux, including v2.4). I''d appreciate you adding the license text to the top of such files though. -- Keir> > Personally, I''m happy to relicense our ''unprivileged guest'' drivers > > under a BSD-style license. This would include at least the following > > files that contain no Linux code: > > > > linux-2.6.8.1-xen-sparse: > > drivers/xen/netfront/* > > drivers/xen/blkfront/* > > drivers/xen/evtchn/* > > drivers/console/* > > arch/xen/i386/mm/hypervisor.c > > arch/xen/kernel/ctrl_if.c > > arch/xen/kernel/evtchn.c > > include/asm-xen/ctrl_if.h > > include/asm-xen/evtchn.h > > include/asm-xen/hypervisor.h > > include/asm-xen/multicall.h > > include/asm-xen/suspend.h > > That would help. > > I think it''s a good idea. > > thanks > > ron------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Rune Johan Andresen
2004-Oct-21 19:12 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
Hi, we have some additional information about our problem benchmarking
Xen in clusters:
Between two Xen dom0 domains (between two physical computers in the
cluster) we got these strange results:
(We use ttcp socketbuffsize, 10^4 -> 10^6)
Kernel 2.4:
Xen Dom0 -> Xen Dom0: ca. 65 000 KB/s
Kernel 2.6:
Xen Dom0 -> Xen Dom0: ca. 80 000 KB/s (?)
Native Linux:
Native Linux -> Native Linux: ca. 114 000 KB/s
What is new and strage is that Xen Dom0 use about 60% of the CPU when
transfering or receiving, while
Native Linux ony use 6-7%(!) It seems like we have a problem with the
DMA here(?). We use Xen 2.0,
Gigabit ethernet.
I tried ''mv /lib/tls /lib/tls.disabled'' on each node without
success
Under is the dmesg for the Xen node:
Linux version 2.6.8.1-xen0 (root@comp-pvfs-0-17.local) (gcc version
3.2.3 20030502 (Red Hat Linux 3.2.3-34)) #1 Tue Oct 12 14:10:47 GMT
2004
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 0000000008000000 (usable)
128MB LOWMEM available.
On node 0 totalpages: 32768
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 28672 pages, LIFO batch:7
HighMem zone: 0 pages, LIFO batch:1
DMI not present.
Built 1 zonelists
Kernel command line: root=/dev/sda1 ro console=tty0
Initializing CPU#0
PID hash table entries: 1024 (order 10: 8192 bytes)
Xen reported: 3400.171 MHz processor.
Using tsc for high-res timesource
Console: colour VGA+ 80x25
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 125260k/131072k available (2636k kernel code, 5624k reserved,
834k data, 396k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor
mode... Ok.
Calibrating delay loop... 6789.52 BogoMIPS
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000
CPU: Trace cache: 12K uops, L1 D cache: 8K
CPU: L2 cache: 512K
CPU: After all inits, caps: beebcbe1 00000000 00000000 00000080
CPU: Intel(R) Pentium(R) 4 CPU 3.40GHz stepping 09
Enabling unmasked SIMD FPU exception support... done.
Checking ''hlt'' instruction... disabled
NET: Registered protocol family 16
PCI: Using configuration type Xen
SCSI subsystem initialized
PCI: Probing PCI hardware
PCI: Probing PCI hardware (bus 00)
PCI: Probing PCI hardware (bus 01)
PCI: Probing PCI hardware (bus 02)
PCI: Probing PCI hardware (bus 03)
PCI: Probing PCI hardware
Initializing Cryptographic API
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)
Using anticipatory io scheduler
nbd: registered device at major 43
Intel(R) PRO/1000 Network Driver - version 5.2.52-k4
Copyright (c) 1999-2004 Intel Corporation.
PCI: Obtained IRQ 18 for device 0000:01:01.0
PCI: Setting latency timer of device 0000:01:01.0 to 64
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
PCI: Obtained IRQ 21 for device 0000:03:02.0
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
pcnet32.c:v1.30i 06.28.2004 tsbogend@alpha.franken.de
e100: Intel(R) PRO/100 Network Driver, 3.0.18
e100: Copyright(c) 1999-2004 Intel Corporation
Xen virtual console successfully installed as ttyS
Event-channel device installed.
Initialising Xen netif backend
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with
idebus=xx
hda: SAMSUNG CD-ROM SN-124, ATAPI CD/DVD-ROM drive
ide1: I/O resource 0x170-0x177 not free.
ide1: ports already in use, skipping probe
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hda: ATAPI 24X CD-ROM drive, 128kB Cache
Uniform CD-ROM driver Revision: 3.20
PCI: Obtained IRQ 24 for device 0000:02:01.0
PCI: Obtained IRQ 25 for device 0000:02:01.1
scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
<Adaptec 3960D Ultra160 SCSI adapter>
aic7899: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs
(scsi0:A:0): 160.000MB/s transfers (80.000MHz DT, offset 63, 16bit)
Vendor: SEAGATE Model: ST336607LW Rev: DS09
Type: Direct-Access ANSI SCSI revision: 03
scsi0:A:0:0: Tagged Queuing enabled. Depth 32
scsi1 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
<Adaptec 3960D Ultra160 SCSI adapter>
aic7899: Ultra160 Wide Channel B, SCSI Id=7, 32/253 SCBs
Red Hat/Adaptec aacraid driver (1.1.2-lk2 Oct 12 2004)
3ware Storage Controller device driver for Linux v1.26.00.039.
3w-xxxx: No cards found.
libata version 1.02 loaded.
ata_piix version 1.02
ata_piix: combined mode detected
PCI: Obtained IRQ 17 for device 0000:00:1f.2
ata: 0x1f0 IDE port busy
PCI: Setting latency timer of device 0000:00:1f.2 to 64
ata1: SATA max UDMA/133 cmd 0x170 ctl 0x376 bmdma 0xFEA8 irq 15
ata1: SATA port has no device.
scsi2 : ata_piix
SCSI device sda: 71132959 512-byte hdwr sectors (36420 MB)
SCSI device sda: drive cache: write through
sda: sda1 sda2 sda3 sda4 < sda5 >
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
mice: PS/2 mouse device common for all mice
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
md: raid0 personality registered as nr 2
md: raid1 personality registered as nr 3
md: raid5 personality registered as nr 4
raid5: automatically using best checksumming function: pIII_sse
pIII_sse : 440.400 MB/sec
raid5: using function: pIII_sse (440.400 MB/sec)
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
device-mapper: 4.1.0-ioctl (2003-12-10) initialised: dm@uk.sistina.com
NET: Registered protocol family 2
IP: routing cache hash table of 1024 buckets, 8Kbytes
TCP: Hash tables configured (established 8192 bind 16384)
NET: Registered protocol family 1
NET: Registered protocol family 17
Bridge firewalling registered
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: sda1: orphan cleanup on readonly fs
ext3_orphan_cleanup: deleting unreferenced inode 4718
EXT3-fs: sda1: 1 orphan inode deleted
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 396k freed
***************************************************************
***************************************************************
** WARNING: Currently emulating unsupported memory accesses **
** in /lib/tls libraries. The emulation is very **
** slow, and may not work correctly with all **
** programs (e.g., some may ''Segmentation fault'').
**
** TO ENSURE FULL PERFORMANCE AND CORRECT FUNCTION, **
** YOU MUST EXECUTE THE FOLLOWING AS ROOT: **
** mv /lib/tls /lib/tls.disabled **
***************************************************************
***************************************************************
Pausing... 5Pausing... 4Pausing...
3Pausing... 2Pausing...
1Continuing...
EXT3 FS on sda1, internal journal
Adding 1020116k swap on /dev/sda3. Priority:-1 extents:1
kjournald starting. Commit interval 5 seconds
EXT3 FS on sda2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on sda5, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex
process `syslogd'' is using obsolete setsockopt SO_BSDCOMPAT
process `snmpd'' is using obsolete setsockopt SO_BSDCOMPAT
Cheers,
Rune
On Oct 15, 2004, at 1:24 AM, Mark A. Williamson wrote:
>> When we benchmark from a virtual domain (running on Xen on a physical
>> node) to an another virtual domain (on another physical node) we get
>> 56.480 MByte/s (1:16)
>
> Ouch. How are you benchmarking this? (what tool, what parameters,
> etc.).
> It''ll help me reproduce this on our test systems. Then
we''ll know if
> it''s
> your config or if there''s something to track down.
>
> We did see some weird performance for small packets at one stage and
> I''m not
> sure if that was ever resolved. If it''s the same problem, I can
do a
> binary
> chop search of changesets in order to locate it.
>
> Cheers,
> Mark
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: IT Product Guide on
> ITManagersJournal
> Use IT products in your business? Tell us what you think of them. Give
> us
> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out
> more
> http://productguide.itmanagersjournal.com/guidepromo.tmpl
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt
2004-Oct-21 20:24 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
> Between two Xen dom0 domains (between two physical computers in the > cluster) we got these strange results: > (We use ttcp socketbuffsize, 10^4 -> 10^6) > > Kernel 2.6: > Xen Dom0 -> Xen Dom0: ca. 80 000 KB/s (?) > > Native Linux: > Native Linux -> Native Linux: ca. 114 000 KB/s > > What is new and strage is that Xen Dom0 use about 60% of the CPU when > transfering or receiving, while > Native Linux ony use 6-7%(!) It seems like we have a problem with the > DMA here(?). We use Xen 2.0, > Gigabit ethernet.dom0 to dom0 performance really shouldn''t be any difference from native. It certainly isn''t on any of our machines. The only thing I can think of is that something stupid might be happening with interrupts on your machines. Can you compare the rate that the relevant interrupts are going up in /proc/interrupts between xenLinux and native. There''s no interrupt sharing or anything daft like that going on? Are you using NAPI on the native e1000 linux driver? Ian ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Rune Johan Andresen
2004-Oct-22 16:36 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the samre
results between
two xen dom0 nodes. I am not sure if these interrupts tells anything:
Native Linux:
CPU0
0: 87290435 XT-PIC timer
1: 2 XT-PIC keyboard
2: 0 XT-PIC cascade
3: 7668994 XT-PIC eth0
7: 0 XT-PIC ehci-hcd
8: 1 XT-PIC rtc
10: 3 XT-PIC usb-uhci
11: 332088 XT-PIC aic7xxx, aic7xxx, usb-uhci
14: 1 XT-PIC ide0
15: 0 XT-PIC libata
NMI: 0
ERR: 0
Xen Dom0:
CPU0
1: 2 Phys-irq keyboard
14: 3 Phys-irq ide0
18: 954304 Phys-irq eth0
24: 7313 Phys-irq aic7xxx
25: 30 Phys-irq aic7xxx
128: 1 Dynamic-irq misdirect
129: 0 Dynamic-irq ctrl-if
130: 241914 Dynamic-irq timer
131: 0 Dynamic-irq timer_dbg, net-be-dbg
132: 0 Dynamic-irq console
NMI: 0
ERR: 0
If you can see anything which is not normal behavior of xen please
tell us :)
Cheers,
Rune
On Oct 21, 2004, at 10:24 PM, Ian Pratt wrote:
>
>> Between two Xen dom0 domains (between two physical computers in the
>> cluster) we got these strange results:
>> (We use ttcp socketbuffsize, 10^4 -> 10^6)
>>
>> Kernel 2.6:
>> Xen Dom0 -> Xen Dom0: ca. 80 000 KB/s (?)
>>
>> Native Linux:
>> Native Linux -> Native Linux: ca. 114 000 KB/s
>>
>> What is new and strage is that Xen Dom0 use about 60% of the CPU when
>> transfering or receiving, while
>> Native Linux ony use 6-7%(!) It seems like we have a problem with the
>> DMA here(?). We use Xen 2.0,
>> Gigabit ethernet.
>
> dom0 to dom0 performance really shouldn''t be any difference from
> native. It certainly isn''t on any of our machines.
>
> The only thing I can think of is that something stupid might be
> happening with interrupts on your machines. Can you compare the
> rate that the relevant interrupts are going up in
> /proc/interrupts between xenLinux and native.
>
> There''s no interrupt sharing or anything daft like that going on?
> Are you using NAPI on the native e1000 linux driver?
>
>
> Ian
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt
2004-Oct-22 17:47 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
> > We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the samre > results between > two xen dom0 nodes. I am not sure if these interrupts tells anything:It''s the different rate at which the eth0 interrupt counts go up during your bandwidth tests that is interesting. e.g. poll it once a second during the test. It looks like your native Linux is not using legacy PIC mode rather than using the ioapic. Have you tried an SMP native kernel to see if it gets the same interrupt layout as Xen? Ian> Native Linux: > > > CPU0 > 0: 87290435 XT-PIC timer > 1: 2 XT-PIC keyboard > 2: 0 XT-PIC cascade > 3: 7668994 XT-PIC eth0 > 7: 0 XT-PIC ehci-hcd > 8: 1 XT-PIC rtc > 10: 3 XT-PIC usb-uhci > 11: 332088 XT-PIC aic7xxx, aic7xxx, usb-uhci > 14: 1 XT-PIC ide0 > 15: 0 XT-PIC libata > NMI: 0 > ERR: 0 > > > Xen Dom0: > > CPU0 > 1: 2 Phys-irq keyboard > 14: 3 Phys-irq ide0 > 18: 954304 Phys-irq eth0 > 24: 7313 Phys-irq aic7xxx > 25: 30 Phys-irq aic7xxx > 128: 1 Dynamic-irq misdirect > 129: 0 Dynamic-irq ctrl-if > 130: 241914 Dynamic-irq timer > 131: 0 Dynamic-irq timer_dbg, net-be-dbg > 132: 0 Dynamic-irq console > NMI: 0 > ERR: 0 > > If you can see anything which is not normal behavior of xen please > tell us :) > > Cheers, > Rune------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Håvard Bjerke
2004-Oct-27 17:11 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
On Fri, Oct 22, 2004 at 06:47:35PM +0100, Ian Pratt wrote:> > > > We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the samre > > results between > > two xen dom0 nodes. I am not sure if these interrupts tells anything: > > It''s the different rate at which the eth0 interrupt counts go up > during your bandwidth tests that is interesting. e.g. poll it > once a second during the test. >We tried sending 1 MB and measured: non-SMP native Linux: ~ 130k interrupts 114 kB/s native Linux with compiled-in SMP support, single CPU: ~ 140k interrupts 114 kB/s Xen0: ~ 180k interrupts 80 kB/s> It looks like your native Linux is not using legacy PIC mode > rather than using the ioapic. Have you tried an SMP native > kernel to see if it gets the same interrupt layout as Xen? >Layout in native linux w/o SMP: CPU0 0: 130710161 XT-PIC timer 1: 2 XT-PIC keyboard 2: 0 XT-PIC cascade 3: 12872557 XT-PIC eth0 7: 0 XT-PIC ehci-hcd 8: 1 XT-PIC rtc 10: 3 XT-PIC usb-uhci 11: 445263 XT-PIC aic7xxx, aic7xxx, usb-uhci 14: 1 XT-PIC ide0 15: 0 XT-PIC libata NMI: 0 ERR: 0 Layout in native linux compiled with SMP: CPU0 0: 121362 IO-APIC-edge timer 1: 2 IO-APIC-edge keyboard 2: 0 XT-PIC cascade 14: 5 IO-APIC-edge ide0 16: 0 IO-APIC-level usb-uhci 18: 177583 IO-APIC-level eth0 19: 0 IO-APIC-level usb-uhci 24: 7587 IO-APIC-level aic7xxx 25: 30 IO-APIC-level aic7xxx NMI: 0 LOC: 121307 ERR: 0 MIS: 0 Layout in Xen0: CPU0 1: 2 Phys-irq keyboard 14: 3 Phys-irq ide0 18: 267274 Phys-irq eth0 24: 15910 Phys-irq aic7xxx 25: 30 Phys-irq aic7xxx 128: 1 Dynamic-irq misdirect 129: 0 Dynamic-irq ctrl-if 130: 771917 Dynamic-irq timer 131: 0 Dynamic-irq timer_dbg, net-be-dbg 132: 0 Dynamic-irq console NMI: 0 ERR: 0 Questions: Do you think interrupt sharing is the problem? Or the use of IO-APIC? Is this an inherent problem with Xen? Is it possible to change the interrupt scheme in Xen in order to achieve the same performance as in native linux? Cheers, Hvard ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_idU88&alloc_id065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt
2004-Oct-27 18:26 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
> On Fri, Oct 22, 2004 at 06:47:35PM +0100, Ian Pratt wrote: > > > > > > We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the samre > > > results between > > > two xen dom0 nodes. I am not sure if these interrupts tells anything: > > > > It''s the different rate at which the eth0 interrupt counts go up > > during your bandwidth tests that is interesting. e.g. poll it > > once a second during the test. > > > > We tried sending 1 MB and measured:1MB isn''t really very much with a 128KB socket buffer. Do you get the same results with larger transfers?> non-SMP native Linux: > ~ 130k interrupts > 114 kB/s > native Linux with compiled-in SMP support, single CPU: > ~ 140k interrupts > 114 kB/s > Xen0: > ~ 180k interrupts > 80 kB/sIt''s pretty odd that Xen''s taking more interrupts. Are you using the same native kernel version as you are for Xen? Also, what happens if you boot xen with ''nosmp'' on the Xen command line. We''ve got Xen tcp performance results from a bunch of machines, and dom0 to dom0 performance has always been almost identical to native for 1500 byte MTU packets. Ian ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark A. Williamson
2004-Oct-27 18:40 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
Just a thought: you are using the BVT scheduler, right? We haven''t tested performance with the other schedulers recently but we know something goes wrong for IO intensive domains on Atropos. Mark On Wednesday 27 Oct 2004 19:26, Ian Pratt wrote:> > On Fri, Oct 22, 2004 at 06:47:35PM +0100, Ian Pratt wrote: > > > > We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the samre > > > > results between > > > > two xen dom0 nodes. I am not sure if these interrupts tells anything: > > > > > > It''s the different rate at which the eth0 interrupt counts go up > > > during your bandwidth tests that is interesting. e.g. poll it > > > once a second during the test. > > > > We tried sending 1 MB and measured: > > 1MB isn''t really very much with a 128KB socket buffer. Do you get > the same results with larger transfers? > > > non-SMP native Linux: > > ~ 130k interrupts > > 114 kB/s > > native Linux with compiled-in SMP support, single CPU: > > ~ 140k interrupts > > 114 kB/s > > Xen0: > > ~ 180k interrupts > > 80 kB/s > > It''s pretty odd that Xen''s taking more interrupts. Are you using > the same native kernel version as you are for Xen? > > Also, what happens if you boot xen with ''nosmp'' on the Xen > command line. > > We''ve got Xen tcp performance results from a bunch of machines, > and dom0 to dom0 performance has always been almost identical to > native for 1500 byte MTU packets. > > > Ian > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader''s Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Håvard Bjerke
2004-Oct-28 11:32 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
On Wed, Oct 27, 2004 at 07:26:10PM +0100, Ian Pratt wrote:> > On Fri, Oct 22, 2004 at 06:47:35PM +0100, Ian Pratt wrote: > > > > > > > > We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the samre > > > > results between > > > > two xen dom0 nodes. I am not sure if these interrupts tells anything: > > > > > > It''s the different rate at which the eth0 interrupt counts go up > > > during your bandwidth tests that is interesting. e.g. poll it > > > once a second during the test. > > > > > > > We tried sending 1 MB and measured: > > 1MB isn''t really very much with a 128KB socket buffer. Do you get > the same results with larger transfers? >With 10 MB transfers the results are roughly the same: native Linux with compiled-in SMP support, single CPU: ~ 1365k interrupts 114 kB/s Xen0 with "nosmp": ~ 1676 k interrupts 76 kB/s> > non-SMP native Linux: > > ~ 130k interrupts > > 114 kB/s > > native Linux with compiled-in SMP support, single CPU: > > ~ 140k interrupts > > 114 kB/s > > Xen0: > > ~ 180k interrupts > > 80 kB/s > > It''s pretty odd that Xen''s taking more interrupts. Are you using > the same native kernel version as you are for Xen? >Yes, both are 2.4.27> Also, what happens if you boot xen with ''nosmp'' on the Xen > command line. >There seems to be no change in behaviour. The interrupt layout and count remains roughly the same. Do you have any more tips? :) Hvard ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_idU88&alloc_id065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Håvard Bjerke
2004-Oct-28 11:51 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
We''re getting these results in domain 0, not in a VM. As I understand, BVT or apropos scheduling do not apply in domain 0? Hvard On Wed, Oct 27, 2004 at 07:40:27PM +0100, Mark A. Williamson wrote:> Just a thought: you are using the BVT scheduler, right? We haven''t tested > performance with the other schedulers recently but we know something goes > wrong for IO intensive domains on Atropos. > > Mark > > On Wednesday 27 Oct 2004 19:26, Ian Pratt wrote: > > > On Fri, Oct 22, 2004 at 06:47:35PM +0100, Ian Pratt wrote: > > > > > We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the samre > > > > > results between > > > > > two xen dom0 nodes. I am not sure if these interrupts tells anything: > > > > > > > > It''s the different rate at which the eth0 interrupt counts go up > > > > during your bandwidth tests that is interesting. e.g. poll it > > > > once a second during the test. > > > > > > We tried sending 1 MB and measured: > > > > 1MB isn''t really very much with a 128KB socket buffer. Do you get > > the same results with larger transfers? > > > > > non-SMP native Linux: > > > ~ 130k interrupts > > > 114 kB/s > > > native Linux with compiled-in SMP support, single CPU: > > > ~ 140k interrupts > > > 114 kB/s > > > Xen0: > > > ~ 180k interrupts > > > 80 kB/s > > > > It''s pretty odd that Xen''s taking more interrupts. Are you using > > the same native kernel version as you are for Xen? > > > > Also, what happens if you boot xen with ''nosmp'' on the Xen > > command line. > > > > We''ve got Xen tcp performance results from a bunch of machines, > > and dom0 to dom0 performance has always been almost identical to > > native for 1500 byte MTU packets. > > > > > > Ian > > > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by: > > Sybase ASE Linux Express Edition - download now for FREE > > LinuxWorld Reader''s Choice Award Winner for best database on Linux. > > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_idU88&alloc_id065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt
2004-Oct-28 12:44 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
> Yes, both are 2.4.27 > > > Also, what happens if you boot xen with ''nosmp'' on the Xen > > command line. > > > > There seems to be no change in behaviour. The interrupt layout and count remains roughly the same. > > Do you have any more tips? :)Please can you try Xen/linux 2.6.9. Also, please can you remind me of the spec of your machines. Ian -=- MIME -=- On Wed, Oct 27, 2004 at 07:26:10PM +0100, Ian Pratt wrote:> > On Fri, Oct 22, 2004 at 06:47:35PM +0100, Ian Pratt wrote: > > > >=20 > > > > We tried to compile xen0 with CONFIG_E1000_NAPI =3D y and got the samre=20 > > > > results between > > > > two xen dom0 nodes. I am not sure if these interrupts tells anything: > > >=20 > > > It''s the different rate at which the eth0 interrupt counts go up > > > during your bandwidth tests that is interesting. e.g. poll it > > > once a second during the test. > > >=20 > >=20 > > We tried sending 1 MB and measured: >=20 > 1MB isn''t really very much with a 128KB socket buffer. Do you get > the same results with larger transfers? >=20With 10 MB transfers the results are roughly the same: native Linux with compiled-in SMP support, single CPU: ~ 1365k interrupts 114 kB/s Xen0 with "nosmp": ~ 1676 k interrupts 76 kB/s> > non-SMP native Linux: > > ~ 130k interrupts > > 114 kB/s > > native Linux with compiled-in SMP support, single CPU: > > ~ 140k interrupts > > 114 kB/s > > Xen0: > > ~ 180k interrupts > > 80 kB/s >=20 > It''s pretty odd that Xen''s taking more interrupts. Are you using > the same native kernel version as you are for Xen? >=20Yes, both are 2.4.27> Also, what happens if you boot xen with ''nosmp'' on the Xen > command line. >=20There seems to be no change in behaviour. The interrupt layout and count remains roughly the same. Do you have any more tips? :) H=E5vard ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark A. Williamson
2004-Oct-28 13:09 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
Domain 0 is (from the POV of Xen) basically Just Another VM and is scheduled pre-emptively. Some capability flags give the Dom0 VM the privileges it needs in order to control devices, screen, Xen management functions etc. If you use the buggy Atropos then you''ll lose performance even with just one domain. HTH, Mark On Thursday 28 October 2004 11:51, Hvard Bjerke wrote:> We''re getting these results in domain 0, not in a VM. As I understand, BVT > or apropos scheduling do not apply in domain 0? > > Håvard > > On Wed, Oct 27, 2004 at 07:40:27PM +0100, Mark A. Williamson wrote: > > Just a thought: you are using the BVT scheduler, right? We haven''t > > tested performance with the other schedulers recently but we know > > something goes wrong for IO intensive domains on Atropos. > > > > Mark > > > > On Wednesday 27 Oct 2004 19:26, Ian Pratt wrote: > > > > On Fri, Oct 22, 2004 at 06:47:35PM +0100, Ian Pratt wrote: > > > > > > We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the > > > > > > samre results between > > > > > > two xen dom0 nodes. I am not sure if these interrupts tells > > > > > > anything: > > > > > > > > > > It''s the different rate at which the eth0 interrupt counts go up > > > > > during your bandwidth tests that is interesting. e.g. poll it > > > > > once a second during the test. > > > > > > > > We tried sending 1 MB and measured: > > > > > > 1MB isn''t really very much with a 128KB socket buffer. Do you get > > > the same results with larger transfers? > > > > > > > non-SMP native Linux: > > > > ~ 130k interrupts > > > > 114 kB/s > > > > native Linux with compiled-in SMP support, single CPU: > > > > ~ 140k interrupts > > > > 114 kB/s > > > > Xen0: > > > > ~ 180k interrupts > > > > 80 kB/s > > > > > > It''s pretty odd that Xen''s taking more interrupts. Are you using > > > the same native kernel version as you are for Xen? > > > > > > Also, what happens if you boot xen with ''nosmp'' on the Xen > > > command line. > > > > > > We''ve got Xen tcp performance results from a bunch of machines, > > > and dom0 to dom0 performance has always been almost identical to > > > native for 1500 byte MTU packets. > > > > > > > > > Ian > > > > > > > > > > > > ------------------------------------------------------- > > > This SF.Net email is sponsored by: > > > Sybase ASE Linux Express Edition - download now for FREE > > > LinuxWorld Reader''s Choice Award Winner for best database on Linux. > > > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_idU88&alloc_id065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Håvard Bjerke
2004-Oct-28 16:45 UTC
Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
On Thu, Oct 28, 2004 at 01:44:44PM +0100, Ian Pratt wrote:> > Please can you try Xen/linux 2.6.9. Also, please can you remind > me of the spec of your machines. >Specs: Ethernet controller: Intel Corp. 82547GI Gigabit Ethernet Controller CPU: Intel(R) Pentium(R) 4 CPU 3.40GHz single cpu RAM: 1 GB The previous results from 2.4.27 are (I wrote kB/s earlier but that was wrong): Native: 130k interrupts 114 MB/s Xen0: 180k interrupts 80 MB/s New results with 2.6.9: Native: bandwidth: 2048000000 bytes in 17.42 real seconds = 114794.42 KB/sec CPU: 0.0user 1.4sys 0:17real 8% interrupts: 135k Xen0: bandwidth: 2048000000 bytes in 21.77 real seconds = 91885.76 KB/sec CPU: 0.0user 16.2sys 0:21real 74% interrupts: 107k I also tried sending localhost-localhost in 2.4.27, and interestingly Native performed 7:1 times better than xen0: Native: bandwidth: 2048000000 bytes in 2.70 real seconds = 741704.96 KB/sec CPU: 0.0user 1.6sys 0:02real 62% Xen0: bandwidth: 2048000000 bytes in 17.12 real seconds = 116838.03 KB/sec CPU: 5.9user 0.0sys 0:17real 34% We''ve also recently benchmarked in an SMP cluster and achieved satisfying results, ie. around 114 MB/s with both Native and Xen0. But we''re still wondering why we''re not achieving full speed in a single-cpu cluster. Hvard ------------------------------------------------------- This Newsletter Sponsored by: Macrovision For reliable Linux application installations, use the industry''s leading setup authoring tool, InstallShield X. Learn more and evaluate today. http://clk.atdmt.com/MSI/go/ins0030000001msi/direct/01/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Rune Johan Andresen
2004-Oct-29 17:05 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
Hi, we have validated the single CPU results in another single CPU cluster, and we still get an performance loss about 30% (ca. 85 000 KB/s between two dom0 nodes (and NOT 114 000 KB/s), Specs: Ethernet controller: Intel Corp. 82547GI Gigabit Ethernet Controller CPU: Intel(R) Pentium(R) 4 CPU 3.40GHz single CPU RAM: 1 GB) But, in a dual CPU cluster, Intel Xenon CPU 2.40 GHz Ethernet controller: Intel Corp. 82547GI Gigabit Ethernet Controller, we get only 1% (from 114 000 KB/s -> 110 000 KB/s) performance loss.) Both the single CPU clusters and the dual cluster are Xen 2.0 beta (2.4.26 kernel) with Red Hat Ent. 3 It seems to me that there is an issue with Xen and Gigabit ethernet controllers, where Xen is optimized for a dual core(?). As mentioned before we have more interrupts with Xen dom0 on eth0 than Native Linux (we use bvt, not atropos) Are there any Xen developers/testers which have tested Xen 2.0 on a single CPU cluster and can confirm (or not) these results? Cheers, Rune ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt
2004-Oct-29 17:24 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
> But, in a dual CPU cluster, Intel Xenon CPU 2.40 GHz Ethernet > controller: Intel Corp. 82547GI Gigabit Ethernet Controller, we get > only 1% > (from 114 000 KB/s -> 110 000 KB/s) performance loss.)Good, that''s the result we expect.> Hi, we have validated the single CPU results in another single CPU > cluster, and we still get an performance loss about 30% > (ca. 85 000 KB/s between two dom0 nodes (and NOT 114 000 KB/s), > Specs: > Ethernet controller: Intel Corp. 82547GI Gigabit Ethernet Controller > CPU: Intel(R) Pentium(R) 4 CPU 3.40GHz single CPU > RAM: 1 GB)Hmm, have your systems got an IOAPIC, or is Xen using the legacy PIC code? The latter probably hasn''t been thoroughly performance tested... Ian ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Håvard Bjerke
2004-Oct-29 18:39 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
On Fri, Oct 29, 2004 at 06:24:35PM +0100, Ian Pratt wrote:> > > But, in a dual CPU cluster, Intel Xenon CPU 2.40 GHz Ethernet > > controller: Intel Corp. 82547GI Gigabit Ethernet Controller, we get > > only 1% > > (from 114 000 KB/s -> 110 000 KB/s) performance loss.) > > Good, that''s the result we expect. > > > Hi, we have validated the single CPU results in another single CPU > > cluster, and we still get an performance loss about 30% > > (ca. 85 000 KB/s between two dom0 nodes (and NOT 114 000 KB/s), > > Specs: > > Ethernet controller: Intel Corp. 82547GI Gigabit Ethernet Controller > > CPU: Intel(R) Pentium(R) 4 CPU 3.40GHz single CPU > > RAM: 1 GB) > > Hmm, have your systems got an IOAPIC, or is Xen using the legacy > PIC code? The latter probably hasn''t been thoroughly performance > tested... >The native systems can have either XT-PIC or IOAPIC, and it seems that both have equal performance. In Xen0, however, by looking through dmesg, there doesn''t seem to be any IOAPIC. Is it possible to enable (or disable) IOAPIC in Xen0? Hvard ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_idU88&alloc_id065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt
2004-Oct-29 19:46 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
> The native systems can have either XT-PIC or IOAPIC, and it seems that both have equal performance. In Xen0, however, by looking through dmesg, there doesn''t seem to be any IOAPIC. Is it possible to enable (or disable) IOAPIC in Xen0?If you boot with ''ignorebiostables'' on the Xen command line Xen will ignore the IOAPIC and use the PIC. I could believe there might be a lurking performance problem with PIC support as it probably hasn''t had the same level of performance testing as IOAPIC. Ian -=- MIME -=- On Fri, Oct 29, 2004 at 06:24:35PM +0100, Ian Pratt wrote:>=20 > > But, in a dual CPU cluster, Intel Xenon CPU 2.40 GHz Ethernet=20 > > controller: Intel Corp. 82547GI Gigabit Ethernet Controller, we get=20 > > only 1% > > (from 114 000 KB/s -> 110 000 KB/s) performance loss.) >=20 > Good, that''s the result we expect. >=20 > > Hi, we have validated the single CPU results in another single CPU=20 > > cluster, and we still get an performance loss about 30% > > (ca. 85 000 KB/s between two dom0 nodes (and NOT 114 000 KB/s), > > Specs: > > Ethernet controller: Intel Corp. 82547GI Gigabit Ethernet Controller > > CPU: Intel(R) Pentium(R) 4 CPU 3.40GHz single CPU > > RAM: 1 GB) >=20 > Hmm, have your systems got an IOAPIC, or is Xen using the legacy > PIC code? The latter probably hasn''t been thoroughly performance > tested... >=20The native systems can have either XT-PIC or IOAPIC, and it seems that both have equal performance. In Xen0, however, by looking through dmesg, there doesn''t seem to be any IOAPIC. Is it possible to enable (or disable) IOAPIC in Xen0? H=E5vard ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Håvard Bjerke
2004-Nov-02 16:13 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
On Fri, Oct 29, 2004 at 08:46:46PM +0100, Ian Pratt wrote:> > The native systems can have either XT-PIC or IOAPIC, and it seems that both have equal performance. In Xen0, however, by looking through dmesg, there doesn''t seem to be any IOAPIC. Is it possible to enable (or disable) IOAPIC in Xen0? > > If you boot with ''ignorebiostables'' on the Xen command line Xen > will ignore the IOAPIC and use the PIC. > > I could believe there might be a lurking performance problem with > PIC support as it probably hasn''t had the same level of > performance testing as IOAPIC. >''ignorebiostables'' did the trick. Thanks for the help :) Now I''m curious about live migration. Is it possible to live migrate an MPI application running on a set of nodes to another set of nodes? Has anyone tried that? Hvard ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_idU88&alloc_id065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark A. Williamson
2004-Nov-02 16:35 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
> Now I''m curious about live migration. Is it possible to live migrate an MPI > application running on a set of nodes to another set of nodes? Has anyone > tried that?If you''re using MPI over TCP/IP (which I imagine you are) then it should Just Work (TM). We have tried live migration with MPI applications but you shouldn''t have any problems moving the VMs around with a cluster. Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark A. Williamson
2004-Nov-02 16:51 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
> If you''re using MPI over TCP/IP (which I imagine you are) then it should > Just Work (TM). We have tried live migration with MPI applications but you > shouldn''t have any problems moving the VMs around with a cluster.Sorry I meant to say we have *not* tried live migration with MPI applications. Note to self: read before clicking send! Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Rune Johan Andresen
2004-Nov-04 17:42 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
Well, after the issue between two xen dom0 domains is solved there is a new case we don''t understand: With two physical domains and 4 guest OSs (2 on each physical node) we get some rare results with ttcp (b=1000000, l = 1000000): Lets say we have two guest OSs on physical node A, A1 and A2, and two guest OSs on physical node B, B1 and B2. Between A1 and B1 I get 110 000 KB/s (which is almost optimal!) Between A1 and B2 I get 81 0000 KB/s Between A2 and B1 I get 94 000 KB/s Do you have any idea why we get less performance in the last two cases? It doesn''t make sense. It cant be a bottleneck in the network either because of case 1.(?) Cheers, Rune On Nov 2, 2004, at 5:51 PM, Mark A. Williamson wrote:>> If you''re using MPI over TCP/IP (which I imagine you are) then it >> should >> Just Work (TM). We have tried live migration with MPI applications >> but you >> shouldn''t have any problems moving the VMs around with a cluster. > > Sorry I meant to say we have *not* tried live migration with MPI > applications. > > Note to self: read before clicking send! > > Cheers, > Mark > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader''s Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Bin Ren
2004-Nov-04 18:41 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
Among A1, A2, B1, B2, which ones are domain 0 and which are unpriviledged? - Bin On Thu, 4 Nov 2004 18:42:34 +0100, Rune Johan Andresen <runejoha@idi.ntnu.no> wrote:> Well, after the issue between two xen dom0 domains is solved there is a > new case we don''t > understand: > > With two physical domains and 4 guest OSs (2 on each physical node) we > get some rare results with ttcp (b=1000000, l = 1000000): > > Lets say we have two guest OSs on physical node A, A1 and A2, and two > guest OSs on physical node B, B1 and B2. > > Between A1 and B1 I get 110 000 KB/s (which is almost optimal!) > Between A1 and B2 I get 81 0000 KB/s > Between A2 and B1 I get 94 000 KB/s > > Do you have any idea why we get less performance in the last two cases? > It doesn''t make sense. It cant be > a bottleneck in the network either because of case 1.(?) > > Cheers, > Rune > > > > > On Nov 2, 2004, at 5:51 PM, Mark A. Williamson wrote: > > >> If you''re using MPI over TCP/IP (which I imagine you are) then it > >> should > >> Just Work (TM). We have tried live migration with MPI applications > >> but you > >> shouldn''t have any problems moving the VMs around with a cluster. > > > > Sorry I meant to say we have *not* tried live migration with MPI > > applications. > > > > Note to self: read before clicking send! > > > > Cheers, > > Mark > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by: > > Sybase ASE Linux Express Edition - download now for FREE > > LinuxWorld Reader''s Choice Award Winner for best database on Linux. > > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xen-devel > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader''s Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel >------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Rune Johan Andresen
2004-Nov-05 14:53 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
A and B are domain 0, A1,A2,B1,B2 are the guest OS systems. I think all are privileged (?), They are created with the same config file, exception of the file loopback configuration and ip config. Cheers, Rune On Nov 4, 2004, at 7:41 PM, Bin Ren wrote:> Among A1, A2, B1, B2, which ones are domain 0 and which are > unpriviledged? > > - Bin > > On Thu, 4 Nov 2004 18:42:34 +0100, Rune Johan Andresen > <runejoha@idi.ntnu.no> wrote: >> Well, after the issue between two xen dom0 domains is solved there is >> a >> new case we don''t >> understand: >> >> With two physical domains and 4 guest OSs (2 on each physical node) we >> get some rare results with ttcp (b=1000000, l = 1000000): >> >> Lets say we have two guest OSs on physical node A, A1 and A2, and two >> guest OSs on physical node B, B1 and B2. >> >> Between A1 and B1 I get 110 000 KB/s (which is almost optimal!) >> Between A1 and B2 I get 81 0000 KB/s >> Between A2 and B1 I get 94 000 KB/s >> >> Do you have any idea why we get less performance in the last two >> cases? >> It doesn''t make sense. It cant be >> a bottleneck in the network either because of case 1.(?) >> >> Cheers, >> Rune >> >> >> >> >> On Nov 2, 2004, at 5:51 PM, Mark A. Williamson wrote: >> >>>> If you''re using MPI over TCP/IP (which I imagine you are) then it >>>> should >>>> Just Work (TM). We have tried live migration with MPI applications >>>> but you >>>> shouldn''t have any problems moving the VMs around with a cluster. >>> >>> Sorry I meant to say we have *not* tried live migration with MPI >>> applications. >>> >>> Note to self: read before clicking send! >>> >>> Cheers, >>> Mark >>> >>> >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: >>> Sybase ASE Linux Express Edition - download now for FREE >>> LinuxWorld Reader''s Choice Award Winner for best database on Linux. >>> http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/xen-devel >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: >> Sybase ASE Linux Express Edition - download now for FREE >> LinuxWorld Reader''s Choice Award Winner for best database on Linux. >> http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/xen-devel >> > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader''s Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Keir Fraser
2004-Nov-05 16:47 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
> > A and B are domain 0, A1,A2,B1,B2 are the guest OS systems. I think all > are privileged (?), They are > created with the same config file, exception of the file loopback > configuration and ip config. > > Cheers, > RuneAre you running on SMP systems? I would guess the performance differs based on whether DOM0 and the unprivileged guest are runnign on the same CPU or different CPUs... -- Keir ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Rune Johan Andresen
2004-Nov-05 17:33 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
Yes, I am running on a SMP system with two CPUs. The nodes are running on different CPUs, but is this a problem? If the two guest OSs are unprivileged and run on separate CPUs, they should perform I/O just as good (or bad) anyway, no? Cheers, Rune On Nov 5, 2004, at 5:47 PM, Keir Fraser wrote:>> >> A and B are domain 0, A1,A2,B1,B2 are the guest OS systems. I think >> all >> are privileged (?), They are >> created with the same config file, exception of the file loopback >> configuration and ip config. >> >> Cheers, >> Rune > > Are you running on SMP systems? I would guess the performance differs > based on whether DOM0 and the unprivileged guest are runnign on the > same CPU or different CPUs... > > -- Keir------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Keir Fraser
2004-Nov-06 08:04 UTC
Re: [Xen-devel] Xen & I/O in clusters - Single Vs. Dual CPU issue
Unless your workload maxes out one of the CPUs, or doesn''t play as well as it could with the CPU scheduler. In general it''s probably wise to run your dom0 on another CPU, or at least HyperThread, from other performance-critical VMs. The situation on a single CPU should be improveable with tweaking, however. -- Keir> > Yes, I am running on a SMP system with two CPUs. The nodes are running > on different CPUs, but is this a problem? If the two > guest OSs are unprivileged and run on separate CPUs, they should > perform I/O just as good (or bad) anyway, no? > > Cheers, > Rune > > > On Nov 5, 2004, at 5:47 PM, Keir Fraser wrote: > > >> > >> A and B are domain 0, A1,A2,B1,B2 are the guest OS systems. I think > >> all > >> are privileged (?), They are > >> created with the same config file, exception of the file loopback > >> configuration and ip config. > >> > >> Cheers, > >> Rune > > > > Are you running on SMP systems? I would guess the performance differs > > based on whether DOM0 and the unprivileged guest are runnign on the > > same CPU or different CPUs... > > > > -- Keir >------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel