Hi, I started to experiment with xen, so I am still new to it. I was impressed how easy I got my first virtual node up and running. I measured the disk speed, created a 1gb file with dd. copying that file on the dom0 always took about 5 seconds, on the domU, it took about 15-20seconds. I used "time cp large_file large_file2" to measure the speed. I only expected a small time difference, but not factor 3-4. My virtual disk is a physical device, configured like this: disk=[ ''phy:/dev/sdv2,xvda,w'', ] As far as I know, using the physical partitions as the virtual disk, should be the fastest solution for virtual disks, compared to files. Are there different ways to present a physical partition from dom0 to a domU, that would influence the speed? Or is the speed factor I have seen above the one to expect? I am on sles10sp1 x86_64. kind regards Sebastian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Dan, "Dan VerWeire" <dverweire@gmail.com> wrote:> Are you using PV or HVM DomU''s? If HVM, this may be expected as the block > device presented to the DomU is an emulated device in Dom0. Someonecorrect> me if I am wrong please. >thanks for your insights. I am running paravirtualized machine, so if I understand correct, I should get sth. different? Sebastian> Dan > > On 10/22/07, Sebastian Reitenbach <sebastia@l00-bugdead-prods.de> wrote: > > > > Hi, > > > > I started to experiment with xen, so I am still new to it. I wasimpressed> > how easy I got my first virtual node up and running. > > I measured the disk speed, created a 1gb file with dd. > > copying that file on the dom0 always took about 5 seconds, on the domU,it> > took about 15-20seconds. I used "time cp large_file large_file2" to > > measure > > the speed. I only expected a small time difference, but not factor 3-4. > > My virtual disk is a physical device, configured like this: > > disk=[ ''phy:/dev/sdv2,xvda,w'', ] > > > > > > As far as I know, using the physical partitions as the virtual disk, > > should > > be the fastest solution for virtual disks, compared to files. > > > > Are there different ways to present a physical partition from dom0 to a > > domU, that would influence the speed? Or is the speed factor I have seen > > above the one to expect? > > > > I am on sles10sp1 x86_64. > > > > kind regards > > Sebastian > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Sebastian, AFAIK, you should be getting some better results. But most of my experience is with Windows HVM domUs, so unfortunately I can''t offer you any more insight into your situation. Good luck! Dan On 10/22/07, Sebastian Reitenbach <sebastia@l00-bugdead-prods.de> wrote:> > Hi Dan, > > "Dan VerWeire" <dverweire@gmail.com> wrote: > > Are you using PV or HVM DomU''s? If HVM, this may be expected as the > block > > device presented to the DomU is an emulated device in Dom0. Someone > correct > > me if I am wrong please. > > > thanks for your insights. I am running paravirtualized machine, so if I > understand correct, I should get sth. different? > > Sebastian > > > Dan > > > > On 10/22/07, Sebastian Reitenbach <sebastia@l00-bugdead-prods.de> wrote: > > > > > > Hi, > > > > > > I started to experiment with xen, so I am still new to it. I was > impressed > > > how easy I got my first virtual node up and running. > > > I measured the disk speed, created a 1gb file with dd. > > > copying that file on the dom0 always took about 5 seconds, on the > domU, > it > > > took about 15-20seconds. I used "time cp large_file large_file2" to > > > measure > > > the speed. I only expected a small time difference, but not factor > 3-4. > > > My virtual disk is a physical device, configured like this: > > > disk=[ ''phy:/dev/sdv2,xvda,w'', ] > > > > > > > > > As far as I know, using the physical partitions as the virtual disk, > > > should > > > be the fastest solution for virtual disks, compared to files. > > > > > > Are there different ways to present a physical partition from dom0 to > a > > > domU, that would influence the speed? Or is the speed factor I have > seen > > > above the one to expect? > > > > > > I am on sles10sp1 x86_64. > > > > > > kind regards > > > Sebastian > > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Oct 22, 2007 at 02:12:39PM +0200, Sebastian Reitenbach wrote:> > I measured the disk speed, created a 1gb file with dd. > copying that file on the dom0 always took about 5 seconds, on the domU, it > took about 15-20seconds. I used "time cp large_file large_file2" to measure > the speed. I only expected a small time difference, but not factor 3-4.We also did some testing like this, writing inside a domU sitting on lvm on local discs took 3.5 times as long as dom0 writes to a filesystem there. Some values here: http://fluxcoil.net/doku.php/xen/docs - but i cant explain some numbers myself and should redo the testing. Also the values vary when testing different xen-packages from suse.> As far as I know, using the physical partitions as the virtual disk, should > be the fastest solution for virtual disks, compared to files.Files when loopbackmounted showed good values, but shouldnt be used for known reasons. Just that using tap:aio still makes trouble for us on those sles10sp1 amd64 boxes.> Are there different ways to present a physical partition from dom0 to a > domU, that would influence the speed? Or is the speed factor I have seen > above the one to expect?When dom0 is involved i dont know of a different way. One could still look into performance of space available via iscsi to the domU, or handing a pci-device like a san- or scsi-card over to the domU (with this trading the better performance for features like live-migration). Christian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Has all the testing that shows this slowness been done with large files? I''d be interested to see if the same is true under more normal use. E.G. copy 10 medium files 10 times each and 100 medium files 1 time each. Caching could make it faster on domU and seeking around could make it slower... Or who knows what other variables might kick in..> On Mon, Oct 22, 2007 at 02:12:39PM +0200, Sebastian Reitenbach wrote: > > > > I measured the disk speed, created a 1gb file with dd. > > copying that file on the dom0 always took about 5 seconds, on the domU, it > > took about 15-20seconds. I used "time cp large_file large_file2" to measure > > the speed. I only expected a small time difference, but not factor 3-4. > We also did some testing like this, writing inside a domU sitting on lvm > on local discs took 3.5 times as long as dom0 writes to a filesystem > there. Some values here: http://fluxcoil.net/doku.php/xen/docs - but i > cant explain some numbers myself and should redo the testing. > Also the values vary when testing different xen-packages from suse. > > > As far as I know, using the physical partitions as the virtual disk, should > > be the fastest solution for virtual disks, compared to files. > Files when loopbackmounted showed good values, but shouldnt be used for > known reasons. Just that using tap:aio still makes trouble for us on those > sles10sp1 amd64 boxes. > > > Are there different ways to present a physical partition from dom0 to a > > domU, that would influence the speed? Or is the speed factor I have seen > > above the one to expect? > When dom0 is involved i dont know of a different way. One could still look > into performance of space available via iscsi to the domU, or handing a > pci-device like a san- or scsi-card over to the domU (with this trading the > better performance for features like live-migration). > > > Christian > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Oct 22, 2007 at 10:45:32AM -0700, Dylan Martin wrote:> Has all the testing that shows this slowness been done with large > files?Yes> I''d be interested to see if the same is true under more normal > use. E.G. copy 10 medium files 10 times each and 100 medium files 1 > time each. Caching could make it faster on domU and seeking around > could make it slower... Or who knows what other variables might kick > in..I wasnt interested in how well caching in the domU works, as its the same algorithms as non-xen-linux. Just the raw i/o was measured, and making sure the data was synced. If you need profiles of how well the caching works you should bench- mark with exactly your usecase anyway... Christian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> ... > If you need profiles of how well the caching works you should bench- > mark with exactly your usecase anyway... > > ChristianAbsolutely true, but I''m just curious and too lazy to do it myself... _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi all, Dylan Martin <dmartin@sccd.ctc.edu> wrote:> Has all the testing that shows this slowness been done with large > files? I''d be interested to see if the same is true under more normal > use. E.G. copy 10 medium files 10 times each and 100 medium files 1 > time each. Caching could make it faster on domU and seeking around > could make it slower... Or who knows what other variables might kick > in..yes, it has been done with these files. In my usecase I have to handle a lot of files of that size. So I do not really care how fast I can handle a million 1k sized files.> > > On Mon, Oct 22, 2007 at 02:12:39PM +0200, Sebastian Reitenbach wrote: > > > > > > I measured the disk speed, created a 1gb file with dd. > > > copying that file on the dom0 always took about 5 seconds, on thedomU, it> > > took about 15-20seconds. I used "time cp large_file large_file2" tomeasure> > > the speed. I only expected a small time difference, but not factor3-4.> > We also did some testing like this, writing inside a domU sitting on lvm > > on local discs took 3.5 times as long as dom0 writes to a filesystem > > there. Some values here: http://fluxcoil.net/doku.php/xen/docs - but i > > cant explain some numbers myself and should redo the testing. > > Also the values vary when testing different xen-packages from suse. > > > > > As far as I know, using the physical partitions as the virtual disk,should> > > be the fastest solution for virtual disks, compared to files. > > Files when loopbackmounted showed good values, but shouldnt be used for > > known reasons. Just that using tap:aio still makes trouble for us onthose> > sles10sp1 amd64 boxes. > > > > > Are there different ways to present a physical partition from dom0 toa> > > domU, that would influence the speed? Or is the speed factor I haveseen> > > above the one to expect? > > When dom0 is involved i dont know of a different way. One could stilllook> > into performance of space available via iscsi to the domU, or handing a > > pci-device like a san- or scsi-card over to the domU (with this tradingthe> > better performance for features like live-migration).Trying iSCSI sounds interesting. Also I did now know yet, that I can hand over the SAN device to the virtual node. I want to use xen in a HA cluster, as long as everything is in a good condition each virtual machine will be on a separate physical machine, but if one of the physical nodes dies, two or more of the xen instances have to share a physical node. Do I can hand over one physical device to more than one virtual instance in that case? If not, then I have to use iSCSI. kind regards Sebastian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi, Sebastian Reitenbach <sebastia@l00-bugdead-prods.de> wrote:> Hi all, > > Dylan Martin <dmartin@sccd.ctc.edu> wrote: > > Has all the testing that shows this slowness been done with large > > files? I''d be interested to see if the same is true under more normal > > use. E.G. copy 10 medium files 10 times each and 100 medium files 1 > > time each. Caching could make it faster on domU and seeking around > > could make it slower... Or who knows what other variables might kick > > in.. > yes, it has been done with these files. In my usecase I have to handle alot> of files of that size. So I do not really care how fast I can handle a > million 1k sized files. > > > > > > On Mon, Oct 22, 2007 at 02:12:39PM +0200, Sebastian Reitenbach wrote: > > > > > > > > I measured the disk speed, created a 1gb file with dd. > > > > copying that file on the dom0 always took about 5 seconds, on the > domU, it > > > > took about 15-20seconds. I used "time cp large_file large_file2" to > measure > > > > the speed. I only expected a small time difference, but not factor > 3-4. > > > We also did some testing like this, writing inside a domU sitting onlvm> > > on local discs took 3.5 times as long as dom0 writes to a filesystem > > > there. Some values here: http://fluxcoil.net/doku.php/xen/docs - but i > > > cant explain some numbers myself and should redo the testing. > > > Also the values vary when testing different xen-packages from suse. > > > > > > > As far as I know, using the physical partitions as the virtual disk, > should > > > > be the fastest solution for virtual disks, compared to files. > > > Files when loopbackmounted showed good values, but shouldnt be usedfor> > > known reasons. Just that using tap:aio still makes trouble for us on > those > > > sles10sp1 amd64 boxes. > > > > > > > Are there different ways to present a physical partition from dom0to> a > > > > domU, that would influence the speed? Or is the speed factor I have > seen > > > > above the one to expect? > > > When dom0 is involved i dont know of a different way. One could still > look > > > into performance of space available via iscsi to the domU, or handinga> > > pci-device like a san- or scsi-card over to the domU (with thistrading> the > > > better performance for features like live-migration). > Trying iSCSI sounds interesting. Also I did now know yet, that I can hand > over the SAN device to the virtual node. > I want to use xen in a HA cluster, as long as everything is in a good > condition each virtual machine will be on a separate physical machine, but > if one of the physical nodes dies, two or more of the xen instances haveto> share a physical node. Do I can hand over one physical device to more than > one virtual instance in that case? If not, then I have to use iSCSI.I just tried to use iscsi, but it does not seem to be faster than the physical disk. I tried to figure out whether there are some parameters that I can set to influence the disk speed. In the end I am now more confused than before. I created the virtual machines with virt-manager GUI. It created a file /etc/xen/vm/sles10, there the disk is configured like this: disk=[ ''phy:/dev/sdv1,sda,w'', ''phy:/dev/sdv2,xvdb,w'', ] in the xen manual example disk configurations look like this: disk = [ ’phy:hda1,sda1,w’ ] especially I am wondering about the differences here between xvdb and sda1, I tried to edit /etc/xen/vm/sles10[.xml] manually and restarted xend, but the virtual machine still has the xvda and xvdb devices. I also changed the file in /var/lib/xend/domains/.../config.sxp, but they were overwritten on restart. Does this configuration makes a difference? How can I tell xen to use sda instead of xvdb. I have a physical partition, where the virtual host creates its own partitions in it. in case I understand the example in the xen manual correctly, a physical partition is mapped one to one to a virtual partition. Could that possibly speed up the disk access? I searched the wiki for disk, and I found the http://wiki.xensource.com/xenwiki/XenStoreReference, but it did not helped me that much. kind regards Sebastian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Dylan Martin wrote:> Has all the testing that shows this slowness been done with large > files? I''d be interested to see if the same is true under more normal > use. E.G. copy 10 medium files 10 times each and 100 medium files 1 >I ran some Bonnie++ tests on a recent server before it went into production. My results are just raw Bonnie++ output with no real analysis here if you''re interested: http://mikebabcock.ca/linux/xen/benchmarks/ -- Michael T. Babcock http://mikebabcock.ca _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users