Xen users - Dec 2005 - Poor hard disk performance on xen-3/dom0

On a vanilla 2.6.12 kernel, I get:
   60MB/s for hdparm -t /dev/sd[a|b] (on an Intel ICH7 SATA controller).
   30MB/s for hdparm -t /dev/hda (Intel ICH7)

On the xen dom0 kernel, I get 20MB/s for all three devices. Xen is 
compiled with the following:

xen_changeset          : Fri Dec  9 11:05:06 2005 +0000 8311:53cff3f88e45
cc_compiler            : gcc version 3.3.5 (Debian 1:3.3.5-13)

Does anyone else face similar problems?

-- 
Regards,
wK (www.doubleukay.com)


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On a vanilla 2.6.12 kernel, I get:
   60MB/s for hdparm -t /dev/sd[a|b] (on an Intel ICH7 SATA controller).
   30MB/s for hdparm -t /dev/hda (Intel ICH7)

On the xen dom0 kernel, I get 20MB/s for all three devices. Xen is 
compiled with the following:

xen_changeset          : Fri Dec  9 11:05:06 2005 +0000 8311:53cff3f88e45
cc_compiler            : gcc version 3.3.5 (Debian 1:3.3.5-13)

Does anyone else face similar problems?

-- 
Regards,
wK (www.doubleukay.com)


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On a vanilla 2.6.12 kernel, I get:

   60MB/s for hdparm -t /dev/sd[a|b] (on an Intel ICH7 SATA controller).
   30MB/s for hdparm -t /dev/hda (Intel ICH7)

While on the xen dom0 kernel, I get 20MB/s for all three devices. I 
confirm that DMA is enabled for the PATA hard disk. I don''t know how to
check that for the SATA hard disks, but I assume that they''re on DMA 
mode too.

Xen is compiled with the following:

xen_changeset          : Fri Dec  9 11:05:06 2005 +0000 8311:53cff3f88e45
cc_compiler            : gcc version 3.3.5 (Debian 1:3.3.5-13)

The dom0 kernel is compiled with almost the same options as the vanilla 
kernel, barring the xen-exclusive and nonxen-exclusive options.

Does anyone else face similar problems?

-- 
Regards,
wK (www.doubleukay.com)


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On a vanilla 2.6.12 kernel, I get:

   60MB/s for hdparm -t /dev/sd[a|b] (on an Intel ICH7 SATA controller).
   30MB/s for hdparm -t /dev/hda (Intel ICH7)

While on the xen dom0 kernel, I get 20MB/s for all three devices. I 
confirm that DMA is enabled for the PATA hard disk. I don''t know how to
check that for the SATA hard disks, but I assume that they''re on DMA 
mode too.

Xen is compiled with the following:

xen_changeset          : Fri Dec  9 11:05:06 2005 +0000 8311:53cff3f88e45
cc_compiler            : gcc version 3.3.5 (Debian 1:3.3.5-13)

The dom0 kernel is compiled with almost the same options as the vanilla 
kernel, barring the xen-exclusive and nonxen-exclusive options.

Does anyone else face similar problems?

-- 
Regards,
wK (www.doubleukay.com)

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Compare the boot messages of vanilla vs. xen -- you''re likely using a
different driver.

I know everyone always ignores me whenever I say this, but users really
should be using the -xen kernel rather than -xen0/U. The latter two are
just to make build times quicker for developers. The -xen kernel has way
more drivers, though built as modules, so you''ll need to make an
initrd.

Ian
> On a vanilla 2.6.12 kernel, I get:
> 
>    60MB/s for hdparm -t /dev/sd[a|b] (on an Intel ICH7 SATA 
> controller).
>    30MB/s for hdparm -t /dev/hda (Intel ICH7)
> 
> While on the xen dom0 kernel, I get 20MB/s for all three 
> devices. I confirm that DMA is enabled for the PATA hard 
> disk. I don''t know how to check that for the SATA hard disks, 
> but I assume that they''re on DMA mode too.
> 
> Xen is compiled with the following:
> 
> xen_changeset          : Fri Dec  9 11:05:06 2005 +0000 
> 8311:53cff3f88e45
> cc_compiler            : gcc version 3.3.5 (Debian 1:3.3.5-13)
> 
> The dom0 kernel is compiled with almost the same options as 
> the vanilla kernel, barring the xen-exclusive and 
> nonxen-exclusive options.
> 
> Does anyone else face similar problems?
> 
> --
> Regards,
> wK (www.doubleukay.com)
> 
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Ian Pratt wrote:
> Compare the boot messages of vanilla vs. xen -- you''re likely
using a
> different driver.
> 
> I know everyone always ignores me whenever I say this, but users really
> should be using the -xen kernel rather than -xen0/U. The latter two are
> just to make build times quicker for developers. The -xen kernel has way
> more drivers, though built as modules, so you''ll need to make an
initrd.
>
I''ve always missed it (I don''t read ALL threads).  What
exactly IS the
-xen kernel?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, 30 Dec 2005, Ian Pratt wrote:
>
> Compare the boot messages of vanilla vs. xen -- you''re likely
using a
> different driver.
>
> I know everyone always ignores me whenever I say this, but users really
> should be using the -xen kernel rather than -xen0/U. The latter two are
> just to make build times quicker for developers. The -xen kernel has way
> more drivers, though built as modules, so you''ll need to make an
initrd.
and building initrd''s is a non trivial process... much simpler to put a
few entries into /dev/, build a xen0 kernel with the required drivers, and
take the initrd out of the picture...

(Maybe there are new tools that make initrd''s simple, but the above is
valid for old farts/admins like me... it also has the advantage of
producing a config that I _understand_ and thus _trust_ .)

-Tom




_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Ian Pratt wrote:
> I know everyone always ignores me whenever I say this, but users really
> should be using the -xen kernel rather than -xen0/U. The latter two are
> just to make build times quicker for developers. The -xen kernel has way
> more drivers, though built as modules, so you''ll need to make an
initrd.

http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html#SECTION02140000000000000000

section 2.4.2 building from source
---

unless I''m missing something, the official docs say nothing about a
-xen
kernel (and modules).  what are more appropriate instructions for building?

--- eric

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Thanks Ian for the comment on -xen. But if -xen kernel has more drivers then
why is the size of vmlinuz small for it? Here is the comparision between
sizes of vmlinuz:

jayesh@sam:~/xen$ ls -lh linux-2.6.12-xen/vmlinuz
-rw-r--r--  1 root root 1.6M 2005-12-17 11:16 linux-2.6.12-xen/vmlinuz
jayesh@sam:~/xen$ ls -lh linux-2.6.12-xen0/vmlinuz
-rw-r--r--  1 root root 2.2M 2005-12-17 11:57 linux-2.6.12-xen0/vmlinuz
jayesh@sam:~/xen$ ls -lh linux-2.6.12-xenU/vmlinuz
-rw-r--r--  1 root root 1.2M 2005-12-17 12:00 linux-2.6.12-xenU/vmlinuz

Shouldn''t vmlinuz image of -xen be bigger if it has more drivers
in-built
than -xen0/U ?

I am having problem (previously discussed at
http://lists.xensource.com/archives/html/xen-users/2005-12/msg00533.html)
getting Xen started on Ubuntu 5.10 (I did it successfully on RHFC4). The
boot process freezes after loading initrd. Ubuntu doesn''t have mkinitrd
by
default, so I am using mkinitramfs. I don''t know if it is initrd that
is
causing problem. So if I could boot without using initrd, that might solve
my problem. Right now if I don''t use initrd the boot process just
freezes.

Thanks for your help.
Jayesh


On 12/30/05, Eric S. Johansson <esj@harvee.org>
wrote:
>
> Ian Pratt wrote:
> > I know everyone always ignores me whenever I say this, but users
really
> > should be using the -xen kernel rather than -xen0/U. The latter two
are
> > just to make build times quicker for developers. The -xen kernel has
way
> > more drivers, though built as modules, so you''ll need to make
an initrd.
>
>
>
>
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html#SECTION02140000000000000000
>
> section 2.4.2 building from source
> ---
>
> unless I''m missing something, the official docs say nothing about
a -xen
> kernel (and modules).  what are more appropriate instructions for
> building?
>
> --- eric
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>


--
Jayesh


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Sat, 31 Dec 2005 13:18:40 -0600
Jayesh Salvi <jayeshsalvi@gmail.com> wrote:
> Thanks Ian for the comment on -xen. But if -xen kernel has more drivers
then
> why is the size of vmlinuz small for it? 
I think because the bulk of the drivers are modules, so you should compare the
vmlinuz file + the appropriate /lib/modules/* directory. 

Tim 



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Ok. In that case I see -xen is far bigger than -xen0/U

jayesh@sam:/lib/modules$ ls | xargs du -sh
44M     2.6.12.6-xen
1.2M    2.6.12.6-xen0
316K    2.6.12.6-xenU
62M     2.6.12-9-686

This doesn''t however yet make me boot without initrd. But I will try
some
more things.

Thanks Tim for clarification.

Jayesh
On 12/31/05, Tim Freeman <tfreeman@mcs.anl.gov>
wrote:
>
> On Sat, 31 Dec 2005 13:18:40 -0600
> Jayesh Salvi <jayeshsalvi@gmail.com> wrote:
>
> > Thanks Ian for the comment on -xen. But if -xen kernel has more
drivers
> then
> > why is the size of vmlinuz small for it?
>
> I think because the bulk of the drivers are modules, so you should compare
> the
> vmlinuz file + the appropriate /lib/modules/* directory.
>
> Tim
>
>
>

--
Jayesh


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Ian Pratt wrote:
> I know everyone always ignores me whenever I say this, but users really
> should be using the -xen kernel rather than -xen0/U.
Humn, maybe it''s just me that''s not ignoring you.  But the
names xen
vs. xen0/U does not infer any obvious difference and I''ve never seen
the difference documented anywhere (although it probably is, who
knows).
> The latter two are
> just to make build times quicker for developers. The -xen kernel has way
> more drivers, though built as modules, so you''ll need to make an
initrd.
Maybe renaming them -xendevel0 and -xendevelU or some such would help?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On 1/5/06, Molle Bestefich <molle.bestefich@gmail.com>
wrote:
> Ian Pratt wrote:
> > I know everyone always ignores me whenever I say this, but users
really
> > should be using the -xen kernel rather than -xen0/U.
>
> Humn, maybe it''s just me that''s not ignoring you.  But
the names xen
> vs. xen0/U does not infer any obvious difference and I''ve never
seen
> the difference documented anywhere (although it probably is, who
> knows).
>
> > The latter two are
> > just to make build times quicker for developers. The -xen kernel has
way
> > more drivers, though built as modules, so you''ll need to make
an initrd.
>
> Maybe renaming them -xendevel0 and -xendevelU or some such would help?
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
Now documented at the following location.

http://wiki.xensource.com/xenwiki/InstallationNotes#head-64f7b0db4fee44809a97a827e92fa9ed27a732a8

David

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Jayesh Salvi wrote:
> I am having problem (previously discussed at 
> http://lists.xensource.com/archives/html/xen-users/2005-12/msg00533.html) 
> getting Xen started on Ubuntu 5.10 (I did it successfully on RHFC4). The 
> boot process freezes after loading initrd. Ubuntu doesn''t have
mkinitrd
> by default, so I am using mkinitramfs. I don''t know if it is
initrd that
> is causing problem. So if I could boot without using initrd, that might 
> solve my problem. Right now if I don''t use initrd the boot process
just
> freezes.
As Ian mentioned I expect booting -xen kernel instead of -xen0 should 
improve things, just make sure you generate an initramfs for that image.

vmlinuz-2.6.12.6-xen is 1.6MB, initrd.img-2.6.12.6-xen is 5.4MB and I''m
faily sure I generated that with mkinitramfs though it''s been quite a 
while so I may be forgetting.

My full time workstation is a laptop running Ubuntu 5.10 on Xen 3 so it 
definitely works.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Thanks Fraser. Yeah I solved my problem. There was some problem with my
serial console(
http://lists.xensource.com/archives/html/xen-users/2005-12/msg00932.html).
But I agree, that -xen is a better option. For e.g. On my RH laptop, my
sound card wasn''t working with -xen0 but it works with -xen, I
attribute
that to all the drivers that come with -xen.

And I got other machine running with Ubuntu 5.10 and Xen 3 with an initrd
that I created from mkinitramfs. So it sure works fine, as you said.

Thanks.
Jayesh
On 1/6/06, Fraser Campbell <fraser@georgetown.wehave.net>
wrote:
>
> Jayesh Salvi wrote:
>
> > I am having problem (previously discussed at
> >
> http://lists.xensource.com/archives/html/xen-users/2005-12/msg00533.html)
> > getting Xen started on Ubuntu 5.10 (I did it successfully on RHFC4).
The
> > boot process freezes after loading initrd. Ubuntu doesn''t
have mkinitrd
> > by default, so I am using mkinitramfs. I don''t know if it is
initrd that
> > is causing problem. So if I could boot without using initrd, that
might
> > solve my problem. Right now if I don''t use initrd the boot
process just
> > freezes.
>
> As Ian mentioned I expect booting -xen kernel instead of -xen0 should
> improve things, just make sure you generate an initramfs for that image.
>
> vmlinuz-2.6.12.6-xen is 1.6MB, initrd.img-2.6.12.6-xen is 5.4MB and
I''m
> faily sure I generated that with mkinitramfs though it''s been
quite a
> while so I may be forgetting.
>
> My full time workstation is a laptop running Ubuntu 5.10 on Xen 3 so it
> definitely works.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>


--
Jayesh


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

In reply to myself, I just discovered a factor to the problem I''m
facing.

# xm mem-set 0 120
# hdparm -t /dev/sda

  /dev/sda:
   Timing buffered disk reads:  180 MB in  3.02 seconds =  59.60 MB/sec

# xm mem-set 0 80
# hdparm -t /dev/sda

  /dev/sda:
   Timing buffered disk reads:   68 MB in  3.08 seconds =  22.08 MB/sec

I tried this in the non-xen kernel with a mem=64M boot flag, and found 
that it does not happen (disk performs as expected). So I think that 
something, somewhere, in the dom0 kernel underperforms with reduced RAM.

I''m fine with giving dom0 an extra 64MB just to avoid this problem, but
I''m also curious as to why this happens.

Ian Pratt wrote:
> Compare the boot messages of vanilla vs. xen -- you''re likely
using a
> different driver.
> 
> I know everyone always ignores me whenever I say this, but users really
> should be using the -xen kernel rather than -xen0/U. The latter two are
> just to make build times quicker for developers. The -xen kernel has way
> more drivers, though built as modules, so you''ll need to make an
initrd.
> 
> Ian
> 
>>On a vanilla 2.6.12 kernel, I get:
>>
>>   60MB/s for hdparm -t /dev/sd[a|b] (on an Intel ICH7 SATA 
>>controller).
>>   30MB/s for hdparm -t /dev/hda (Intel ICH7)
>>
>>While on the xen dom0 kernel, I get 20MB/s for all three 
>>devices. I confirm that DMA is enabled for the PATA hard 
>>disk. I don''t know how to check that for the SATA hard disks, 
>>but I assume that they''re on DMA mode too.
>>
>>Xen is compiled with the following:
>>
>>xen_changeset          : Fri Dec  9 11:05:06 2005 +0000 
>>8311:53cff3f88e45
>>cc_compiler            : gcc version 3.3.5 (Debian 1:3.3.5-13)
>>
>>The dom0 kernel is compiled with almost the same options as 
>>the vanilla kernel, barring the xen-exclusive and 
>>nonxen-exclusive options.
>>
>>Does anyone else face similar problems?
-- 
Regards,
wK (www.doubleukay.com)

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Jan 19, 2006 at 03:14:50AM +1100, Woon Wai Keen
wrote:
> In reply to myself, I just discovered a factor to the problem I''m
facing.
> 
> # xm mem-set 0 120
> # hdparm -t /dev/sda
> 
>  /dev/sda:
>   Timing buffered disk reads:  180 MB in  3.02 seconds =  59.60 MB/sec
> 
> # xm mem-set 0 80
> # hdparm -t /dev/sda
> 
>  /dev/sda:
>   Timing buffered disk reads:   68 MB in  3.08 seconds =  22.08 MB/sec
> 
> I tried this in the non-xen kernel with a mem=64M boot flag, and found 
> that it does not happen (disk performs as expected). So I think that 
> something, somewhere, in the dom0 kernel underperforms with reduced RAM.
> 
> I''m fine with giving dom0 an extra 64MB just to avoid this
problem, but
> I''m also curious as to why this happens.
Seems like the benchmark is tainted because of buffer cache
interactions. Please try using a real benchmark program like bonnie++
and post your results.

Marcin
-- 
Marcin Owsiany
porridge@expro.pl

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users