Xen users - Sep 2005 - XEN, SLES9, DELL 6800 and ORACLE experience

We started to play XEN a few weeks go. Now I would like to share our experience.
Our DELL 6800 have four 3Ghz ht 64bit Potomac processors, 12Gb of RAM, dual
network card and plenty of disk space.

Our goal was using this machine for several SUSE Linux Enterprise Server v.9
virtual machines with Oracle RDBMS. The dom0 machine was to carry a production
Oracle10g database. We would like to configure the dom0 Linux for the
production. Production was to use only 4Gb of memory, all the 4(8 virtual)
processors and some 100Gb of disk space. The rest of resources we planned to
split into a few additional virtual SLES9 development and test sites with Oracle
too.

We used xen 2.0.7 from source tree and SLES9 SP2. First we successfully
installed all we wanted onto a PC with only one CPU and 512Mb of memory just to
polish all the installation sequence because the installation guides of XEN is
far from straightforward and the related XEN documentation is not satisfactory
at. We managed it all on a PC: SLES9, XEN2.0.7 and Oracle10gR1. Everything went
OK: Oracle runs test databases on both dom0 and domU Linuxes. Nevertheless we
had to have previously removed 2 of 3 network cards from the PC to make Linux
configure networking properly in every booting kernel. We need only one
interface but the two spare network cards confused XEN.

Then we went to the dell6800 and installed SLES9.sp2 on it. We were glad seing
full 12Gb in the output of Linux ''free'' command. (SLES8 with
Linux kernel 2.4 showed only 4Gb there) The following RPMs we had to install
before XEN:
	bridge-utils, 
	python, 
	python-devel, 
	curl-devel, 
	openssl-devel
Also we installed Twisted-1.3.0. (though not succeded with the newer version
2.0)

Next we configured, compiled and installed pristine 2.6.11 linux kernel just to
make sure it works ok with our DELL and with SLES9. It was.

Then we have the 2.6.11 patched with the XEN patch and make ARCH=xen menuconfig.
The menu shows no ways to choose SMP support and (worse) no ways to choose SCSI
low-level drivers we need!! Seems that they simply disappeared!

It took time to discover many error messages those ''make''
generated just before showing the configuration menu up. They where like these:

 HOSTCC  scripts/lxdialog/yesno.o
 HOSTLD  scripts/lxdialog/lxdialog
scripts/kconfig/mconf arch/xen/Kconfig
/boot/config-2.6.5-7.191-bigsmp:26: trying to assign nonexistent symbol EVLOG
/boot/config-2.6.5-7.191-bigsmp:27: trying to assign nonexistent symbol
EVLOG_FWPRINTK
/boot/config-2.6.5-7.191-bigsmp:35: trying to assign nonexistent symbol CKRM

and so on.

Fortunately different (but why so?) result where archived on the PC where we
managed to compile the kernel for del6800 and its modules ok.

When at last the newlybuilt xen0 kernel was booted at del6800 it shows only one
CPU in /proc/cpuinfo and only 1.5Gb of memory in the ''free''
output. And the xen0 kernel was unable to do networking. Again XEN was confused
by the "extra" network card. Networking was impossible because no
default gateway was given to the interface. Though it could be fixed manually.

Later we have found from the XEN FAQ that XEN shows no "extra"
processors but does use it running virtual machines one-by-one.

As a result we decided to postpone our playing with XEN (for some months I
think) until the project gets mature. The reasons are the following:
 1) The documentation lacks of proper installation instructions
 2) The documentation lacks of list of supported OSes and hardware
configurations
 3) The documentation lacks of clear list of desired BUT STILL UNSUPPORTED
features.
 4) The documentation lacks of clear list of desired BUT STILL UNSUPPORTED
configurations.
 5) Limited support of SMP
 6) Limited support of large memory machines and x86/64
 7) Strange behaviour when more than one network card

Conclusion: Today XEN can not provide a platform for a few ORACLE servers.
Mainly because they all must use more resources than a WEB server.

---
Sergey Lukashevich




_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Mon, 5 Sep 2005, Sergey Lukashevich wrote:
> Conclusion: Today XEN can not provide a platform for a few ORACLE 
> servers. Mainly because they all must use more resources than a WEB 
> server.
Aww, c''mon, that''s not fair! Xen 3.0/"Unstable"
should take care of most
of your functionality complaints; it supports SMP and > 4gb on X86-64. I 
have a hunch that the network error is more a configuration issue than an 
actual Xen problem.

Just out of curiosity, did you try asking the list about any of your 
issues, before just saying "Ahh! It doesn''t work?" Looking
back in the
archives, I''m not seeing any previous posts from you.. the people here 
really are rather helpful.  :)

In any case, if I were in your shoes, I''d give Xen another shot once
3.0
is officially released - it should support everything you need. If you run 
into any stumbling blocks, just ask!

------------------------------------------------------------------------
| nate carlson | natecars@natecarlson.com | http://www.natecarlson.com |
|       depriving some poor village of its idiot since 1981            |
------------------------------------------------------------------------

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> -----Original Message-----
> From: Nate Carlson
> Sent: Tuesday, 6 September 2005 2:52 p.m.
> 
> On Mon, 5 Sep 2005, Sergey Lukashevich wrote:
> > Conclusion: Today XEN can not provide a platform for a few ORACLE 
> > servers. Mainly because they all must use more resources than a WEB 
> > server.
> 
> Aww, c''mon, that''s not fair! Xen 3.0/"Unstable"
should take care of most
> of your functionality complaints; it supports SMP and > 4gb on X86-64. I
> have a hunch that the network error is more a configuration issue than an 
> actual Xen problem.
Hopefully! And in the meantime if Sergey wants to run multiple Oracle
databases in separate virtual environments he could always use either
Solaris 10 x86 containers or the Linux-Vserver patch (www.linux-vserver.org)
or (some really expensive) Vmware GSX server product. All of those handle
SMP and large amounts of memory, I think.

The only problem (if it is a problem) is that AFAIK all of these approaches
are uncertified Oracle configurations and anyone using them will be running
a database with no Oracle Support. Now this may or may not be a bad thing!

SuSE SLES 10 (due for release Q1 2006?) should have Xen fully integrated
into it. It would be nice to assume that Oracle will eventually support SuSE
SLES 10 customers running Oracle *inside* Xen on SLES 10, but who knows? I
don''t believe Oracle currently supports running Databases inside
Solaris 10
containers (ok those are not virtual machines, but it does provide for handy
environment seperation...)

It will be very cool to one day be able to consolidate all my servers into
"one box" (including the Oracle database server) but it may well be
next
year (or later!) before I''d be able to do this and still have a
"certified"
configuration that meant I could log support calls with Oracle! Of course if
Oracle had less bugs in it, who would need support????? :) /wink

Cheers
Mike


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users