Hi everyone, I am very new to Xen but will migrate 3 SAP ERP servers plus 1 SolMan into a Xen environment next year, probably running Novell''s SLES (as we use SLES right now). The machines will be app servers only, no DB - the DB is "outsourced" to a common Oracle RAC this year (maintained by a colleague). So I will get 2 boxes and now fiddle around with possible configuration of storage. Due to SAP notes the SAP binaries must be stored on the RAC and made accessible from there, for example via NFS or SAN. In our company, we are used to SANing ;-), so my boxes will have some FC boards to connect to the RAC''s 2 FC switches. Am I right that I can access the block device from SAN (will be one big thing with OCFS2) inside dom0 and pass them via phy: to the domUs? Next: the OS of the domU. I plan to have some local disk space (about 300 GB per box). I think on something like a drbd drive (RAID-1) that is separated into chunks for each domU. The dom0 forwards the respective part to the domU. The domUs use this disk space for their operating system. Somewhere (like in /bigthing/) the big shared storage is mounted and the necessary symlinks for SAP are set. Can this become a working layout? Regards, Werner _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
2009/6/11 Werner Flamme <werner.flamme@ufz.de>:> Hi everyone, > > I am very new to Xen but will migrate 3 SAP ERP servers plus 1 SolMan > into a Xen environment next year, probably running Novell''s SLES (as we > use SLES right now). >Wow, you''re the first person I know that will be running SAP on top of Xen. Let us know how it goes. AFAIK since SAP license cost is much more expensive compared to hardware cost, hardware savings (thru virtualization, for example) will be insignificant compared to license cost, so we focus saving efforts on license.> The machines will be app servers only, no DB - the DB is "outsourced" to > a common Oracle RAC this year (maintained by a colleague). > > So I will get 2 boxes and now fiddle around with possible configuration > of storage. Due to SAP notes the SAP binaries must be stored on the RAC > and made accessible from there, for example via NFS or SAN. In our > company, we are used to SANing ;-), so my boxes will have some FC boards > to connect to the RAC''s 2 FC switches. Am I right that I can access the > block device from SAN (will be one big thing with OCFS2) inside dom0 and > pass them via phy: to the domUs?And use OCFS2 on domU? should be possible. Although it''d be simpler if you import it via nfs on domU.> > Next: the OS of the domU. I plan to have some local disk space (about > 300 GB per box). I think on something like a drbd drive (RAID-1) that is > separated into chunks for each domU. The dom0 forwards the respective > part to the domU. The domUs use this disk space for their operating > system. Somewhere (like in /bigthing/) the big shared storage is mounted > and the necessary symlinks for SAP are set. >I still have doubts against drbd, but other user has reported it gives acceptable performance, so you may want to search the list archive for his particular setup. If it were me I''d simply put domU''s storage on SAN if I need migration (you already have SAN anyway, right?) or on mirrored local disk (for maximum I/O throughput) -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha [11.06.2009 15:55]:> 2009/6/11 Werner Flamme <werner.flamme@ufz.de>: >> Hi everyone, >> >> I am very new to Xen but will migrate 3 SAP ERP servers plus 1 SolMan >> into a Xen environment next year, probably running Novell''s SLES (as we >> use SLES right now). >> > > Wow, you''re the first person I know that will be running SAP on top of > Xen. Let us know how it goes.*gulp* :-) For us, it is "power consumption". 3 Boxes plus 1 fat RAID use more power from the wall outlets that two without RAID ;-)> AFAIK since SAP license cost is much more expensive compared to > hardware cost, hardware savings (thru virtualization, for example) > will be insignificant compared to license cost, so we focus saving > efforts on license.Our company decided to give SAP access to everyone who has a contract for at least one year. So we have about 700 users in average, since most scientists stay for 3 to 7 years...>> The machines will be app servers only, no DB - the DB is "outsourced" to >> a common Oracle RAC this year (maintained by a colleague). >> >> So I will get 2 boxes and now fiddle around with possible configuration >> of storage. Due to SAP notes the SAP binaries must be stored on the RAC >> and made accessible from there, for example via NFS or SAN. In our >> company, we are used to SANing ;-), so my boxes will have some FC boards >> to connect to the RAC''s 2 FC switches. Am I right that I can access the >> block device from SAN (will be one big thing with OCFS2) inside dom0 and >> pass them via phy: to the domUs? > > And use OCFS2 on domU? should be possible. Although it''d be simpler if > you import it via nfs on domU.Yes, use OCFS2 on domU. I hate NFS ;-) Used it once and it was terribly slow. Maybe it has changed in the meantime. At least it is a fallback then.>> Next: the OS of the domU. I plan to have some local disk space (about >> 300 GB per box). I think on something like a drbd drive (RAID-1) that is >> separated into chunks for each domU. The dom0 forwards the respective >> part to the domU. The domUs use this disk space for their operating >> system. Somewhere (like in /bigthing/) the big shared storage is mounted >> and the necessary symlinks for SAP are set. >> > > I still have doubts against drbd, but other user has reported it gives > acceptable performance, so you may want to search the list archive for > his particular setup.Ah, if it is only for performance... ;-) the data on this drbd disk is used for OS only. So it will mainly be used for startup - and if it adds some 10 seconds, who cares? :-) I prefer stability over performance, and if drbd is reliable it fulfills its purpose OK.> If it were me I''d simply put domU''s storage on SAN if I need migration > (you already have SAN anyway, right?) or on mirrored local disk (for > maximum I/O throughput)Yes, I have SAN - an own SAN for SAP now, not shared with the RAC. For DB migration I have to copy all data files from my storage to RAC''s storage. Except for the OS everything is on the SAN now. The drbd part is for the OS only, to boot from. Hm, I''m still thinking too much hardware-bound ;-) Of course I can have some files containing the OS on the SAN - just that SAP warns doing so. In SAPnote 962334, you find: ---snip--- SAP strongly advises to place Xen virtual machines in raw devices or partitions. Do not use a file as virtual device for a virtual machine. If you run an SAP database instance on such a file based virtual machine the I/O performance will drop dramatically compared to a raw setup. Nevertheless, you may also use logical volumes instead of raw devices. We also advise you to use an external storage for the virtual machines when using a SAP database instance performance wise. For SAP application servers, local disks are sufficient. ---pins--- Since I will be dealing with SAP application servers only... :-) Thanks for your input, Fajar! Regards, Werner _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Zoran Popović [11.06.2009 16:52]:> 2009/6/11 Werner Flamme <werner.flamme@ufz.de> > >> Hi everyone, >> >> I am very new to Xen but will migrate 3 SAP ERP servers plus 1 SolMan >> into a Xen environment next year, probably running Novell''s SLES (as we >> use SLES right now). >> >> The machines will be app servers only, no DB - the DB is "outsourced" to >> a common Oracle RAC this year (maintained by a colleague). >> >> So I will get 2 boxes and now fiddle around with possible configuration >> of storage. Due to SAP notes the SAP binaries must be stored on the RAC >> and made accessible from there, for example via NFS or SAN. In our >> company, we are used to SANing ;-), so my boxes will have some FC boards >> to connect to the RAC''s 2 FC switches. Am I right that I can access the >> block device from SAN (will be one big thing with OCFS2) inside dom0 and >> pass them via phy: to the domUs? >> > > Ok, I am just doing some testing on RHEL5.3 with Xen 3.0.3 (somewhat) on > Itanium (IA64, not x64) Dom0 and HVM Windows Server 2003 HVM DomUs. > Dom0 is booting and completely using FC SAN (HBAs), DomUs also use SAN - > actually, I am using snapshots (EVA HP split mirror terminology) of the > production database which I easily present to Dom0 and use it directly as > phy drive on DomU. There I have SAP ECC6.0 EnhP3 installed together with > Oracle 10.2.0.4.6 db. I am very happy with results !! The only problem is > I/O on HVM because IA64 lacks PV support at the moment. Currently, that is > the only reason not to keep DB on HVM DomU (though there are many ways to > somehow circumvent or lessen problems, apart from getting SAP''s official > permission/acceptance for the platform. Oracle VM is Xen, should I say more > ...So I am not the first one who tries to fiddle together SAP an Xen :-) SAP does not allow Oracle VM for a productive instance, see SAPnote https://service.sap.com/sap/support/notes/1122387, near the end. CitrixXen is Xen also, but not allowed, a few lines above Oracle VM ;-). They just don''t pay SAP to test it, I think. Novell and RedHat did... We use CitrixXen in our company for a variety of things, and if it was allowed, I''d use it...> But, one thing puzzles me - why would you access SAP binaries lying on > Oracle RAC disks ? Standard failover SAP HA (High Availability scenario, > either Windows or HPUX, most likely it is same with Linux, I didn''t use or > read about that so far) is using separate cluster (shared storage) disks. > Using NFS for binaries is not an idea I would be eager to use. Also, > accessing raw OCFS2 drive wihtout Oracle OCFS software on DomU (where dialog > or centrral instances reside) makes no sense. If it is about Central > scheduling for SAP (note 895253) NFS is ok, but for SAP kernel I couldn''t > find any note claiming this - can you show me the note ?This is something I read on one of the various PDFs on <https://www.sdn.sap.com/irj/sdn/ora?rid=/webcontent/uuid/e9071e42-0d01-0010-4bae-dafd2e524b57> - I think it was the first one, "SAP NetWeaver with Oracle 10g R2 RAC (UNIX & Linux)". This document is spread by Oracle, too (not only by SAP ;-)). I have a paragraph on page 4 that reads: ---snip--- This document assumes that the planned installation uses a cluster filesystem for the voting disks, CRS repository, database files, control files, redo log files, archived redo log files and executables under $ORACLE_HOME (software installation directory for the Oracle RDBMS software), log files and executables under $CRS_HOME (software installation directory for Oracle CRS software) as well as all SAP executables in the /sapmnt directory. ---pins--- I understand this text so that /sapmnt has to be on the SAN too. What''s the big deal? I have it already running this way now with my SAP SAN ;-) BTW, we have only one instance per system, no separate dialogue instances.> Using SAN FC > usually involves availability and reliability mechanisms (multipathing, raid > levels, etc), and OCFS is justified for other purposes e.g. you want > patching kernel on one place, directory - but you have sapcpe for that, and > you loose availability - concurent maintenance (all application servers must > be put down). I would simply use Dom0''s phy (SAN) drives in the DomU''s most > usual and convenient file system (you can even create f/s in DomU). > I would be glad to help you if can or to know more about this, > ZP.Zoran, unfortunately the shared storage I will use is already formatted with OCFS2 and used by the RAC for other DBs. I am not allowed to change the partitions or filesystems in that thing, though I might be able to do so ;-). SAPnote 527843 (Oracle RAC support in the SAP environment) tells me that - when I am using Linux - I have to use "Oracle Clusterware + OCFS2 1.4.x", so this is not a point of discussion here :-( <mode="brag">Of course multipathing is (and will be) used, and the shared storage isn''t RAID-0 :-) There are two RAID boxes (12 Disks with 300 GB each) mirroring each other, to be more exact...</mode> Thanks for your input! Regards, Werner _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users