Hi, I''m trying to install Oracle 10g RAC using xen virtual servers and a Network Appliance F880 filer as the shared storage. I''m using Xen 3.0.2, kernel version 2.6.16u, and my VBD is created from a Redhat 4u3 distribution. I would like to use raw devices (disk LUNs on the NetApp filer) in conjunction with Oracle ASM to store the oracle database files. This configuration works well on standard linux servers, but I''m unable to get it working on Xen because I can''t get the iSCSI initiator to work in the guest domains. I attempted to compile the core-iscsi initiator against the 2.6.16U kernel, and it seems to compile ok, but I get "unknown symbol in module" error when I attempt to load via insmod in the guest domain. I tried open-scsi as well, but also receive errors. So my question is - can you launch the iscsi-initiator and deamons within the guest domain to establish iscsi connetivity to the iSCSI target (netapp filer), or does iscsi have to be started on the xen host (dom0). If iSCSI can be launched from the guest, which initiator should I use with the 2.6.16U kernel, and are there any tips on how to compile this initiator to work with Xen ? Thanks in advance. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thu, Oct 19, 2006 at 04:01:26PM -0700, Mike Leatherman wrote:> > Hi, > I''m trying to install Oracle 10g RAC using xen virtual servers and a > Network Appliance F880 filer as the shared storage. I''m using Xen 3.0.2, > kernel version 2.6.16u, and my VBD is created from a Redhat 4u3 > distribution. I would like to use raw devices (disk LUNs on the NetApp > filer) in conjunction with Oracle ASM to store the oracle database files. > This configuration works well on standard linux servers, but I''m unable to > get it working on Xen because I can''t get the iSCSI initiator to work in > the guest domains. > > I attempted to compile the core-iscsi initiator against the 2.6.16U > kernel, and it seems to compile ok, but I get "unknown symbol in module" > error when I attempt to load via insmod in the guest domain. I tried > open-scsi as well, but also receive errors. > > So my question is - > > can you launch the iscsi-initiator and deamons within the guest domain to > establish iscsi connetivity to the iSCSI target (netapp filer), or does > iscsi have to be started on the xen host (dom0). If iSCSI can be launched > from the guest, which initiator should I use with the 2.6.16U kernel, and > are there any tips on how to compile this initiator to work with Xen ? > > Thanks in advance. >Yes, you can use iscsi-initiator in domU. With 2.6.16 (domU) kernel you should use open-iscsi initiator, which does not (yet) come with RHEL. It will be in RHEL5 (and in FC6). You can (should) download the latest SVN version of open-iscsi, as it contains some bugfixes over the latest released version (2.0). Compile it and remember to use the kernel modules from the open-iscsi, not the ones in the kernel. -- Pasi ^ . . Linux / - \ Choice.of.the .Next.Generation. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Thanks for the response. I''m still encountering issues with iSCSI - maybe an extra hint or two will get me over the hump. I''m trying to compile open-scsi in my domU virtual domain - Here are the versions - open-iscsi-1.0-485 2.6.16-xenU Nothing was done on the dom0 kernel or in the dom0 domain. All actions were performed in domU. To configure Xen domU kernel, I performed the following: nfs mounted the Xen source files (3.0.2-2) in the /Xen mount point, modified the Makefile with KERNELS ?= linux-2.6-xenU I then ran "make world", which created the linux-2.6.16-xenU directory. I modifed the linux-2.6.16-xenU/.config file with my required changes, inluding enabling the iSCSI over TCP (CONFIG_ISCSI_TCP=m). After these changes, I ran make, make install and created the vmlinuz-2.6.16-xenUfile. I copied this to my xen host (domain), and booted a new domU with this newly created kernel file. To attempt to create the open-isci module, I did the following: Once my domU server came online, I remounted the Xen source files on the domU vm, then changed directories to /tmp/open-iscsi-1.0-485 I ran: make KSRC=/XEN/linux-2.6.16-xenU it appears to compile successfully - here''s what I see make -C usr make[1]: Entering directory `/tmp/open-iscsi-1.0-485/usr'' make[1]: Nothing to be done for `all''. make[1]: Leaving directory `/tmp/open-iscsi-1.0-485/usr'' make -C kernel make[1]: Entering directory `/tmp/open-iscsi-1.0-485/kernel'' kernel check... OTHER KERNEL DETECTED make -C /XEN/linux-2.6.16-xenU M=`pwd` V=0 modules make[2]: Entering directory `/XEN/linux-2.6.16-xenU'' Building modules, stage 2. MODPOST make[2]: Leaving directory `/XEN/linux-2.6.16-xenU'' make[1]: Leaving directory `/tmp/open-iscsi-1.0-485/kernel'' Compilation complete Output file ----------------------------------- ---------------- Built iSCSI Open Interface module: kernel/scsi_transport_iscsi.ko Built iSCSI over TCP kernel module: kernel/iscsi_tcp.ko Built iSCSI daemon: usr/iscsid Built management application: usr/iscsiadm Read README file for detailed information. However, when I attempt to load the iscsi_tcp.ko or scsi_transport_iscsi.ko modules in domU, I get errors - FATAL: Error inserting scsi_transport_iscsi (/lib/modules/2.6.16-xenU/kernel/drivers/scsi/scsi_transport_iscsi.ko): Unknown symbol i n module, or unknown parameter (see dmesg) dmesg shows - scsi_transport_iscsi: Unknown symbol scsi_remove_host scsi_transport_iscsi: Unknown symbol scsi_host_get scsi_transport_iscsi: Unknown symbol scsi_host_put scsi_transport_iscsi: Unknown symbol scsi_add_host scsi_transport_iscsi: Unknown symbol scsi_is_host_device scsi_transport_iscsi: Unknown symbol scsi_host_alloc Did I miss a step, or run a step incorrectly. I realize this email is very long, but I wanted to include as much detail as possible. Thanks in advance. - Michael On 10/19/06, Mike Leatherman <michael.leatherman@gmail.com> wrote:> > > Hi, > I''m trying to install Oracle 10g RAC using xen virtual servers and a > Network Appliance F880 filer as the shared storage. I''m using Xen 3.0.2, > kernel version 2.6.16u, and my VBD is created from a Redhat 4u3 > distribution. I would like to use raw devices (disk LUNs on the NetApp > filer) in conjunction with Oracle ASM to store the oracle database files. > This configuration works well on standard linux servers, but I''m unable to > get it working on Xen because I can''t get the iSCSI initiator to work in > the guest domains. > > I attempted to compile the core-iscsi initiator against the 2.6.16U > kernel, and it seems to compile ok, but I get "unknown symbol in module" > error when I attempt to load via insmod in the guest domain. I tried > open-scsi as well, but also receive errors. > > So my question is - > > can you launch the iscsi-initiator and deamons within the guest domain to > establish iscsi connetivity to the iSCSI target (netapp filer), or does > iscsi have to be started on the xen host (dom0). If iSCSI can be launched > from the guest, which initiator should I use with the 2.6.16U kernel, and > are there any tips on how to compile this initiator to work with Xen ? > > Thanks in advance. > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Tue, Oct 24, 2006 at 03:33:38PM -0700, Michael Leatherman wrote:> Thanks for the response. I''m still encountering issues with iSCSI - maybe > an extra hint or two will get me over the hump. > > I''m trying to compile open-scsi in my domU virtual domain - > Here are the versions - > > open-iscsi-1.0-485 > 2.6.16-xenU >Don''t use 1.0-485, use latest SVN version (2.0-something). And be sure to use the compiled modules from open-iscsi tree.. not the ones already in kernel. So after you install the modules to /lib/modules/<version>, check that you don''t have the modules in two places.. remove the ones that come with the kernel and run "depmod -a". If I remember correctly, open-iscsi does not overwrite the modules already in the kernel, but installs copies to another location which can cause problems (modprobe loads the original modules, not the newly compiled ones). -- Pasi> Nothing was done on the dom0 kernel or in the dom0 domain. All actions were > performed in domU. > > > To configure Xen domU kernel, I performed the following: > nfs mounted the Xen source files (3.0.2-2) in the /Xen mount point, modified > the Makefile with > KERNELS ?= linux-2.6-xenU > > I then ran "make world", which created the linux-2.6.16-xenU directory. I > modifed the linux-2.6.16-xenU/.config file with my required changes, > inluding enabling the iSCSI over TCP (CONFIG_ISCSI_TCP=m). After these > changes, I ran make, make install and created the > vmlinuz-2.6.16-xenUfile. I copied this to my xen host (domain), and > booted a new domU with > this newly created kernel file. > > To attempt to create the open-isci module, I did the following: > Once my domU server came online, I remounted the Xen source files on the > domU vm, then changed directories to /tmp/open-iscsi-1.0-485 > > I ran: > > make KSRC=/XEN/linux-2.6.16-xenU > > it appears to compile successfully - here''s what I see > > make -C usr > make[1]: Entering directory `/tmp/open-iscsi-1.0-485/usr'' > make[1]: Nothing to be done for `all''. > make[1]: Leaving directory `/tmp/open-iscsi-1.0-485/usr'' > make -C kernel > make[1]: Entering directory `/tmp/open-iscsi-1.0-485/kernel'' > kernel check... OTHER KERNEL DETECTED > make -C /XEN/linux-2.6.16-xenU M=`pwd` V=0 modules > make[2]: Entering directory `/XEN/linux-2.6.16-xenU'' > Building modules, stage 2. > MODPOST > make[2]: Leaving directory `/XEN/linux-2.6.16-xenU'' > make[1]: Leaving directory `/tmp/open-iscsi-1.0-485/kernel'' > > Compilation complete Output file > ----------------------------------- ---------------- > Built iSCSI Open Interface module: kernel/scsi_transport_iscsi.ko > Built iSCSI over TCP kernel module: kernel/iscsi_tcp.ko > Built iSCSI daemon: usr/iscsid > Built management application: usr/iscsiadm > > Read README file for detailed information. > > However, when I attempt to load the iscsi_tcp.ko or scsi_transport_iscsi.ko > modules in domU, I get errors - > > FATAL: Error inserting scsi_transport_iscsi > (/lib/modules/2.6.16-xenU/kernel/drivers/scsi/scsi_transport_iscsi.ko): > Unknown symbol i > n module, or unknown parameter (see dmesg) > > > dmesg shows - > > scsi_transport_iscsi: Unknown symbol scsi_remove_host > scsi_transport_iscsi: Unknown symbol scsi_host_get > scsi_transport_iscsi: Unknown symbol scsi_host_put > scsi_transport_iscsi: Unknown symbol scsi_add_host > scsi_transport_iscsi: Unknown symbol scsi_is_host_device > scsi_transport_iscsi: Unknown symbol scsi_host_alloc > > > Did I miss a step, or run a step incorrectly. I realize this email is very > long, but I wanted to include as much detail as possible. Thanks in > advance. > > - Michael > > > > > On 10/19/06, Mike Leatherman <michael.leatherman@gmail.com> wrote: > > > > > >Hi, > >I''m trying to install Oracle 10g RAC using xen virtual servers and a > >Network Appliance F880 filer as the shared storage. I''m using Xen 3.0.2, > >kernel version 2.6.16u, and my VBD is created from a Redhat 4u3 > >distribution. I would like to use raw devices (disk LUNs on the NetApp > >filer) in conjunction with Oracle ASM to store the oracle database files. > >This configuration works well on standard linux servers, but I''m unable to > >get it working on Xen because I can''t get the iSCSI initiator to work in > >the guest domains. > > > >I attempted to compile the core-iscsi initiator against the 2.6.16U > >kernel, and it seems to compile ok, but I get "unknown symbol in module" > >error when I attempt to load via insmod in the guest domain. I tried > >open-scsi as well, but also receive errors. > > > >So my question is - > > > >can you launch the iscsi-initiator and deamons within the guest domain to > >establish iscsi connetivity to the iSCSI target (netapp filer), or does > >iscsi have to be started on the xen host (dom0). If iSCSI can be launched > >from the guest, which initiator should I use with the 2.6.16U kernel, and > >are there any tips on how to compile this initiator to work with Xen ? > > > >Thanks in advance. > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users