thr3ads.net - libvirt users - Re: [libvirt-users] Oracle RAC in libvirt+KVM environment [Aug 2013]

If this information is useful, please help other people find it:
Share via:

Timon Wang

2013-Aug-20 06:00 UTC

Re: [libvirt-users] Oracle RAC in libvirt+KVM environment

My domain xml is like this:

<domain type='kvm'
xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>2008-2</name>
  <uuid>6325d8a5-468d-42e9-b5cb-9a04f5f34e80</uuid>
  <memory unit='KiB'>524288</memory>
  <currentMemory unit='KiB'>524288</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <os>
    <type arch='x86_64'
machine='pc-i440fx-1.4'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='localtime'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='floppy'>
      <driver name='qemu' type='raw'
cache='none'/>
      <target dev='fda' bus='fdc'/>
      <readonly/>
      <address type='drive' controller='0' bus='0'
target='0' unit='0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'
cache='none'/>
      <source file='/home/images/win2008_2_sys'/>
      <target dev='hda' bus='ide'/>
      <boot order='3'/>
      <address type='drive' controller='0' bus='0'
target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/isos/windows2008_64r2.iso'/>
      <target dev='sdc' bus='ide'/>
      <readonly/>
      <boot order='1'/>
      <address type='drive' controller='0' bus='1'
target='0' unit='0'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw'/>
      <source dev='/dev/fedora/q_disk'/>
      <target dev='sda' bus='virtio'/>
      <shareable/>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x0a'
function='0x0'/>
    </disk>
    <controller type='fdc' index='0'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x06'
function='0x0'/>
    </controller>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x2'/>
    </controller>
    <controller type='pci' index='0'
model='pci-root'/>
    <controller type='scsi' index='0'
model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x07'
function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:71:20:ae'/>
      <source bridge='br0'/>
      <target dev='vport2'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x03'
function='0x0'/>
    </interface>
    <interface type='network'>
      <mac address='52:54:00:12:a0:fd'/>
      <source network='default'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x04'
function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <graphics type='spice' autoport='yes'
listen='0.0.0.0'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <sound model='ac97'>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x05'
function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='32768'
heads='2'/>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x02'
function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x08'
function='0x0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-rtc-td-hack'/>
  </qemu:commandline>
</domain>



On 8/19/13, Paolo Bonzini <pbonzini@redhat.com>
wrote:> Il 15/08/2013 12:01, Timon Wang ha scritto:
>> Thanks.
>>
>> I have read the link you provide, there is another link which tells me
>> to pass a NPIV discovery lun as a disk, this is seen as a local direct
>> access disk in windows. RAC and Failure Cluster both consider this
>> pass through disk as local disk, not a share disk, and the setup
>> process failed.
>>
>> Hyper-v provides a virtual Fiber Channel implementation, so I
>> wondering if kvm has the same solution like it.
>
> Can you include the XML file you are using for the domain?
>
> Paolo
>
>

-- 
Focus on: Server Vitualization, Network security,Scanner,NodeJS,JAVA,WWW
Blog: http://www.nohouse.net

Timon Wang

2013-Aug-20 06:05 UTC

head link

Re: [libvirt-users] Oracle RAC in libvirt+KVM environment

Right now, I found that Windows Failover Cluster needs SCSI-3
Persistent Reservation, I don't know where if virtio-scsi supports
this, according to http://www.ovirt.org/Features/Virtio-SCSI I found
this:

limited flexibility: virtio-blk does not support all possible storage
scenarios. For example, it does not allow SCSI passthrough or
persistent reservations. In principle, virtio-scsi provides anything
that the underlying SCSI target (be it physical storage, iSCSI or the
in-kernel target) supports.

virtio-blk does not support persistent reservations, but virtio-scsi
may surpport.

Another web page which archive mail lists
http://www.spinics.net/lists/target-devel/msg01813.html, which request
virtio-scsi to implement SPC-3 (persistent reservation) feature, but I
can't found any more information about this.

On Tue, Aug 20, 2013 at 2:00 PM, Timon Wang <timonwst@gmail.com>
wrote:> My domain xml is like this:
>
> <domain type='kvm'
xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
>   <name>2008-2</name>
>   <uuid>6325d8a5-468d-42e9-b5cb-9a04f5f34e80</uuid>
>   <memory unit='KiB'>524288</memory>
>   <currentMemory unit='KiB'>524288</currentMemory>
>   <vcpu placement='static'>2</vcpu>
>   <os>
>     <type arch='x86_64'
machine='pc-i440fx-1.4'>hvm</type>
>   </os>
>   <features>
>     <acpi/>
>     <apic/>
>     <pae/>
>   </features>
>   <clock offset='localtime'/>
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>destroy</on_crash>
>   <devices>
>     <emulator>/usr/bin/qemu-kvm</emulator>
>     <disk type='file' device='floppy'>
>       <driver name='qemu' type='raw'
cache='none'/>
>       <target dev='fda' bus='fdc'/>
>       <readonly/>
>       <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>     </disk>
>     <disk type='file' device='disk'>
>       <driver name='qemu' type='raw'
cache='none'/>
>       <source file='/home/images/win2008_2_sys'/>
>       <target dev='hda' bus='ide'/>
>       <boot order='3'/>
>       <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>     </disk>
>     <disk type='file' device='cdrom'>
>       <driver name='qemu' type='raw'/>
>       <source file='/home/isos/windows2008_64r2.iso'/>
>       <target dev='sdc' bus='ide'/>
>       <readonly/>
>       <boot order='1'/>
>       <address type='drive' controller='0'
bus='1' target='0' unit='0'/>
>     </disk>
>     <disk type='block' device='disk'>
>       <driver name='qemu' type='raw'/>
>       <source dev='/dev/fedora/q_disk'/>
>       <target dev='sda' bus='virtio'/>
>       <shareable/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x0a'
> function='0x0'/>
>     </disk>
>     <controller type='fdc' index='0'/>
>     <controller type='ide' index='0'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x01'
> function='0x1'/>
>     </controller>
>     <controller type='virtio-serial' index='0'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x06'
> function='0x0'/>
>     </controller>
>     <controller type='usb' index='0'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x01'
> function='0x2'/>
>     </controller>
>     <controller type='pci' index='0'
model='pci-root'/>
>     <controller type='scsi' index='0'
model='virtio-scsi'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x07'
> function='0x0'/>
>     </controller>
>     <interface type='bridge'>
>       <mac address='52:54:00:71:20:ae'/>
>       <source bridge='br0'/>
>       <target dev='vport2'/>
>       <model type='rtl8139'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x03'
> function='0x0'/>
>     </interface>
>     <interface type='network'>
>       <mac address='52:54:00:12:a0:fd'/>
>       <source network='default'/>
>       <model type='rtl8139'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x04'
> function='0x0'/>
>     </interface>
>     <serial type='pty'>
>       <target port='0'/>
>     </serial>
>     <console type='pty'>
>       <target type='serial' port='0'/>
>     </console>
>     <input type='tablet' bus='usb'/>
>     <input type='mouse' bus='ps2'/>
>     <graphics type='spice' autoport='yes'
listen='0.0.0.0'>
>       <listen type='address' address='0.0.0.0'/>
>     </graphics>
>     <sound model='ac97'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x05'
> function='0x0'/>
>     </sound>
>     <video>
>       <model type='qxl' ram='65536' vram='32768'
heads='2'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
>     <memballoon model='virtio'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x08'
> function='0x0'/>
>     </memballoon>
>   </devices>
>   <qemu:commandline>
>     <qemu:arg value='-rtc-td-hack'/>
>   </qemu:commandline>
> </domain>
>
>
>
> On 8/19/13, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> Il 15/08/2013 12:01, Timon Wang ha scritto:
>>> Thanks.
>>>
>>> I have read the link you provide, there is another link which tells
me
>>> to pass a NPIV discovery lun as a disk, this is seen as a local
direct
>>> access disk in windows. RAC and Failure Cluster both consider this
>>> pass through disk as local disk, not a share disk, and the setup
>>> process failed.
>>>
>>> Hyper-v provides a virtual Fiber Channel implementation, so I
>>> wondering if kvm has the same solution like it.
>>
>> Can you include the XML file you are using for the domain?
>>
>> Paolo
>>
>>
>
>
> --
> Focus on: Server Vitualization, Network security,Scanner,NodeJS,JAVA,WWW
> Blog: http://www.nohouse.net


-- 
Focus on: Server Vitualization, Network security,Scanner,NodeJS,JAVA,WWW
Blog: http://www.nohouse.net

Paolo Bonzini

2013-Aug-20 08:33 UTC

head link

Re: [libvirt-users] Oracle RAC in libvirt+KVM environment

Il 20/08/2013 08:00, Timon Wang ha scritto:>     <disk type='file' device='disk'>
>       <driver name='qemu' type='raw'
cache='none'/>
>       <source file='/home/images/win2008_2_sys'/>
>       <target dev='hda' bus='ide'/>
>       <boot order='3'/>
>       <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>     </disk>
>     <disk type='file' device='cdrom'>
>       <driver name='qemu' type='raw'/>
>       <source file='/home/isos/windows2008_64r2.iso'/>
>       <target dev='sdc' bus='ide'/>
>       <readonly/>
>       <boot order='1'/>
>       <address type='drive' controller='0'
bus='1' target='0' unit='0'/>
>     </disk>
>     <disk type='block' device='disk'>
I'm not sure this will be enough, but if you want passthrough to the
host device you should use device='lun' here.  However, you still would
not be able to issue SCSI reservations unless you run QEMU with the
CAP_SYS_RAWIO capability (using "<disk ...
rawio='yes'>").

Most important, it still would be unsafe to do this if the same device
is passed to multiple virtual machines on the same host.  You need to
have NPIV and create separate virtual HBAs.  Then each virtual machine
should get a separate virtual HBA.  Otherwise, persistent reservations
are not attached to a particular virtual machine, but generically to the
host.
>       <driver name='qemu' type='raw'/>
>       <source dev='/dev/fedora/q_disk'/>
>       <target dev='sda' bus='virtio'/>
You are not exposing a virtio-scsi disk here.  You are exposing a
virtio-blk disk.  You can see this from the type='pci' address that
libvirt gave to the disk.

If you use bus='scsi', you will see that libvirt will use
type='drive'
for the address.
>     <controller type='scsi' index='0'
model='virtio-scsi'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x07'
> function='0x0'/>
>     </controller>
This is okay.
>   <qemu:commandline>
>     <qemu:arg value='-rtc-td-hack'/>
>   </qemu:commandline>
FWIW, this can be replaced with

  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
  </clock>

(you already have the <clock> element, but no <timer> element
inside).

Paolo
> </domain>
> 
> 
> 
> On 8/19/13, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> Il 15/08/2013 12:01, Timon Wang ha scritto:
>>> Thanks.
>>>
>>> I have read the link you provide, there is another link which tells
me
>>> to pass a NPIV discovery lun as a disk, this is seen as a local
direct
>>> access disk in windows. RAC and Failure Cluster both consider this
>>> pass through disk as local disk, not a share disk, and the setup
>>> process failed.
>>>
>>> Hyper-v provides a virtual Fiber Channel implementation, so I
>>> wondering if kvm has the same solution like it.
>>
>> Can you include the XML file you are using for the domain?
>>
>> Paolo
>>
>>
> 
>

Timon Wang

2013-Aug-20 09:59 UTC

head link

Re: [libvirt-users] Oracle RAC in libvirt+KVM environment

On Tue, Aug 20, 2013 at 4:33 PM, Paolo Bonzini <pbonzini@redhat.com>
wrote:> Il 20/08/2013 08:00, Timon Wang ha scritto:
>>     <disk type='file' device='disk'>
>>       <driver name='qemu' type='raw'
cache='none'/>
>>       <source file='/home/images/win2008_2_sys'/>
>>       <target dev='hda' bus='ide'/>
>>       <boot order='3'/>
>>       <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>>     </disk>
>>     <disk type='file' device='cdrom'>
>>       <driver name='qemu' type='raw'/>
>>       <source file='/home/isos/windows2008_64r2.iso'/>
>>       <target dev='sdc' bus='ide'/>
>>       <readonly/>
>>       <boot order='1'/>
>>       <address type='drive' controller='0'
bus='1' target='0' unit='0'/>
>>     </disk>
>>     <disk type='block' device='disk'>
>
> I'm not sure this will be enough, but if you want passthrough to the
> host device you should use device='lun' here.  However, you still
would
> not be able to issue SCSI reservations unless you run QEMU with the
> CAP_SYS_RAWIO capability (using "<disk ...
rawio='yes'>").
>
After change the libvirt xml like this:
<disk type='block' device='lun' rawio='yes'>
      <driver name='qemu' type='raw'
cache='none'/>
      <source dev='/dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk'/>
      <target dev='sda' bus='scsi'/>
      <shareable/>
      <address type='drive' controller='0' bus='0'
target='0' unit='0'/>
    </disk>
I got these errors:
char device redirected to /dev/pts/1 (label charserial0)
qemu-system-x86_64: -device
scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0:
scsi-block: INQUIRY failed
qemu-system-x86_64: -device
scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0:
Device 'scsi-block' could not be initialized

> Most important, it still would be unsafe to do this if the same device
> is passed to multiple virtual machines on the same host.  You need to
> have NPIV and create separate virtual HBAs.  Then each virtual machine
> should get a separate virtual HBA.  Otherwise, persistent reservations
> are not attached to a particular virtual machine, but generically to the
> host.
How to use NPIV virtual HBAs with libvirt xml configurations? I can
define nodedev, but have no idea about how to pass the nodedev to the
vm.
>
>>       <driver name='qemu' type='raw'/>
>>       <source dev='/dev/fedora/q_disk'/>
>>       <target dev='sda' bus='virtio'/>
>
> You are not exposing a virtio-scsi disk here.  You are exposing a
> virtio-blk disk.  You can see this from the type='pci' address that
> libvirt gave to the disk.
>
> If you use bus='scsi', you will see that libvirt will use
type='drive'
> for the address.
>
>>     <controller type='scsi' index='0'
model='virtio-scsi'>
>>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x07'
>> function='0x0'/>
>>     </controller>
>
> This is okay.
>
>>   <qemu:commandline>
>>     <qemu:arg value='-rtc-td-hack'/>
>>   </qemu:commandline>
>
> FWIW, this can be replaced with
>
>   <clock offset='localtime'>
>     <timer name='rtc' tickpolicy='catchup'/>
>   </clock>
>
> (you already have the <clock> element, but no <timer> element
inside).
Thanks for this tip.
>
> Paolo
>
>> </domain>
>>
>>
>>
>> On 8/19/13, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>> Il 15/08/2013 12:01, Timon Wang ha scritto:
>>>> Thanks.
>>>>
>>>> I have read the link you provide, there is another link which
tells me
>>>> to pass a NPIV discovery lun as a disk, this is seen as a local
direct
>>>> access disk in windows. RAC and Failure Cluster both consider
this
>>>> pass through disk as local disk, not a share disk, and the
setup
>>>> process failed.
>>>>
>>>> Hyper-v provides a virtual Fiber Channel implementation, so I
>>>> wondering if kvm has the same solution like it.
>>>
>>> Can you include the XML file you are using for the domain?
>>>
>>> Paolo
>>>
>>>
>>
>>
>
-- 
Focus on: Server Vitualization, Network security,Scanner,NodeJS,JAVA,WWW
Blog: http://www.nohouse.net

Apparently Analagous Threads

Search for more possibly parallel threads

libvirt users - Aug 2013 - Re: Oracle RAC in libvirt+KVM environment

Re: [libvirt-users] Oracle RAC in libvirt+KVM environment

Re: [libvirt-users] Oracle RAC in libvirt+KVM environment

Re: [libvirt-users] Oracle RAC in libvirt+KVM environment

Re: [libvirt-users] Oracle RAC in libvirt+KVM environment

Apparently Analagous Threads