libvirt users - Aug 2013 - Re: Oracle RAC in libvirt+KVM environment

On Tue, Aug 20, 2013 at 4:33 PM, Paolo Bonzini <pbonzini@redhat.com>
wrote:
> Il 20/08/2013 08:00, Timon Wang ha scritto:
>>     <disk type='file' device='disk'>
>>       <driver name='qemu' type='raw'
cache='none'/>
>>       <source file='/home/images/win2008_2_sys'/>
>>       <target dev='hda' bus='ide'/>
>>       <boot order='3'/>
>>       <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>>     </disk>
>>     <disk type='file' device='cdrom'>
>>       <driver name='qemu' type='raw'/>
>>       <source file='/home/isos/windows2008_64r2.iso'/>
>>       <target dev='sdc' bus='ide'/>
>>       <readonly/>
>>       <boot order='1'/>
>>       <address type='drive' controller='0'
bus='1' target='0' unit='0'/>
>>     </disk>
>>     <disk type='block' device='disk'>
>
> I'm not sure this will be enough, but if you want passthrough to the
> host device you should use device='lun' here.  However, you still
would
> not be able to issue SCSI reservations unless you run QEMU with the
> CAP_SYS_RAWIO capability (using "<disk ...
rawio='yes'>").
>
After change the libvirt xml like this:
<disk type='block' device='lun' rawio='yes'>
      <driver name='qemu' type='raw'
cache='none'/>
      <source dev='/dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk'/>
      <target dev='sda' bus='scsi'/>
      <shareable/>
      <address type='drive' controller='0' bus='0'
target='0' unit='0'/>
    </disk>
I got these errors:
char device redirected to /dev/pts/1 (label charserial0)
qemu-system-x86_64: -device
scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0:
scsi-block: INQUIRY failed
qemu-system-x86_64: -device
scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0:
Device 'scsi-block' could not be initialized

> Most important, it still would be unsafe to do this if the same device
> is passed to multiple virtual machines on the same host.  You need to
> have NPIV and create separate virtual HBAs.  Then each virtual machine
> should get a separate virtual HBA.  Otherwise, persistent reservations
> are not attached to a particular virtual machine, but generically to the
> host.
How to use NPIV virtual HBAs with libvirt xml configurations? I can
define nodedev, but have no idea about how to pass the nodedev to the
vm.
>
>>       <driver name='qemu' type='raw'/>
>>       <source dev='/dev/fedora/q_disk'/>
>>       <target dev='sda' bus='virtio'/>
>
> You are not exposing a virtio-scsi disk here.  You are exposing a
> virtio-blk disk.  You can see this from the type='pci' address that
> libvirt gave to the disk.
>
> If you use bus='scsi', you will see that libvirt will use
type='drive'
> for the address.
>
>>     <controller type='scsi' index='0'
model='virtio-scsi'>
>>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x07'
>> function='0x0'/>
>>     </controller>
>
> This is okay.
>
>>   <qemu:commandline>
>>     <qemu:arg value='-rtc-td-hack'/>
>>   </qemu:commandline>
>
> FWIW, this can be replaced with
>
>   <clock offset='localtime'>
>     <timer name='rtc' tickpolicy='catchup'/>
>   </clock>
>
> (you already have the <clock> element, but no <timer> element
inside).
Thanks for this tip.
>
> Paolo
>
>> </domain>
>>
>>
>>
>> On 8/19/13, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>> Il 15/08/2013 12:01, Timon Wang ha scritto:
>>>> Thanks.
>>>>
>>>> I have read the link you provide, there is another link which
tells me
>>>> to pass a NPIV discovery lun as a disk, this is seen as a local
direct
>>>> access disk in windows. RAC and Failure Cluster both consider
this
>>>> pass through disk as local disk, not a share disk, and the
setup
>>>> process failed.
>>>>
>>>> Hyper-v provides a virtual Fiber Channel implementation, so I
>>>> wondering if kvm has the same solution like it.
>>>
>>> Can you include the XML file you are using for the domain?
>>>
>>> Paolo
>>>
>>>
>>
>>
>
-- 
Focus on: Server Vitualization, Network security,Scanner,NodeJS,JAVA,WWW
Blog: http://www.nohouse.net

Il 20/08/2013 11:59, Timon Wang ha scritto:
> On Tue, Aug 20, 2013 at 4:33 PM, Paolo Bonzini <pbonzini@redhat.com>
wrote:
>> Il 20/08/2013 08:00, Timon Wang ha scritto:
>>>     <disk type='file' device='disk'>
>>>       <driver name='qemu' type='raw'
cache='none'/>
>>>       <source file='/home/images/win2008_2_sys'/>
>>>       <target dev='hda' bus='ide'/>
>>>       <boot order='3'/>
>>>       <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>>>     </disk>
>>>     <disk type='file' device='cdrom'>
>>>       <driver name='qemu' type='raw'/>
>>>       <source
file='/home/isos/windows2008_64r2.iso'/>
>>>       <target dev='sdc' bus='ide'/>
>>>       <readonly/>
>>>       <boot order='1'/>
>>>       <address type='drive' controller='0'
bus='1' target='0' unit='0'/>
>>>     </disk>
>>>     <disk type='block' device='disk'>
>>
>> I'm not sure this will be enough, but if you want passthrough to
the
>> host device you should use device='lun' here.  However, you
still would
>> not be able to issue SCSI reservations unless you run QEMU with the
>> CAP_SYS_RAWIO capability (using "<disk ...
rawio='yes'>").
>>
> 
> After change the libvirt xml like this:
> <disk type='block' device='lun' rawio='yes'>
>       <driver name='qemu' type='raw'
cache='none'/>
>       <source
dev='/dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk'/>
>       <target dev='sda' bus='scsi'/>
>       <shareable/>
>       <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>     </disk>
> I got these errors:
> char device redirected to /dev/pts/1 (label charserial0)
> qemu-system-x86_64: -device
>
scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0:
> scsi-block: INQUIRY failed
> qemu-system-x86_64: -device
>
scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0:
> Device 'scsi-block' could not be initialized
Can you do

# ls -l /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk
# sg_inq /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk

?

Paolo

[root@localhost /]# ls -l /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk
lrwxrwxrwx. 1 root root 8 8月  20 17:38
/dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk -> ../dm-13
[root@localhost /]# sg_inq /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk
standard INQUIRY:
  PQual=0  Device_type=0  RMB=0  version=0x05  [SPC-3]
  [AERC=0]  [TrmTsk=0]  NormACA=0  HiSUP=0  Resp_data_format=0
  SCCS=1  ACC=0  TPGS=1  3PC=0  Protect=0  [BQue=0]
  EncServ=0  MultiP=0  [MChngr=0]  [ACKREQQ=0]  Addr16=0
  [RelAdr=0]  WBus16=1  Sync=1  Linked=0  [TranDis=0]  CmdQue=1
    length=36 (0x24)   Peripheral device type: disk
 Vendor identification: MacroSAN
 Product identification: LU
 Product revision level: 1.0
 Unit serial number: 0d9281ae-aea4-6da0-0000-02180142b300

This lun is from a vg build based on iscsi target.

[root@localhost /]# libvirtd --version
libvirtd (libvirt) 1.0.5
[root@localhost /]# qemu-kvm --version
QEMU emulator version 1.4.1, Copyright (c) 2003-2008 Fabrice Bellard
[root@localhost /]# uname -a
Linux localhost.localdomain 3.9.2-301.fc19.x86_64 #1 SMP Mon May 13
12:36:24 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux


On Tue, Aug 20, 2013 at 6:16 PM, Paolo Bonzini <pbonzini@redhat.com>
wrote:
> Il 20/08/2013 11:59, Timon Wang ha scritto:
>> On Tue, Aug 20, 2013 at 4:33 PM, Paolo Bonzini
<pbonzini@redhat.com> wrote:
>>> Il 20/08/2013 08:00, Timon Wang ha scritto:
>>>>     <disk type='file' device='disk'>
>>>>       <driver name='qemu' type='raw'
cache='none'/>
>>>>       <source file='/home/images/win2008_2_sys'/>
>>>>       <target dev='hda' bus='ide'/>
>>>>       <boot order='3'/>
>>>>       <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>>>>     </disk>
>>>>     <disk type='file' device='cdrom'>
>>>>       <driver name='qemu' type='raw'/>
>>>>       <source
file='/home/isos/windows2008_64r2.iso'/>
>>>>       <target dev='sdc' bus='ide'/>
>>>>       <readonly/>
>>>>       <boot order='1'/>
>>>>       <address type='drive' controller='0'
bus='1' target='0' unit='0'/>
>>>>     </disk>
>>>>     <disk type='block' device='disk'>
>>>
>>> I'm not sure this will be enough, but if you want passthrough
to the
>>> host device you should use device='lun' here.  However, you
still would
>>> not be able to issue SCSI reservations unless you run QEMU with the
>>> CAP_SYS_RAWIO capability (using "<disk ...
rawio='yes'>").
>>>
>>
>> After change the libvirt xml like this:
>> <disk type='block' device='lun'
rawio='yes'>
>>       <driver name='qemu' type='raw'
cache='none'/>
>>       <source
dev='/dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk'/>
>>       <target dev='sda' bus='scsi'/>
>>       <shareable/>
>>       <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>>     </disk>
>> I got these errors:
>> char device redirected to /dev/pts/1 (label charserial0)
>> qemu-system-x86_64: -device
>>
scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0:
>> scsi-block: INQUIRY failed
>> qemu-system-x86_64: -device
>>
scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0:
>> Device 'scsi-block' could not be initialized
>
> Can you do
>
> # ls -l /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk
> # sg_inq /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk
>
> ?
>
> Paolo
>


-- 
Focus on: Server Vitualization, Network security,Scanner,NodeJS,JAVA,WWW
Blog: http://www.nohouse.net