Timon Wang
2013-Aug-20 11:43 UTC
Re: [libvirt-users] Oracle RAC in libvirt+KVM environment
Thanks, the whole iSCSI LUN have been passed to the VM. But I test it with scsicmd, and found that the driver may be not support SPC-3, but if i use this by microsoft iscsi initiator, I can pass all the scsi3_test tests. Tool can be found here: http://www.symantec.com/business/support/index?page=content&id=TECH72086 It's this means that the scsi passthrough windows driver does not support SPC-3 feature, I have read a post about this, it says if support this we should change both the implementation and the documents in virtio spec. I am new to this list, so I don't know what is the situation right now? Would somebody please give me some advise on it? On Tue, Aug 20, 2013 at 6:49 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:> Il 20/08/2013 12:42, Timon Wang ha scritto: >> [root@localhost /]# ls -l /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk >> lrwxrwxrwx. 1 root root 8 8月 20 17:38 >> /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk -> ../dm-13 >> [root@localhost /]# sg_inq /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk >> standard INQUIRY: >> PQual=0 Device_type=0 RMB=0 version=0x05 [SPC-3] >> [AERC=0] [TrmTsk=0] NormACA=0 HiSUP=0 Resp_data_format=0 >> SCCS=1 ACC=0 TPGS=1 3PC=0 Protect=0 [BQue=0] >> EncServ=0 MultiP=0 [MChngr=0] [ACKREQQ=0] Addr16=0 >> [RelAdr=0] WBus16=1 Sync=1 Linked=0 [TranDis=0] CmdQue=1 >> length=36 (0x24) Peripheral device type: disk >> Vendor identification: MacroSAN >> Product identification: LU >> Product revision level: 1.0 >> Unit serial number: 0d9281ae-aea4-6da0-0000-02180142b300 >> >> This lun is from a vg build based on iscsi target. > > If it is a logical volume, you cannot pass it as a LUN to the guest. > Only the whole iSCSI LUN can be passed as a LUN. > > Paolo > >> [root@localhost /]# libvirtd --version >> libvirtd (libvirt) 1.0.5 >> [root@localhost /]# qemu-kvm --version >> QEMU emulator version 1.4.1, Copyright (c) 2003-2008 Fabrice Bellard >> [root@localhost /]# uname -a >> Linux localhost.localdomain 3.9.2-301.fc19.x86_64 #1 SMP Mon May 13 >> 12:36:24 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux >> >> >> On Tue, Aug 20, 2013 at 6:16 PM, Paolo Bonzini <pbonzini@redhat.com> wrote: >>> Il 20/08/2013 11:59, Timon Wang ha scritto: >>>> On Tue, Aug 20, 2013 at 4:33 PM, Paolo Bonzini <pbonzini@redhat.com> wrote: >>>>> Il 20/08/2013 08:00, Timon Wang ha scritto: >>>>>> <disk type='file' device='disk'> >>>>>> <driver name='qemu' type='raw' cache='none'/> >>>>>> <source file='/home/images/win2008_2_sys'/> >>>>>> <target dev='hda' bus='ide'/> >>>>>> <boot order='3'/> >>>>>> <address type='drive' controller='0' bus='0' target='0' unit='0'/> >>>>>> </disk> >>>>>> <disk type='file' device='cdrom'> >>>>>> <driver name='qemu' type='raw'/> >>>>>> <source file='/home/isos/windows2008_64r2.iso'/> >>>>>> <target dev='sdc' bus='ide'/> >>>>>> <readonly/> >>>>>> <boot order='1'/> >>>>>> <address type='drive' controller='0' bus='1' target='0' unit='0'/> >>>>>> </disk> >>>>>> <disk type='block' device='disk'> >>>>> >>>>> I'm not sure this will be enough, but if you want passthrough to the >>>>> host device you should use device='lun' here. However, you still would >>>>> not be able to issue SCSI reservations unless you run QEMU with the >>>>> CAP_SYS_RAWIO capability (using "<disk ... rawio='yes'>"). >>>>> >>>> >>>> After change the libvirt xml like this: >>>> <disk type='block' device='lun' rawio='yes'> >>>> <driver name='qemu' type='raw' cache='none'/> >>>> <source dev='/dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk'/> >>>> <target dev='sda' bus='scsi'/> >>>> <shareable/> >>>> <address type='drive' controller='0' bus='0' target='0' unit='0'/> >>>> </disk> >>>> I got these errors: >>>> char device redirected to /dev/pts/1 (label charserial0) >>>> qemu-system-x86_64: -device >>>> scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0: >>>> scsi-block: INQUIRY failed >>>> qemu-system-x86_64: -device >>>> scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0: >>>> Device 'scsi-block' could not be initialized >>> >>> Can you do >>> >>> # ls -l /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk >>> # sg_inq /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk >>> >>> ? >>> >>> Paolo >>> >> >> >> >-- Focus on: Server Vitualization, Network security,Scanner,NodeJS,JAVA,WWW Blog: http://www.nohouse.net
Timon Wang
2013-Aug-20 11:57 UTC
Re: [libvirt-users] Oracle RAC in libvirt+KVM environment
I found when I use "scsicmd -d1 -s13" test command to test the
"controller bus reset" request, there will be a blue screen on windows
2008 r2.
The error code is :
BugCheck D1, {4, a, 0, fffff8800154dd06}
1: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000004, memory referenced
Arg2: 000000000000000a, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff8800154dd06, address which referenced memory
Debugging Details:
------------------
Page 17c41 not present in the dump file. Type ".hh dbgerr004" for
details
READ_ADDRESS: 0000000000000004
CURRENT_IRQL: a
FAULTING_IP:
vioscsi+1d06
fffff880`0154dd06 458b4804 mov r9d,dword ptr [r8+4]
DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT
BUGCHECK_STR: 0xD1
PROCESS_NAME: scsicmd.exe
TRAP_FRAME: fffff880009f7670 -- (.trap 0xfffff880009f7670)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000000002 rbx=0000000000000000 rcx=fffffa800065e738
rdx=fffffa800065e8f8 rsi=0000000000000000 rdi=0000000000000000
rip=fffff8800154dd06 rsp=fffff880009f7800 rbp=fffffa800065e8f8
r8=0000000000000000 r9=0000000000000000 r10=fffffa80009155b0
r11=fffff880009f7848 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl zr na po nc
vioscsi+0x1d06:
fffff880`0154dd06 458b4804 mov r9d,dword ptr [r8+4]
ds:e630:0004=????????
Resetting default scope
LAST_CONTROL_TRANSFER: from fffff800016ca469 to fffff800016caf00
STACK_TEXT:
fffff880`009f7528 fffff800`016ca469 : 00000000`0000000a
00000000`00000004 00000000`0000000a 00000000`00000000 :
nt!KeBugCheckEx
fffff880`009f7530 fffff800`016c90e0 : 00000000`00000000
fffffa80`009151b0 fffffa80`0155a290 fffff880`01339110 :
nt!KiBugCheckDispatch+0x69
fffff880`009f7670 fffff880`0154dd06 : 00000000`00000001
fffff880`0154dcec fffffa80`009151b0 fffff880`01323934 :
nt!KiPageFault+0x260
fffff880`009f7800 fffff880`0132abcf : fffffa80`009151b0
fffffa80`0065e8f8 fffffa80`0065e738 00000000`00000001 : vioscsi+0x1d06
fffff880`009f7850 fffff880`0154d971 : 00000000`00000001
00000000`00000001 00000000`002d5000 fffffa80`00925000 :
storport!StorPortSynchronizeAccess+0x4f
fffff880`009f7890 fffff880`01323a0c : fffffa80`00000fb1
fffffa80`0155a200 00000000`002d5000 fffffa80`01576010 : vioscsi+0x1971
fffff880`009f78d0 fffff880`01333adf : fffffa80`006eeb30
fffffa80`006e2070 00000000`00000000 00000000`00000801 :
storport!RaCallMiniportResetBus+0x1c
fffff880`009f7900 fffff880`01333b68 : fffffa80`0155a290
fffffa80`006b39f0 00000040`00000000 00000000`00000000 :
storport!RaidAdapterResetBus+0x2f
fffff880`009f7950 fffff880`0136de0b : 00000000`20206f49
00000000`00000001 00000000`00000001 00000000`20206f49 :
storport!RaidAdapterStorageResetBusIoctl+0x28
fffff880`009f7980 fffff880`0136d1d0 : fffff880`01339110
fffffa80`00915060 00000000`00000000 fffffa80`006e2070 : storport! ??
::NNGAKEGL::`string'+0x3c8
fffff880`009f79d0 fffff800`019e33a7 : fffffa80`006e2070
fffff880`009f7ca0 fffffa80`006e2070 fffffa80`0155a290 :
storport!RaDriverDeviceControlIrp+0x90
fffff880`009f7a10 fffff800`019e3c06 : 00000000`00000000
00000000`00000000 00000000`00000000 00000000`00000000 :
nt!IopXxxControlFile+0x607
fffff880`009f7b40 fffff800`016ca153 : 00000000`001aeb01
00000000`00000001 00000000`001aeba0 fffff800`019db152 :
nt!NtDeviceIoControlFile+0x56
fffff880`009f7bb0 00000000`77a2ff2a : 00000000`00000000
00000000`00000000 00000000`00000000 00000000`00000000 :
nt!KiSystemServiceCopyEnd+0x13
00000000`001af1d8 00000000`00000000 : 00000000`00000000
00000000`00000000 00000000`00000000 00000000`00000000 : 0x77a2ff2a
STACK_COMMAND: kb
FOLLOWUP_IP:
vioscsi+1d06
fffff880`0154dd06 458b4804 mov r9d,dword ptr [r8+4]
SYMBOL_STACK_INDEX: 3
SYMBOL_NAME: vioscsi+1d06
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: vioscsi
IMAGE_NAME: vioscsi.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 5200724f
FAILURE_BUCKET_ID: X64_0xD1_vioscsi+1d06
BUCKET_ID: X64_0xD1_vioscsi+1d06
Followup: MachineOwner
---------
On Tue, Aug 20, 2013 at 7:43 PM, Timon Wang <timonwst@gmail.com>
wrote:> Thanks, the whole iSCSI LUN have been passed to the VM.
>
> But I test it with scsicmd, and found that the driver may be not
> support SPC-3, but if i use this by microsoft iscsi initiator, I can
> pass all the scsi3_test tests.
>
> Tool can be found here:
>
http://www.symantec.com/business/support/index?page=content&id=TECH72086
>
> It's this means that the scsi passthrough windows driver does not
> support SPC-3 feature, I have read a post about this, it says if
> support this we should change both the implementation and the
> documents in virtio spec.
>
> I am new to this list, so I don't know what is the situation right now?
>
> Would somebody please give me some advise on it?
>
>
> On Tue, Aug 20, 2013 at 6:49 PM, Paolo Bonzini <pbonzini@redhat.com>
wrote:
>> Il 20/08/2013 12:42, Timon Wang ha scritto:
>>> [root@localhost /]# ls -l
/dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk
>>> lrwxrwxrwx. 1 root root 8 8月 20 17:38
>>> /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk -> ../dm-13
>>> [root@localhost /]# sg_inq
/dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk
>>> standard INQUIRY:
>>> PQual=0 Device_type=0 RMB=0 version=0x05 [SPC-3]
>>> [AERC=0] [TrmTsk=0] NormACA=0 HiSUP=0 Resp_data_format=0
>>> SCCS=1 ACC=0 TPGS=1 3PC=0 Protect=0 [BQue=0]
>>> EncServ=0 MultiP=0 [MChngr=0] [ACKREQQ=0] Addr16=0
>>> [RelAdr=0] WBus16=1 Sync=1 Linked=0 [TranDis=0] CmdQue=1
>>> length=36 (0x24) Peripheral device type: disk
>>> Vendor identification: MacroSAN
>>> Product identification: LU
>>> Product revision level: 1.0
>>> Unit serial number: 0d9281ae-aea4-6da0-0000-02180142b300
>>>
>>> This lun is from a vg build based on iscsi target.
>>
>> If it is a logical volume, you cannot pass it as a LUN to the guest.
>> Only the whole iSCSI LUN can be passed as a LUN.
>>
>> Paolo
>>
>>> [root@localhost /]# libvirtd --version
>>> libvirtd (libvirt) 1.0.5
>>> [root@localhost /]# qemu-kvm --version
>>> QEMU emulator version 1.4.1, Copyright (c) 2003-2008 Fabrice
Bellard
>>> [root@localhost /]# uname -a
>>> Linux localhost.localdomain 3.9.2-301.fc19.x86_64 #1 SMP Mon May 13
>>> 12:36:24 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>>>
>>>
>>> On Tue, Aug 20, 2013 at 6:16 PM, Paolo Bonzini
<pbonzini@redhat.com> wrote:
>>>> Il 20/08/2013 11:59, Timon Wang ha scritto:
>>>>> On Tue, Aug 20, 2013 at 4:33 PM, Paolo Bonzini
<pbonzini@redhat.com> wrote:
>>>>>> Il 20/08/2013 08:00, Timon Wang ha scritto:
>>>>>>> <disk type='file'
device='disk'>
>>>>>>> <driver name='qemu'
type='raw' cache='none'/>
>>>>>>> <source
file='/home/images/win2008_2_sys'/>
>>>>>>> <target dev='hda'
bus='ide'/>
>>>>>>> <boot order='3'/>
>>>>>>> <address type='drive'
controller='0' bus='0' target='0' unit='0'/>
>>>>>>> </disk>
>>>>>>> <disk type='file'
device='cdrom'>
>>>>>>> <driver name='qemu'
type='raw'/>
>>>>>>> <source
file='/home/isos/windows2008_64r2.iso'/>
>>>>>>> <target dev='sdc'
bus='ide'/>
>>>>>>> <readonly/>
>>>>>>> <boot order='1'/>
>>>>>>> <address type='drive'
controller='0' bus='1' target='0' unit='0'/>
>>>>>>> </disk>
>>>>>>> <disk type='block'
device='disk'>
>>>>>>
>>>>>> I'm not sure this will be enough, but if you want
passthrough to the
>>>>>> host device you should use device='lun' here.
However, you still would
>>>>>> not be able to issue SCSI reservations unless you run
QEMU with the
>>>>>> CAP_SYS_RAWIO capability (using "<disk ...
rawio='yes'>").
>>>>>>
>>>>>
>>>>> After change the libvirt xml like this:
>>>>> <disk type='block' device='lun'
rawio='yes'>
>>>>> <driver name='qemu' type='raw'
cache='none'/>
>>>>> <source
dev='/dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk'/>
>>>>> <target dev='sda' bus='scsi'/>
>>>>> <shareable/>
>>>>> <address type='drive'
controller='0' bus='0' target='0' unit='0'/>
>>>>> </disk>
>>>>> I got these errors:
>>>>> char device redirected to /dev/pts/1 (label charserial0)
>>>>> qemu-system-x86_64: -device
>>>>>
scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0:
>>>>> scsi-block: INQUIRY failed
>>>>> qemu-system-x86_64: -device
>>>>>
scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0:
>>>>> Device 'scsi-block' could not be initialized
>>>>
>>>> Can you do
>>>>
>>>> # ls -l /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk
>>>> # sg_inq /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk
>>>>
>>>> ?
>>>>
>>>> Paolo
>>>>
>>>
>>>
>>>
>>
>
>
>
> --
> Focus on: Server Vitualization, Network security,Scanner,NodeJS,JAVA,WWW
> Blog: http://www.nohouse.net
--
Focus on: Server Vitualization, Network security,Scanner,NodeJS,JAVA,WWW
Blog: http://www.nohouse.net
Paolo Bonzini
2013-Aug-20 12:09 UTC
Re: [libvirt-users] Oracle RAC in libvirt+KVM environment
Il 20/08/2013 13:43, Timon Wang ha scritto:> Thanks, the whole iSCSI LUN have been passed to the VM. > > But I test it with scsicmd, and found that the driver may be not > support SPC-3, but if i use this by microsoft iscsi initiator, I can > pass all the scsi3_test tests.If you are passing the LUN to the VM with device='lun', the driver and VMM do not interpret any SCSI command. You should see exactly the same data as in the host, which includes support for SPC-3:>>> [root@localhost /]# sg_inq /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk >>> standard INQUIRY: >>> PQual=0 Device_type=0 RMB=0 version=0x05 [SPC-3] >>> [AERC=0] [TrmTsk=0] NormACA=0 HiSUP=0 Resp_data_format=0 >>> SCCS=1 ACC=0 TPGS=1 3PC=0 Protect=0 [BQue=0] >>> EncServ=0 MultiP=0 [MChngr=0] [ACKREQQ=0] Addr16=0 >>> [RelAdr=0] WBus16=1 Sync=1 Linked=0 [TranDis=0] CmdQue=1 >>> length=36 (0x24) Peripheral device type: disk >>> Vendor identification: MacroSAN >>> Product identification: LU >>> Product revision level: 1.0 >>> Unit serial number: 0d9281ae-aea4-6da0-0000-02180142b300Can you try using a Linux VM and executing sg_inq in the VM? Paolo
Timon Wang
2013-Aug-21 02:11 UTC
Re: [libvirt-users] Oracle RAC in libvirt+KVM environment
>From the fedora 19 host:[root@fedora ~]# sg_inq /dev/sdc standard INQUIRY: PQual=0 Device_type=0 RMB=0 version=0x05 [SPC-3] [AERC=0] [TrmTsk=0] NormACA=0 HiSUP=0 Resp_data_format=0 SCCS=1 ACC=0 TPGS=1 3PC=0 Protect=0 [BQue=0] EncServ=0 MultiP=0 [MChngr=0] [ACKREQQ=0] Addr16=0 [RelAdr=0] WBus16=1 Sync=1 Linked=0 [TranDis=0] CmdQue=1 length=36 (0x24) Peripheral device type: disk Vendor identification: MacroSAN Product identification: LU Product revision level: 1.0 Unit serial number: fd01ece6-8540-f4c7-0000-fe170142b300>From the fedora 19 vm:[root@fedoravm ~]# sg_inq /dev/sdb standard INQUIRY: PQual=0 Device_type=0 RMB=0 version=0x05 [SPC-3] [AERC=0] [TrmTsk=0] NormACA=0 HiSUP=0 Resp_data_format=0 SCCS=1 ACC=0 TPGS=1 3PC=0 Protect=0 [BQue=0] EncServ=0 MultiP=0 [MChngr=0] [ACKREQQ=0] Addr16=0 [RelAdr=0] WBus16=1 Sync=1 Linked=0 [TranDis=0] CmdQue=1 length=36 (0x24) Peripheral device type: disk Vendor identification: MacroSAN Product identification: LU Product revision level: 1.0 Unit serial number: fd01ece6-8540-f4c7-0000-fe170142b300 The result from fedora 19 host and fedora 19 vm are the same. It's that means I got a wrong windows vm scsi pass-through driver? Or is there any tool like sg_inq in windows 2008? On Tue, Aug 20, 2013 at 8:09 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:> Il 20/08/2013 13:43, Timon Wang ha scritto: >> Thanks, the whole iSCSI LUN have been passed to the VM. >> >> But I test it with scsicmd, and found that the driver may be not >> support SPC-3, but if i use this by microsoft iscsi initiator, I can >> pass all the scsi3_test tests. > > If you are passing the LUN to the VM with device='lun', the driver and > VMM do not interpret any SCSI command. You should see exactly the same > data as in the host, which includes support for SPC-3: > >>>> [root@localhost /]# sg_inq /dev/VM-IMAGES-BACKUP-DO-NOT-REMOVE/q_disk >>>> standard INQUIRY: >>>> PQual=0 Device_type=0 RMB=0 version=0x05 [SPC-3] >>>> [AERC=0] [TrmTsk=0] NormACA=0 HiSUP=0 Resp_data_format=0 >>>> SCCS=1 ACC=0 TPGS=1 3PC=0 Protect=0 [BQue=0] >>>> EncServ=0 MultiP=0 [MChngr=0] [ACKREQQ=0] Addr16=0 >>>> [RelAdr=0] WBus16=1 Sync=1 Linked=0 [TranDis=0] CmdQue=1 >>>> length=36 (0x24) Peripheral device type: disk >>>> Vendor identification: MacroSAN >>>> Product identification: LU >>>> Product revision level: 1.0 >>>> Unit serial number: 0d9281ae-aea4-6da0-0000-02180142b300 > > Can you try using a Linux VM and executing sg_inq in the VM? > > Paolo >-- Focus on: Server Vitualization, Network security,Scanner,NodeJS,JAVA,WWW Blog: http://www.nohouse.net