libvirt users - Oct 2019 - Re: [libvirt] Add support for vhost-user-scsi-pci/vhost-user-blk-pci

Hi Cole & Michal,

I'm sorry for my late response, I just end my journey today.
Thank your response, your suggestion is very helpful to me.

I have added Michal in this mail, Michal helps me review my initial patchset.
(https://www.spinics.net/linux/fedora/libvir/msg191339.html)

All concern about this feature is the XML design.
My original XML design exposes more details of Qemu.

    <vhost-user-blk-pci type='unix'>
        <source type='bind' path='/tmp/vhost-blk.sock'>
            <reconnect enabled='yes' timeout='5' />
        </source>
        <queue num='4'/>
    </vhost-user-blk-pci>

As Cole's suggestion, the better design with all vhost-user-scsi/blk
features would like this:

vhost-user-blk:

<disk type='vhostuser' device='disk'>
   <source type='unix' path='/path/to/vhost-user-blk.sock'
mode='client'>
       <reconnect enabled='yes' timeout='5' />
    </source>
   <target dev='vda' bus='virtio'/>
   <queue num='4'/>
    <address type='pci' domain='0x0000' bus='0x00'
slot='0x07' function='0x0'/>
</disk>

vhost-user-scsi:

<disk type='vhostuser' device='disk'>
   <source type='unix' path='/path/to/vhost-user-scsi.sock'
mode='client'>
       <reconnect enabled='yes' timeout='5' />
   </source>
   <target dev='sda' bus='scsi'/>
   <queue num='4'/>
</disk>

Conclusion:
1. Add new type(vhostuser) for disk label;
2. Add queue sub-label for disk to support multiqueue(<queue
num='4'/>) or reusing the driver label
(<driver name='vhostuser' queues='4'), which one is better?
Qemu support multiqueue like this:
-device vhost-user-scsi-pci,id=scsi0,chardev=spdk_vhost_scsi0,num_queues=4
-device vhost-user-blk-pci,chardev=spdk_vhost_blk0,num-queues=4

Another question:
When qemu is connecting to a vhost-user-scsi controller[1],  there may
exist multiple LUNs under one target,
then one disklabel(<disk/>) will represent multiple SCSI LUNs,
the 'dev' property(<target dev='sda' bus='scsi'/>)
will be ignored, right?
In other words, for vhost-user-scsi disk, it more likes a controller,
maybe the controller label is suitable.

I look forward to hearing from you as soon as possible.

[1]: https://spdk.io/doc/vhost.html

Feng Li

Cole Robinson <crobinso@redhat.com> 于2019年10月10日周四
上午6:48写道：
>
> Sorry for the late reply, and thanks Jano for pointing out elsewhere
> that this didn't receive a response.
>
> On 8/12/19 5:56 AM, Li Feng wrote:
> > Hi Guys,
> >
> > And I want to add the vhost-user-scsi-pci/vhost-user-blk-pci support
> > for libvirt.
> >
> > The usage in qemu like this:
> >
> > Vhost-SCSI
> > -chardev socket,id=char0,path=/var/tmp/vhost.0
> > -device vhost-user-scsi-pci,id=scsi0,chardev=char0
> > Vhost-BLK
> > -chardev socket,id=char1,path=/var/tmp/vhost.1
> > -device vhost-user-blk-pci,id=blk0,chardev=char1
> >
>
> Indeed that matches what I see for the qemu commits too:
>
> https://git.qemu.org/?p=qemu.git;a=commit;h=00343e4b54b
> https://git.qemu.org/?p=qemu.git;a=commit;h=f12c1ebddf7
>
> > What type should I add for libvirt.
> > Type1:
> >      <hostdev mode='subsystem'
type='vhost-user'>
> >          <source protocol='vhost-user-scsi'
path='/tmp/vhost-scsi.sock'></source>
> >          <alias name="vhost-user-scsi-disk1"/>
> >      </hostdev>
> >
> >
> > Type2:
> >
> >      <disk type='network' device='disk'>
> >        <driver name='qemu' type='raw'
cache='none' io='native'/>
> >        <source protocol='vhost-user'
path='/tmp/vhost-scsi.sock'>
> >        </source>
> >        <target dev='sdb' bus='vhost-user-scsi'/>
> >        <boot order='3'/>
> >        <alias name='scsi0-0-0-1'/>
> >        <address type='drive' controller='0'
bus='0' target='0' unit='1'/>
> >      </disk>
> >
> >
> >      <disk type='network' device='disk'>
> >        <driver name='qemu' type='raw'
cache='none' io='native'/>
> >        <source protocol='vhost-user'
path='/tmp/vhost-blk.sock'>
> >        </source>
> >        <target dev='vda' bus='vhost-user-blk'/>
> >        <boot order='1'/>
> >        <alias name='virtio-disk0'/>
> >        <address type='pci' domain='0x0000'
bus='0x00' slot='0x07'
> > function='0x0'/>
> >      </disk>
> >
>
> I think wiring this into <disk> makes more sense. <hostdev> is
really an
> abstraction for assigning a (typically) physical host device to the VM,
> so it handles things like hiding a PCI device from the host, and passing
> that exact device to the VM.
>
> In the vhost-user-scsi/blk case, the host device is just a special
> process running on the other side of a socket, and the device
> represented to the guest is a typical virtio device. So to me it makes
> more sense as a <disk> with a <source> that points at that
socket.
>
> target bus=virtio vs bus=scsi is already used to distinguish between
> virtio-blk and virtio-scsi, so I think we can keep that bit as is, with
> the <address type=drive|pci> to match. We just need to differentiate
> between plain virtio and vhost-user
>
> network devices already have vhostuser support:
>
>    <interface type='vhostuser'>
>      <source type='unix' path='/tmp/vhost1.sock'
mode='server|client'/>
>      <model type='virtio'/>
>    </interface>
>
> Internally that <source> is a virDomainChrSourceDefPtr which is our
> internal representation of a chardev. So I think something akin to this
> is the way to go. It will likely require updating a LOT of places in the
> code that check disk type= field, probably most places that care about
> whether type=NETWORK or type=!NETWORK will need to be mirrored for the
> new type.
>
> <disk type='vhostuser' device='disk'>
>    <source type='unix'
path='/path/to/vhost-user-blk.sock' mode='client'/>
>    <target dev='vda' bus='virtio'/>
> </disk>
>
> <disk type='vhostuser' device='disk'>
>    <source type='unix'
path='/path/to/vhost-user-scsi.sock' mode='client'/>
>    <target dev='sda' bus='scsi'/>
> </disk>
>
> - Cole
-- 
The SmartX email address is only for business purpose. Any sent message 
that is not related to the business is not authorized or permitted by 
SmartX.
本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.

On 10/14/19 3:12 AM, Li Feng wrote:
> Hi Cole & Michal,
> 
> I'm sorry for my late response, I just end my journey today.
> Thank your response, your suggestion is very helpful to me.
> 
> I have added Michal in this mail, Michal helps me review my initial
patchset.
> (https://www.spinics.net/linux/fedora/libvir/msg191339.html)
> 
Whoops I missed that posting, I didn't realize you had sent patches!
> All concern about this feature is the XML design.
> My original XML design exposes more details of Qemu.
> 
>      <vhost-user-blk-pci type='unix'>
>          <source type='bind'
path='/tmp/vhost-blk.sock'>
>              <reconnect enabled='yes' timeout='5' />
>          </source>
>          <queue num='4'/>
>      </vhost-user-blk-pci>
> 
> As Cole's suggestion, the better design with all vhost-user-scsi/blk
> features would like this:
> 
> vhost-user-blk:
> 
> <disk type='vhostuser' device='disk'>
>     <source type='unix'
path='/path/to/vhost-user-blk.sock' mode='client'>
>         <reconnect enabled='yes' timeout='5' />
>      </source>
>     <target dev='vda' bus='virtio'/>
>     <queue num='4'/>
>      <address type='pci' domain='0x0000'
bus='0x00' slot='0x07' function='0x0'/>
> </disk>
> 
> vhost-user-scsi:
> 
> <disk type='vhostuser' device='disk'>
>     <source type='unix'
path='/path/to/vhost-user-scsi.sock' mode='client'>
>         <reconnect enabled='yes' timeout='5' />
>     </source>
>     <target dev='sda' bus='scsi'/>
>     <queue num='4'/>
> </disk>
> 
I think my SCSI idea is wrong, sorry. vhost-user-scsi is for passing a
scsi host adapter to the VM, correct? If so, then it's not really a
<disk>, and so using the existing vhost-scsi support in <hostdev> is
probably better. <hostdev> could possible be used for vhost-user-blk as
well

Can you provide some examples of full qemu command lines using
vhost-user-blk and vhost-user-scsi? Just linking to examples else where
is fine, but I'm wondering if there's more context

Internally we already have an abstraction for vhost-scsi:

    <hostdev mode='subsystem' type='scsi_host'>
      <source protocol='vhost' wwpn='XXX'/>
    </hostdev>


The obvious extension would be

    <hostdev mode='subsystem' type='scsi_host'>
      <source protocol='vhostuser' type='unix'
path='/path/to/vhost-user-scsi.sock' mode='client'/>
    </hostdev>

Internally implementing this will be weird. The <source> parameters are
only dictated by the hostdev type= field, but in this case they would be
dictated by the <source protocol=> field, and we would want to reuse the
internal chardev abstraction.

vhost-user-blk could be implemented similarly, but with type='storage'
which is the way we pass through block devices to LXC guests, but it
isn't currently supported in the qemu driver.

I dunno. Maybe Michal or danpb can provide guidance

> Conclusion:
> 1. Add new type(vhostuser) for disk label;
> 2. Add queue sub-label for disk to support multiqueue(<queue
> num='4'/>) or reusing the driver label
> (<driver name='vhostuser' queues='4'), which one is
better?
> Qemu support multiqueue like this:
> -device vhost-user-scsi-pci,id=scsi0,chardev=spdk_vhost_scsi0,num_queues=4
> -device vhost-user-blk-pci,chardev=spdk_vhost_blk0,num-queues=4
> 
num-queues is already supported by libvirt for both <disk> and
<hostdev>
with <driver queues=X/>, so whether we use <disk> or <hostdev>
you won't
need to add any new XML here.
> Another question:
> When qemu is connecting to a vhost-user-scsi controller[1],  there may
> exist multiple LUNs under one target,
> then one disklabel(<disk/>) will represent multiple SCSI LUNs,
> the 'dev' property(<target dev='sda'
bus='scsi'/>) will be ignored, right?
> In other words, for vhost-user-scsi disk, it more likes a controller,
> maybe the controller label is suitable.
> 
Yes you are right, and this was my understanding. But then its not
really a <controller> in libvirt's sense because we can't attach
emulated devices to it, so it's more a fit for the existing <hostdev>
vhost-user support. Unfortunately it's not really a clean fit anywhere,
there will have to be some kind of compromise. And I'm not sure if
<disk> or <hostdev> is right for vhost-user-blk, but hopefully
others
have more clear opinions.

Thanks,
Cole
> I look forward to hearing from you as soon as possible.
> 
> [1]: https://spdk.io/doc/vhost.html
> 
> Feng Li
> 
> Cole Robinson <crobinso@redhat.com> 于2019年10月10日周四 上午6:48写道：
>>
>> Sorry for the late reply, and thanks Jano for pointing out elsewhere
>> that this didn't receive a response.
>>
>> On 8/12/19 5:56 AM, Li Feng wrote:
>>> Hi Guys,
>>>
>>> And I want to add the vhost-user-scsi-pci/vhost-user-blk-pci
support
>>> for libvirt.
>>>
>>> The usage in qemu like this:
>>>
>>> Vhost-SCSI
>>> -chardev socket,id=char0,path=/var/tmp/vhost.0
>>> -device vhost-user-scsi-pci,id=scsi0,chardev=char0
>>> Vhost-BLK
>>> -chardev socket,id=char1,path=/var/tmp/vhost.1
>>> -device vhost-user-blk-pci,id=blk0,chardev=char1
>>>
>>
>> Indeed that matches what I see for the qemu commits too:
>>
>> https://git.qemu.org/?p=qemu.git;a=commit;h=00343e4b54b
>> https://git.qemu.org/?p=qemu.git;a=commit;h=f12c1ebddf7
>>
>>> What type should I add for libvirt.
>>> Type1:
>>>       <hostdev mode='subsystem'
type='vhost-user'>
>>>           <source protocol='vhost-user-scsi'
path='/tmp/vhost-scsi.sock'></source>
>>>           <alias name="vhost-user-scsi-disk1"/>
>>>       </hostdev>
>>>
>>>
>>> Type2:
>>>
>>>       <disk type='network' device='disk'>
>>>         <driver name='qemu' type='raw'
cache='none' io='native'/>
>>>         <source protocol='vhost-user'
path='/tmp/vhost-scsi.sock'>
>>>         </source>
>>>         <target dev='sdb'
bus='vhost-user-scsi'/>
>>>         <boot order='3'/>
>>>         <alias name='scsi0-0-0-1'/>
>>>         <address type='drive' controller='0'
bus='0' target='0' unit='1'/>
>>>       </disk>
>>>
>>>
>>>       <disk type='network' device='disk'>
>>>         <driver name='qemu' type='raw'
cache='none' io='native'/>
>>>         <source protocol='vhost-user'
path='/tmp/vhost-blk.sock'>
>>>         </source>
>>>         <target dev='vda'
bus='vhost-user-blk'/>
>>>         <boot order='1'/>
>>>         <alias name='virtio-disk0'/>
>>>         <address type='pci' domain='0x0000'
bus='0x00' slot='0x07'
>>> function='0x0'/>
>>>       </disk>
>>>
>>
>> I think wiring this into <disk> makes more sense. <hostdev>
is really an
>> abstraction for assigning a (typically) physical host device to the VM,
>> so it handles things like hiding a PCI device from the host, and
passing
>> that exact device to the VM.
>>
>> In the vhost-user-scsi/blk case, the host device is just a special
>> process running on the other side of a socket, and the device
>> represented to the guest is a typical virtio device. So to me it makes
>> more sense as a <disk> with a <source> that points at that
socket.
>>
>> target bus=virtio vs bus=scsi is already used to distinguish between
>> virtio-blk and virtio-scsi, so I think we can keep that bit as is, with
>> the <address type=drive|pci> to match. We just need to
differentiate
>> between plain virtio and vhost-user
>>
>> network devices already have vhostuser support:
>>
>>     <interface type='vhostuser'>
>>       <source type='unix' path='/tmp/vhost1.sock'
mode='server|client'/>
>>       <model type='virtio'/>
>>     </interface>
>>
>> Internally that <source> is a virDomainChrSourceDefPtr which is
our
>> internal representation of a chardev. So I think something akin to this
>> is the way to go. It will likely require updating a LOT of places in
the
>> code that check disk type= field, probably most places that care about
>> whether type=NETWORK or type=!NETWORK will need to be mirrored for the
>> new type.
>>
>> <disk type='vhostuser' device='disk'>
>>     <source type='unix'
path='/path/to/vhost-user-blk.sock' mode='client'/>
>>     <target dev='vda' bus='virtio'/>
>> </disk>
>>
>> <disk type='vhostuser' device='disk'>
>>     <source type='unix'
path='/path/to/vhost-user-scsi.sock' mode='client'/>
>>     <target dev='sda' bus='scsi'/>
>> </disk>
>>
>> - Cole
> 

- Cole

Cole Robinson <crobinso@redhat.com> 于2019年10月15日周二
上午1:48写道：
>
> On 10/14/19 3:12 AM, Li Feng wrote:
> > Hi Cole & Michal,
> >
> > I'm sorry for my late response, I just end my journey today.
> > Thank your response, your suggestion is very helpful to me.
> >
> > I have added Michal in this mail, Michal helps me review my initial
patchset.
> > (https://www.spinics.net/linux/fedora/libvir/msg191339.html)
> >
>
> Whoops I missed that posting, I didn't realize you had sent patches!
>
> > All concern about this feature is the XML design.
> > My original XML design exposes more details of Qemu.
> >
> >      <vhost-user-blk-pci type='unix'>
> >          <source type='bind'
path='/tmp/vhost-blk.sock'>
> >              <reconnect enabled='yes' timeout='5'
/>
> >          </source>
> >          <queue num='4'/>
> >      </vhost-user-blk-pci>
> >
> > As Cole's suggestion, the better design with all
vhost-user-scsi/blk
> > features would like this:
> >
> > vhost-user-blk:
> >
> > <disk type='vhostuser' device='disk'>
> >     <source type='unix'
path='/path/to/vhost-user-blk.sock' mode='client'>
> >         <reconnect enabled='yes' timeout='5' />
> >      </source>
> >     <target dev='vda' bus='virtio'/>
> >     <queue num='4'/>
> >      <address type='pci' domain='0x0000'
bus='0x00' slot='0x07' function='0x0'/>
> > </disk>
> >
> > vhost-user-scsi:
> >
> > <disk type='vhostuser' device='disk'>
> >     <source type='unix'
path='/path/to/vhost-user-scsi.sock' mode='client'>
> >         <reconnect enabled='yes' timeout='5' />
> >     </source>
> >     <target dev='sda' bus='scsi'/>
> >     <queue num='4'/>
> > </disk>
> >
>
> I think my SCSI idea is wrong, sorry. vhost-user-scsi is for passing a
> scsi host adapter to the VM, correct? If so, then it's not really a
> <disk>, and so using the existing vhost-scsi support in
<hostdev> is
> probably better. <hostdev> could possible be used for vhost-user-blk
as well
>
> Can you provide some examples of full qemu command lines using
> vhost-user-blk and vhost-user-scsi? Just linking to examples else where
> is fine, but I'm wondering if there's more context
You could check the vhost-user-scsi/blk examples from SPDK pages:
https://spdk.io/doc/vhost.html
>
> Internally we already have an abstraction for vhost-scsi:
>
>     <hostdev mode='subsystem' type='scsi_host'>
>       <source protocol='vhost' wwpn='XXX'/>
>     </hostdev>
>
>
> The obvious extension would be
>
>     <hostdev mode='subsystem' type='scsi_host'>
>       <source protocol='vhostuser' type='unix'
> path='/path/to/vhost-user-scsi.sock' mode='client'/>
>     </hostdev>
>
> Internally implementing this will be weird. The <source> parameters
are
> only dictated by the hostdev type= field, but in this case they would be
> dictated by the <source protocol=> field, and we would want to reuse
the
> internal chardev abstraction.
>
> vhost-user-blk could be implemented similarly, but with
type='storage'
> which is the way we pass through block devices to LXC guests, but it
> isn't currently supported in the qemu driver.
>
> I dunno. Maybe Michal or danpb can provide guidance
>
@Michal, @danpb, could you give some guidance?
>
> > Conclusion:
> > 1. Add new type(vhostuser) for disk label;
> > 2. Add queue sub-label for disk to support multiqueue(<queue
> > num='4'/>) or reusing the driver label
> > (<driver name='vhostuser' queues='4'), which one is
better?
> > Qemu support multiqueue like this:
> > -device
vhost-user-scsi-pci,id=scsi0,chardev=spdk_vhost_scsi0,num_queues=4
> > -device vhost-user-blk-pci,chardev=spdk_vhost_blk0,num-queues=4
> >
>
> num-queues is already supported by libvirt for both <disk> and
<hostdev>
> with <driver queues=X/>, so whether we use <disk> or
<hostdev> you won't
> need to add any new XML here.
Got it.
>
> > Another question:
> > When qemu is connecting to a vhost-user-scsi controller[1],  there may
> > exist multiple LUNs under one target,
> > then one disklabel(<disk/>) will represent multiple SCSI LUNs,
> > the 'dev' property(<target dev='sda'
bus='scsi'/>) will be ignored, right?
> > In other words, for vhost-user-scsi disk, it more likes a controller,
> > maybe the controller label is suitable.
> >
>
> Yes you are right, and this was my understanding. But then its not
> really a <controller> in libvirt's sense because we can't
attach
> emulated devices to it, so it's more a fit for the existing
<hostdev>
> vhost-user support. Unfortunately it's not really a clean fit anywhere,
> there will have to be some kind of compromise. And I'm not sure if
> <disk> or <hostdev> is right for vhost-user-blk, but hopefully
others
> have more clear opinions.
I'm also confused about it.
Thanks for your reply.

Thanks,
Feng Li
>
> Thanks,
> Cole
>
> > I look forward to hearing from you as soon as possible.
> >
> > [1]: https://spdk.io/doc/vhost.html
> >
> > Feng Li
> >
> > Cole Robinson <crobinso@redhat.com> 于2019年10月10日周四 上午6:48写道：
> >>
> >> Sorry for the late reply, and thanks Jano for pointing out
elsewhere
> >> that this didn't receive a response.
> >>
> >> On 8/12/19 5:56 AM, Li Feng wrote:
> >>> Hi Guys,
> >>>
> >>> And I want to add the vhost-user-scsi-pci/vhost-user-blk-pci
support
> >>> for libvirt.
> >>>
> >>> The usage in qemu like this:
> >>>
> >>> Vhost-SCSI
> >>> -chardev socket,id=char0,path=/var/tmp/vhost.0
> >>> -device vhost-user-scsi-pci,id=scsi0,chardev=char0
> >>> Vhost-BLK
> >>> -chardev socket,id=char1,path=/var/tmp/vhost.1
> >>> -device vhost-user-blk-pci,id=blk0,chardev=char1
> >>>
> >>
> >> Indeed that matches what I see for the qemu commits too:
> >>
> >> https://git.qemu.org/?p=qemu.git;a=commit;h=00343e4b54b
> >> https://git.qemu.org/?p=qemu.git;a=commit;h=f12c1ebddf7
> >>
> >>> What type should I add for libvirt.
> >>> Type1:
> >>>       <hostdev mode='subsystem'
type='vhost-user'>
> >>>           <source protocol='vhost-user-scsi'
path='/tmp/vhost-scsi.sock'></source>
> >>>           <alias
name="vhost-user-scsi-disk1"/>
> >>>       </hostdev>
> >>>
> >>>
> >>> Type2:
> >>>
> >>>       <disk type='network'
device='disk'>
> >>>         <driver name='qemu' type='raw'
cache='none' io='native'/>
> >>>         <source protocol='vhost-user'
path='/tmp/vhost-scsi.sock'>
> >>>         </source>
> >>>         <target dev='sdb'
bus='vhost-user-scsi'/>
> >>>         <boot order='3'/>
> >>>         <alias name='scsi0-0-0-1'/>
> >>>         <address type='drive'
controller='0' bus='0' target='0' unit='1'/>
> >>>       </disk>
> >>>
> >>>
> >>>       <disk type='network'
device='disk'>
> >>>         <driver name='qemu' type='raw'
cache='none' io='native'/>
> >>>         <source protocol='vhost-user'
path='/tmp/vhost-blk.sock'>
> >>>         </source>
> >>>         <target dev='vda'
bus='vhost-user-blk'/>
> >>>         <boot order='1'/>
> >>>         <alias name='virtio-disk0'/>
> >>>         <address type='pci' domain='0x0000'
bus='0x00' slot='0x07'
> >>> function='0x0'/>
> >>>       </disk>
> >>>
> >>
> >> I think wiring this into <disk> makes more sense.
<hostdev> is really an
> >> abstraction for assigning a (typically) physical host device to
the VM,
> >> so it handles things like hiding a PCI device from the host, and
passing
> >> that exact device to the VM.
> >>
> >> In the vhost-user-scsi/blk case, the host device is just a special
> >> process running on the other side of a socket, and the device
> >> represented to the guest is a typical virtio device. So to me it
makes
> >> more sense as a <disk> with a <source> that points at
that socket.
> >>
> >> target bus=virtio vs bus=scsi is already used to distinguish
between
> >> virtio-blk and virtio-scsi, so I think we can keep that bit as is,
with
> >> the <address type=drive|pci> to match. We just need to
differentiate
> >> between plain virtio and vhost-user
> >>
> >> network devices already have vhostuser support:
> >>
> >>     <interface type='vhostuser'>
> >>       <source type='unix'
path='/tmp/vhost1.sock' mode='server|client'/>
> >>       <model type='virtio'/>
> >>     </interface>
> >>
> >> Internally that <source> is a virDomainChrSourceDefPtr which
is our
> >> internal representation of a chardev. So I think something akin to
this
> >> is the way to go. It will likely require updating a LOT of places
in the
> >> code that check disk type= field, probably most places that care
about
> >> whether type=NETWORK or type=!NETWORK will need to be mirrored for
the
> >> new type.
> >>
> >> <disk type='vhostuser' device='disk'>
> >>     <source type='unix'
path='/path/to/vhost-user-blk.sock' mode='client'/>
> >>     <target dev='vda' bus='virtio'/>
> >> </disk>
> >>
> >> <disk type='vhostuser' device='disk'>
> >>     <source type='unix'
path='/path/to/vhost-user-scsi.sock' mode='client'/>
> >>     <target dev='sda' bus='scsi'/>
> >> </disk>
> >>
> >> - Cole
> >
>
>
> - Cole
-- 
The SmartX email address is only for business purpose. Any sent message 
that is not related to the business is not authorized or permitted by 
SmartX.
本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.