Xen devel - Oct 2007 - [PATCH 0/5] pvSCSI (SCSI pass through) driver

Hi all,

I will send total five patches on following e-mails.

These patches provide functionality that guest domain can issue
complete SCSI command to FC-SCSI HBA(Fibre Channel SCSI Host Bus
Adapter) assigned to the guest domain.
Using this functionality, guest domain can control such as tape device
for backup and so on.

We hope these patches will be merged into Xen 3.2.0.

Thanks,

-----
Jun Kamada



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This seems to have been hanging around for ages. If there''s no argument
otherwise I''ll give it a read through and stick it in for 3.2.0.

 -- Keir

On 19/10/07 06:15, "Jun Kamada" <kama@jp.fujitsu.com> wrote:
> Hi all,
> 
> I will send total five patches on following e-mails.
> 
> These patches provide functionality that guest domain can issue
> complete SCSI command to FC-SCSI HBA(Fibre Channel SCSI Host Bus
> Adapter) assigned to the guest domain.
> Using this functionality, guest domain can control such as tape device
> for backup and so on.
> 
> We hope these patches will be merged into Xen 3.2.0.
> 
> Thanks,
> 
> -----
> Jun Kamada
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> This seems to have been hanging around for ages. If there''s no
argument
> otherwise I''ll give it a read through and stick it in for 3.2.0.
Yep, its certainly a candidate for 3.2, but I''d class the API as
"experimental" at this stage and not commit to supporting it going
forward as-is just yet.

Things I''d like to see: 
 * performance results (hopefully at the summit!)
 * variable sized comms rings (fixed at connect time) so we can queue
more descriptors if req''d.
 * support for pass through of individual scsi logical units to the
dom0''s linux scsi-generic layer rather than whole HBAs -- I think most
users will want to hide all that FC nastiness and just present a LUN.

Best,
Ian

> > These patches provide functionality that guest domain can issue
> > complete SCSI command to FC-SCSI HBA(Fibre Channel SCSI Host Bus
> > Adapter) assigned to the guest domain.
> > Using this functionality, guest domain can control such as tape
> device
> > for backup and so on.
> >
> > We hope these patches will be merged into Xen 3.2.0.
> >
> > Thanks,
> >
> > -----
> > Jun Kamada
> >
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> Hi all,
> 
> I will send total five patches on following e-mails.
> 
> These patches provide functionality that guest domain can issue
> complete SCSI command to FC-SCSI HBA(Fibre Channel SCSI Host Bus
> Adapter) assigned to the guest domain.
> Using this functionality, guest domain can control such as tape device
> for backup and so on.
> 
> We hope these patches will be merged into Xen 3.2.0.
> 
I am very interested in this functionality, but mainly from a Windows
point of view (doing test restores of backups for customers). What would
be involved in getting this working under a HVM domain in the absence of
PV drivers?

Failing that, I''ll see if I can incorporate it into the Windows PV
drivers I''m working on...

Thanks

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Keir-san, Ian-san,

Thank you so much for your positive response.

On Fri, 19 Oct 2007 11:20:28 +0100
"Ian Pratt" <Ian.Pratt@cl.cam.ac.uk>
wrote:
> > This seems to have been hanging around for ages. If there''s
no
> argument
> > otherwise I''ll give it a read through and stick it in for
3.2.0.
> 
> Yep, its certainly a candidate for 3.2, but I''d class the API as
> "experimental" at this stage and not commit to supporting it
going
> forward as-is just yet.
Do you mean that the API is "xm" command interface such as "xm 
scsihost-attach" or Xen-API?
If the former, we also consider the expansion of the interface is
needed. The interface should conform to VM migration that the VM can
continue to use same ID before and after the migration. On FC-SCSI,
the ID may be WWPN (World Wide Port Name).

> Things I''d like to see: 
>  * performance results (hopefully at the summit!)
We are planning to report performance data on next Xen Summit.

>  * variable sized comms rings (fixed at connect time) so we can queue
> more descriptors if req''d.
>  * support for pass through of individual scsi logical units to the
> dom0''s linux scsi-generic layer rather than whole HBAs -- I think
most
> users will want to hide all that FC nastiness and just present a LUN.
We also recognize the usefulness of above feature. However, I heard that
the method for specifying LUN is vender dependent. Implementation may be
difficult by non-technical reason.

-----
Jun Kamada
kama@jp.fujitsu.com



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

What''s the CONFIG_XEN_FC stuff all about? It doesn''t appear to
be
configurable (so it''s always off), it''s ifdef''ed all
over the place, and
there are dead files (e.g., fcfront.c) that presumably would depend on that
option.

Either it needs to be plumbed in properly, or the XEN_FC stuff needs to be
stripped out entirely from this patchset.

 -- Keir

On 19/10/07 06:15, "Jun Kamada" <kama@jp.fujitsu.com> wrote:
> Hi all,
> 
> I will send total five patches on following e-mails.
> 
> These patches provide functionality that guest domain can issue
> complete SCSI command to FC-SCSI HBA(Fibre Channel SCSI Host Bus
> Adapter) assigned to the guest domain.
> Using this functionality, guest domain can control such as tape device
> for backup and so on.
> 
> We hope these patches will be merged into Xen 3.2.0.
> 
> Thanks,
> 
> -----
> Jun Kamada
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Dear Keir-san,

On Mon, 22 Oct 2007 09:58:17 +0100
Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
> What''s the CONFIG_XEN_FC stuff all about? It doesn''t
appear to be
> configurable (so it''s always off), it''s ifdef''ed
all over the place, and
> there are dead files (e.g., fcfront.c) that presumably would depend on that
> option.
> 
> Either it needs to be plumbed in properly, or the XEN_FC stuff needs to be
> stripped out entirely from this patchset.
The CONFIG_XEN_FC stuff is used for activating FC(Fibre Channel)
transport layer functionality. I will attach additional patch for the
stuff.


Please note:

The FC transport functionality for PV driver on HVM domain is not
activated even if this stuff is set. It is only available for 2.6.18
kernel for PV domain.
This is why definition of "struct fc_function_template" used for the
FC transport functionality are differrent for each kernel version.
Therefore "#ifdef" switch is needed for each kernel version. We
didn''t
implement the switch on current implementation.

Best regards,


-----
Jun Kamada


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi James-san,

Thank you for your interests in our pvSCSI driver.

On Mon, 22 Oct 2007 10:07:40 +1000
"James Harper" <james.harper@bendigoit.com.au>
wrote:
> I am very interested in this functionality, but mainly from a Windows
> point of view (doing test restores of backups for customers). What would
> be involved in getting this working under a HVM domain in the absence of
> PV drivers?
> 
> Failing that, I''ll see if I can incorporate it into the Windows PV
> drivers I''m working on...
I appreciate your proposal because I don''t have skill for developing
Windows driver. I suppose that the pvSCSI functionality is useful for
Windows too.

Thanks,

-----
Jun Kamada



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> The CONFIG_XEN_FC stuff is used for activating FC(Fibre Channel)
> transport layer functionality. I will attach additional patch for the
> stuff.
Okay, thanks.

Another question about the original patches: did you test that a 64-bit
backend can communicate with 32-bit frontend (and vice versa)? Either you
need to make sure that all the shared-memory structures defined in vscsiif.h
compile identically for 32-bit and 64-bit mode, or you need to detect the
frontend''s ''bitness'' in scsiback and optionally do
32-to-64 or 64-to-32
conversion. I think it''s obvious which is less hassle. ;-)

Handling this is really a prerequisite for putting these patches into a
stable Xen release.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir-san,

On Tue, 23 Oct 2007 09:50:38 +0100
Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
> Another question about the original patches: did you test that a 64-bit
> backend can communicate with 32-bit frontend (and vice versa)? Either you
> need to make sure that all the shared-memory structures defined in
vscsiif.h
> compile identically for 32-bit and 64-bit mode, or you need to detect the
> frontend''s ''bitness'' in scsiback and optionally
do 32-to-64 or 64-to-32
> conversion. I think it''s obvious which is less hassle. ;-)
I suppose that the VBD uses the later method. (Is it right?) However,
the later method requires ''optimize level of frontend on compile
time''
in addition to the ''bitness''. If the frontend is optimized for
space,
members in structure are packed. On the other hand, if the frontend is
optimized for speed, each members are aligned on word boundary. (padding
will be inserted.)
How do you think about this?
> Handling this is really a prerequisite for putting these patches into a
> stable Xen release.
I would like to modify and post an additional patch ASAP.

-----
Jun Kamada



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 23/10/07 12:35, "Jun Kamada" <kama@jp.fujitsu.com> wrote:
>> Another question about the original patches: did you test that a 64-bit
>> backend can communicate with 32-bit frontend (and vice versa)? Either
you
>> need to make sure that all the shared-memory structures defined in
vscsiif.h
>> compile identically for 32-bit and 64-bit mode, or you need to detect
the
>> frontend''s ''bitness'' in scsiback and
optionally do 32-to-64 or 64-to-32
>> conversion. I think it''s obvious which is less hassle. ;-)
> 
> I suppose that the VBD uses the later method. (Is it right?) However,
> the later method requires ''optimize level of frontend on compile
time''
> in addition to the ''bitness''. If the frontend is
optimized for space,
> members in structure are packed. On the other hand, if the frontend is
> optimized for speed, each members are aligned on word boundary. (padding
> will be inserted.)
> How do you think about this?
The layout of structures is part of the C ABI on the target system:
optimising compilation for space cannot change that (without explicit
pragmas or attributes being added to the code).

Anyway, the right answer is to ensure that your structures compile the same
whether compiled with 32-bit or 64-bit gcc. Check the structure sizes,
fields sizes and field offsets.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Dear Keir-san,

Can I ask one question for my correct understanding?

On Tue, 23 Oct 2007 13:39:24 +0100
Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
> Anyway, the right answer is to ensure that your structures compile the same
> whether compiled with 32-bit or 64-bit gcc. Check the structure sizes,
> fields sizes and field offsets.
You showed me two options in previous email; i.) all the shared-memory
structures defined in vscsiif.h compile identically for 32-bit and
64-bit mode, ii.) to detect the frontend''s ''bitness''
in scsiback and
optionally do 32-to-64 or 64-to-32 conversion.

I consider that current VBD implementation takes option ii.), however
you recommended me to take option i.) for pvSCSI driver.
Is my understanding right?

Best regards,

-----
Jun Kamada
Linux Technology Development Div.
Server Systems Unit
Fujitsu Ltd.
kama@jp.fujitsu.com



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 24/10/07 07:22, "Jun Kamada" <kama@jp.fujitsu.com> wrote:
>> Anyway, the right answer is to ensure that your structures compile the
same
>> whether compiled with 32-bit or 64-bit gcc. Check the structure sizes,
>> fields sizes and field offsets.
> 
> You showed me two options in previous email; i.) all the shared-memory
> structures defined in vscsiif.h compile identically for 32-bit and
> 64-bit mode, ii.) to detect the frontend''s
''bitness'' in scsiback and
> optionally do 32-to-64 or 64-to-32 conversion.
> 
> I consider that current VBD implementation takes option ii.), however
> you recommended me to take option i.) for pvSCSI driver.
> Is my understanding right?
Yes. It was an accident we ended up with structures compiling differently on
32- vs 64-bit. By the time we discovered it was an issue, we couldn''t
break
backward compatibility.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir-san,

On Wed, 24 Oct 2007 08:42:38 +0100
Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
> On 24/10/07 07:22, "Jun Kamada" <kama@jp.fujitsu.com>
wrote:
> 
> >> Anyway, the right answer is to ensure that your structures compile
the same
> >> whether compiled with 32-bit or 64-bit gcc. Check the structure
sizes,
> >> fields sizes and field offsets.
> > 
> > You showed me two options in previous email; i.) all the shared-memory
> > structures defined in vscsiif.h compile identically for 32-bit and
> > 64-bit mode, ii.) to detect the frontend''s
''bitness'' in scsiback and
> > optionally do 32-to-64 or 64-to-32 conversion.
> > 
> > I consider that current VBD implementation takes option ii.), however
> > you recommended me to take option i.) for pvSCSI driver.
> > Is my understanding right?
> 
> Yes. It was an accident we ended up with structures compiling differently
on
> 32- vs 64-bit. By the time we discovered it was an issue, we
couldn''t break
> backward compatibility.
I understood. I''ll try to modify the pvSCSI driver by option i.).

Best regards,

-----
Jun Kamada



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir-san,

I have finished the modification of the driver by option i.).
The pvSCSI driver can work on heterogeneous (x86_32 and x86_64)
environment.

I will attach total six patches on following mail.

Best regards,


On Thu, 25 Oct 2007 10:59:31 +0900
Jun Kamada <kama@jp.fujitsu.com> wrote:
> Keir-san,
> 
> On Wed, 24 Oct 2007 08:42:38 +0100
> Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
> > On 24/10/07 07:22, "Jun Kamada" <kama@jp.fujitsu.com>
wrote:
> > 
> > >> Anyway, the right answer is to ensure that your structures
compile the same
> > >> whether compiled with 32-bit or 64-bit gcc. Check the
structure sizes,
> > >> fields sizes and field offsets.
> > > 
> > > You showed me two options in previous email; i.) all the
shared-memory
> > > structures defined in vscsiif.h compile identically for 32-bit
and
> > > 64-bit mode, ii.) to detect the frontend''s
''bitness'' in scsiback and
> > > optionally do 32-to-64 or 64-to-32 conversion.
> > > 
> > > I consider that current VBD implementation takes option ii.),
however
> > > you recommended me to take option i.) for pvSCSI driver.
> > > Is my understanding right?
> > 
> > Yes. It was an accident we ended up with structures compiling
differently on
> > 32- vs 64-bit. By the time we discovered it was an issue, we
couldn''t break
> > backward compatibility.
> 
> I understood. I''ll try to modify the pvSCSI driver by option i.).
> 
> Best regards,
> 
> -----
> Jun Kamada
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
-----
Jun Kamada



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This patch is for frontend driver.

Signed-off-by: Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>
Signed-off-by: Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>
Signed-off-by: Jun Kamada <kama@jp.fujitsu.com>
Signed-off-by: Akira Hayakawa <hayakawa.akira@jp.fujitsu.com>

-----
Jun Kamada


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This patch is for backend driver.

Signed-off-by: Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>
Signed-off-by: Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>
Signed-off-by: Jun Kamada <kama@jp.fujitsu.com>
Signed-off-by: Akira Hayakawa <hayakawa.akira@jp.fujitsu.com>

-----
Jun Kamada


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This patch is for vscsiif.h.

Signed-off-by: Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>
Signed-off-by: Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>
Signed-off-by: Jun Kamada <kama@jp.fujitsu.com>
Signed-off-by: Akira Hayakawa <hayakawa.akira@jp.fujitsu.com>

-----
Jun Kamada


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This patch is for Makefile and Kconfig file for scsifront and scsiback
driver.

Signed-off-by: Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>
Signed-off-by: Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>
Signed-off-by: Jun Kamada <kama@jp.fujitsu.com>
Signed-off-by: Akira Hayakawa <hayakawa.akira@jp.fujitsu.com>

-----
Jun Kamada


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This patch is for installing scsifront driver into pv-driver.

Signed-off-by: Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>
Signed-off-by: Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>
Signed-off-by: Jun Kamada <kama@jp.fujitsu.com>
Signed-off-by: Akira Hayakawa <hayakawa.akira@jp.fujitsu.com>

-----
Jun Kamada


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This patch is for modifying xend in order to use pvscsi driver.
(attach/detach HBA, etc.)

Signed-off-by: Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>
Signed-off-by: Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>
Signed-off-by: Jun Kamada <kama@jp.fujitsu.com>
Signed-off-by: Akira Hayakawa <hayakawa.akira@jp.fujitsu.com>

-----
Jun Kamada


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Vscsiif.h still needs clean up:
 1. You cannot have Linux-specific stuff in that header file. It''s
supposed
to be generic. Get rid of references to CONFIG_XEN_FC.
 2. You cannot pollute the global namespace with unqualified names. All your
names, macros, etc should have a vscsiif_ or VSCSIIF_ prefix. Names like
DEFAULT_CAN_QUEUE and SG_TABLESIZE pollute the namespace and it''s not
immediately obvious which header they come from (which it would be if they
were prefixed).
 3. Do you really mean for vscsiif_btf_response to be empty? And neither
response structure has an identifier to match the request it responds to.
Can requests not be handled out of order by your drivers? That would seem
very odd.

Also CONFIG_XEN_FC is ifdef''ed all over scsifront and scsiback.
You''ll have
to find some way to clean that up, perhaps by providing null stub
implementations of fc functions in a header file if !CONFIG_XEN_FC.

FIXME comments in the connection setup code, accompanied by hard-coded
constants, don''t bode well. Is the connection setup protocol fully
baked
yet?

All this, coupled with the fact that you can only export whole HBAs, which
doesn''t seem likely to be a very popular usage scenario, makes me think
this
driver should wait until after 3.2.0.

 --Keir

On 30/10/07 10:39, "Jun Kamada" <kama@jp.fujitsu.com> wrote:
> Keir-san,
> 
> I have finished the modification of the driver by option i.).
> The pvSCSI driver can work on heterogeneous (x86_32 and x86_64)
> environment.
> 
> I will attach total six patches on following mail.
> 
> Best regards,
> 
> 
> On Thu, 25 Oct 2007 10:59:31 +0900
> Jun Kamada <kama@jp.fujitsu.com> wrote:
> 
>> Keir-san,
>> 
>> On Wed, 24 Oct 2007 08:42:38 +0100
>> Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
>>> On 24/10/07 07:22, "Jun Kamada"
<kama@jp.fujitsu.com> wrote:
>>> 
>>>>> Anyway, the right answer is to ensure that your structures
compile the
>>>>> same
>>>>> whether compiled with 32-bit or 64-bit gcc. Check the
structure sizes,
>>>>> fields sizes and field offsets.
>>>> 
>>>> You showed me two options in previous email; i.) all the
shared-memory
>>>> structures defined in vscsiif.h compile identically for 32-bit
and
>>>> 64-bit mode, ii.) to detect the frontend''s
''bitness'' in scsiback and
>>>> optionally do 32-to-64 or 64-to-32 conversion.
>>>> 
>>>> I consider that current VBD implementation takes option ii.),
however
>>>> you recommended me to take option i.) for pvSCSI driver.
>>>> Is my understanding right?
>>> 
>>> Yes. It was an accident we ended up with structures compiling
differently on
>>> 32- vs 64-bit. By the time we discovered it was an issue, we
couldn''t break
>>> backward compatibility.
>> 
>> I understood. I''ll try to modify the pvSCSI driver by option
i.).
>> 
>> Best regards,
>> 
>> -----
>> Jun Kamada
>> 
>> 
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
> 
> -----
> Jun Kamada
> 
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Keir-san,

Thank you for your comments.

On Tue, 30 Oct 2007 10:56:57 +0000
Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
> Vscsiif.h still needs clean up:
>  1. You cannot have Linux-specific stuff in that header file. It''s
supposed
> to be generic.
I understood. I would like to remove Linux-specific definitions, such
like below.

    #include <scsi/scsi_transport.h>
    enum fc_port_state
>  2. You cannot pollute the global namespace with unqualified names. All
your
> names, macros, etc should have a vscsiif_ or VSCSIIF_ prefix. Names like
> DEFAULT_CAN_QUEUE and SG_TABLESIZE pollute the namespace and it''s
not
> immediately obvious which header they come from (which it would be if they
> were prefixed).
I would like to add the prefix to all the definitions.
>  3. Do you really mean for vscsiif_btf_response to be empty? And neither
> response structure has an identifier to match the request it responds to.
> Can requests not be handled out of order by your drivers? That would seem
> very odd.
The pvSCSI driver uses two rings for frontend-to-backend (ftb) and
backend-to-frontend (btf) communication. The reason why the
architecture is taken is as follows.

- The ftb_request packet was relatively big, so we could not allocate 
  enough entries on the ring. It caused a performance problem.
- By dividing into the two rings ("ftb" ring and "btf"
ring), and using
  original queueing mechanism on both frontend and backend, the entries
  on the ring is immediately freed after queued. It contributes a good
  performance.

Therefore, the ftb_response and btf_response carries no information.
It is only used as a trigger of freeing the entry on the ring.

We also would like to discuss about design of the ring architecture
described above. Any comment is welcome.
> Also CONFIG_XEN_FC is ifdef''ed all over scsifront and scsiback.
You''ll have
> to find some way to clean that up, perhaps by providing null stub
> implementations of fc functions in a header file if !CONFIG_XEN_FC.
I think that one of the way to reduce CONFIG_XEN_FC is to ifdef''ing
intialization of FC transport functionality. (However, object modules
still include FC modules.) How do you think about this modification?
Could you tell me more detail about your advice?

Please note why the CONFIG_XEN_FC is needed are,

- Old Linux kernel has different definition of
  "struct fc_function_template". 
- In order to support the old Linux kernel on HVM guest domain, simple
  way is disabling FC functionality.
> FIXME comments in the connection setup code, accompanied by hard-coded
> constants, don''t bode well. Is the connection setup protocol fully
baked
> yet?
FIXME comment indicates tunable point of the parameter for performance
and so on. I understood the word "FIXME'' was not appropriate. I
will
remove the word.
> All this, coupled with the fact that you can only export whole HBAs, which
> doesn''t seem likely to be a very popular usage scenario, makes me
think this
> driver should wait until after 3.2.0.
We consider that SCSI support on guest domain is mandatory for bussiness
use. Using a NPIV technology on host domain, each guest domain can have
own virtual HBA. We have already evaluated on our environment the
conbination of NPIV and pvSCSI work well.
On the other hand, we undestand the usefullness of LUN assignment to
guest domain. We would like to develop it on next step in our current
plan.

Best regards,

-----
Jun Kamada



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 31/10/07 08:37, "Jun Kamada" <kama@jp.fujitsu.com> wrote:
>> All this, coupled with the fact that you can only export whole HBAs,
which
>> doesn''t seem likely to be a very popular usage scenario, makes
me think this
>> driver should wait until after 3.2.0.
> 
> We consider that SCSI support on guest domain is mandatory for bussiness
> use. Using a NPIV technology on host domain, each guest domain can have
> own virtual HBA. We have already evaluated on our environment the
> conbination of NPIV and pvSCSI work well.
> On the other hand, we undestand the usefullness of LUN assignment to
> guest domain. We would like to develop it on next step in our current
> plan.
Okay, well re-spin the patches and I''ll take another look.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir-san,

On Wed, 31 Oct 2007 09:10:34 +0000
Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
> On 31/10/07 08:37, "Jun Kamada" <kama@jp.fujitsu.com>
wrote:
> 
> >> All this, coupled with the fact that you can only export whole
HBAs, which
> >> doesn''t seem likely to be a very popular usage scenario,
makes me think this
> >> driver should wait until after 3.2.0.
> > 
> > We consider that SCSI support on guest domain is mandatory for
bussiness
> > use. Using a NPIV technology on host domain, each guest domain can
have
> > own virtual HBA. We have already evaluated on our environment the
> > conbination of NPIV and pvSCSI work well.
> > On the other hand, we undestand the usefullness of LUN assignment to
> > guest domain. We would like to develop it on next step in our current
> > plan.
> 
> Okay, well re-spin the patches and I''ll take another look.
Can I ask one question in order to avoid my misunderstanding?

Do you mean that the pvSCSI driver is never commited until it supports
LUN assignment to geust domain, or anything else?

Best regards,

-----
Jun Kamada



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 31/10/07 10:56, "Jun Kamada" <kama@jp.fujitsu.com> wrote:
>> Okay, well re-spin the patches and I''ll take another look.
> 
> Can I ask one question in order to avoid my misunderstanding?
> 
> Do you mean that the pvSCSI driver is never commited until it supports
> LUN assignment to geust domain, or anything else?
The virtual HBA usage scenario sounds useful.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi,

On Tue, 2007-10-30 at 19:39 +0900, Jun Kamada wrote:

Is this using the same syntax for HBA reservation that we used to have,
ie. simply enumerated using the scsi host number on the dom0?

If so, that is rather fragile.  The scsi host number can change from
kernel to kernel depending on HBA driver probe order and PCI bus
enumeration order.  It changes when hardware is rearranged.  And how do
we deal with migrate, when we''ll need to select a completely different
HBA on a different host?

Selecting the HBA by scsi host number is good enough for a demonstration
that the code works, but I don''t think it''s going to be robust
enough
for production deployment.  

Also, has there been any progress towards exposing individual LUNs to
the guest rather than a whole HBA?

Cheers, 
 Stephen



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> Hi,
> 
> On Tue, 2007-10-30 at 19:39 +0900, Jun Kamada wrote:
> 
> Is this using the same syntax for HBA reservation that we used to
have,
> ie. simply enumerated using the scsi host number on the dom0?
> 
> If so, that is rather fragile.  The scsi host number can change from
> kernel to kernel depending on HBA driver probe order and PCI bus
> enumeration order.  It changes when hardware is rearranged.  And how
do
> we deal with migrate, when we''ll need to select a completely
different
> HBA on a different host?
> 
> Selecting the HBA by scsi host number is good enough for a
demonstration
> that the code works, but I don''t think it''s going to be
robust enough
> for production deployment.
> 
> Also, has there been any progress towards exposing individual LUNs to
> the guest rather than a whole HBA?
People keep saying individual LUN... do they actually mean SCSI Target
devices? Splitting up devices on the same HBA should be reasonably
simple, but due to the way the scsi protocol works (AFAIK), splitting up
LUN''s on a single SCSI device could be a bit trickier...

Or maybe LUN doesn''t mean what I think it means in this case... my
understanding is:

HBA = Host Bus Adapter (eg a PCI SCSI adapter)
SCSI Target Device = one of the physical devices that sits on the SCSI
bus (eg a tape drive or a disk drive)
LUN (Logical Unit Number) = a sub unit of the SCSI Target Device, eg a
single disk drive in a RAID array, a tape device in an multi tape drive
autoloader, a CD ROM in a CDROM tower.

...

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Stephen-san,

On Thu, 01 Nov 2007 12:14:41 +0000
"Stephen C. Tweedie" <sct@redhat.com>
wrote:
> On Tue, 2007-10-30 at 19:39 +0900, Jun Kamada wrote:
> 
> Is this using the same syntax for HBA reservation that we used to have,
> ie. simply enumerated using the scsi host number on the dom0?
Yes.
> If so, that is rather fragile.  The scsi host number can change from
> kernel to kernel depending on HBA driver probe order and PCI bus
> enumeration order.  It changes when hardware is rearranged.  And how do
> we deal with migrate, when we''ll need to select a completely
different
> HBA on a different host?
> 
> Selecting the HBA by scsi host number is good enough for a demonstration
> that the code works, but I don''t think it''s going to be
robust enough
> for production deployment.  
We also recognize that more sophisticated interface is needed. WWPN can
be used for FC-SCSI, however pure SCSI does not have any unique ID.
Do you have any idea?
> Also, has there been any progress towards exposing individual LUNs to
> the guest rather than a whole HBA?
> 
> Cheers, 
>  Stephen
> 
Jun Kamada
Linux Technology Development Div.
Server Systems Unit
Fujitsu Ltd.
kama@jp.fujitsu.com



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi James-san,

On Fri, 2 Nov 2007 11:23:50 +1100
"James Harper" <james.harper@bendigoit.com.au> wrote:
> > Hi,
> > 
> > On Tue, 2007-10-30 at 19:39 +0900, Jun Kamada wrote:
> > 
> > Is this using the same syntax for HBA reservation that we used to
> have,
> > ie. simply enumerated using the scsi host number on the dom0?
> > 
> > If so, that is rather fragile.  The scsi host number can change from
> > kernel to kernel depending on HBA driver probe order and PCI bus
> > enumeration order.  It changes when hardware is rearranged.  And how
> do
> > we deal with migrate, when we''ll need to select a completely
different
> > HBA on a different host?
> > 
> > Selecting the HBA by scsi host number is good enough for a
> demonstration
> > that the code works, but I don''t think it''s going to
be robust enough
> > for production deployment.
> > 
> > Also, has there been any progress towards exposing individual LUNs to
> > the guest rather than a whole HBA?
> 
> People keep saying individual LUN... do they actually mean SCSI Target
> devices? Splitting up devices on the same HBA should be reasonably
> simple, but due to the way the scsi protocol works (AFAIK), splitting up
> LUN''s on a single SCSI device could be a bit trickier...
The reason why we have to support SCSI on guest domain is that we would
like to use commercial appilications.
My understanding is as follows.

1.) There is a vender specific area in SCSI command frame. 
2.) There is a case that some data in the area depends on SCSI-ID, and
    some storage management software uses tha data. And sometimes the
    software requires one-to-one bindings to HBA. (This fuct is not good
    for standardization point of view.)


Possible solution may be 

a.) Assign a physical HBA for the guest domain the software, described
    in 2.), is running. And also multiple guest domains can share one
    physical HBA by using NPIV technology in this case.
b.) Assign LUNs for the other guests.

> Or maybe LUN doesn''t mean what I think it means in this case... my
> understanding is:
> 
> HBA = Host Bus Adapter (eg a PCI SCSI adapter)
> SCSI Target Device = one of the physical devices that sits on the SCSI
> bus (eg a tape drive or a disk drive)
> LUN (Logical Unit Number) = a sub unit of the SCSI Target Device, eg a
> single disk drive in a RAID array, a tape device in an multi tape drive
> autoloader, a CD ROM in a CDROM tower.
My understanding is same as yours.


Best regards,

-----
Jun Kamada



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stephen C. Tweedie wrote:
> Hi,
>
> On Tue, 2007-10-30 at 19:39 +0900, Jun Kamada wrote:
>
> Is this using the same syntax for HBA reservation that we used to have,
> ie. simply enumerated using the scsi host number on the dom0?
>
> If so, that is rather fragile.  The scsi host number can change from
> kernel to kernel depending on HBA driver probe order and PCI bus
> enumeration order.  It changes when hardware is rearranged.  And how do
> we deal with migrate, when we''ll need to select a completely
different
> HBA on a different host?
>
> Selecting the HBA by scsi host number is good enough for a demonstration
> that the code works, but I don''t think it''s going to be
robust enough
> for production deployment.  
>
> Also, has there been any progress towards exposing individual LUNs to
> the guest rather than a whole HBA?
>   
NPIV can help here for FibreChannel devices.  You can create a virtual 
NPIV adapter, and export exactly the LUNs/devices you want to that adapter.

It''s a different approach then exporting LUNs, but similar result.
> Cheers, 
>  Stephen
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi,

On Mon, 2007-11-05 at 11:05 +0900, Jun Kamada wrote:
> We also recognize that more sophisticated interface is needed. WWPN can
> be used for FC-SCSI, however pure SCSI does not have any unique ID.
> Do you have any idea?
Well... Linux does try to use address-by-content to some extent, but
that''s going to be fragile here, as well.  It''s complicated
even more by
the difficulty of having potential multipath in the mix.

libblkid has some helpful functions for certain cases, but that doesn''t
help at all for the bootstrap cases when you''re dealing with completely
unformatted disks.

The libvirt folks have been wrestling with some of the same questions
recently, libvirt-devel may have some useful discussion.

--Stephen



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Stephen-san,

Thank you for your helpful information. I will access the information.

Thanks,

On Thu, 08 Nov 2007 21:33:50 +0000
"Stephen C. Tweedie" <sct@redhat.com> wrote:
> Hi,
> 
> On Mon, 2007-11-05 at 11:05 +0900, Jun Kamada wrote:
> 
> > We also recognize that more sophisticated interface is needed. WWPN
can
> > be used for FC-SCSI, however pure SCSI does not have any unique ID.
> > Do you have any idea?
> 
> Well... Linux does try to use address-by-content to some extent, but
> that''s going to be fragile here, as well.  It''s
complicated even more by
> the difficulty of having potential multipath in the mix.
> 
> libblkid has some helpful functions for certain cases, but that
doesn''t
> help at all for the bootstrap cases when you''re dealing with
completely
> unformatted disks.
> 
> The libvirt folks have been wrestling with some of the same questions
> recently, libvirt-devel may have some useful discussion.
> 
> --Stephen
> 
> 
Jun Kamada
Linux Technology Development Div.
Server Systems Unit
Fujitsu Ltd.
kama@jp.fujitsu.com



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel