Lustre discuss - Mar 2007 - OST_IO_PORTAL / OST_REQUEST_PORTAL ?!?

Hi,

Since the last documentation, OST_IO_PORTAL has been created and has 
replaced OST_REQUEST_PORTAL 6, which became OST_REQUEST_PORTAL 28.

What is the purpose of OST_IO_PORTAL ?
How does it work ? ( opcodes associated, ...., how many buffers.. with 
which information, etc..)

What is the new purpose of OST_REQUEST_PORTAL ?

Regards,

Thibault DONADINI

On Mar 05, 2007  14:22 +0100, Donadini Thibault wrote:
> Since the last documentation, OST_IO_PORTAL has been created and has 
> replaced OST_REQUEST_PORTAL 6, which became OST_REQUEST_PORTAL 28.
> 
> What is the purpose of OST_IO_PORTAL ?
> How does it work ? ( opcodes associated, ...., how many buffers.. with 
> which information, etc..)
> 
> What is the new purpose of OST_REQUEST_PORTAL ?
The reason for this change is that small/fast RPC requests like getattr
were being blocked behind many GB of IO RPCs and this was causing
responsiveness problems.

For reasons of compatibility, the service threads running on portal #6
were kept for doing the large IO requests (because they have pre-allocated
1MB buffers for doing IO without allocations).  The small request RPCs
were moved to a new portal (#28) if both the client and server understand
the OBD_CONNECT_REQPORTAL feature.  Both services handle the same RPC
requests over the wire.

The same is true for the MDS_*_PORTAL - the format of the requests and
the back-end handling code is exactly the same, but they are split into
different portals because of latency or deadlock-avoidance issues.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Andreas Dilger a ?crit :
> On Mar 05, 2007  14:22 +0100, Donadini Thibault wrote:
>   
>> Since the last documentation, OST_IO_PORTAL has been created and has 
>> replaced OST_REQUEST_PORTAL 6, which became OST_REQUEST_PORTAL 28.
>>
>> What is the purpose of OST_IO_PORTAL ?
>> How does it work ? ( opcodes associated, ...., how many buffers.. with 
>> which information, etc..)
>>
>> What is the new purpose of OST_REQUEST_PORTAL ?
>>     
>
> The reason for this change is that small/fast RPC requests like getattr
> were being blocked behind many GB of IO RPCs and this was causing
> responsiveness problems.
>
> For reasons of compatibility, the service threads running on portal #6
> were kept for doing the large IO requests (because they have pre-allocated
> 1MB buffers for doing IO without allocations).  The small request RPCs
> were moved to a new portal (#28) if both the client and server understand
> the OBD_CONNECT_REQPORTAL feature.  Both services handle the same RPC
> requests over the wire.
>
> The same is true for the MDS_*_PORTAL - the format of the requests and
> the back-end handling code is exactly the same, but they are split into
> different portals because of latency or deadlock-avoidance issues.
>
> Cheers, Andreas
> --
> Andreas Dilger
> Principal Software Engineer
> Cluster File Systems, Inc.
>
>
>   
Ok thank you, but what do you want to say with :

Both services handle the same RPC
requests over the wire.

Because if you speak about "opcodes", i didn''t get the same
for
OST_REQUEST_PORTAL ( i got OST_PUNCH, LDLM_ENQUEUE) and OST_IO_PORTAL ( 
i got OST_WRITE,OST_READ) in tests i did.

Moreover, what do you mean with MDS_*_PORTAL, i didnt see very well .. 
because there isn''t MDS_IO_PORTAL ?

Thibault DONADINI

On Mar 05, 2007  16:39 +0100, Donadini Thibault wrote:
> Andreas Dilger a ?crit :
> >For reasons of compatibility, the service threads running on portal #6
> >were kept for doing the large IO requests (because they have
pre-allocated
> >1MB buffers for doing IO without allocations).  The small request RPCs
> >were moved to a new portal (#28) if both the client and server
understand
> >the OBD_CONNECT_REQPORTAL feature.  Both services handle the same RPC
> >requests over the wire.
> >
> >The same is true for the MDS_*_PORTAL - the format of the requests and
> >the back-end handling code is exactly the same, but they are split into
> >different portals because of latency or deadlock-avoidance issues.
>
> what do you want to say with :
> 
> Both services handle the same RPC
> requests over the wire.
> 
> Because if you speak about "opcodes", i didn''t get the
same for
> OST_REQUEST_PORTAL ( i got OST_PUNCH, LDLM_ENQUEUE) and OST_IO_PORTAL ( 
> i got OST_WRITE,OST_READ) in tests i did.
If you are running, say, 1.4.6 clients they don''t know anything about
portal 28, and will send all OST opcodes to portal 6.  The same RPC
handler function (ost_handler()) is handling requests on both portals,
so the RPC decoding code may as well treat them identically.  See
ost_setup() for the ptlrpc_init_svc() calls for the handler registers. 
> Moreover, what do you mean with MDS_*_PORTAL, i didnt see very well .. 
> because there isn''t MDS_IO_PORTAL ?
There is the MDS_REQUEST_PORTAL, MDS_SETATTR_PORTAL, MDS_READPAGE_PORTAL.
The clients will normally send different request opcodes to those portals,
but in fact the back-end mds_handler() is the same for all of them.  See
mdt_setup() for the handler setup.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.