crossbow discuss - Feb 2009 - relaxing dlpi_enabnotify() restrictions

Folks,

Currently, dlpi_enabnotify(3DLPI) requires that the DLPI stream be DL_IDLE
(i.e., bound to a SAP) to succeed.  IIRC, this was done because there were
known to be some non-GLDv3 drivers that (erroneously) required the stream
to be DL_IDLE, and Clearview UV was still under development.

However, looking at the current codebase, softmac internally deals with
this issue (see softmac_lower_setup()), and indeed GLDv3 allows
DL_NOTIFY_REQ to work as long as the stream is attached.  Further, the
restriction in dlpi_enabnotify() impacts performance, as applications
(such as in.mpathd) must issue dlpi_bind() on a DLPI endpoint they have no
intention of receiving packets on.  This in turn leads to optimizations
inside GLDv3 being disabled.

As such, I''d like to remove this restriction.  I''ve done some
basic
testing and it seems to work fine.  If I don''t hear any objections by
COB Monday, I''ll proceed with a closed-approved-automatic PSARC case.

Thanks,
-- 
meem

On 2009?02?21? 13:05, Peter Memishian wrote:
> Folks,
> 
> Currently, dlpi_enabnotify(3DLPI) requires that the DLPI stream be DL_IDLE
> (i.e., bound to a SAP) to succeed.  IIRC, this was done because there were
> known to be some non-GLDv3 drivers that (erroneously) required the stream
> to be DL_IDLE, and Clearview UV was still under development.
> 
> However, looking at the current codebase, softmac internally deals with
> this issue (see softmac_lower_setup()),
As we discussed offline, this will be soon changed by the UV fastpath 
project. The DL_BIND_REQ will be defered until 
mac_start()->softmac_m_start() so that we will not always have a DL_IDLE 
shared-lower-stream which would definitely hurt the performance.

At the very least, we''d need to file a bug against ce.

Thanks
- Cathy

  and indeed GLDv3 allows
> DL_NOTIFY_REQ to work as long as the stream is attached.  Further, the
> restriction in dlpi_enabnotify() impacts performance, as applications
> (such as in.mpathd) must issue dlpi_bind() on a DLPI endpoint they have no
> intention of receiving packets on.  This in turn leads to optimizations
> inside GLDv3 being disabled.
> 
> As such, I''d like to remove this restriction.  I''ve done
some basic
> testing and it seems to work fine.  If I don''t hear any objections
by
> COB Monday, I''ll proceed with a closed-approved-automatic PSARC
case.
> 
> Thanks,

> As we discussed offline, this will be soon changed by the UV fastpath 
 > project. The DL_BIND_REQ will be defered until 
 > mac_start()->softmac_m_start() so that we will not always have a
DL_IDLE
 > shared-lower-stream which would definitely hurt the performance.
 > 
 > At the very least, we''d need to file a bug against ce.

I''ve filed 6809576 and 6809577 against ce and gem respectively.  I
presume
we can propagate the error from ce/gem up to the caller?  This would at
least allow the caller to try the dlpi_enabnotify() without doing a
dlpi_bind(), and if that fails they can do the dlpi_bind() and reissue the
dlpi_enabnotify(), which means we only have the performance issue with
ce/gem (until the bug is fixed) as opposed to with every driver.

-- 
meem

On 2009?02?24? 14:11, Peter Memishian wrote:
>  > As we discussed offline, this will be soon changed by the UV fastpath
>  > project. The DL_BIND_REQ will be defered until 
>  > mac_start()->softmac_m_start() so that we will not always have a
DL_IDLE
>  > shared-lower-stream which would definitely hurt the performance.
>  > 
>  > At the very least, we''d need to file a bug against ce.
> 
> I''ve filed 6809576 and 6809577 against ce and gem respectively.  I
presume
> we can propagate the error from ce/gem up to the caller?  
Not right now. I guess we could add some mac driver functions (or another 
softmac legacy capability) to get such information and returns error in 
proto_notify_req(). I don''t know whether it worth the code to work
around
the bugs in the driver.

Thanks
- Cathy
> This would at
> least allow the caller to try the dlpi_enabnotify() without doing a
> dlpi_bind(), and if that fails they can do the dlpi_bind() and reissue the
> dlpi_enabnotify(), which means we only have the performance issue with
> ce/gem (until the bug is fixed) as opposed to with every driver.
>

> >  > At the very least, we''d need to file a bug against ce.
 > > 
 > > I''ve filed 6809576 and 6809577 against ce and gem
respectively.  I presume
 > > we can propagate the error from ce/gem up to the caller?  
 > 
 > Not right now. I guess we could add some mac driver functions (or another 
 > softmac legacy capability) to get such information and returns error in 
 > proto_notify_req(). I don''t know whether it worth the code to
work around
 > the bugs in the driver.

So in general DLPI requests that are passed through softmac will be
successful if softmac/GLDv3 likes them, even if the underlying driver
does not like them?  That seems problematic.

--
meem

On 2009?02?24? 15:25, Peter Memishian wrote:
>  > >  > At the very least, we''d need to file a bug
against ce.
>  > > 
>  > > I''ve filed 6809576 and 6809577 against ce and gem
respectively.  I presume
>  > > we can propagate the error from ce/gem up to the caller?  
>  > 
>  > Not right now. I guess we could add some mac driver functions (or
another
>  > softmac legacy capability) to get such information and returns error
in
>  > proto_notify_req(). I don''t know whether it worth the code
to work around
>  > the bugs in the driver.
> 
> So in general DLPI requests that are passed through softmac will be
> successful if softmac/GLDv3 likes them, even if the underlying driver
> does not like them?  That seems problematic.
> 
Softmac does propogate the notification types the underlying driver does 
not support, but that information is got when DL_BIND_REQ is sent to the 
underlying driver during the softmac discovery time. And today, the 
NOTIFY_IND will only start to be sent upstream when the softmac is started.

Yes, I admit that it is problematic. I just don''t know whether it is
worth
the complexity to consider those buggy legacy drivers, or I should simply 
try to do the right thing now - Send NOTIFY_REQ even when the underlying 
driver is not bound, if that fails, it simply means the softmac does not 
support such notifications.

- Cathy

> Yes, I admit that it is problematic. I just don''t know whether it
is worth
 > the complexity to consider those buggy legacy drivers, or I should simply 
 > try to do the right thing now - Send NOTIFY_REQ even when the underlying 
 > driver is not bound, if that fails, it simply means the softmac does not 
 > support such notifications.

Doing the right thing now seems a good idea.

-- 
meem

On 2009?02?24? 15:50, Peter Memishian wrote:
>  > Yes, I admit that it is problematic. I just don''t know
whether it is worth
>  > the complexity to consider those buggy legacy drivers, or I should
simply
>  > try to do the right thing now - Send NOTIFY_REQ even when the
underlying
>  > driver is not bound, if that fails, it simply means the softmac does
not
>  > support such notifications.
> 
> Doing the right thing now seems a good idea.
> 
I talked to Meem offline, and here is the summary:

I will talk to Joyce to see whether the two bugs 6809576 and 6809577 are 
easy to fix, if so, I will change the softmac to send NOTIFY_REQ when the 
underlying driver is not bound. Note that after this change and before the 
two driver bugs are being fixed, ce and gem will not be able to support 
NOTIFY_IND at all, and IPMP over them won''t be able to detect duplicate
addresses.

If fixing the two bugs involves more effort, I will include the logic in 
softmac to detect whether the underlying driver has to be DL_IDLE to 
support DL_NOTIFY_REQ, and will return error_ack when the driver is not 
DL_IDLE, so that IPMP can do a BIND_REQ and retry.

Thanks
- Cathy