dtrace discuss - Feb 2009 - no Dtrace hooks for monitoring ddi put/get

There doesn''t seem to be any Dtrace hooks for monitoring ddi put/get. I
think this would be a great feature for developers as they would be able to
monitor hardware access at the register level. Is there any reason why these
hooks do not exist already?

(bofi is not really suitable because it requires that the driver is taken
off-line)
-- 
This message posted from opensolaris.org

J wrote:
> There doesn''t seem to be any Dtrace hooks for monitoring ddi
put/get.
Yes, it''s because they''re leaf routines without standard
prologues (for speed).
I''ve often wanted them too, but that''s why they''re
not instrumented.

J writes:
> There doesn''t seem to be any Dtrace hooks for monitoring ddi
put/get. I think this would be a great feature for developers as they would be
able to monitor hardware access at the register level. Is there any reason why
these hooks do not exist already?
> 
> (bofi is not really suitable because it requires that the driver is taken
off-line)
This is CR 4969919.

Yes, it''s extremely annoying when debugging drivers, and the RE has
suggested that it be closed as "will not fix."

-- 
James Carlson, Solaris Networking              <james.d.carlson at
sun.com>
Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677

On Thu, 12 Feb 2009, James Carlson wrote:
> J writes:
>> There doesn''t seem to be any Dtrace hooks for monitoring ddi
put/get. I
>> think this would be a great feature for developers as they would be 
>> able to monitor hardware access at the register level. Is there any 
>> reason why these hooks do not exist already?
>
> This is CR 4969919.
>
> Yes, it''s extremely annoying when debugging drivers, and the RE
has
> suggested that it be closed as "will not fix."
>
For the sun4v hypervisor hcalls, I''ve built special versions that use 
save/restore instead of leafs so I could simply dtrace them.  Could we add 
a build flag and extra code for low speed versions of ddi_put/get?

The people who really want it probably don''t mind switching kernels,
since
they''re probably running debug anyways.

-- Dave

David Finberg writes:
> On Thu, 12 Feb 2009, James Carlson wrote:
> 
> > J writes:
> >> There doesn''t seem to be any Dtrace hooks for monitoring
ddi put/get. I
> >> think this would be a great feature for developers as they would
be
> >> able to monitor hardware access at the register level. Is there
any
> >> reason why these hooks do not exist already?
> >
> > This is CR 4969919.
> >
> > Yes, it''s extremely annoying when debugging drivers, and the
RE has
> > suggested that it be closed as "will not fix."
> >
> 
> For the sun4v hypervisor hcalls, I''ve built special versions that
use
> save/restore instead of leafs so I could simply dtrace them.  Could we add 
> a build flag and extra code for low speed versions of ddi_put/get?
> 
> The people who really want it probably don''t mind switching
kernels, since
> they''re probably running debug anyways.
I think the argument against that idea is that the whole point of
dtrace is to be able to instrument a production system.  If you have
to rebuild the kernel, then you''ve wandered off the reservation.

I guess I don''t quite see why these special functions can''t be
instrumented deliberately with SDTs instead of relying on FBT.  It
should be possible even though this is assembly code.

Maybe there''s a good reason that wasn''t considered as an
option to
solve the underlying problem, and the RE just wants to close out the
bug instead.  It''d be helpful if he''d comment here, as I
don''t
understand enough of the issues, so I''ve cc''d him.  ;-}

(Instead of call records, as the RE''d hoped, what usually happens here
is that people waste a few hours re-discovering that there are some
functions that dtrace can''t see, grouse privately about it, and then
either write their own dirty little wrappers for the DDI functions, or
otherwise litter their code with private debug features.  We''re just
not very good at keeping track of these sorts of creeping problems
across the universe of developers.)

-- 
James Carlson, Solaris Networking              <james.d.carlson at
sun.com>
Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677

Hi,

James Carlson wrote:
> David Finberg writes:
>   
>> On Thu, 12 Feb 2009, James Carlson wrote:
>>
>>     
>>> J writes:
>>>       
>>>> There doesn''t seem to be any Dtrace hooks for
monitoring ddi put/get. I
>>>> think this would be a great feature for developers as they
would be
>>>> able to monitor hardware access at the register level. Is there
any
>>>> reason why these hooks do not exist already?
>>>>         
>>> This is CR 4969919.
>>>
>>> Yes, it''s extremely annoying when debugging drivers, and
the RE has
>>> suggested that it be closed as "will not fix."
>>>
>>>       
>> For the sun4v hypervisor hcalls, I''ve built special versions
that use
>> save/restore instead of leafs so I could simply dtrace them.  Could we
add
>> a build flag and extra code for low speed versions of ddi_put/get?
>>
>> The people who really want it probably don''t mind switching
kernels, since
>> they''re probably running debug anyways.
>>     
>
> I think the argument against that idea is that the whole point of
> dtrace is to be able to instrument a production system.  If you have
> to rebuild the kernel, then you''ve wandered off the reservation.
>
> I guess I don''t quite see why these special functions
can''t be
> instrumented deliberately with SDTs instead of relying on FBT.  It
> should be possible even though this is assembly code.
>
> Maybe there''s a good reason that wasn''t considered as an
option to
> solve the underlying problem, and the RE just wants to close out the
> bug instead.  It''d be helpful if he''d comment here, as I
don''t
> understand enough of the issues, so I''ve cc''d him.  ;-}
>
>   
I agree with James.  The mutex_enter code is a good place to look to see a
way to do this in assembler.  I don''t have access to source at the
minute,
but I think the lockstat provider replaces the ret at the end of mutex_enter
and simply falls into lockstat_wrapper code.  You wouldn''t catch the 
call into
the function, but you could see what is happening at the return.

max

>> There doesn''t seem to be any Dtrace hooks for monitoring ddi
put/get.
>
> Yes, it''s because they''re leaf routines without standard
prologues
> (for speed).
> I''ve often wanted them too, but that''s why
they''re not instrumented.
Is there actually a compelling reason for these to be so highly  
optimized?
Do drivers call these in the hot path or could we safely force them to  
be
instrumentable?

Adam

--
Adam Leventhal, Fishworks                        http://blogs.sun.com/ahl

Adam Leventhal wrote:
>>> There doesn''t seem to be any Dtrace hooks for monitoring
ddi put/get.
>>
>> Yes, it''s because they''re leaf routines without
standard prologues
>> (for speed).
>> I''ve often wanted them too, but that''s why
they''re not instrumented.
> 
> Is there actually a compelling reason for these to be so highly optimized?
> Do drivers call these in the hot path or could we safely force them to be
> instrumentable?
These routines are the lowest-level ddi-compliant way to do actual 
device I/O.  There are drivers that bypass them, of course, as there 
will always be, but for a compliant, portable driver, they are the 
"inner loop"; that''s why so much has been put into making
them as
lightweight as possible.

hotpatching might well be a reasonable option.