dtrace discuss - Oct 2005 - dtrace: failed to initialize dtrace: DTrace device not available on system

I have a number of systems running solaris10 and i see the package and binary
for dtrace installed however whenever we try to run anything we get this error

dtrace: failed to initialize dtrace: DTrace device not available on system

the only system in which i dont have this error is the development server that
has the full solaris 10 install while others are minimized, do i need additional
packages besides
system      SUNWdtrc                         DTrace Clients
system      SUNWdtrp                         DTrace Providers

The minimized versions are the core with some added packages as needed but these
server do not create the devices found under /devices/pseudo/ as you can see via
a truss on a dtrace.

open("/devices/pseudo/fasttrap at 0:fasttrap", O_RDONLY) Err#2 ENOENT
open("/devices/pseudo/fbt at 0:fbt", O_RDONLY)     Err#2 ENOENT
open("/devices/pseudo/lockstat at 0:lockstat", O_RDONLY) Err#2 ENOENT
open("/devices/pseudo/profile at 0:profile", O_RDONLY) Err#2 ENOENT
open("/devices/pseudo/sdt at 0:sdt", O_RDONLY)     Err#2 ENOENT
open("/devices/pseudo/systrace at 0:systrace", O_RDONLY) Err#2 ENOENT


And i confirm these devices dont exist in any of the production or staging
servers which have the minimized set of packages.

anyone have an idea what we have setup wrong?
and when i run a truss on dtrace i see it errors with not found for all the
devices.


thanks
This message posted from opensolaris.org

Hi Gary,

That''s awfully strange -- is this a stock installation of Solaris 10?
Can you confirm that the DTrace pseudo drivers are loaded by looking
in /kernel/drv and /kernel/dtrace for the fasttrap binary. This is what I
see:

$ find /kernel/drv /kernel/dtrace -name fasttrap
/kernel/drv/amd64/fasttrap
/kernel/drv/fasttrap
/kernel/dtrace/amd64/fasttrap
/kernel/dtrace/fasttrap

Can you also see that the providers are listed in /etc/name_to_major:

$ grep fasttrap /etc/name_to_major
fasttrap 167

Thanks.

Adam

On Tue, Oct 11, 2005 at 10:37:24AM -0700, gary wrote:
> I have a number of systems running solaris10 and i see the package and
binary for dtrace installed however whenever we try to run anything we get this
error
> 
> dtrace: failed to initialize dtrace: DTrace device not available on system
> 
> the only system in which i dont have this error is the development server
that has the full solaris 10 install while others are minimized, do i need
additional packages besides
> system      SUNWdtrc                         DTrace Clients
> system      SUNWdtrp                         DTrace Providers
> 
> The minimized versions are the core with some added packages as needed but
these server do not create the devices found under /devices/pseudo/ as you can
see via a truss on a dtrace.
> 
> open("/devices/pseudo/fasttrap at 0:fasttrap", O_RDONLY) Err#2
ENOENT
> open("/devices/pseudo/fbt at 0:fbt", O_RDONLY)     Err#2 ENOENT
> open("/devices/pseudo/lockstat at 0:lockstat", O_RDONLY) Err#2
ENOENT
> open("/devices/pseudo/profile at 0:profile", O_RDONLY) Err#2
ENOENT
> open("/devices/pseudo/sdt at 0:sdt", O_RDONLY)     Err#2 ENOENT
> open("/devices/pseudo/systrace at 0:systrace", O_RDONLY) Err#2
ENOENT
> 
> 
> And i confirm these devices dont exist in any of the production or staging
servers which have the minimized set of packages.
> 
> anyone have an idea what we have setup wrong?
> and when i run a truss on dtrace i see it errors with not found for all the
devices.
> 
> 
> thanks
> This message posted from opensolaris.org
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org
-- 
Adam Leventhal, Solaris Kernel Development       http://blogs.sun.com/ahl

I have the exact same experience and same output from truss running dtrace -l
after adding SUNWdtrc and SUNWdtrp to a core Solaris install.  I did verify that
all the dependant packages are on the system.

To answer Adam''s questions:
my output is identical to yours for the find and grep tests you supplied.

Do I need to reboot the system- since it is in production at remote host that is
highly frowned upon, but will do it if necessary.

-John
This message posted from opensolaris.org

Hi John,

You should be able to use add_drv to load the DTrace modules.

Adam

On Thu, Nov 10, 2005 at 09:57:00AM -0800, John Hoffmann
wrote:
> I have the exact same experience and same output from truss running dtrace
-l after adding SUNWdtrc and SUNWdtrp to a core Solaris install.  I did verify
that all the dependant packages are on the system.
> 
> To answer Adam''s questions:
> my output is identical to yours for the find and grep tests you supplied.
> 
> Do I need to reboot the system- since it is in production at remote host
that is highly frowned upon, but will do it if necessary.
> 
> -John
> This message posted from opensolaris.org
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org
-- 
Adam Leventhal, Solaris Kernel Development       http://blogs.sun.com/ahl

> I have the exact same experience and same output from truss running dtrace
-l
> after adding SUNWdtrc and SUNWdtrp to a core Solaris install.  I did verify
> that all the dependant packages are on the system.
>
> To answer Adam''s questions:
> my output is identical to yours for the find and grep tests you supplied.
>
> Do I need to reboot the system- since it is in production at remote host
> that is highly frowned upon, but will do it if necessary.
>
> -John
If you added the packages after install, and haven''t yet rebooted the
system,
then you need to tell the kernel to load the driver which is now present.
Try doing this:

# devfsadm -v -i dtrace
# modinfo | grep dtrace
  5 fffffffff411c000  24c40 155   1  dtrace (Dynamic Tracing)

If you see errors from devfsadm or you don''t see DTrace show up in the
modinfo output, let us know what you get.  If you see that it successfully
loaded, then things should begin working at that point.  If not, let
us know what you see.

-Mike

-- 
Mike Shapiro, Solaris Kernel Development. blogs.sun.com/mws/

Thanks Mike, I got no errors and the modinfo reports:
  5 ffffffffef902000  15f10 155   1  dtrace (Dynamic Tracing)

However, when I do /usr/sbin/dtrace -l, I only see three probes:

cwp-ptl-id3# /usr/sbin/dtrace -l
   ID   PROVIDER            MODULE                          FUNCTION NAME
    1     dtrace                                                     BEGIN
    2     dtrace                                                     END
    3     dtrace                                                     ERROR

And my dscript fails with:

: probe description vminfo:::execpgin does not match any probes

Do I need to do more along the lines of what Adam suggested about add_drv ?
Is there a step in greenline I can invoke to load them all?

Hi John,

You should be able to use add_drv to load the DTrace modules.

Adam

---------------------
Mike Shapiro wrote:

If you added the packages after install, and haven''t yet rebooted the
system,
then you need to tell the kernel to load the driver which is now present.
Try doing this:

# devfsadm -v -i dtrace
# modinfo | grep dtrace
  5 fffffffff411c000  24c40 155   1  dtrace (Dynamic Tracing)

If you see errors from devfsadm or you don''t see DTrace show up in the
modinfo output, let us know what you get.  If you see that it successfully
loaded, then things should begin working at that point.  If not, let
us know what you see.

-Mike

-- Mike Shapiro, Solaris Kernel Development. blogs.sun.com/mws/
This message posted from opensolaris.org

> Thanks Mike, I got no errors and the modinfo reports:
>   5 ffffffffef902000  15f10 155   1  dtrace (Dynamic Tracing)
> 
> However, when I do /usr/sbin/dtrace -l, I only see three probes:
> 
> cwp-ptl-id3# /usr/sbin/dtrace -l
>    ID   PROVIDER            MODULE                          FUNCTION NAME
>     1     dtrace                                                     BEGIN
>     2     dtrace                                                     END
>     3     dtrace                                                     ERROR
> 
> And my dscript fails with:
> 
> : probe description vminfo:::execpgin does not match any probes
> 
> Do I need to do more along the lines of what Adam suggested about add_drv ?
> Is there a step in greenline I can invoke to load them all?
Ok, super: you''ve got it working now.  Now you just need to load
the rest of your DTrace provider modules, and then all their probes
will show up.  Do as follows:

# devfsadm -v -i fasttrap
# devfsadm -v -i fbt
# devfsadm -v -i lockstat
# devfsadm -v -i profile
# devfsadm -v -i sdt
# devfsadm -v -i systrace

The one-shot way of doing this is just "devfsadm", but this is a bit
lighter-weight and allows us to be sure we''re catching any issues.

-Mike
 
-- 
Mike Shapiro, Solaris Kernel Development. blogs.sun.com/mws/

Perfect.  dvmstat.d is running great.

FYI, there was only one line of output from adding the providers:

# devfsadm -v -i fasttrap
# devfsadm -v -i fbt
# devfsadm -v -i lockstat
devfsadm[9685]: verbose: symlink /dev/lockstat -> ../devices/pseudo/lockstat
at 0:lockstat
# devfsadm -v -i profile
# devfsadm -v -i sdt
# devfsadm -v -i systrace

Thanks for all your help.
This message posted from opensolaris.org