The Solaris Dtrace guide says "The most common use of anonymous tracing is to permit device driver developers to debug and trace activity that occurs during system boot" The anonymous tracing worked fine if my device is not the boot device. What if the device driver I want to debug using anonymous tracing happens to be the driver that control the boot device? So I tried to create an anonymous enabling following the Dtrace guide and /etc/system was modified to have some necessary forceload statements added. The problem is that the dtrace driver was loaded way after my driver was loaded and attached and did its work. Obviously forceload xxx in /etc/system for anonymous tracing is not enough. I guess I have to make my boot device driver dependent on all these modules(driver) using -dy -N to make sure they are all loaded before my HBA driver got loaded. Any suggestions? -- This message posted from opensolaris.org
Qiang Liu wrote:> The Solaris Dtrace guide says "The most common use of anonymous tracing is to permit device driver developers to debug and trace activity that occurs during system boot" > The anonymous tracing worked fine if my device is not the boot device. > > What if the device driver I want to debug using anonymous tracing happens to be the driver that control the boot device? So I tried to create an anonymous enabling following the Dtrace guide and /etc/system was modified to have some necessary forceload statements added. > > The problem is that the dtrace driver was loaded way after my driver was loaded and attached and did its work. Obviously forceload xxx in /etc/system for anonymous tracing is not enough. I guess I have to make my boot device driver dependent on all these modules(driver) using -dy -N to make sure they are all loaded before my HBA driver got loaded. > > Any suggestions?probably can''t do that.
I can make the dtrace related modules (drivers) loaded before boot device
driver, but dtrace probing only happens after dtrace related pseudo devices got
created.
"The most common use of anonymous tracing is to permit device driver
developers to debug and trace activity that occurs during system boot"
--- Here ***during system boot*** actually means ever since the time *after*
dtrace probes are enabled during system boot.
"To allow tracing during boot, the dtrace(7D) driver must be loaded as
early as possible. dtrace adds the necessary forceload statements to /etc/system
for each required DTrace provider and for dtrace(7D) itself"
--- Here ***as early as possible*** does not mean too much at least to those who
want to trace the behavior of boot device driver.
any comments will be appreciated.
----------------------
elfdump of my boot device driver.
[0] NEEDED 0x1 misc/md5
[1] NEEDED 0xa misc/sha1
[2] NEEDED 0x14 misc/bignum
[3] NEEDED 0x20 drv/systrace
[4] NEEDED 0x2d drv/sdt
[5] NEEDED 0x35 drv/profile
[6] NEEDED 0x41 drv/lx_systrace
[7] NEEDED 0x51 drv/lockstat
[8] NEEDED 0x5e drv/fbt
[9] NEEDED 0x66 drv/fasttrap
[10] NEEDED 0x73 drv/dtrace
...
load ''misc/kcf'' id 51 loaded @
0xfffffffff79e9000/0xfffffffffbdc7e08 size 155736/3160
load ''misc/md5'' id 50 loaded @
0xfffffffff79e6000/0xfffffffffbdc7b30 size 10528/728
load ''misc/sha1'' id 52 loaded @
0xfffffffff7a10000/0xfffffffffbdd18e0 size 12880/728
load ''misc/bignum'' id 53 loaded @
0xfffffffff7a14000/0xfffffffffbdd1bb8 size 29712/280
load ''drv/dtrace'' id 7 loaded @
0xfffffffff7a1c000/0xfffffffffbdd1e80 size 116728/992
load ''drv/systrace'' id 54 loaded @
0xfffffffff7a1b410/0xfffffffffbdd1cd0 size 2768/432
load ''drv/sdt'' id 55 loaded @
0xfffffffff7a39000/0xfffffffffbdd2280 size 9416/21096
load ''drv/profile'' id 56 loaded @
0xfffffffff7a3c000/0xfffffffffbdd74e8 size 4376/600
load ''exec/elfexec'' id 59 loaded @
0xfffffffff7a46000/0xfffffffffbddaa48 size 42944/328
load ''brand/lx_brand'' id 58 loaded @
0xfffffffff7a3e000/0xfffffffffbdd78f8 size 31016/12624
load ''drv/lx_systrace'' id 57 loaded @
0xfffffffff7956670/0xfffffffffbdd7740 size 2304/440
load ''drv/lockstat'' id 60 loaded @
0xfffffffff7a387f8/0xfffffffffbddabb0 size 1928/992
load ''drv/fbt'' id 61 loaded @
0xfffffffff7a51000/0xfffffffffbddaf90 size 4264/456
load ''drv/fasttrap'' id 62 loaded @
0xfffffffff7a53000/0xfffffffffbddb158 size 23736/664
load ''drv/emlxs'' id 49 loaded @
0xfffffffff7957000/0xfffffffffbdb5f90 size 584832/72608 <----------- my boot
device driver
....
installing emlxs, module id 49.
installing dtrace, module id 7.
installing md5, module id 50.
installing kcf, module id 51.
installing sha1, module id 52.
installing bignum, module id 53.
installing systrace, module id 54.
installing sdt, module id 55.
installing profile, module id 56.
installing lx_systrace, module id 57.
installing lx_brand, module id 58.
installing elfexec, module id 59.
installing lockstat, module id 60.
installing fbt, module id 61.
installing fasttrap, module id 62.
load ''misc/fctl'' id 63 loaded @
0xfffffffff7a59000/0xfffffffffbdde1e4 size 41240/3384
installing fctl, module id 63.
....
Jan 22 10:37:18 sfx4600 genunix: [ID 308332 kern.notice] root on /scsi_vhci/disk
at g20000011c6818e6c:a fstype ufs
Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load
''fs/ctfs'' id 69 loaded @ 0xfffffffff7aeb000/0xfffffffffbde2101
size 16040/888
Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing ctfs, module
id 69.
Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load
''fs/procfs'' id 70 loaded @
0xfffffffff7aef000/0xfffffffffbde24d9 size 132488/268
8
Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing procfs,
module id 70.
Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load
''fs/mntfs'' id 71 loaded @
0xfffffffff7b10000/0xfffffffffbde2f6d size 12024/176
Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing mntfs,
module id 71.
Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load
''fs/tmpfs'' id 72 loaded @
0xfffffffff7b13000/0xffffffffc0000000 size 29408/66432
Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing tmpfs,
module id 72.
Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load
''fs/objfs'' id 73 loaded @
0xfffffffff7b1b000/0xffffffffc00103a8 size 9488/1000
Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing objfs,
module id 73.
Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load
''fs/sharefs'' id 74 loaded @
0xfffffffff7b1e000/0xffffffffc00107b0 size 9344/1344
Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing sharefs,
module id 74.
Jan 22 10:37:18 sfx4600 unix: [ID 190185 kern.notice] SMBIOS v2.3 loaded (3938
bytes)
Jan 22 10:37:18 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: systrace0
Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] systrace0 is
/pseudo/systrace at 0
Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: sdt0
Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] sdt0 is /pseudo/sdt at
0
Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: profile0
Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] profile0 is
/pseudo/profile at 0
Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device:
lx_systrace0
Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] lx_systrace0 is
/pseudo/lx_systrace at 0
Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: lockstat0
Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] lockstat0 is
/pseudo/lockstat at 0
Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: fbt0
Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] fbt0 is /pseudo/fbt at
0
Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: fasttrap0
Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] fasttrap0 is
/pseudo/fasttrap at 0
Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: dtrace0
Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] dtrace0 is
/pseudo/dtrace at 0
Jan 22 10:37:19 sfx4600 dtrace: [ID 566105 kern.notice] NOTICE: enabling probe 0
(fbt::emlxs_attach:entry)
Jan 22 10:37:19 sfx4600 dtrace: [ID 566105 kern.notice] NOTICE: enabling probe 1
(fbt:pcplusmp::)
Jan 22 10:37:19 sfx4600 dtrace: [ID 566105 kern.notice] NOTICE: enabling probe 2
(fbt::emlxs_attach:return)
Jan 22 10:37:19 sfx4600 dtrace: [ID 566105 kern.notice] NOTICE: enabling probe 3
(dtrace:::ERROR)
.....
--
This message posted from opensolaris.org
Qiang Liu wrote:> I can make the dtrace related modules (drivers) loaded before boot device driver, but dtrace probing only happens after dtrace related pseudo devices got created. > > "The most common use of anonymous tracing is to permit device driver developers to debug and trace activity that occurs during system boot" > > --- Here ***during system boot*** actually means ever since the time *after* dtrace probes are enabled during system boot. > > "To allow tracing during boot, the dtrace(7D) driver must be loaded as early as possible. dtrace adds the necessary forceload statements to /etc/system for each required DTrace provider and for dtrace(7D) itself" > > --- Here ***as early as possible*** does not mean too much at least to those who want to trace the behavior of boot device driver. > > any comments will be appreciated.Yes. I believe you are correct. kmdb exists.> > ---------------------- > > elfdump of my boot device driver. > > [0] NEEDED 0x1 misc/md5 > [1] NEEDED 0xa misc/sha1 > [2] NEEDED 0x14 misc/bignum > [3] NEEDED 0x20 drv/systrace > [4] NEEDED 0x2d drv/sdt > [5] NEEDED 0x35 drv/profile > [6] NEEDED 0x41 drv/lx_systrace > [7] NEEDED 0x51 drv/lockstat > [8] NEEDED 0x5e drv/fbt > [9] NEEDED 0x66 drv/fasttrap > [10] NEEDED 0x73 drv/dtrace > ... > load ''misc/kcf'' id 51 loaded @ 0xfffffffff79e9000/0xfffffffffbdc7e08 size 155736/3160 > load ''misc/md5'' id 50 loaded @ 0xfffffffff79e6000/0xfffffffffbdc7b30 size 10528/728 > load ''misc/sha1'' id 52 loaded @ 0xfffffffff7a10000/0xfffffffffbdd18e0 size 12880/728 > load ''misc/bignum'' id 53 loaded @ 0xfffffffff7a14000/0xfffffffffbdd1bb8 size 29712/280 > load ''drv/dtrace'' id 7 loaded @ 0xfffffffff7a1c000/0xfffffffffbdd1e80 size 116728/992 > load ''drv/systrace'' id 54 loaded @ 0xfffffffff7a1b410/0xfffffffffbdd1cd0 size 2768/432 > load ''drv/sdt'' id 55 loaded @ 0xfffffffff7a39000/0xfffffffffbdd2280 size 9416/21096 > load ''drv/profile'' id 56 loaded @ 0xfffffffff7a3c000/0xfffffffffbdd74e8 size 4376/600 > load ''exec/elfexec'' id 59 loaded @ 0xfffffffff7a46000/0xfffffffffbddaa48 size 42944/328 > load ''brand/lx_brand'' id 58 loaded @ 0xfffffffff7a3e000/0xfffffffffbdd78f8 size 31016/12624 > load ''drv/lx_systrace'' id 57 loaded @ 0xfffffffff7956670/0xfffffffffbdd7740 size 2304/440 > load ''drv/lockstat'' id 60 loaded @ 0xfffffffff7a387f8/0xfffffffffbddabb0 size 1928/992 > load ''drv/fbt'' id 61 loaded @ 0xfffffffff7a51000/0xfffffffffbddaf90 size 4264/456 > load ''drv/fasttrap'' id 62 loaded @ 0xfffffffff7a53000/0xfffffffffbddb158 size 23736/664 > load ''drv/emlxs'' id 49 loaded @ 0xfffffffff7957000/0xfffffffffbdb5f90 size 584832/72608 <----------- my boot device driver > .... > installing emlxs, module id 49. > installing dtrace, module id 7. > installing md5, module id 50. > installing kcf, module id 51. > installing sha1, module id 52. > installing bignum, module id 53. > installing systrace, module id 54. > installing sdt, module id 55. > installing profile, module id 56. > installing lx_systrace, module id 57. > installing lx_brand, module id 58. > installing elfexec, module id 59. > installing lockstat, module id 60. > installing fbt, module id 61. > installing fasttrap, module id 62. > load ''misc/fctl'' id 63 loaded @ 0xfffffffff7a59000/0xfffffffffbdde1e4 size 41240/3384 > installing fctl, module id 63. > .... > Jan 22 10:37:18 sfx4600 genunix: [ID 308332 kern.notice] root on /scsi_vhci/disk at g20000011c6818e6c:a fstype ufs > Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load ''fs/ctfs'' id 69 loaded @ 0xfffffffff7aeb000/0xfffffffffbde2101 size 16040/888 > Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing ctfs, module id 69. > Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load ''fs/procfs'' id 70 loaded @ 0xfffffffff7aef000/0xfffffffffbde24d9 size 132488/268 > 8 > Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing procfs, module id 70. > Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load ''fs/mntfs'' id 71 loaded @ 0xfffffffff7b10000/0xfffffffffbde2f6d size 12024/176 > Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing mntfs, module id 71. > Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load ''fs/tmpfs'' id 72 loaded @ 0xfffffffff7b13000/0xffffffffc0000000 size 29408/66432 > Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing tmpfs, module id 72. > Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load ''fs/objfs'' id 73 loaded @ 0xfffffffff7b1b000/0xffffffffc00103a8 size 9488/1000 > Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing objfs, module id 73. > Jan 22 10:37:18 sfx4600 genunix: [ID 943528 kern.notice] load ''fs/sharefs'' id 74 loaded @ 0xfffffffff7b1e000/0xffffffffc00107b0 size 9344/1344 > Jan 22 10:37:18 sfx4600 genunix: [ID 131579 kern.notice] installing sharefs, module id 74. > Jan 22 10:37:18 sfx4600 unix: [ID 190185 kern.notice] SMBIOS v2.3 loaded (3938 bytes) > Jan 22 10:37:18 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: systrace0 > Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] systrace0 is /pseudo/systrace at 0 > Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: sdt0 > Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] sdt0 is /pseudo/sdt at 0 > Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: profile0 > Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] profile0 is /pseudo/profile at 0 > Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: lx_systrace0 > Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] lx_systrace0 is /pseudo/lx_systrace at 0 > Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: lockstat0 > Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] lockstat0 is /pseudo/lockstat at 0 > Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: fbt0 > Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] fbt0 is /pseudo/fbt at 0 > Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: fasttrap0 > Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] fasttrap0 is /pseudo/fasttrap at 0 > Jan 22 10:37:19 sfx4600 pseudo: [ID 129642 kern.notice] pseudo-device: dtrace0 > Jan 22 10:37:19 sfx4600 genunix: [ID 936769 kern.notice] dtrace0 is /pseudo/dtrace at 0 > Jan 22 10:37:19 sfx4600 dtrace: [ID 566105 kern.notice] NOTICE: enabling probe 0 (fbt::emlxs_attach:entry) > Jan 22 10:37:19 sfx4600 dtrace: [ID 566105 kern.notice] NOTICE: enabling probe 1 (fbt:pcplusmp::) > Jan 22 10:37:19 sfx4600 dtrace: [ID 566105 kern.notice] NOTICE: enabling probe 2 (fbt::emlxs_attach:return) > Jan 22 10:37:19 sfx4600 dtrace: [ID 566105 kern.notice] NOTICE: enabling probe 3 (dtrace:::ERROR) > .....