Dear all:
Just an FYI that I've gone ahead and merged userspace DTrace support to
FreeBSD 8.x from 9.x. While it appeared to pass build tests locally, boot and
run, etc, this is a non-trivial merge, and it's possible I've messed up.
If
so, apologies in advance, and I'll try to resolve any problems as quickly as
I
can!
And of course, many thanks go to Rui Paulo, who did the port of userspace
DTrace to FreeBSD 9.x with support from the FreeBSD Foundation!
Thanks,
Robert N M Watson
Computer Laboratory
University of Cambridge
---------- Forwarded message ----------
Date: Mon, 28 Feb 2011 23:28:35 +0000 (UTC)
From: Robert Watson <rwatson@FreeBSD.org>
To: src-committers@freebsd.org, svn-src-all@freebsd.org,
svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org
Subject: svn commit: r219107 - in stable/8/sys: amd64/amd64 amd64/include
boot/common cddl/compat/opensolaris/kern cddl/compat/opensolaris/sys
cddl/contrib/opensolaris/uts/common/dtrace cddl/contrib/opensol...
Author: rwatson
Date: Mon Feb 28 23:28:35 2011
New Revision: 219107
URL: http://svn.freebsd.org/changeset/base/219107
Log:
Merge userspace DTrace support from head to stable/8:
r209721:
Merge from vendor-sys/opensolaris:
* add fasttrap files
r209731:
Introduce USD_{SET,GET}{BASE,LIMIT}. These help setting up the user
segment descriptor hi and lo values. Idea from Solaris.
Reviewed by: kib
r209763:
Fix style issues with the previous commit, namely
use-tab-instead-of-space and don't use underscores in macro variables.
Pointed out by: bde
r210292:
Fix typo in comment.
r210357:
MFamd64:
Add USD_GETBASE(), USD_SETBASE(), USD_GETLIMIT() and USD_SETLIMIT().
r210611:
Bump the witness pendlist to 768 to accomodate the increased number of
spinlocks.
r211553:
Add sysname to struct opensolaris_utsname. This is needed by one DTrace
test.
r211566:
Add a sysname char * to struct opensolaris_utsname.
r211606:
Add the FreeBSD definition for the fasttrap ioctls.
r211607:
Add a function compatibility function dtrace_instr_size_isa() that on
FreeBSD does the same as dtrace_dis_isize().
r211608:
Kernel DTrace support for:
o uregs (sson@)
o ustack (sson@)
o /dev/dtrace/helper device (needed for USDT probes)
r211610:
Add more compatibility structure members needed by the upcoming fasttrap
DTrace device.
r211611:
Destroy the helper device when unloading.
r211613:
Fix style issues.
r211614:
Bump KDTRACE_THREAD_ZERO and use M_ZERO as a malloc flag instead of
calling bzero.
r211615:
Remove an elif and add an or-clause.
r211616:
Add an extra comment to the SDT probes definition. This allows us to get
use '-' in probe names, matching the probe names in Solaris.
Add userland SDT probes definitions to sys/sdt.h.
r211617:
Call the systrace_probe_func() when the error value.
r211618:
Port this to FreeBSD. We miss some suword functions, so we use copyout.
r211738:
Port the fasttrap provider to FreeBSD. This provider is responsible for
injecting debugging probes in the userland programs and is the basis for
the pid provider and the usdt provider.
r211744:
MD fasttrap implementation.
r211745:
Replace a pksignal() call with tdksignal().
Pointed out by: kib
r211746:
Update for the recent location of the fasttrap code.
r211747:
Replace structure assignments with explicity memcpy calls. This allows
Clang to compile this file: it was using the builtin memcpy and we want
to use the memcpy defined in gptboot.c. (Clang can't compile boot2
yet).
Submitted by: Dimitry Andric <dimitry at andric.com>
Reviewed by: jhb
r211751:
Add a trap code for DTrace induced traps.
r211752:
Add two DTrace trap type values. Used by fasttrap.
r211753:
Enable fasttrap and make dtraceall depend on fasttrap when building i386
or amd64.
r211804:
Call the necessary DTrace function pointers when we have different kinds
of traps.
r211813:
Add the necessary DTrace function pointers.
r211839:
Sync DTrace bits with amd64 and fix the build.
r211924:
Register an interrupt vector for DTrace return probes. There is some
code missing in lapic to make sure that we don't overwrite this entry,
but this will be done on a sequent commit.
r211925:
Replace a memory barrier with a mutex barrier.
r211926:
Add the path necessary to find fasttrap_isa.h to CFLAGS.
r211929:
Remove debugging.
r212004:
When DTrace is enabled, make sure we don't overwrite the IDT_DTRACE_RET
entry with an IRQ for some hardware component.
Reviewed by: jhb
r212093:
Make the /dev/dtrace/helper node have the mode 0660. This allows
programs that refuse to run as root (pgsql) to install probes when their
user is part of the wheel group.
r212357:
Fix two bugs in DTrace:
* when the process exits, remove the associated USDT probes
* when the process forks, duplicate the USDT probes.
r212465:
Avoid a LOR (sleepable after non-sleepable) in
fasttrap_tracepoint_enable().
r212494:
Revamp locking a bit. This fixes three problems:
* processes now can't go away while we are inserting probes (fixes a
panic)
* if a trap happens, we won't be holding the process lock (fixes a
hang)
* fix a LOR between the process lock and the fasttrap bucket list lock
Thanks to kib for pointing some problems.
r212568:
Bump __FreeBSD_version to reflect the userland DTrace changes
Sponsored by: The FreeBSD Foundation
Userspace DTrace work by: rpaulo
Added:
stable/8/sys/cddl/contrib/opensolaris/uts/common/sys/fasttrap_impl.h
- copied, changed from r209721,
head/sys/cddl/contrib/opensolaris/uts/common/sys/fasttrap_impl.h
stable/8/sys/cddl/contrib/opensolaris/uts/intel/dtrace/
- copied from r209721, head/sys/cddl/contrib/opensolaris/uts/intel/dtrace/
stable/8/sys/cddl/contrib/opensolaris/uts/sparc/dtrace/
- copied from r209721, head/sys/cddl/contrib/opensolaris/uts/sparc/dtrace/
stable/8/sys/cddl/dev/dtrace/amd64/regset.h
- copied unchanged from r211608, head/sys/cddl/dev/dtrace/amd64/regset.h
stable/8/sys/cddl/dev/dtrace/i386/regset.h
- copied unchanged from r211608, head/sys/cddl/dev/dtrace/i386/regset.h
Modified:
stable/8/sys/amd64/amd64/exception.S
stable/8/sys/amd64/amd64/machdep.c
stable/8/sys/amd64/amd64/trap.c
stable/8/sys/amd64/include/segments.h
stable/8/sys/amd64/include/trap.h
stable/8/sys/boot/common/ufsread.c
stable/8/sys/cddl/compat/opensolaris/kern/opensolaris_misc.c
stable/8/sys/cddl/compat/opensolaris/sys/misc.h
stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c
stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c
stable/8/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h
stable/8/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace_impl.h
stable/8/sys/cddl/contrib/opensolaris/uts/common/sys/fasttrap.h
stable/8/sys/cddl/contrib/opensolaris/uts/intel/dtrace/fasttrap_isa.c
stable/8/sys/cddl/dev/dtrace/amd64/dtrace_isa.c
stable/8/sys/cddl/dev/dtrace/amd64/instr_size.c
stable/8/sys/cddl/dev/dtrace/dtrace_cddl.h
stable/8/sys/cddl/dev/dtrace/dtrace_ioctl.c
stable/8/sys/cddl/dev/dtrace/dtrace_load.c
stable/8/sys/cddl/dev/dtrace/dtrace_unload.c
stable/8/sys/cddl/dev/dtrace/i386/dtrace_isa.c
stable/8/sys/cddl/dev/dtrace/i386/instr_size.c
stable/8/sys/cddl/dev/systrace/systrace.c
stable/8/sys/i386/i386/exception.s
stable/8/sys/i386/i386/machdep.c
stable/8/sys/i386/i386/trap.c
stable/8/sys/i386/include/segments.h
stable/8/sys/i386/include/trap.h
stable/8/sys/kern/kern_dtrace.c
stable/8/sys/kern/kern_exec.c
stable/8/sys/kern/kern_exit.c
stable/8/sys/kern/kern_fork.c
stable/8/sys/kern/kern_priv.c
stable/8/sys/kern/kern_proc.c
stable/8/sys/kern/kern_sig.c
stable/8/sys/kern/kern_timeout.c
stable/8/sys/kern/subr_trap.c
stable/8/sys/kern/subr_witness.c
stable/8/sys/kern/vfs_cache.c
stable/8/sys/kern/vfs_lookup.c
stable/8/sys/kern/vfs_syscalls.c
stable/8/sys/modules/dtrace/Makefile
stable/8/sys/modules/dtrace/dtrace/Makefile
stable/8/sys/modules/dtrace/dtraceall/dtraceall.c
stable/8/sys/modules/dtrace/fasttrap/Makefile
stable/8/sys/net/vnet.c
stable/8/sys/opencrypto/deflate.c
stable/8/sys/security/mac/mac_framework.c
stable/8/sys/security/mac/mac_internal.h
stable/8/sys/sys/dtrace_bsd.h
stable/8/sys/sys/param.h
stable/8/sys/sys/priv.h
stable/8/sys/sys/sdt.h
stable/8/sys/sys/signal.h
stable/8/sys/sys/sysent.h
stable/8/sys/tools/vnode_if.awk
stable/8/sys/x86/x86/local_apic.c
Directory Properties:
stable/8/sys/ (props changed)
stable/8/sys/amd64/include/xen/ (props changed)
stable/8/sys/cddl/contrib/opensolaris/ (props changed)
stable/8/sys/contrib/dev/acpica/ (props changed)
stable/8/sys/contrib/pf/ (props changed)
Modified: stable/8/sys/amd64/amd64/exception.S
=============================================================================---
stable/8/sys/amd64/amd64/exception.S Mon Feb 28 21:33:26 2011 (r219106)
+++ stable/8/sys/amd64/amd64/exception.S Mon Feb 28 23:28:35 2011 (r219107)
@@ -108,6 +108,10 @@ IDTVEC(dbg)
TRAP_NOEN(T_TRCTRAP)
IDTVEC(bpt)
TRAP_NOEN(T_BPTFLT)
+#ifdef KDTRACE_HOOKS
+IDTVEC(dtrace_ret)
+ TRAP_NOEN(T_DTRACE_RET)
+#endif
/* Regular traps; The cpu does not supply tf_err for these. */
#define TRAP(a) \
Modified: stable/8/sys/amd64/amd64/machdep.c
=============================================================================---
stable/8/sys/amd64/amd64/machdep.c Mon Feb 28 21:33:26 2011 (r219106)
+++ stable/8/sys/amd64/amd64/machdep.c Mon Feb 28 23:28:35 2011 (r219107)
@@ -54,6 +54,7 @@ __FBSDID("$FreeBSD$");
#include "opt_msgbuf.h"
#include "opt_perfmon.h"
#include "opt_sched.h"
+#include "opt_kdtrace.h"
#include <sys/param.h>
#include <sys/proc.h>
@@ -1094,6 +1095,9 @@ extern inthand_t
IDTVEC(tss), IDTVEC(missing), IDTVEC(stk), IDTVEC(prot),
IDTVEC(page), IDTVEC(mchk), IDTVEC(rsvd), IDTVEC(fpu), IDTVEC(align),
IDTVEC(xmm), IDTVEC(dblfault),
+#ifdef KDTRACE_HOOKS
+ IDTVEC(dtrace_ret),
+#endif
IDTVEC(fast_syscall), IDTVEC(fast_syscall32);
#ifdef DDB
@@ -1624,6 +1628,9 @@ hammer_time(u_int64_t modulep, u_int64_t
setidt(IDT_AC, &IDTVEC(align), SDT_SYSIGT, SEL_KPL, 0);
setidt(IDT_MC, &IDTVEC(mchk), SDT_SYSIGT, SEL_KPL, 0);
setidt(IDT_XF, &IDTVEC(xmm), SDT_SYSIGT, SEL_KPL, 0);
+#ifdef KDTRACE_HOOKS
+ setidt(IDT_DTRACE_RET, &IDTVEC(dtrace_ret), SDT_SYSIGT, SEL_UPL, 0);
+#endif
r_idt.rd_limit = sizeof(idt0) - 1;
r_idt.rd_base = (long) idt;
Modified: stable/8/sys/amd64/amd64/trap.c
=============================================================================---
stable/8/sys/amd64/amd64/trap.c Mon Feb 28 21:33:26 2011 (r219106)
+++ stable/8/sys/amd64/amd64/trap.c Mon Feb 28 23:28:35 2011 (r219107)
@@ -113,6 +113,13 @@ dtrace_doubletrap_func_t dtrace_doubletr
* implementation opaque.
*/
systrace_probe_func_t systrace_probe_func;
+
+/*
+ * These hooks are necessary for the pid, usdt and fasttrap providers.
+ */
+dtrace_fasttrap_probe_ptr_t dtrace_fasttrap_probe_ptr;
+dtrace_pid_probe_ptr_t dtrace_pid_probe_ptr;
+dtrace_return_probe_ptr_t dtrace_return_probe_ptr;
#endif
extern void trap(struct trapframe *frame);
@@ -243,6 +250,55 @@ trap(struct trapframe *frame)
if (dtrace_trap_func != NULL)
if ((*dtrace_trap_func)(frame, type))
goto out;
+ if (type == T_DTRACE_PROBE || type == T_DTRACE_RET ||
+ type == T_BPTFLT) {
+ struct reg regs;
+
+ regs.r_r15 = frame->tf_r15;
+ regs.r_r14 = frame->tf_r14;
+ regs.r_r13 = frame->tf_r13;
+ regs.r_r12 = frame->tf_r12;
+ regs.r_r11 = frame->tf_r11;
+ regs.r_r10 = frame->tf_r10;
+ regs.r_r9 = frame->tf_r9;
+ regs.r_r8 = frame->tf_r8;
+ regs.r_rdi = frame->tf_rdi;
+ regs.r_rsi = frame->tf_rsi;
+ regs.r_rbp = frame->tf_rbp;
+ regs.r_rbx = frame->tf_rbx;
+ regs.r_rdx = frame->tf_rdx;
+ regs.r_rcx = frame->tf_rcx;
+ regs.r_rax = frame->tf_rax;
+ regs.r_rip = frame->tf_rip;
+ regs.r_cs = frame->tf_cs;
+ regs.r_rflags = frame->tf_rflags;
+ regs.r_rsp = frame->tf_rsp;
+ regs.r_ss = frame->tf_ss;
+ if (frame->tf_flags & TF_HASSEGS) {
+ regs.r_ds = frame->tf_ds;
+ regs.r_es = frame->tf_es;
+ regs.r_fs = frame->tf_fs;
+ regs.r_gs = frame->tf_gs;
+ } else {
+ regs.r_ds = 0;
+ regs.r_es = 0;
+ regs.r_fs = 0;
+ regs.r_gs = 0;
+ }
+ if (type == T_DTRACE_PROBE &&
+ dtrace_fasttrap_probe_ptr != NULL &&
+ dtrace_fasttrap_probe_ptr(®s) == 0)
+ goto out;
+ if (type == T_BPTFLT &&
+ dtrace_pid_probe_ptr != NULL &&
+ dtrace_pid_probe_ptr(®s) == 0)
+ goto out;
+ if (type == T_DTRACE_RET &&
+ dtrace_return_probe_ptr != NULL &&
+ dtrace_return_probe_ptr(®s) == 0)
+ goto out;
+
+ }
#endif
if ((frame->tf_rflags & PSL_I) == 0) {
Modified: stable/8/sys/amd64/include/segments.h
=============================================================================---
stable/8/sys/amd64/include/segments.h Mon Feb 28 21:33:26 2011 (r219106)
+++ stable/8/sys/amd64/include/segments.h Mon Feb 28 23:28:35 2011 (r219107)
@@ -74,6 +74,13 @@ struct user_segment_descriptor {
u_int64_t sd_hibase:8; /* segment base address (msb) */
} __packed;
+#define USD_GETBASE(sd) (((sd)->sd_lobase) | (sd)->sd_hibase <<
24)
+#define USD_SETBASE(sd, b) (sd)->sd_lobase = (b); \
+ (sd)->sd_hibase = ((b) >> 24);
+#define USD_GETLIMIT(sd) (((sd)->sd_lolimit) | (sd)->sd_hilimit <<
16)
+#define USD_SETLIMIT(sd, l) (sd)->sd_lolimit = (l); \
+ (sd)->sd_hilimit = ((l) >> 16);
+
/*
* System segment descriptors (128 bit wide)
*/
@@ -207,6 +214,7 @@ struct region_descriptor {
#define IDT_XF 19 /* #XF: SIMD Floating-Point Exception */
#define IDT_IO_INTS NRSVIDT /* Base of IDT entries for I/O interrupts. */
#define IDT_SYSCALL 0x80 /* System Call Interrupt Vector */
+#define IDT_DTRACE_RET 0x92 /* DTrace pid provider Interrupt Vector */
/*
* Entries in the Global Descriptor Table (GDT)
Modified: stable/8/sys/amd64/include/trap.h
=============================================================================---
stable/8/sys/amd64/include/trap.h Mon Feb 28 21:33:26 2011 (r219106)
+++ stable/8/sys/amd64/include/trap.h Mon Feb 28 23:28:35 2011 (r219107)
@@ -62,6 +62,8 @@
#define T_MCHK 28 /* machine check trap */
#define T_XMMFLT 29 /* SIMD floating-point exception */
#define T_RESERVED 30 /* reserved (unknown) */
+#define T_DTRACE_RET 31 /* DTrace pid return */
+#define T_DTRACE_PROBE 32 /* DTrace fasttrap probe */
/* XXX most of the following codes aren't used, but could be. */
Modified: stable/8/sys/boot/common/ufsread.c
=============================================================================---
stable/8/sys/boot/common/ufsread.c Mon Feb 28 21:33:26 2011 (r219106)
+++ stable/8/sys/boot/common/ufsread.c Mon Feb 28 23:28:35 2011 (r219107)
@@ -223,14 +223,19 @@ fsread(ino_t inode, void *buf, size_t nb
return -1;
n = INO_TO_VBO(n, inode);
#if defined(UFS1_ONLY)
- dp1 = ((struct ufs1_dinode *)blkbuf)[n];
+ memcpy(&dp1, (struct ufs1_dinode *)blkbuf + n,
+ sizeof(struct ufs1_dinode));
#elif defined(UFS2_ONLY)
- dp2 = ((struct ufs2_dinode *)blkbuf)[n];
+ memcpy(&dp2, (struct ufs2_dinode *)blkbuf + n,
+ sizeof(struct ufs2_dinode));
#else
if (fs->fs_magic == FS_UFS1_MAGIC)
- dp1 = ((struct ufs1_dinode *)blkbuf)[n];
+ memcpy(&dp1, (struct ufs1_dinode *)blkbuf + n,
+ sizeof(struct ufs1_dinode));
else
- dp2 = ((struct ufs2_dinode *)blkbuf)[n];
+ memcpy(&dp2, (struct ufs2_dinode *)blkbuf + n,
+ sizeof(struct ufs2_dinode));
+
#endif
inomap = inode;
fs_off = 0;
Modified: stable/8/sys/cddl/compat/opensolaris/kern/opensolaris_misc.c
=============================================================================---
stable/8/sys/cddl/compat/opensolaris/kern/opensolaris_misc.c Mon Feb 28 21:33:26
2011 (r219106)
+++ stable/8/sys/cddl/compat/opensolaris/kern/opensolaris_misc.c Mon Feb 28
23:28:35 2011 (r219107)
@@ -38,7 +38,8 @@ __FBSDID("$FreeBSD$");
char hw_serial[11] = "0";
struct opensolaris_utsname utsname = {
- .nodename = "unset"
+ .nodename = "unset",
+ .sysname = "SunOS"
};
int
Modified: stable/8/sys/cddl/compat/opensolaris/sys/misc.h
=============================================================================---
stable/8/sys/cddl/compat/opensolaris/sys/misc.h Mon Feb 28 21:33:26 2011
(r219106)
+++ stable/8/sys/cddl/compat/opensolaris/sys/misc.h Mon Feb 28 23:28:35 2011
(r219107)
@@ -46,6 +46,7 @@
#ifdef _KERNEL
struct opensolaris_utsname {
char *nodename;
+ char *sysname;
};
extern char hw_serial[11];
Modified: stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c
=============================================================================---
stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c Mon Feb 28
21:33:26 2011 (r219106)
+++ stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c Mon Feb 28
23:28:35 2011 (r219107)
@@ -551,20 +551,16 @@ static void dtrace_enabling_provide(dtra
static int dtrace_enabling_match(dtrace_enabling_t *, int *);
static void dtrace_enabling_matchall(void);
static dtrace_state_t *dtrace_anon_grab(void);
-#if defined(sun)
static uint64_t dtrace_helper(int, dtrace_mstate_t *,
dtrace_state_t *, uint64_t, uint64_t);
static dtrace_helpers_t *dtrace_helpers_create(proc_t *);
-#endif
static void dtrace_buffer_drop(dtrace_buffer_t *);
static intptr_t dtrace_buffer_reserve(dtrace_buffer_t *, size_t, size_t,
dtrace_state_t *, dtrace_mstate_t *);
static int dtrace_state_option(dtrace_state_t *, dtrace_optid_t,
dtrace_optval_t);
static int dtrace_ecb_create_enable(dtrace_probe_t *, void *);
-#if defined(sun)
static void dtrace_helper_provider_destroy(dtrace_helper_provider_t *);
-#endif
uint16_t dtrace_load16(uintptr_t);
uint32_t dtrace_load32(uintptr_t);
uint64_t dtrace_load64(uintptr_t);
@@ -2784,6 +2780,21 @@ dtrace_dif_variable(dtrace_mstate_t *mst
return (dtrace_getreg(lwp->lwp_regs, ndx));
return (0);
}
+#else
+ case DIF_VAR_UREGS: {
+ struct trapframe *tframe;
+
+ if (!dtrace_priv_proc(state))
+ return (0);
+
+ if ((tframe = curthread->td_frame) == NULL) {
+ DTRACE_CPUFLAG_SET(CPU_DTRACE_BADADDR);
+ cpu_core[curcpu].cpuc_dtrace_illval = 0;
+ return (0);
+ }
+
+ return (dtrace_getreg(tframe, ndx));
+ }
#endif
case DIF_VAR_CURTHREAD:
@@ -2839,7 +2850,6 @@ dtrace_dif_variable(dtrace_mstate_t *mst
}
return (mstate->dtms_stackdepth);
-#if defined(sun)
case DIF_VAR_USTACKDEPTH:
if (!dtrace_priv_proc(state))
return (0);
@@ -2859,7 +2869,6 @@ dtrace_dif_variable(dtrace_mstate_t *mst
mstate->dtms_present |= DTRACE_MSTATE_USTACKDEPTH;
}
return (mstate->dtms_ustackdepth);
-#endif
case DIF_VAR_CALLER:
if (!dtrace_priv_kernel(state))
@@ -2896,7 +2905,6 @@ dtrace_dif_variable(dtrace_mstate_t *mst
}
return (mstate->dtms_caller);
-#if defined(sun)
case DIF_VAR_UCALLER:
if (!dtrace_priv_proc(state))
return (0);
@@ -2920,7 +2928,6 @@ dtrace_dif_variable(dtrace_mstate_t *mst
}
return (mstate->dtms_ucaller);
-#endif
case DIF_VAR_PROBEPROV:
ASSERT(mstate->dtms_present & DTRACE_MSTATE_PROBE);
@@ -5736,7 +5743,6 @@ dtrace_action_chill(dtrace_mstate_t *mst
cpu->cpu_dtrace_chilled += val;
}
-#if defined(sun)
static void
dtrace_action_ustack(dtrace_mstate_t *mstate, dtrace_state_t *state,
uint64_t *buf, uint64_t arg)
@@ -5849,7 +5855,6 @@ dtrace_action_ustack(dtrace_mstate_t *ms
out:
mstate->dtms_scratch_ptr = old;
}
-#endif
/*
* If you're looking for the epicenter of DTrace, you just found it. This
@@ -6172,7 +6177,6 @@ dtrace_probe(dtrace_id_t id, uintptr_t a
(uint32_t *)arg0);
continue;
-#if defined(sun)
case DTRACEACT_JSTACK:
case DTRACEACT_USTACK:
if (!dtrace_priv_proc(state))
@@ -6214,7 +6218,6 @@ dtrace_probe(dtrace_id_t id, uintptr_t a
DTRACE_USTACK_NFRAMES(rec->dtrd_arg) + 1);
DTRACE_CPUFLAG_CLEAR(CPU_DTRACE_NOFAULT);
continue;
-#endif
default:
break;
@@ -8141,7 +8144,6 @@ dtrace_helper_provide(dof_helper_t *dhp,
dtrace_enabling_matchall();
}
-#if defined(sun)
static void
dtrace_helper_provider_remove_one(dof_helper_t *dhp, dof_sec_t *sec, pid_t
pid)
{
@@ -8189,7 +8191,6 @@ dtrace_helper_provider_remove(dof_helper
dtrace_helper_provider_remove_one(dhp, sec, pid);
}
}
-#endif
/*
* DTrace Meta Provider-to-Framework API Functions
@@ -8729,7 +8730,6 @@ dtrace_difo_validate(dtrace_difo_t *dp,
return (err);
}
-#if defined(sun)
/*
* Validate a DTrace DIF object that it is to be used as a helper. Helpers
* are much more constrained than normal DIFOs. Specifically, they may
@@ -8887,7 +8887,6 @@ dtrace_difo_validate_helper(dtrace_difo_
return (err);
}
-#endif
/*
* Returns 1 if the expression in the DIF object can be cached on a per-thread
@@ -9219,7 +9218,6 @@ dtrace_difo_init(dtrace_difo_t *dp, dtra
dtrace_difo_hold(dp);
}
-#if defined(sun)
static dtrace_difo_t *
dtrace_difo_duplicate(dtrace_difo_t *dp, dtrace_vstate_t *vstate)
{
@@ -9263,7 +9261,6 @@ dtrace_difo_duplicate(dtrace_difo_t *dp,
dtrace_difo_init(new, vstate);
return (new);
}
-#endif
static void
dtrace_difo_destroy(dtrace_difo_t *dp, dtrace_vstate_t *vstate)
@@ -13791,7 +13788,6 @@ dtrace_anon_property(void)
}
}
-#if defined(sun)
/*
* DTrace Helper Functions
*/
@@ -13855,9 +13851,7 @@ dtrace_helper_trace(dtrace_helper_action
((uint64_t *)(uintptr_t)svar->dtsv_data)[curcpu];
}
}
-#endif
-#if defined(sun)
static uint64_t
dtrace_helper(int which, dtrace_mstate_t *mstate,
dtrace_state_t *state, uint64_t arg0, uint64_t arg1)
@@ -13865,7 +13859,7 @@ dtrace_helper(int which, dtrace_mstate_t
uint16_t *flags = &cpu_core[curcpu].cpuc_dtrace_flags;
uint64_t sarg0 = mstate->dtms_arg[0];
uint64_t sarg1 = mstate->dtms_arg[1];
- uint64_t rval;
+ uint64_t rval = 0;
dtrace_helpers_t *helpers = curproc->p_dtrace_helpers;
dtrace_helper_action_t *helper;
dtrace_vstate_t *vstate;
@@ -14056,9 +14050,7 @@ dtrace_helper_destroygen(int gen)
return (0);
}
-#endif
-#if defined(sun)
static int
dtrace_helper_validate(dtrace_helper_action_t *helper)
{
@@ -14073,9 +14065,7 @@ dtrace_helper_validate(dtrace_helper_act
return (err == 0);
}
-#endif
-#if defined(sun)
static int
dtrace_helper_action_add(int which, dtrace_ecbdesc_t *ep)
{
@@ -14622,12 +14612,17 @@ dtrace_helpers_create(proc_t *p)
return (help);
}
-static void
-dtrace_helpers_destroy(void)
+#if defined(sun)
+static
+#endif
+void
+dtrace_helpers_destroy(proc_t *p)
{
dtrace_helpers_t *help;
dtrace_vstate_t *vstate;
+#if defined(sun)
proc_t *p = curproc;
+#endif
int i;
mutex_enter(&dtrace_lock);
@@ -14714,7 +14709,10 @@ dtrace_helpers_destroy(void)
mutex_exit(&dtrace_lock);
}
-static void
+#if defined(sun)
+static
+#endif
+void
dtrace_helpers_duplicate(proc_t *from, proc_t *to)
{
dtrace_helpers_t *help, *newhelp;
@@ -14795,7 +14793,6 @@ dtrace_helpers_duplicate(proc_t *from, p
if (hasprovs)
dtrace_helper_provider_register(to, newhelp, NULL);
}
-#endif
#if defined(sun)
/*
@@ -16466,6 +16463,7 @@ _fini(void)
#else
static d_ioctl_t dtrace_ioctl;
+static d_ioctl_t dtrace_ioctl_helper;
static void dtrace_load(void *);
static int dtrace_unload(void);
#if __FreeBSD_version < 800039
@@ -16474,6 +16472,7 @@ static struct clonedevs *dtrace_clones;
static eventhandler_tag eh_tag; /* Event handler tag. */
#else
static struct cdev *dtrace_dev;
+static struct cdev *helper_dev;
#endif
void dtrace_invop_init(void);
@@ -16488,6 +16487,13 @@ static struct cdevsw dtrace_cdevsw = {
.d_name = "dtrace",
};
+static struct cdevsw helper_cdevsw = {
+ .d_version = D_VERSION,
+ .d_flags = D_TRACKCLOSE | D_NEEDMINOR,
+ .d_ioctl = dtrace_ioctl_helper,
+ .d_name = "helper",
+};
+
#include <dtrace_anon.c>
#if __FreeBSD_version < 800039
#include <dtrace_clone.c>
Modified: stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c
=============================================================================---
stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c Mon Feb 28
21:33:26 2011 (r219106)
+++ stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c Mon Feb
28 23:28:35 2011 (r219107)
@@ -17,6 +17,10 @@
* information: Portions Copyright [yyyy] [name of copyright owner]
*
* CDDL HEADER END
+ *
+ * Portions Copyright 2010 The FreeBSD Foundation
+ *
+ * $FreeBSD$
*/
/*
@@ -24,7 +28,9 @@
* Use is subject to license terms.
*/
+#if defined(sun)
#pragma ident "%Z%%M% %I% %E% SMI"
+#endif
#include <sys/atomic.h>
#include <sys/errno.h>
@@ -32,11 +38,15 @@
#include <sys/modctl.h>
#include <sys/conf.h>
#include <sys/systm.h>
+#if defined(sun)
#include <sys/ddi.h>
+#endif
#include <sys/sunddi.h>
#include <sys/cpuvar.h>
#include <sys/kmem.h>
+#if defined(sun)
#include <sys/strsubr.h>
+#endif
#include <sys/fasttrap.h>
#include <sys/fasttrap_impl.h>
#include <sys/fasttrap_isa.h>
@@ -44,9 +54,17 @@
#include <sys/dtrace_impl.h>
#include <sys/sysmacros.h>
#include <sys/proc.h>
-#include <sys/priv.h>
#include <sys/policy.h>
+#if defined(sun)
#include <util/qsort.h>
+#endif
+#include <sys/mutex.h>
+#include <sys/kernel.h>
+#if !defined(sun)
+#include <sys/user.h>
+#include <sys/dtrace_bsd.h>
+#include <cddl/dev/dtrace/dtrace_cddl.h>
+#endif
/*
* User-Land Trap-Based Tracing
@@ -125,11 +143,20 @@
* never hold the provider lock and creation lock simultaneously
*/
-static dev_info_t *fasttrap_devi;
+static d_open_t fasttrap_open;
+static d_ioctl_t fasttrap_ioctl;
+
+static struct cdevsw fasttrap_cdevsw = {
+ .d_version = D_VERSION,
+ .d_open = fasttrap_open,
+ .d_ioctl = fasttrap_ioctl,
+ .d_name = "fasttrap",
+};
+static struct cdev *fasttrap_cdev;
static dtrace_meta_provider_id_t fasttrap_meta_id;
-static timeout_id_t fasttrap_timeout;
-static kmutex_t fasttrap_cleanup_mtx;
+static struct callout fasttrap_timeout;
+static struct mtx fasttrap_cleanup_mtx;
static uint_t fasttrap_cleanup_work;
/*
@@ -181,6 +208,10 @@ static void fasttrap_proc_release(fasttr
#define FASTTRAP_PROCS_INDEX(pid) ((pid) & fasttrap_procs.fth_mask)
+#if !defined(sun)
+static kmutex_t fasttrap_cpuc_pid_lock[MAXCPU];
+#endif
+
static int
fasttrap_highbit(ulong_t i)
{
@@ -229,6 +260,7 @@ fasttrap_hash_str(const char *p)
void
fasttrap_sigtrap(proc_t *p, kthread_t *t, uintptr_t pc)
{
+#if defined(sun)
sigqueue_t *sqp = kmem_zalloc(sizeof (sigqueue_t), KM_SLEEP);
sqp->sq_info.si_signo = SIGTRAP;
@@ -241,6 +273,17 @@ fasttrap_sigtrap(proc_t *p, kthread_t *t
if (t != NULL)
aston(t);
+#else
+ ksiginfo_t *ksi = kmem_zalloc(sizeof (ksiginfo_t), KM_SLEEP);
+
+ ksiginfo_init(ksi);
+ ksi->ksi_signo = SIGTRAP;
+ ksi->ksi_code = TRAP_DTRACE;
+ ksi->ksi_addr = (caddr_t)pc;
+ PROC_LOCK(p);
+ (void) tdksignal(t, SIGTRAP, ksi);
+ PROC_UNLOCK(p);
+#endif
}
/*
@@ -257,9 +300,9 @@ fasttrap_mod_barrier(uint64_t gen)
fasttrap_mod_gen++;
- for (i = 0; i < NCPU; i++) {
- mutex_enter(&cpu_core[i].cpuc_pid_lock);
- mutex_exit(&cpu_core[i].cpuc_pid_lock);
+ CPU_FOREACH(i) {
+ mutex_enter(&fasttrap_cpuc_pid_lock[i]);
+ mutex_exit(&fasttrap_cpuc_pid_lock[i]);
}
}
@@ -274,16 +317,15 @@ fasttrap_pid_cleanup_cb(void *data)
fasttrap_provider_t **fpp, *fp;
fasttrap_bucket_t *bucket;
dtrace_provider_id_t provid;
- int i, later;
+ int i, later = 0;
static volatile int in = 0;
ASSERT(in == 0);
in = 1;
- mutex_enter(&fasttrap_cleanup_mtx);
while (fasttrap_cleanup_work) {
fasttrap_cleanup_work = 0;
- mutex_exit(&fasttrap_cleanup_mtx);
+ mtx_unlock(&fasttrap_cleanup_mtx);
later = 0;
@@ -349,10 +391,12 @@ fasttrap_pid_cleanup_cb(void *data)
mutex_exit(&bucket->ftb_mtx);
}
- mutex_enter(&fasttrap_cleanup_mtx);
+ mtx_lock(&fasttrap_cleanup_mtx);
}
+#if 0
ASSERT(fasttrap_timeout != 0);
+#endif
/*
* If we were unable to remove a retired provider, try again after
@@ -364,14 +408,17 @@ fasttrap_pid_cleanup_cb(void *data)
* get a chance to do that work if and when the timeout is reenabled
* (if detach fails).
*/
- if (later > 0 && fasttrap_timeout != (timeout_id_t)1)
- fasttrap_timeout = timeout(&fasttrap_pid_cleanup_cb, NULL, hz);
+ if (later > 0 && callout_active(&fasttrap_timeout))
+ callout_reset(&fasttrap_timeout, hz, &fasttrap_pid_cleanup_cb,
+ NULL);
else if (later > 0)
fasttrap_cleanup_work = 1;
- else
- fasttrap_timeout = 0;
+ else {
+#if !defined(sun)
+ /* Nothing to be done for FreeBSD */
+#endif
+ }
- mutex_exit(&fasttrap_cleanup_mtx);
in = 0;
}
@@ -381,11 +428,11 @@ fasttrap_pid_cleanup_cb(void *data)
static void
fasttrap_pid_cleanup(void)
{
- mutex_enter(&fasttrap_cleanup_mtx);
+
+ mtx_lock(&fasttrap_cleanup_mtx);
fasttrap_cleanup_work = 1;
- if (fasttrap_timeout == 0)
- fasttrap_timeout = timeout(&fasttrap_pid_cleanup_cb, NULL, 1);
- mutex_exit(&fasttrap_cleanup_mtx);
+ callout_reset(&fasttrap_timeout, 1, &fasttrap_pid_cleanup_cb, NULL);
+ mtx_unlock(&fasttrap_cleanup_mtx);
}
/*
@@ -400,9 +447,35 @@ fasttrap_fork(proc_t *p, proc_t *cp)
pid_t ppid = p->p_pid;
int i;
+#if defined(sun)
ASSERT(curproc == p);
ASSERT(p->p_proc_flag & P_PR_LOCK);
+#else
+ PROC_LOCK_ASSERT(p, MA_OWNED);
+#endif
+#if defined(sun)
ASSERT(p->p_dtrace_count > 0);
+#else
+ if (p->p_dtrace_helpers) {
+ /*
+ * dtrace_helpers_duplicate() allocates memory.
+ */
+ _PHOLD(cp);
+ PROC_UNLOCK(p);
+ PROC_UNLOCK(cp);
+ dtrace_helpers_duplicate(p, cp);
+ PROC_LOCK(cp);
+ PROC_LOCK(p);
+ _PRELE(cp);
+ }
+ /*
+ * This check is purposely here instead of in kern_fork.c because,
+ * for legal resons, we cannot include the dtrace_cddl.h header
+ * inside kern_fork.c and insert if-clause there.
+ */
+ if (p->p_dtrace_count == 0)
+ return;
+#endif
ASSERT(cp->p_dtrace_count == 0);
/*
@@ -419,9 +492,13 @@ fasttrap_fork(proc_t *p, proc_t *cp)
* We don't have to worry about the child process disappearing
* because we're in fork().
*/
- mutex_enter(&cp->p_lock);
+#if defined(sun)
+ mtx_lock_spin(&cp->p_slock);
sprlock_proc(cp);
- mutex_exit(&cp->p_lock);
+ mtx_unlock_spin(&cp->p_slock);
+#else
+ _PHOLD(cp);
+#endif
/*
* Iterate over every tracepoint looking for ones that belong to the
@@ -451,8 +528,12 @@ fasttrap_fork(proc_t *p, proc_t *cp)
mutex_exit(&bucket->ftb_mtx);
}
+#if defined(sun)
mutex_enter(&cp->p_lock);
sprunlock(cp);
+#else
+ _PRELE(cp);
+#endif
}
/*
@@ -463,24 +544,30 @@ fasttrap_fork(proc_t *p, proc_t *cp)
static void
fasttrap_exec_exit(proc_t *p)
{
+#if defined(sun)
ASSERT(p == curproc);
- ASSERT(MUTEX_HELD(&p->p_lock));
-
- mutex_exit(&p->p_lock);
+#endif
+ PROC_LOCK_ASSERT(p, MA_OWNED);
+ _PHOLD(p);
+ PROC_UNLOCK(p);
/*
* We clean up the pid provider for this process here; user-land
* static probes are handled by the meta-provider remove entry point.
*/
fasttrap_provider_retire(p->p_pid, FASTTRAP_PID_NAME, 0);
-
- mutex_enter(&p->p_lock);
+#if !defined(sun)
+ if (p->p_dtrace_helpers)
+ dtrace_helpers_destroy(p);
+#endif
+ PROC_LOCK(p);
+ _PRELE(p);
}
/*ARGSUSED*/
static void
-fasttrap_pid_provide(void *arg, const dtrace_probedesc_t *desc)
+fasttrap_pid_provide(void *arg, dtrace_probedesc_t *desc)
{
/*
* There are no "default" pid probes.
@@ -504,7 +591,9 @@ fasttrap_tracepoint_enable(proc_t *p, fa
ASSERT(probe->ftp_tps[index].fit_tp->ftt_pid == pid);
+#if defined(sun)
ASSERT(!(p->p_flag & SVFORK));
+#endif
/*
* Before we make any modifications, make sure we've imposed a barrier
@@ -610,7 +699,9 @@ again:
* Increment the count of the number of tracepoints active in
* the victim process.
*/
+#if defined(sun)
ASSERT(p->p_proc_flag & P_PR_LOCK);
+#endif
p->p_dtrace_count++;
return (rc);
@@ -666,7 +757,7 @@ fasttrap_tracepoint_disable(proc_t *p, f
fasttrap_bucket_t *bucket;
fasttrap_provider_t *provider = probe->ftp_prov;
fasttrap_tracepoint_t **pp, *tp;
- fasttrap_id_t *id, **idp;
+ fasttrap_id_t *id, **idp = NULL;
pid_t pid;
uintptr_t pc;
@@ -800,7 +891,9 @@ fasttrap_tracepoint_disable(proc_t *p, f
* Decrement the count of the number of tracepoints active
* in the victim process.
*/
+#if defined(sun)
ASSERT(p->p_proc_flag & P_PR_LOCK);
+#endif
p->p_dtrace_count--;
}
@@ -851,26 +944,31 @@ fasttrap_enable_callbacks(void)
static void
fasttrap_disable_callbacks(void)
{
+#if defined(sun)
ASSERT(MUTEX_HELD(&cpu_lock));
+#endif
+
mutex_enter(&fasttrap_count_mtx);
ASSERT(fasttrap_pid_count > 0);
fasttrap_pid_count--;
if (fasttrap_pid_count == 0) {
+#if defined(sun)
cpu_t *cur, *cpu = CPU;
for (cur = cpu->cpu_next_onln; cur != cpu;
cur = cur->cpu_next_onln) {
rw_enter(&cur->cpu_ft_lock, RW_WRITER);
}
-
+#endif
dtrace_pid_probe_ptr = NULL;
dtrace_return_probe_ptr = NULL;
-
+#if defined(sun)
for (cur = cpu->cpu_next_onln; cur != cpu;
cur = cur->cpu_next_onln) {
rw_exit(&cur->cpu_ft_lock);
}
+#endif
}
mutex_exit(&fasttrap_count_mtx);
}
@@ -880,13 +978,16 @@ static void
fasttrap_pid_enable(void *arg, dtrace_id_t id, void *parg)
{
fasttrap_probe_t *probe = parg;
- proc_t *p;
+ proc_t *p = NULL;
int i, rc;
+
ASSERT(probe != NULL);
ASSERT(!probe->ftp_enabled);
ASSERT(id == probe->ftp_id);
+#if defined(sun)
ASSERT(MUTEX_HELD(&cpu_lock));
+#endif
/*
* Increment the count of enabled probes on this probe's provider;
@@ -911,6 +1012,7 @@ fasttrap_pid_enable(void *arg, dtrace_id
* a fork in which the traced process is being born and we're copying
* USDT probes. Otherwise, the process is gone so bail.
*/
+#if defined(sun)
if ((p = sprlock(probe->ftp_pid)) == NULL) {
if ((curproc->p_flag & SFORKING) == 0)
return;
@@ -934,12 +1036,23 @@ fasttrap_pid_enable(void *arg, dtrace_id
ASSERT(!(p->p_flag & SVFORK));
mutex_exit(&p->p_lock);
+#else
+ if ((p = pfind(probe->ftp_pid)) == NULL)
+ return;
+#endif
/*
* We have to enable the trap entry point before any user threads have
* the chance to execute the trap instruction we're about to place
* in their process's text.
*/
+#ifdef __FreeBSD__
+ /*
+ * pfind() returns a locked process.
+ */
+ _PHOLD(p);
+ PROC_UNLOCK(p);
+#endif
fasttrap_enable_callbacks();
/*
@@ -967,8 +1080,12 @@ fasttrap_pid_enable(void *arg, dtrace_id
i--;
}
+#if defined(sun)
mutex_enter(&p->p_lock);
sprunlock(p);
+#else
+ PRELE(p);
+#endif
/*
* Since we're not actually enabling this probe,
@@ -978,9 +1095,12 @@ fasttrap_pid_enable(void *arg, dtrace_id
return;
}
}
-
+#if defined(sun)
mutex_enter(&p->p_lock);
sprunlock(p);
+#else
+ PRELE(p);
+#endif
probe->ftp_enabled = 1;
}
@@ -996,18 +1116,22 @@ fasttrap_pid_disable(void *arg, dtrace_i
ASSERT(id == probe->ftp_id);
+ mutex_enter(&provider->ftp_mtx);
+
/*
* We won't be able to acquire a /proc-esque lock on the process
* iff the process is dead and gone. In this case, we rely on the
* provider lock as a point of mutual exclusion to prevent other
* DTrace consumers from disabling this probe.
*/
- if ((p = sprlock(probe->ftp_pid)) != NULL) {
- ASSERT(!(p->p_flag & SVFORK));
- mutex_exit(&p->p_lock);
+ if ((p = pfind(probe->ftp_pid)) == NULL) {
+ mutex_exit(&provider->ftp_mtx);
+ return;
}
-
- mutex_enter(&provider->ftp_mtx);
+#ifdef __FreeBSD__
+ _PHOLD(p);
+ PROC_UNLOCK(p);
+#endif
/*
* Disable all the associated tracepoints (for fully enabled probes).
@@ -1030,9 +1154,6 @@ fasttrap_pid_disable(void *arg, dtrace_i
if (provider->ftp_retired && !provider->ftp_marked)
whack = provider->ftp_marked = 1;
mutex_exit(&provider->ftp_mtx);
-
- mutex_enter(&p->p_lock);
- sprunlock(p);
} else {
/*
* If the process is dead, we're just waiting for the
@@ -1046,12 +1167,17 @@ fasttrap_pid_disable(void *arg, dtrace_i
if (whack)
fasttrap_pid_cleanup();
+#ifdef __FreeBSD__
+ PRELE(p);
+#endif
if (!probe->ftp_enabled)
return;
probe->ftp_enabled = 0;
+#if defined(sun)
ASSERT(MUTEX_HELD(&cpu_lock));
+#endif
fasttrap_disable_callbacks();
}
@@ -1163,6 +1289,7 @@ fasttrap_proc_lookup(pid_t pid)
fasttrap_bucket_t *bucket;
fasttrap_proc_t *fprc, *new_fprc;
+
bucket = &fasttrap_procs.fth_table[FASTTRAP_PROCS_INDEX(pid)];
mutex_enter(&bucket->ftb_mtx);
@@ -1189,6 +1316,10 @@ fasttrap_proc_lookup(pid_t pid)
new_fprc->ftpc_pid = pid;
new_fprc->ftpc_rcount = 1;
new_fprc->ftpc_acount = 1;
+#if !defined(sun)
+ mutex_init(&new_fprc->ftpc_mtx, "fasttrap proc mtx",
MUTEX_DEFAULT,
+ NULL);
+#endif
*** DIFF OUTPUT TRUNCATED AT 1000 LINES ***