To All (This is mainly for the Mac DTrace 3, Adam Leventhal, Bryan Cantrill, Mike Shapiro), My name is Blake Sawyer and I am currently doing some research at Virginia Tech. One of the goals of this project is to track meaningful user interactions such as viewing any file, webpage, or mail document. I love developing for the Mac and knew that AppleScript would do this easily, but also knew that DTrace has been ported to the Mac. To move away from AppleScript, I started to learn DTrace. As I am fairly new to DTrace, both on the Mac and any Solaris environment, I have a few questions about DTrace''s capabilities on Mac OS X. What I want to know is if DTrace _can_ do this, not _how_ to do it. I need to decide if I should spend time learning DTrace for this research project. From what I have learned from DTrace is that you can monitor the system calls of either the OS probes or individual processes. Also, on the Mac you are able to monitor Objective-C calls by specifying an individual Cocoa application. For my particular project I need to track 3 tasks: when any arbitrary application accesses a file, when any web page is accessed, and when any mail client assesses an email. My knowledge of DTrace tells me that I need to create probes for every application that can do these three types of tasks. Is there another approach that does not need to know every type of application or process that can do these 3 tasks? I appreciate any feedback one could give. Thanks in advanced, Blake
On 09/02/08 08:07, Blake Sawyer wrote:> To All (This is mainly for the Mac DTrace 3, Adam Leventhal, Bryan > Cantrill, Mike Shapiro), > > My name is Blake Sawyer and I am currently doing some research at > Virginia Tech. One of the goals of this project is to track > meaningful user interactions such as viewing any file, webpage, or > mail document. I love developing for the Mac and knew that > AppleScript would do this easily, but also knew that DTrace has been > ported to the Mac. To move away from AppleScript, I started to learn > DTrace. As I am fairly new to DTrace, both on the Mac and any Solaris > environment, I have a few questions about DTrace''s capabilities on Mac > OS X. What I want to know is if DTrace _can_ do this, not _how_ to do > it. I need to decide if I should spend time learning DTrace for this > research project. > > From what I have learned from DTrace is that you can monitor the > system calls of either the OS probes or individual processes. Also, > on the Mac you are able to monitor Objective-C calls by specifying an > individual Cocoa application. For my particular project I need to > track 3 tasks: when any arbitrary application accesses a file, when > any web page is accessed, and when any mail client assesses an email. > My knowledge of DTrace tells me that I need to create probes for every > application that can do these three types of tasks. Is there another > approach that does not need to know every type of application or > process that can do these 3 tasks?I don''t have (any) Mac-OS specific knowledge, but I were to try this on Solaris, I''d start with the syscall and/or fbt provider(s) (they''re not specific to any process, unlike the pid provider) and try to identify the probes that fire for the events you''re looking at; once that''s done, you can, if you still wish, run a secondary probe on the application(s) of interest the first set of probes identified ... like this: syscall::open*:entry / set of condition(s) / { system(app-specific probe); } it''s probably going to be a little more complicated for what you want to do, and it may look different on the Mac, but you get the idea, I hope. HTH Michael -- Michael Schuster http://blogs.sun.com/recursion Recursion, n.: see ''Recursion''
> To All (This is mainly for the Mac DTrace 3, Adam Leventhal, Bryan > Cantrill, Mike Shapiro).. >We appreciate your desire to go to the source. It''s a lot like posting a question on relativity, and indicating you''d really like an answer from Einstein.... :^) That said, there''s great news - the DTrace community is growing all the time, and there are many, many DTrace experts that can help. So Brian, Mike and Adam (and Eric, and Brendan, and Jon), can spend more time catching up on People magazine, tracking alien landings in the National Enquirer, and playing FishPong.... :^)> From what I have learned from DTrace is that you can monitor the > system calls of either the OS probes or individual processes.Let''s start with terminology here. Systems calls are not issued by the OS - application processes (threads) issue system calls to enter the OS for a privileged operation (e.g. open a file). Please clarify what you mean by "OS probes" - are you referring to DTrace probes, or OS X IOKit probes? Are you interested in dtrace''ing dtrace? I''m not breaking your you-know-whats....I''m sincerely interested in making sure we agree on terminology. The notion of "OS probes" issuing system calls is not clear to me.> Also, > on the Mac you are able to monitor Objective-C calls by specifying an > individual Cocoa application. For my particular project I need to > track 3 tasks: when any arbitrary application accesses a file, when > any web page is accessed, and when any mail client assesses an email. > My knowledge of DTrace tells me that I need to create probes for every > application that can do these three types of tasks. Is there another > approach that does not need to know every type of application or > process that can do these 3 tasks? >"... any arbitrary application accesses a file" - Are you interested in tracking access to one specific file by any possible process? I assume the machine you''re monitoring is running a WEB server, and an email server? There''s a couple ways to do this, depending somewhat on your configuration. For tracking access to a file, you can use the system call provider and a predicate on arg0, which is a pointer to a pathname string. For the second two items, I''d start with monitoring the httpd process and mail server process with the syscall provider to determine the syscall path to the patterns you''re interested in. Once you''ve established that, you can whittle all this down to a relatively simple script that does what you want. Make sense? (In case it does not) - In other words, as a general methodology, I sometimes use dtrace to get a broad view of what a particular application process is doing, e.g. grab a system call profile of a httpd process, to establish the specific calls (and, in some cases, args) used when something of interest is happening (e.g. accessing a static WEB page). From there I use that information to create a dtrace script that is intended to hone-in on that particular flow of activity. HTH, /jim> I appreciate any feedback one could give. > > Thanks in advanced, > > Blake > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org >
> and there are many, many DTrace experts that can help. So Brian, Mike and >Sorry; s/Brian/Bryan
Jim, Thanks so much for your help. Also thanks for understanding my terminology mistakes, I am new to a lot of this. Let me clear some of these things up by explaining the idea of my project. I want to create an indexing application that will work with a location/context aware system. This application will automatically index media files(documents, video, audio, webpages, mail, etc.) socially and episodically. For example, in a meeting scenario, the system will track media files accessed and tag them with who attended the meeting and the time and location of the meeting. I am trying to prove that giving a user the ability to reference files socially and episodically, they will be better at re-contextualization. You can think of it as a new personal information manager. Therefore, the purposes of using DTrace is not to run any kind of test on a web server or email server. I want this to run on an individual''s computer that determines when meaningful media access occurs. It will then query the location/context aware system to tag the media appropriately. I am just unsure if DTrace is the right approach here. This is not really the main purpose of DTrace, but my professors and I agree that this may be a good starting point. As I mentioned Mac supports AppleScript that can do this, though it is quite inefficient. It seems to me that if I want to create an application like this, I would need to monitor each application using DTrace. This approach would not be very extensible for future versions. I hope this clears up what I am trying to accomplish. Again, thanks so much for the help. -Blake On Sep 2, 2008, at [Sep 2]1:06 PM, Jim Mauro wrote:> >> To All (This is mainly for the Mac DTrace 3, Adam Leventhal, Bryan >> Cantrill, Mike Shapiro).. >> > We appreciate your desire to go to the source. It''s a lot like > posting a question on > relativity, and indicating you''d really like an answer from > Einstein.... :^) > > That said, there''s great news - the DTrace community is growing all > the time, > and there are many, many DTrace experts that can help. So Brian, > Mike and > Adam (and Eric, and Brendan, and Jon), can spend more time catching up > on People magazine, tracking alien landings in the National > Enquirer, and > playing FishPong.... > > :^) >> From what I have learned from DTrace is that you can monitor the >> system calls of either the OS probes or individual processes. > Let''s start with terminology here. Systems calls are not issued by > the OS - application > processes (threads) issue system calls to enter the OS for a > privileged operation (e.g. > open a file). Please clarify what you mean by "OS probes" - are you > referring to DTrace > probes, or OS X IOKit probes? > > Are you interested in dtrace''ing dtrace? > > I''m not breaking your you-know-whats....I''m sincerely interested in > making sure > we agree on terminology. The notion of "OS probes" issuing system > calls is not > clear to me. >> Also, on the Mac you are able to monitor Objective-C calls by >> specifying an individual Cocoa application. For my particular >> project I need to track 3 tasks: when any arbitrary application >> accesses a file, when any web page is accessed, and when any mail >> client assesses an email. My knowledge of DTrace tells me that I >> need to create probes for every application that can do these >> three types of tasks. Is there another approach that does not >> need to know every type of application or process that can do >> these 3 tasks? >> > "... any arbitrary application accesses a file" - Are you interested > in tracking access > to one specific file by any possible process? > > I assume the machine you''re monitoring is running a WEB server, and > an email server? > > There''s a couple ways to do this, depending somewhat on your > configuration. > For tracking access to a file, you can use the system call provider > and a predicate > on arg0, which is a pointer to a pathname string. > > For the second two items, I''d start with monitoring the httpd > process and mail server > process with the syscall provider to determine the syscall path to > the patterns you''re > interested in. Once you''ve established that, you can whittle all > this down to a > relatively simple script that does what you want. Make sense? > > (In case it does not) - In other words, as a general methodology, I > sometimes use > dtrace to get a broad view of what a particular application process > is doing, e.g. > grab a system call profile of a httpd process, to establish the > specific calls (and, in > some cases, args) used when something of interest is happening (e.g. > accessing > a static WEB page). From there I use that information to create a > dtrace script > that is intended to hone-in on that particular flow of activity. > > HTH, > /jim > > >> I appreciate any feedback one could give. >> >> Thanks in advanced, >> >> Blake >> _______________________________________________ >> dtrace-discuss mailing list >> dtrace-discuss at opensolaris.org >>
Blake,> Thanks so much for your help. Also thanks for understanding my > terminology mistakes, I am new to a lot of this. Let me clear some of > these things up by explaining the idea of my project. > > I want to create an indexing application that will work with a > location/context aware system. This application will automatically > index media files(documents, video, audio, webpages, mail, etc.) > socially and episodically. For example, in a meeting scenario, the > system will track media files accessed and tag them with who attended > the meeting and the time and location of the meeting. I am trying to > prove that giving a user the ability to reference files socially and > episodically, they will be better at re-contextualization. You can > think of it as a new personal information manager. > > Therefore, the purposes of using DTrace is not to run any kind of test > on a web server or email server. I want this to run on an > individual''s computer that determines when meaningful media access > occurs. It will then query the location/context aware system to tag > the media appropriately. I am just unsure if DTrace is the right > approach here. This is not really the main purpose of DTrace, but my > professors and I agree that this may be a good starting point. As I > mentioned Mac supports AppleScript that can do this, though it is > quite inefficient.It''s an interesting idea, though as you are rightly inferring, outside the design center of DTrace. There are two potential reasons to use DTrace here: the first is if there is not a reliable way of getting this information other than DTrace. If, for example, you want to support many different kinds of "meaningful media accessing" applications (however you are defining those applications), it might be impossible to rely on modifying or otherwise interposing on them -- and you may be forced to go the kernel (and hence DTrace) as the source of Truth. The second reason to use DTrace is that while you perhaps _can_ modify or interpose on those applications, DTrace represents an easy way to prototype that system and flesh out the semantics that you wish to implement. Either way, there is a clear downside to using DTrace: it was designed to instrument arbitrary contexts, not to audit reliably (one cannot have it both ways), and it is therefore always be possible to have records dropped (for example, on buffer overflow). Now, you can size your buffers and otherwise construct your enablins such that you can put a bound on this (e.g., not lossy for less than n accesses/sec for some n), but it doesn''t negate the fact that lossiness is always a possibility. Regardless, good luck -- and if you do choose to implement or prototype on DTrace (or even if you don''t), it would be interesting to hear about it at dtrace.conf(09)! - Bryan -------------------------------------------------------------------------- Bryan Cantrill, Sun Microsystems Fishworks. http://blogs.sun.com/bmc
| I want to create an indexing application that will work with a | location/context aware system. This application will automatically | index media files(documents, video, audio, webpages, mail, etc.) | socially and episodically. For example, in a meeting scenario, the | system will track media files accessed and tag them with who attended | the meeting and the time and location of the meeting. I am trying to | prove that giving a user the ability to reference files socially and | episodically, they will be better at re-contextualization. You can | think of it as a new personal information manager. Is there a reason you are trying to use DTrace for this, instead of extending Spotlight (which already provides robust content indexing)? -- bda Cyberpunk is dead. Long live cyberpunk. http://mirrorshades.org
I''m not sure I can add anything beyond Bryan''s response. I''m not an expert in WEB servers or the multimedia server stack (other software components that get integrated for building these cool sites), but I would first look at add-ons, compiler flags, options, etc, for the actual software components running the service. It''s not unusual to find additional logging/tracking features that can be enabled. I see DTrace as a fit here to help you understand how to build the software you''re interested in building (which is what Bryan suggested in his "second reason" response), but not as a component of the final application. If you have specific questions on how to do that, let''s continue the dialogue. Thanks, /jim Blake Sawyer wrote:> Jim, > > Thanks so much for your help. Also thanks for understanding my > terminology mistakes, I am new to a lot of this. Let me clear some of > these things up by explaining the idea of my project. > > I want to create an indexing application that will work with a > location/context aware system. This application will automatically > index media files(documents, video, audio, webpages, mail, etc.) > socially and episodically. For example, in a meeting scenario, the > system will track media files accessed and tag them with who attended > the meeting and the time and location of the meeting. I am trying to > prove that giving a user the ability to reference files socially and > episodically, they will be better at re-contextualization. You can > think of it as a new personal information manager. > > Therefore, the purposes of using DTrace is not to run any kind of test > on a web server or email server. I want this to run on an > individual''s computer that determines when meaningful media access > occurs. It will then query the location/context aware system to tag > the media appropriately. I am just unsure if DTrace is the right > approach here. This is not really the main purpose of DTrace, but my > professors and I agree that this may be a good starting point. As I > mentioned Mac supports AppleScript that can do this, though it is > quite inefficient. > > It seems to me that if I want to create an application like this, I > would need to monitor each application using DTrace. This approach > would not be very extensible for future versions. > > I hope this clears up what I am trying to accomplish. > > Again, thanks so much for the help. > > -Blake > > On Sep 2, 2008, at [Sep 2]1:06 PM, Jim Mauro wrote: > >> >>> To All (This is mainly for the Mac DTrace 3, Adam Leventhal, Bryan >>> Cantrill, Mike Shapiro).. >>> >> We appreciate your desire to go to the source. It''s a lot like >> posting a question on >> relativity, and indicating you''d really like an answer from >> Einstein.... :^) >> >> That said, there''s great news - the DTrace community is growing all >> the time, >> and there are many, many DTrace experts that can help. So Brian, Mike >> and >> Adam (and Eric, and Brendan, and Jon), can spend more time catching up >> on People magazine, tracking alien landings in the National Enquirer, >> and >> playing FishPong.... >> >> :^) >>> From what I have learned from DTrace is that you can monitor the >>> system calls of either the OS probes or individual processes. >> Let''s start with terminology here. Systems calls are not issued by >> the OS - application >> processes (threads) issue system calls to enter the OS for a >> privileged operation (e.g. >> open a file). Please clarify what you mean by "OS probes" - are you >> referring to DTrace >> probes, or OS X IOKit probes? >> >> Are you interested in dtrace''ing dtrace? >> >> I''m not breaking your you-know-whats....I''m sincerely interested in >> making sure >> we agree on terminology. The notion of "OS probes" issuing system >> calls is not >> clear to me. >>> Also, on the Mac you are able to monitor Objective-C calls by >>> specifying an individual Cocoa application. For my particular >>> project I need to track 3 tasks: when any arbitrary application >>> accesses a file, when any web page is accessed, and when any mail >>> client assesses an email. My knowledge of DTrace tells me that I >>> need to create probes for every application that can do these three >>> types of tasks. Is there another approach that does not need to >>> know every type of application or process that can do these 3 tasks? >>> >> "... any arbitrary application accesses a file" - Are you interested >> in tracking access >> to one specific file by any possible process? >> >> I assume the machine you''re monitoring is running a WEB server, and >> an email server? >> >> There''s a couple ways to do this, depending somewhat on your >> configuration. >> For tracking access to a file, you can use the system call provider >> and a predicate >> on arg0, which is a pointer to a pathname string. >> >> For the second two items, I''d start with monitoring the httpd process >> and mail server >> process with the syscall provider to determine the syscall path to >> the patterns you''re >> interested in. Once you''ve established that, you can whittle all this >> down to a >> relatively simple script that does what you want. Make sense? >> >> (In case it does not) - In other words, as a general methodology, I >> sometimes use >> dtrace to get a broad view of what a particular application process >> is doing, e.g. >> grab a system call profile of a httpd process, to establish the >> specific calls (and, in >> some cases, args) used when something of interest is happening (e.g. >> accessing >> a static WEB page). From there I use that information to create a >> dtrace script >> that is intended to hone-in on that particular flow of activity. >> >> HTH, >> /jim >> >> >>> I appreciate any feedback one could give. >>> >>> Thanks in advanced, >>> >>> Blake >>> _______________________________________________ >>> dtrace-discuss mailing list >>> dtrace-discuss at opensolaris.org >>> >