dtrace discuss - Jun 2009 - Sharing memory between user and kernel space

Hi all,
I have a question on dtrace buffer management.I am kind of new to  
dtrace development.I am looking into optimizing dtrace from the point  
of view of reducing the memory copies from user to kernel space and  
vice versa.
Isn''t it possible to share memory between user and kernel space by  
preallocating and using that shared memory to transfer data between  
user and kernel space? This would prevent those expensive memory copies.Is 
  there any problem in doing that.Sorry for my lack of expertise with  
dtrace development,I would like it to know whether it can be done or  
not?
Thanks a lot.
With Regards,
Abhinav.

Hey Abhinav,

We''ve taken a different tack with DTrace: rather than optimize the  
data flow
from kernel to user-land, we''ve given the user primitives to prune and
accumulate data as close to the source as possible to minimize data  
flow and
present the user with a concise answer.

May I ask why you''ve chosen this project?

Adam

On Jun 25, 2009, at 2:37 PM, abhinav duggal wrote:
> Hi all,
> I have a question on dtrace buffer management.I am kind of new to  
> dtrace development.I am looking into optimizing dtrace from the  
> point of view of reducing the memory copies from user to kernel  
> space and vice versa.
> Isn''t it possible to share memory between user and kernel space by
> preallocating and using that shared memory to transfer data between  
> user and kernel space? This would prevent those expensive memory  
> copies.Is there any problem in doing that.Sorry for my lack of  
> expertise with dtrace development,I would like it to know whether it  
> can be done or not?
> Thanks a lot.
> With Regards,
> Abhinav.
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org

--
Adam Leventhal, Fishworks                        http://blogs.sun.com/ahl

Hi Adam,
Ok I understand your point.The user space primitives like aggregates  
etc help prune and accumulate data.
I am looking at dtrace from the performance point of view.The idea is  
to look at various parts of dtrace,find potential places of  
optimization and see if anything can be improved.I am benchmarking  
parts of dtrace and looking for the bottlenecks.
I know that aggregates help in reducing the transfer of data between  
user and kernel space. But in case of running the system with lot of  
probes enabled it is still some overhead in terms of copy,or say not  
using aggregate functions or firing multiple dtrace sessions  
simultaneously. So I was thinking on reducing these overheads, if  
possible.One example I see is data transfer in megabytes from kernel  
to user for the dt_consume with DTRACEIOC_BUFSNAP ioctl(in case of  
heavy probing).Why not mmap those buffers and avoid that copy?
There are cases in which one wants to collect long kernel logs and  
analyze them,like for example looking for things like race conditions  
or atomicity violations. So if the overhead is reduced and we can have  
better performance it would be nice.
Thanks & Regards,
Abhinav.

On Jun 30, 2009, at 12:10 PM, Adam Leventhal wrote:
> Hey Abhinav,
>
> We''ve taken a different tack with DTrace: rather than optimize the
> data flow
> from kernel to user-land, we''ve given the user primitives to prune
and
> accumulate data as close to the source as possible to minimize data  
> flow and
> present the user with a concise answer.
>
> May I ask why you''ve chosen this project?
>
> Adam
>
> On Jun 25, 2009, at 2:37 PM, abhinav duggal wrote:
>
>> Hi all,
>> I have a question on dtrace buffer management.I am kind of new to  
>> dtrace development.I am looking into optimizing dtrace from the  
>> point of view of reducing the memory copies from user to kernel  
>> space and vice versa.
>> Isn''t it possible to share memory between user and kernel
space by
>> preallocating and using that shared memory to transfer data between  
>> user and kernel space? This would prevent those expensive memory
copies.Is
>>  there any problem in doing that.Sorry for my lack of expertise  
>> with dtrace development,I would like it to know whether it can be  
>> done or not?
>> Thanks a lot.
>> With Regards,
>> Abhinav.
>> _______________________________________________
>> dtrace-discuss mailing list
>> dtrace-discuss at opensolaris.org
>
>
> --
> Adam Leventhal, Fishworks                        http://blogs.sun.com/ahl
>