I am currently converting a single threaded process into a multi-threaded process by using pthreads. I have currently increased the stack size for each thread to 4MB, but since I am going to have a lot of threads I would like to reduce this to ~1/4MB. So how do I get there? Does anyone have any good ideas on how I can detect the stack requirements for the different functions (so I can start by analyzing the functions consuming the most stack space)... Thanks Trond This message posted from opensolaris.org
Trond Norbye wrote:> I am currently converting a single threaded process into a multi-threaded process by using pthreads. I have currently increased the stack size for each thread to 4MB, but since I am going to have a lot of threads I would like to reduce this to ~1/4MB. > > So how do I get there? Does anyone have any good ideas on how I can detect the stack requirements for the different functions (so I can start by analyzing the functions consuming the most stack space)... > > Thanks > > Trond > > > This message posted from opensolaris.org > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org >you can get and set the stack size using getrlimit, setrlimit system calls. <sys/resource.h> header file and man page for the same can give more details. -Deepti
>you can get and set the stack size using getrlimit, setrlimit system >calls. <sys/resource.h>Not for threads and not for the current process. In order to get stack use you can use "pmap -x" or just measure the difference between the deepest stack frame and the top of the stack. (Which requires identifying that point) You might be able to entise all the zero-fill-on-demand page faults on the thread stacks and get a stack trace at those points. If you needed to grow the thread stacks to 4MB, then there already is some issue. Casper
Casper.Dik at Sun.COM wrote:>> you can get and set the stack size using getrlimit, setrlimit system >> calls. <sys/resource.h> >> > > Not for threads and not for the current process. >why not? as far as I understand, getrlimit and setrlimit are in standard C lib as well and they have been used to dynamically get the soft an hard limit of the resources in the context of the current process. I vaguely recall there are per thread specific system calls as well available with posix.1c i.e. pthread_attr_setstacksize() or so to set specific attribute on thread such as thread stacks. one can do demand based stack/resource limiting for threads. am I missing something?> In order to get stack use you can use "pmap -x" or just measure > the difference between the deepest stack frame and the top of the > stack. (Which requires identifying that point) >above is manual. I guess user asked how to do it in application. although I don''t know how question is relevant to dtrace-discuss alias unless he wanted to trace it thru dtrace script, but email wanting to do it thru dtrace. also, user wants to trace functions stack usage, since is application based, I thought it is more straightforward to do it dynamically in application itself. his original email : >Trond Norbye wrote: > I am currently converting a single threaded process into a multi-threaded process by using pthreads. > I have currently increased the stack size for each thread to 4MB, but since I am going to have a > lot of threads I would like to reduce this to ~1/4MB. > So how do I get there? Does anyone have any good ideas on how I can detect the stack requirements > for the different functions (so I can start by analyzing the functions consuming the most stack space)... >Thanks >Trond> You might be able to entise all the zero-fill-on-demand page faults on > the thread stacks and get a stack trace at those points. > > If you needed to grow the thread stacks to 4MB, then there already > is some issue. >user wanted to reduce it to 1/4 MB. anyways... Trond can explain the details of his original questions. -Deepti> Casper > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org >
>why not? as far as I understand, getrlimit and setrlimit are in standard >C lib as welland they have been used to dynamically get the soft an hard >limit of the resources in the context of the current process.Yes, but they don''t work for the stack size parameters. The stack limit determines the area of virtual memory reserved for the stack; Solaris maps the lshared libraries directly under it, so it is *impossible* to grow the stack limit for the current process. The thread stacks are mapped at thread started; they are not governed by any stack resource limits . Casper
Casper.Dik at Sun.COM wrote:>> why not? as far as I understand, getrlimit and setrlimit are in standard >> C lib as welland they have been used to dynamically get the soft an hard >> limit of the resources in the context of the current process. >> > > Yes, but they don''t work for the stack size parameters. >they work for stack size as well as cpu time,maximum file size, data size, core file size etc. are all resources that are allowed to be tuned in the given process''s context. I recall only use of RLIMIT_STACK resource so far.> The stack limit determines the area of virtual memory reserved for > the stack;agree. stack grows within the allocated virtual memory /process address space for the given process.> Solaris maps the lshared libraries directly under it, so > it is *impossible* to grow the stack limit for the current process. >actually the syscall implementation cares for this detail such as if resource allocation is resulting intruding into shared library region or any other region, then further growing of stack size would result into error (-1) and stack will not be allowed to be grown (thus setrlimit would return -1 with errno EINVAL). since shared libraries are mapped into one of the processes (and not all) and all other user processes keep address of this shared lib, not all processes are giong to have lshared lib in their address space. so you are right, it is impossible to grow beyond hard limit (hard limits are settable only by su). and sucess of setrlimit system call is limited by per attribute specific hard limit as well total address space of the process. but still its a better option for processes/theads to efficiently utilize the whatever space available.> The thread stacks are mapped at thread started; they are > not governed by any stack resource limits . >it is implementation dependent. also, threads are initialized with default stack size, and newer libs and standards allow threads/processes to tweak these attributes/resources. pthread_attr_g/setstacksize and/or g/setrlimit are useful syscalls. -Deepti> Casper > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org >
deepti dhokte wrote:> Casper.Dik at Sun.COM wrote: >>> why not? as far as I understand, getrlimit and setrlimit are in >>> standard C lib as welland they have been used to dynamically get the >>> soft an hard >>> limit of the resources in the context of the current process. >>> >> >> Yes, but they don''t work for the stack size parameters. >> > they work for stack size as well as cpu time,maximum file size, data > size, core file size etc. > are all resources that are allowed to be tuned in the given process''s > context. > I recall only use of RLIMIT_STACK resource so far.I just checked solaris''s man page on getrlimit. apart from typical RLIMIT_STACK and company that are implemented by typical unices, since version 8, Solaris has RLIMIT_AS and RLIMIT_VMEM to increase process''s total address space. cool. -Deepti> >> The stack limit determines the area of virtual memory reserved for >> the stack; > agree. stack grows within the allocated virtual memory /process > address space for the given process. >> Solaris maps the lshared libraries directly under it, so >> it is *impossible* to grow the stack limit for the current process. >> > actually the syscall implementation cares for this detail such as if > resource allocation is resulting > intruding into shared library region or any other region, then further > growing of stack size > would result into error (-1) and stack will not be allowed to be grown > (thus setrlimit would return -1 > with errno EINVAL). > since shared libraries are mapped into one of the processes (and not > all) and all other user processes > keep address of this shared lib, not all processes are giong to have > lshared lib in their address space. > > so you are right, it is impossible to grow beyond hard limit (hard > limits are settable only by su). > and sucess of setrlimit system call is limited by per attribute > specific hard limit as well total address > space of the process. > but still its a better option for processes/theads to efficiently > utilize the whatever space available. >> The thread stacks are mapped at thread started; they are >> not governed by any stack resource limits . >> > it is implementation dependent. also, threads are initialized with > default stack size, > and newer libs and standards allow threads/processes to tweak these > attributes/resources. > pthread_attr_g/setstacksize and/or g/setrlimit are useful syscalls. > -Deepti >> Casper >> _______________________________________________ >> dtrace-discuss mailing list >> dtrace-discuss at opensolaris.org >> > >
>Casper.Dik at Sun.COM wrote: >>> why not? as far as I understand, getrlimit and setrlimit are in standard >>> C lib as welland they have been used to dynamically get the soft an hard >>> limit of the resources in the context of the current process. >>> >> >> Yes, but they don''t work for the stack size parameters. >> >they work for stack size as well as cpu time,maximum file size, data >size, core file size etc. >are all resources that are allowed to be tuned in the given process''s >context. >I recall only use of RLIMIT_STACK resource so far.Perhaps you should read the manual page: RLIMIT_STACK The maximum size of a process''s stack in bytes. The system will not automatically grow the stack beyond this limit. Within a process, setrlimit() will increase the limit on the size of your stack, but will not move current memory segments to allow for that growth. To guarantee that the process stack can grow to the limit, the limit must be altered prior to the execution of the process in which the new stack size is to be used. Within a multithreaded process, setrlimit() has no impact on the stack size limit for the calling thread if the calling thread is not the main thread. A call to setrlimit() for RLIMIT_STACK impacts only the main thread''s stack, and should be made only from the main thread, if at all. In short: no effect and thread stacks and little or no effect on the stack of the currently executing process (see the second hald of the second paragraph. Casper
>> Casper wrote: >> In order to get stack use you can use "pmap -x" or >> just measure the difference between the deepest stack frame and >> the top of the stack. (Which requires identifying that point)It was something like the last part there I was hoping to do with DTrace. Since the pid-provider can fire a probe on function calls, I was hoping there was some kind of magic that I could do to get the stack pointer ;-) Deepti wrote:> above is manual. I guess user asked how to do it in > application. although I don''t know how question is relevant to > dtrace-discuss alias unless he wanted to trace it thru dtrace script, but > email wanting to do it thru dtrace.See above.. I will of course appreciate all hints and tips ;) Casper wrote:> > You might be able to entise all the zero-fill-on-demand page faults on > > the thread stacks and get a stack trace at those points.How do I do such a thing? any pointers?> > If you needed to grow the thread stacks to 4MB, then there already > > is some issue.I know :-( The problem is that this is a big and complex daemon process so I don''t know where to start to look ;-) Trond This message posted from opensolaris.org
Trond Norbye writes:
> >> Casper wrote:
> >> In order to get stack use you can use "pmap -x" or
> >> just measure the difference between the deepest stack frame and
> >> the top of the stack. (Which requires identifying that point)
>
> It was something like the last part there I was hoping to do with
> DTrace. Since the pid-provider can fire a probe on function calls, I
> was hoping there was some kind of magic that I could do to get the
> stack pointer ;-)
>
> Deepti wrote:
> > above is manual. I guess user asked how to do it in
> > application. although I don''t know how question is relevant
to
> > dtrace-discuss alias unless he wanted to trace it thru dtrace script,
but
> > email wanting to do it thru dtrace.
>
> See above.. I will of course appreciate all hints and tips ;)
>
> Casper wrote:
> > > You might be able to entise all the zero-fill-on-demand page
faults on
> > > the thread stacks and get a stack trace at those points.
>
> How do I do such a thing? any pointers?
>
If you have the source, I believe the Sun Studio compilers
can generate code for this using -xcheck=stkovf.
-xcheck[=<a>[,<a>]] Generate runtime checks for error condition
<a>={%all|%none|stkovf|no%stkovf}
On every function entry (if compiled with the option), code is generated
to poke the stack endpoint.
-r
> > > If you needed to grow the thread stacks to 4MB, then there
already
> > > is some issue.
>
> I know :-( The problem is that this is a big and complex daemon process so
I don''t know where to start to look ;-)
>
> Trond
>
>
> This message posted from opensolaris.org
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org
Casper.Dik at Sun.COM wrote:>> Casper.Dik at Sun.COM wrote: >> >>>> why not? as far as I understand, getrlimit and setrlimit are in standard >>>> C lib as welland they have been used to dynamically get the soft an hard >>>> limit of the resources in the context of the current process. >>>> >>>> >>> Yes, but they don''t work for the stack size parameters. >>> >>> >> they work for stack size as well as cpu time,maximum file size, data >> size, core file size etc. >> are all resources that are allowed to be tuned in the given process''s >> context. >> I recall only use of RLIMIT_STACK resource so far. >> > > > Perhaps you should read the manual page: >yeah. this is how solaris man page reads. I said the same in earlier email, that beyond system allocated stack space it will not grow. but whatever stack space is available, given system calls it will let you efficiently utilize and manipulate the available stack space within process''s address space.> RLIMIT_STACK The maximum size of a process''s stack in > bytes. The system will not automatically > grow the stack beyond this limit. > >yeah, I said the same in earlier email, btw, trond at norbye.org results in email delivery failure. seems this email address stopped working after original email posting. :-|> Within a process, setrlimit() will increase > the limit on the size of your stack,no surprise... I said the same.> but > will not move current memory segments to > allow for that growth.makes sense. as it supposed to _not_.> To guarantee that the > process stack can grow to the limit, the > limit must be altered prior to the execution > of the process in which the new stack size > is to be used. >ok, above is system call invocation detail.> Within a multithreaded process, setrlimit() > has no impact on the stack size limit for > the calling thread if the calling thread is > not the main thread. A call to setrlimit() > for RLIMIT_STACK impacts only the main > thread''s stack, and should be made only from > the main thread, if at all. >I know setrlimit will not have impact on thread specific stack manipulations. I said, POSIX has pthread_attr_setstacksize to manipulate thread specific attributes.> > In short: no effect and thread stacksI never claimed setrlimit manipulates thread''s stack size. it affects processes. I always said, pthread_attr-setstacksize is the one to be used with thread stack manipulations.> and littlehmmn.. it''s a moot point. anyways, its something not relevant for dtrace-discuss alias also the email address of the poster seems invalid for future discussion, since the original email-poster is quiet for a while now and I get mail delivery errors with that email. so this stack manipulations discussion, if any further, we can take offline thanks, Deepti> or no effect > on the stack of the currently executing process (see the second > hald of the second paragraph. > > Casper >
>I said the same in earlier email, that beyond system allocated stack >space it will not grow.I said that *rlimit was completely useless for the control of thread stacks in the context of the discussion we were having. I am somewhat puzzled that the discussion on *rlimit continued after that point. Casper
Casper.Dik at Sun.COM wrote:>> I said the same in earlier email, that beyond system allocated stack >> space it will not grow. >> > > I said that *rlimit was completely useless for the control > of thread stacks in the context of the discussion we were > having. I am somewhat puzzled that the discussion on *rlimit > continued after that point. > >I see where the confusion started. The original email mentioned s/he has to change application and wants to modify stack size. my earlier response wasn''t an answer but just a pointer to look for a system call for thread stack size manipulation by telling in process context we do it using s/getrlimit etc. in the later emails, I was responding to many other rambling discussions we had but I had also told Trond about actual system call that can help him manipulate _thread_ _stacksize_ . anyways. Its a pointless discussion when we both agree. thanks, -deepti> Casper > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org >