dtrace discuss - Oct 2006 - Why do we need clause-local variables?

I''ve been explaining the difference between global variables, 
thread-local variables, and clause-local variables in an article I''m 
writing, and I''m getting hung up on the rationale for clause-local 
variables.  As I explain this to you all, perhaps I''ll see the flaw in 
my thinking, and not need to send this E-mail. :-)

In scope and functionality, global variables seem to be a superset of 
clause-local variables.  If you use clause-local variables to accumulate 
intermediate values in a set of probe clauses, you could just as easily 
have used globals.

At first entry into a set of clauses with the same probe, you have to 
initialize a clause-local variables before you use them, because you 
can''t rely on the value carrying over from the last firing of the 
probe.  If you used a global variable instead, you''d have to initialize
it because you _can_ count on the value carrying over from the last 
firing of the probe.

If you use the same named clause-local variable in two probe clauses 
associated with different probes, you''re suppose to treat them as two 
different variables, which means you have to initialize the variables at 
the start of each clause.  However, the two variables still have to have 
be the same type.  If you used a global variable in that situation, it 
would work the same way, and you''d have to live with the same
restrictions.

And then there''s the fact that in every test I''ve tried,
clause-local
variables seem to behave just like global variables, despite the 
warnings in the manual that the value is unreliable.  If a clause-local 
variable has never been initialized, it still seems to contain an 
initial zero.  If it''s referenced on the next probe firing without 
initializing it, it seems to still have its previous value.  If you set 
it in one probe clause, and then reference it in another probe clause 
associated with a different probe, it still has the previous value.  I 
realize that this could be coincidental, because D is just not bothering 
to clear the storage location, but I''ve tried making D reuse the 
location by making it create storage for other variables, but it just 
doesn''t seem to fail.

Now the last paragraph was more about how interesting it is that 
thread-locals behave like globals, but I don''t want to steer away from 
my real question.  If clause-locals don''t give you any additional 
functionality over globals, then why do we need them?

Thanks,
Chip

Hey Chip,

Clause-local variables are very different than global variables. Global
variables get their name from the fact that there is a single instance for
the D program; if probes on two different CPUs fire at about the same time
attempts to read or modify the global variable can interfere with one another.
Global variables store global, persistent state while clause-local variables
store temporary values for the duration of a probe firing (they might better
be descriped as probe-local, but that''s not much clearer).

If you use a global variable in place of a clause-local variable you will
eventually get incorrect data on an MP system if the activity being traced
can happen on different CPUs simultaneously.

Adam

On Thu, Oct 26, 2006 at 09:11:43PM -0500, Chip Bennett
wrote:
> I''ve been explaining the difference between global variables, 
> thread-local variables, and clause-local variables in an article
I''m
> writing, and I''m getting hung up on the rationale for clause-local
> variables.  As I explain this to you all, perhaps I''ll see the
flaw in
> my thinking, and not need to send this E-mail. :-)
> 
> In scope and functionality, global variables seem to be a superset of 
> clause-local variables.  If you use clause-local variables to accumulate 
> intermediate values in a set of probe clauses, you could just as easily 
> have used globals.
> 
> At first entry into a set of clauses with the same probe, you have to 
> initialize a clause-local variables before you use them, because you 
> can''t rely on the value carrying over from the last firing of the 
> probe.  If you used a global variable instead, you''d have to
initialize
> it because you _can_ count on the value carrying over from the last 
> firing of the probe.
> 
> If you use the same named clause-local variable in two probe clauses 
> associated with different probes, you''re suppose to treat them as
two
> different variables, which means you have to initialize the variables at 
> the start of each clause.  However, the two variables still have to have 
> be the same type.  If you used a global variable in that situation, it 
> would work the same way, and you''d have to live with the same
restrictions.
> 
> And then there''s the fact that in every test I''ve tried,
clause-local
> variables seem to behave just like global variables, despite the 
> warnings in the manual that the value is unreliable.  If a clause-local 
> variable has never been initialized, it still seems to contain an 
> initial zero.  If it''s referenced on the next probe firing without
> initializing it, it seems to still have its previous value.  If you set 
> it in one probe clause, and then reference it in another probe clause 
> associated with a different probe, it still has the previous value.  I 
> realize that this could be coincidental, because D is just not bothering 
> to clear the storage location, but I''ve tried making D reuse the 
> location by making it create storage for other variables, but it just 
> doesn''t seem to fail.
> 
> Now the last paragraph was more about how interesting it is that 
> thread-locals behave like globals, but I don''t want to steer away
from
> my real question.  If clause-locals don''t give you any additional 
> functionality over globals, then why do we need them?
> 
> Thanks,
> Chip
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org
-- 
Adam Leventhal, Solaris Kernel Development       http://blogs.sun.com/ahl

Hi Adam,

Let''s suppose two threads hit the same probe at the same time on a 
multiprocessor system.  Also, let''s assume the probe clause for this 
probe has a global variable and a clause-local variable being 
manipulated.  If the global variable can be corrupted by two instances 
of the one DIF program running simultaneously, then why couldn''t the 
clause-local variable, for which there is only one copy for a clause, 
also be corrupted for the same reason.

And even if the potential corruption is limited to global variables, at 
what level of corruption are we talking about.  For example, are strings 
written with lock, or could two threads be changing a string at the same 
time.  Or are we just talking about two clauses racing to change a 
variable and messing each other up.

Thanks,
Chip

Adam Leventhal wrote:
> Hey Chip,
>
> Clause-local variables are very different than global variables. Global
> variables get their name from the fact that there is a single instance for
> the D program; if probes on two different CPUs fire at about the same time
> attempts to read or modify the global variable can interfere with one
another.
> Global variables store global, persistent state while clause-local
variables
> store temporary values for the duration of a probe firing (they might
better
> be descriped as probe-local, but that''s not much clearer).
>
> If you use a global variable in place of a clause-local variable you will
> eventually get incorrect data on an MP system if the activity being traced
> can happen on different CPUs simultaneously.
>
> Adam
>
> On Thu, Oct 26, 2006 at 09:11:43PM -0500, Chip Bennett wrote:
>   
>> I''ve been explaining the difference between global variables, 
>> thread-local variables, and clause-local variables in an article
I''m
>> writing, and I''m getting hung up on the rationale for
clause-local
>> variables.  As I explain this to you all, perhaps I''ll see the
flaw in
>> my thinking, and not need to send this E-mail. :-)
>>
>> In scope and functionality, global variables seem to be a superset of 
>> clause-local variables.  If you use clause-local variables to
accumulate
>> intermediate values in a set of probe clauses, you could just as easily
>> have used globals.
>>
>> At first entry into a set of clauses with the same probe, you have to 
>> initialize a clause-local variables before you use them, because you 
>> can''t rely on the value carrying over from the last firing of
the
>> probe.  If you used a global variable instead, you''d have to
initialize
>> it because you _can_ count on the value carrying over from the last 
>> firing of the probe.
>>
>> If you use the same named clause-local variable in two probe clauses 
>> associated with different probes, you''re suppose to treat them
as two
>> different variables, which means you have to initialize the variables
at
>> the start of each clause.  However, the two variables still have to
have
>> be the same type.  If you used a global variable in that situation, it 
>> would work the same way, and you''d have to live with the same
restrictions.
>>
>> And then there''s the fact that in every test I''ve
tried, clause-local
>> variables seem to behave just like global variables, despite the 
>> warnings in the manual that the value is unreliable.  If a clause-local
>> variable has never been initialized, it still seems to contain an 
>> initial zero.  If it''s referenced on the next probe firing
without
>> initializing it, it seems to still have its previous value.  If you set
>> it in one probe clause, and then reference it in another probe clause 
>> associated with a different probe, it still has the previous value.  I 
>> realize that this could be coincidental, because D is just not
bothering
>> to clear the storage location, but I''ve tried making D reuse
the
>> location by making it create storage for other variables, but it just 
>> doesn''t seem to fail.
>>
>> Now the last paragraph was more about how interesting it is that 
>> thread-locals behave like globals, but I don''t want to steer
away from
>> my real question.  If clause-locals don''t give you any
additional
>> functionality over globals, then why do we need them?
>>
>> Thanks,
>> Chip
>> _______________________________________________
>> dtrace-discuss mailing list
>> dtrace-discuss at opensolaris.org
>>     
>
>

If you use a global variable in place of a clause-local variable you will
  eventually get incorrect data on an MP system if the activity being traced
  can happen on different CPUs simultaneously.

  Adam

In a D program, If you replace clause-local (this->a) with
thread-local (self->a), can this break some programs ?

Is the difference that the internals are more efficient when
appropriately using clause-local ? Or the program semantic
are just more explicit ?

-r

Hey Chip,
> Let''s suppose two threads hit the same probe at the same time on a
> multiprocessor system.  Also, let''s assume the probe clause for
this
> probe has a global variable and a clause-local variable being 
> manipulated.  If the global variable can be corrupted by two instances 
> of the one DIF program running simultaneously, then why couldn''t
the
> clause-local variable, for which there is only one copy for a clause, 
> also be corrupted for the same reason.
Clause-local variables have per-CPU storage so two concurrent probes
can''t
interfere with each other (only one probe can fire at a time on a given
virtual CPU).
> And even if the potential corruption is limited to global variables, at 
> what level of corruption are we talking about.  For example, are strings 
> written with lock, or could two threads be changing a string at the same 
> time.  Or are we just talking about two clauses racing to change a 
> variable and messing each other up.
Everything is done without locks when you''re in probe context so, yes,
though unlikely, you could end up with a string which was a combination of
two strings. You could also end up with two concurrent probe firings racing
for a variable as you suggest.

Adam

On Thu, Oct 26, 2006 at 11:43:01PM -0500, Chip Bennett wrote:
...
> 
> Adam Leventhal wrote:
> >Hey Chip,
> >
> >Clause-local variables are very different than global variables. Global
> >variables get their name from the fact that there is a single instance
for
> >the D program; if probes on two different CPUs fire at about the same
time
> >attempts to read or modify the global variable can interfere with one 
> >another.
> >Global variables store global, persistent state while clause-local 
> >variables
> >store temporary values for the duration of a probe firing (they might 
> >better
> >be descriped as probe-local, but that''s not much clearer).
> >
> >If you use a global variable in place of a clause-local variable you
will
> >eventually get incorrect data on an MP system if the activity being
traced
> >can happen on different CPUs simultaneously.
> >
> >Adam
> >
> >On Thu, Oct 26, 2006 at 09:11:43PM -0500, Chip Bennett wrote:
> >  
> >>I''ve been explaining the difference between global
variables,
> >>thread-local variables, and clause-local variables in an article
I''m
> >>writing, and I''m getting hung up on the rationale for
clause-local
> >>variables.  As I explain this to you all, perhaps I''ll see
the flaw in
> >>my thinking, and not need to send this E-mail. :-)
> >>
> >>In scope and functionality, global variables seem to be a superset
of
> >>clause-local variables.  If you use clause-local variables to
accumulate
> >>intermediate values in a set of probe clauses, you could just as
easily
> >>have used globals.
> >>
> >>At first entry into a set of clauses with the same probe, you have
to
> >>initialize a clause-local variables before you use them, because
you
> >>can''t rely on the value carrying over from the last firing
of the
> >>probe.  If you used a global variable instead, you''d have
to initialize
> >>it because you _can_ count on the value carrying over from the last
> >>firing of the probe.
> >>
> >>If you use the same named clause-local variable in two probe
clauses
> >>associated with different probes, you''re suppose to treat
them as two
> >>different variables, which means you have to initialize the
variables at
> >>the start of each clause.  However, the two variables still have to
have
> >>be the same type.  If you used a global variable in that situation,
it
> >>would work the same way, and you''d have to live with the
same
> >>restrictions.
> >>
> >>And then there''s the fact that in every test I''ve
tried, clause-local
> >>variables seem to behave just like global variables, despite the 
> >>warnings in the manual that the value is unreliable.  If a
clause-local
> >>variable has never been initialized, it still seems to contain an 
> >>initial zero.  If it''s referenced on the next probe firing
without
> >>initializing it, it seems to still have its previous value.  If you
set
> >>it in one probe clause, and then reference it in another probe
clause
> >>associated with a different probe, it still has the previous value.
I
> >>realize that this could be coincidental, because D is just not
bothering
> >>to clear the storage location, but I''ve tried making D
reuse the
> >>location by making it create storage for other variables, but it
just
> >>doesn''t seem to fail.
> >>
> >>Now the last paragraph was more about how interesting it is that 
> >>thread-locals behave like globals, but I don''t want to
steer away from
> >>my real question.  If clause-locals don''t give you any
additional
> >>functionality over globals, then why do we need them?
> >>
> >>Thanks,
> >>Chip
> >>_______________________________________________
> >>dtrace-discuss mailing list
> >>dtrace-discuss at opensolaris.org
> >>    
> >
> >  
-- 
Adam Leventhal, Solaris Kernel Development       http://blogs.sun.com/ahl

On Fri, Oct 27, 2006 at 10:24:06AM +0200, Roch - PAE
wrote:
> In a D program, If you replace clause-local (this->a) with
> thread-local (self->a), can this break some programs ?
>
> Is the difference that the internals are more efficient when
> appropriately using clause-local ? Or the program semantic
> are just more explicit ?
Thread-local variables are, indeed, less efficient, but you also have to
remember to always deallocate them (by assigning them a value of zero).
If you were to swap in a thread-local variable in place of a clause-local,
the only problem in terms of correctness would come if you didn''t
deallocate the variable and as a result ran out of dynamic variable space
(you''d see a warning from dtrace(1M) in that case).

Adam

-- 
Adam Leventhal, Solaris Kernel Development       http://blogs.sun.com/ahl

Adam Leventhal wrote:
> Clause-local variables have per-CPU storage so two concurrent probes
can''t
> interfere with each other (only one probe can fire at a time on a given
> virtual CPU).
>
>   
I was not aware that some things (other than the trace buffers), had 
per-CPU copies.  I can see why that would make clause-local variables a 
more reliable choice for intermediate values.

Thanks,
Chip