Hi All,
In order to track down some high memory usage problems I have in a
server process that is using the IronRuby engine I have run a test using
a single line script and have encountered an issue that looks
suspiciously like a memory leak. I''m scouring through the IronRuby code
commenting out a few static lists as I find them to see if they make any
difference but to date haven''t come across anything useful.
The test I''m running is:
static void Main(string[] args)
{
ScriptRuntime scriptRuntime = IronRuby.CreateRuntime();
for (int index = 0; index < 1000; index++)
{
ScriptScope rubyScope = scriptRuntime.CreateScope("IronRuby");
rubyScope.Execute("print ''execute\n''");
}
Console.WriteLine("finished");
Console.ReadLine();
}
Executing this three times on 3 separate runs the resultant process had
memory allocations of the folowing amounts as shown in the Windows Task
Manager:
31,940k
31,936k
31,936k
Executing the loop 10,000 times on 3 separate runs resulted in memory
usage of:
133,648k
134,036k
133,808k
Changing to
static void Main(string[] args)
{
for (int index = 0; index < 1000; index++)
{
Console.WriteLine("execute");
}
Console.WriteLine("finished");
Console.ReadLine();
}
For 1,000 iterations memory usage was 8,402k.
For 10,000 iterations memory usage was 8,704k.
For 1,000,000 iterations memory usage was 9,348.
The reason I''m looking into this is that I am having to jeep a close
eye
on my server process and re-start it every 3 or 4 days when the memory
usage gets up to 300 to 400MB.
Regards,
Aaron
--
Posted via http://www.ruby-forum.com/.
Additional info...
If you move the scriptRuntime.CreateScope method outside the loop then there
is no increase in memory.
Pete
-----Original Message-----
From: ironruby-core-bounces at rubyforge.org
[mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of Aaron Clauson
Sent: Saturday,12 July 12, 2008 14:00
To: ironruby-core at rubyforge.org
Subject: [Ironruby-core] Memory Usage Tests
Hi All,
In order to track down some high memory usage problems I have in a
server process that is using the IronRuby engine I have run a test using
a single line script and have encountered an issue that looks
suspiciously like a memory leak. I''m scouring through the IronRuby code
commenting out a few static lists as I find them to see if they make any
difference but to date haven''t come across anything useful.
The test I''m running is:
static void Main(string[] args)
{
ScriptRuntime scriptRuntime = IronRuby.CreateRuntime();
for (int index = 0; index < 1000; index++)
{
ScriptScope rubyScope = scriptRuntime.CreateScope("IronRuby");
rubyScope.Execute("print ''execute\n''");
}
Console.WriteLine("finished");
Console.ReadLine();
}
Executing this three times on 3 separate runs the resultant process had
memory allocations of the folowing amounts as shown in the Windows Task
Manager:
31,940k
31,936k
31,936k
Executing the loop 10,000 times on 3 separate runs resulted in memory
usage of:
133,648k
134,036k
133,808k
Changing to
static void Main(string[] args)
{
for (int index = 0; index < 1000; index++)
{
Console.WriteLine("execute");
}
Console.WriteLine("finished");
Console.ReadLine();
}
For 1,000 iterations memory usage was 8,402k.
For 10,000 iterations memory usage was 8,704k.
For 1,000,000 iterations memory usage was 9,348.
The reason I''m looking into this is that I am having to jeep a close
eye
on my server process and re-start it every 3 or 4 days when the memory
usage gets up to 300 to 400MB.
Regards,
Aaron
--
Posted via http://www.ruby-forum.com/.
_______________________________________________
Ironruby-core mailing list
Ironruby-core at rubyforge.org
http://rubyforge.org/mailman/listinfo/ironruby-core
Also...
If you only have the CreateScope method and not the Execute method in the
loop then there is no memory increase.
With both in the loop you get a linear increase in memory usage, even if you
regularly force a Garbage Collection, of approximately 10K per iteration.
Pete
-----Original Message-----
From: ironruby-core-bounces at rubyforge.org
[mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of Aaron Clauson
Sent: Saturday,12 July 12, 2008 14:00
To: ironruby-core at rubyforge.org
Subject: [Ironruby-core] Memory Usage Tests
Hi All,
In order to track down some high memory usage problems I have in a
server process that is using the IronRuby engine I have run a test using
a single line script and have encountered an issue that looks
suspiciously like a memory leak. I''m scouring through the IronRuby code
commenting out a few static lists as I find them to see if they make any
difference but to date haven''t come across anything useful.
The test I''m running is:
static void Main(string[] args)
{
ScriptRuntime scriptRuntime = IronRuby.CreateRuntime();
for (int index = 0; index < 1000; index++)
{
ScriptScope rubyScope = scriptRuntime.CreateScope("IronRuby");
rubyScope.Execute("print ''execute\n''");
}
Console.WriteLine("finished");
Console.ReadLine();
}
Executing this three times on 3 separate runs the resultant process had
memory allocations of the folowing amounts as shown in the Windows Task
Manager:
31,940k
31,936k
31,936k
Executing the loop 10,000 times on 3 separate runs resulted in memory
usage of:
133,648k
134,036k
133,808k
Changing to
static void Main(string[] args)
{
for (int index = 0; index < 1000; index++)
{
Console.WriteLine("execute");
}
Console.WriteLine("finished");
Console.ReadLine();
}
For 1,000 iterations memory usage was 8,402k.
For 10,000 iterations memory usage was 8,704k.
For 1,000,000 iterations memory usage was 9,348.
The reason I''m looking into this is that I am having to jeep a close
eye
on my server process and re-start it every 3 or 4 days when the memory
usage gets up to 300 to 400MB.
Regards,
Aaron
--
Posted via http://www.ruby-forum.com/.
_______________________________________________
Ironruby-core mailing list
Ironruby-core at rubyforge.org
http://rubyforge.org/mailman/listinfo/ironruby-core
Peter Bacon Darwin wrote:> Additional info... > If you move the scriptRuntime.CreateScope method outside the loop then > there > is no increase in memory.But from my understanding of script scopes I need a new scope when using local variables. I can have multiple scripts executing on different threads with different local variables so as far as I''m aware creating a new scope each time is the only way to allow that? static void Main(string[] args) { ScriptRuntime scriptRuntime = IronRuby.CreateRuntime(); for (int index = 0; index < 1000; index++) { ScriptScope rubyScope = scriptRuntime.CreateScope("IronRuby"); rubyScope.SetVariable("local1", "hello"); rubyScope.SetVariable("local2", "world"); rubyScope.Execute("print \"#{local1} #{local2}\n\""); } Console.WriteLine("finished"); Console.ReadLine(); } Regards, Aaron -- Posted via http://www.ruby-forum.com/.
Peter Bacon Darwin wrote:> Also... > If you only have the CreateScope method and not the Execute method in > the > loop then there is no memory increase. > With both in the loop you get a linear increase in memory usage, even if > you > regularly force a Garbage Collection, of approximately 10K per > iteration. > PeteHi Pete, If I take the Execute method out of the loop then I don''t get any significant increase in memory usage. However as per my post above I think I do need the CreateScope and Execute calls in the loop to allow for multiple simultaneous script executions. Regards, Aaron -- Posted via http://www.ruby-forum.com/.
Looks like keeping a limit on the number of ScriptScope''s being used is
the best approach for a long running process which makes sense. The code
below does not have a memory usage problem.
class Program
{
private static Queue<ScriptScope> m_scriptScopes = new
Queue<ScriptScope>();
private static AutoResetEvent m_scriptScopeAvailable = new
AutoResetEvent(false);
static void Main(string[] args)
{
ScriptRuntime m_scriptRuntime = IronRuby.CreateRuntime();
m_scriptScopes.Enqueue(m_scriptRuntime.CreateScope("IronRuby"));
m_scriptScopes.Enqueue(m_scriptRuntime.CreateScope("IronRuby"));
m_scriptScopes.Enqueue(m_scriptRuntime.CreateScope("IronRuby"));
for (int index = 0; index < 10000; index++)
{
ScriptScope rubyScope = null;
while (rubyScope == null)
{
lock (m_scriptScopes)
{
if (m_scriptScopes.Count > 0)
{
rubyScope = m_scriptScopes.Dequeue();
}
else
{
m_scriptScopeAvailable.Reset();
}
}
if (rubyScope == null)
{
m_scriptScopeAvailable.WaitOne();
}
}
ThreadPool.QueueUserWorkItem(new WaitCallback(RunRubyScript), new
object[] {rubyScope, index});
}
Console.ReadLine();
}
private static void RunRubyScript(object state)
{
ScriptScope rubyScope = (ScriptScope)((object[])state)[0];
int count = (int)((object[])state)[1];
rubyScope.ClearVariables();
rubyScope.SetVariable("local1", count);
rubyScope.SetVariable("local2", count);
rubyScope.Execute("print \"#{local1} - #{local2}\n\"");
lock (m_scriptScopes)
{
m_scriptScopes.Enqueue(rubyScope);
m_scriptScopeAvailable.Set();
}
}
}
Regards,
Aaron
--
Posted via http://www.ruby-forum.com/.
Hi Aaron, Sorry I wasn''t trying to suggest a solution for your problem. I was just trying to narrow down where the increase in memory is coming from. I suspect that the cause is initializing or calling into the core libraries but I can''t isolate that bit. Pete -----Original Message----- From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of Aaron Clauson Sent: Saturday,12 July 12, 2008 15:33 To: ironruby-core at rubyforge.org Subject: Re: [Ironruby-core] Memory Usage Tests Peter Bacon Darwin wrote:> Also... > If you only have the CreateScope method and not the Execute method in > the > loop then there is no memory increase. > With both in the loop you get a linear increase in memory usage, even if > you > regularly force a Garbage Collection, of approximately 10K per > iteration. > PeteHi Pete, If I take the Execute method out of the loop then I don''t get any significant increase in memory usage. However as per my post above I think I do need the CreateScope and Execute calls in the loop to allow for multiple simultaneous script executions. Regards, Aaron -- Posted via http://www.ruby-forum.com/. _______________________________________________ Ironruby-core mailing list Ironruby-core at rubyforge.org http://rubyforge.org/mailman/listinfo/ironruby-core
I''m working on working set issues right now. It should be better soon. Will look at this one especially. Thanks, Tomas -----Original Message----- From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of Peter Bacon Darwin Sent: Saturday, July 12, 2008 9:44 AM To: ironruby-core at rubyforge.org Subject: Re: [Ironruby-core] Memory Usage Tests Hi Aaron, Sorry I wasn''t trying to suggest a solution for your problem. I was just trying to narrow down where the increase in memory is coming from. I suspect that the cause is initializing or calling into the core libraries but I can''t isolate that bit. Pete -----Original Message----- From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of Aaron Clauson Sent: Saturday,12 July 12, 2008 15:33 To: ironruby-core at rubyforge.org Subject: Re: [Ironruby-core] Memory Usage Tests Peter Bacon Darwin wrote:> Also... > If you only have the CreateScope method and not the Execute method in > the > loop then there is no memory increase. > With both in the loop you get a linear increase in memory usage, even if > you > regularly force a Garbage Collection, of approximately 10K per > iteration. > PeteHi Pete, If I take the Execute method out of the loop then I don''t get any significant increase in memory usage. However as per my post above I think I do need the CreateScope and Execute calls in the loop to allow for multiple simultaneous script executions. Regards, Aaron -- Posted via http://www.ruby-forum.com/. _______________________________________________ Ironruby-core mailing list Ironruby-core at rubyforge.org http://rubyforge.org/mailman/listinfo/ironruby-core _______________________________________________ Ironruby-core mailing list Ironruby-core at rubyforge.org http://rubyforge.org/mailman/listinfo/ironruby-core
Peter Bacon Darwin wrote:> Hi Aaron, > Sorry I wasn''t trying to suggest a solution for your problem. I was > just > trying to narrow down where the increase in memory is coming from. > I suspect that the cause is initializing or calling into the core > libraries > but I can''t isolate that bit. > PeteHi Pete, Thanks for the suggestion in the first place! I realise IronRuby is still under heavy development so expect to have to do the odd workaround. Tomas thanks for taking a look. Regards, Aaron -- Posted via http://www.ruby-forum.com/.
Tomas Matousek wrote:> I''m working on working set issues right now. It should be better soon. > Will look at this one especially. > > Thanks, > TomasHi Tomas, I didn''t quite nail down where the high memory usage was coming from either but I did get to a stage where I could determine it was happening after the RubyScope was being created. So up until the call to InvokeTarget in the ScriptCode everything was fine. That obersvation is probably not particularly useful since all the magic seems to happen when InvokeTarget is called. Another observation is that in the SymbolTable class in Microsoft.Scripting.Core there are only objects being put onto the two static dictionaries. Nothing is ever taken off them. I would have thought the SymbolTable would need to somehow know to remove objects after the ScriptScope that puts them on has diesd. Regards, Aaron -- Posted via http://www.ruby-forum.com/.