Hello Everyone, This is an announcement for version 0.4.3 of SCGI Rails Runner. This release has a slight redesign which should fix some people''s memory leak problems. It also updates all the documentation on the site to reflect the actual functionality of the latest version. Please grab it when you can and send me feedback: http://www.zedshaw.com/projects/scgi_rails/ == Memory Leaks Some people were reporting memory leaks. These were quite odd so I decided to do a "clean" thread management system. Before I just let the GC manage the threads since everything I read said the GC would leave the thread alone until it finished. It turns out that this is partially true. The new design keeps all the threads in a Queue and there''s a collector thread that simply joins each one and reports any exceptions. This should at a minimum remove SCGI::Processor as the main culprit in any memory leak issues. It''s about 100 lines of pure ruby so there''s not many ways it could be leaking memory. The down side to this safety is that it''s about a 2% performance drop. But, the advantage is that I can now start to expose the actual connected clients to controllers so they can control individual connections. == Documentation My website reflects the majority of the functionality I have in the latest release. There''s additional documentation I''d like to write for doing a good Apache configuration and for performance tuning. I''d appreciate anyone who can give me some brief instructions and sample configurations for Apache. Especially for making Apache fast. There is a sample cluster configuration for lighttpd in the documentation as well. == Config Generator Proposal I''m considering a RoR generator that would read the SCGI configuration and write out configuration files for Apache or lighttpd. The idea would be that the generator would create config files which you can include into your main config. Anyone interested in helping on this should contact me (or just write it). Thanks again! Zed A. Shaw http://www.zedshaw.com/
Ezra Zygmuntowicz
2005-Oct-24 03:09 UTC
Re: [ANN] SCGI Rails Runner 0.4.3 (Docs and Mem Leaks)
On Oct 23, 2005, at 6:47 PM, Zed A. Shaw wrote:> Hello Everyone, > > This is an announcement for version 0.4.3 of SCGI Rails Runner. This > release has a slight redesign which should fix some people''s memory > leak problems. It also updates all the documentation on the site to > reflect the actual functionality of the latest version. > > Please grab it when you can and send me feedback: > > http://www.zedshaw.com/projects/scgi_rails/ > > > == Memory Leaks > > Some people were reporting memory leaks. These were quite odd so I > decided to do a "clean" thread management system. Before I just let > the GC manage the threads since everything I read said the GC would > leave the thread alone until it finished. It turns out that this is > partially true. The new design keeps all the threads in a Queue > and there''s a collector thread that simply joins each one and reports > any exceptions. > > This should at a minimum remove SCGI::Processor as the main culprit in > any memory leak issues. It''s about 100 lines of pure ruby so there''s > not many ways it could be leaking memory. > > The down side to this safety is that it''s about a 2% performance drop. > But, the advantage is that I can now start to expose the actual > connected clients to controllers so they can control individual > connections. > > == Documentation > > My website reflects the majority of the functionality I have in the > latest release. There''s additional documentation I''d like to write > for > doing a good Apache configuration and for performance tuning. I''d > appreciate anyone who can give me some brief instructions and sample > configurations for Apache. Especially for making Apache fast. > > There is a sample cluster configuration for lighttpd in the > documentation as well. > > == Config Generator Proposal > > I''m considering a RoR generator that would read the SCGI configuration > and write out configuration files for Apache or lighttpd. The idea > would be that the generator would create config files which you can > include into your main config. Anyone interested in helping on this > should contact me (or just write it). > > Thanks again! > > Zed A. Shaw > http://www.zedshaw.com/ > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails > >Great stuff Zed- I''d be happy to help with the lighttpd generator stuff. What do you have in mind for me to do? Get back to me on or off list. Cheers- -Ezra Zygmuntowicz WebMaster Yakima Herald-Republic Newspaper ezra-gdxLOakOTQ9oetBuM9ipNAC/G2K4zDHf@public.gmane.org 509-577-7732
Zed,
I am testing the new version you released,and I am getting the same
memory leak in linux, the runner started in the normal way:
[INF][5937] Running in production mode on 127.0.0.1:21000
And I got a production.log (so my settings are not for development)
the application works but memory started in 26 and grew to over 60 in
just an hour.
In windows the memory the process crashes after a few
minutes/requests with a:
(erb):22: warning: don''t put space before argument parentheses
(erb):12: warning: don''t put space before argument parentheses
(erb):22: warning: don''t put space before argument parentheses
(erb):12: warning: don''t put space before argument parentheses
(erb):22: [BUG] Segmentation fault
ruby 1.8.2 (2004-12-25) [i386-mswin32]
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application''s support team for more information.
Is there any other information that could help you?
--
Aníbal Rojas
anibalrojas-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
zedshaw-dd7LMGGEL7NBDgjK7y7TUQ@public.gmane.org
2005-Oct-24 18:13 UTC
Re: [ANN] SCGI Rails Runner 0.4.3 (Docs and Mem Leaks)
Hey, I can''t really think of any other way that SRR could be causing a memory leak. I''ve scrubbed out all possible things I could do, and it''s pure ruby so no extensions or anything. Also that segfault looks like a major problem with your install. If anything''s being stored in thread-local storage then it should get cleared now with the new thread collector. Is it possible for you to give me the code for your project (off line) so I can test it myself and hunt down the bug. I''m seriously stumped at this point. I''ll also look at sending you a special scgi.rb file with some extra debugging in it. Zed A. Shaw http://www.zedshaw.com/> Zed, > > I am testing the new version you released,and I am getting the same > memory leak in linux, the runner started in the normal way: > > [INF][5937] Running in production mode on 127.0.0.1:21000 > > And I got a production.log (so my settings are not for development) > the application works but memory started in 26 and grew to over 60 in > just an hour. > > In windows the memory the process crashes after a few > minutes/requests with a: > > (erb):22: warning: don''t put space before argument parentheses > (erb):12: warning: don''t put space before argument parentheses > (erb):22: warning: don''t put space before argument parentheses > (erb):12: warning: don''t put space before argument parentheses > (erb):22: [BUG] Segmentation fault > ruby 1.8.2 (2004-12-25) [i386-mswin32] > > This application has requested the Runtime to terminate it in an unusual > way. > Please contact the application''s support team for more information. > > Is there any other information that could help you? > > -- > Aníbal Rojas > anibalrojas-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
Once I set a breakpoint and I''m reviewing the state in the Breakpointer, is it possible to trace my Rail''s app line by line? If not what methodology are people using to trace their code? Thanks, Joe
Zed,
Send me the debug-fearured version, I''ll test it as soon as I get
it. I''ll set up another Linux installation different from the Fedora
box were the problem us arising, and let you know, ok?
I think the problem is not in the code because the earlier version
0.3.1 in Windows was rock solid, no crash, and steady memory
comsumption. We managed to reproduce each behavior just switching SCGI
Rails Runner versions... But I can hand you the code, anyway it''s
GPLed.
Thanks for your help.
--
Aníbal Rojas
anibalrojas-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
On 10/24/05, zedshaw-dd7LMGGEL7NBDgjK7y7TUQ@public.gmane.org
<zedshaw-dd7LMGGEL7NBDgjK7y7TUQ@public.gmane.org>
wrote:> Hey,
>
> I can''t really think of any other way that SRR could be causing a
memory
> leak. I''ve scrubbed out all possible things I could do, and
it''s pure
> ruby so no extensions or anything. Also that segfault looks like a major
> problem with your install. If anything''s being stored in
thread-local
> storage then it should get cleared now with the new thread collector.
>
> Is it possible for you to give me the code for your project (off line) so
> I can test it myself and hunt down the bug. I''m seriously stumped
at this
> point.
>
> I''ll also look at sending you a special scgi.rb file with some
extra
> debugging in it.
>
> Zed A. Shaw
> http://www.zedshaw.com/
>
>
> > Zed,
> >
> > I am testing the new version you released,and I am getting the same
> > memory leak in linux, the runner started in the normal way:
> >
> > [INF][5937] Running in production mode on 127.0.0.1:21000
> >
> > And I got a production.log (so my settings are not for development)
> > the application works but memory started in 26 and grew to over 60 in
> > just an hour.
> >
> > In windows the memory the process crashes after a few
> > minutes/requests with a:
> >
> > (erb):22: warning: don''t put space before argument
parentheses
> > (erb):12: warning: don''t put space before argument
parentheses
> > (erb):22: warning: don''t put space before argument
parentheses
> > (erb):12: warning: don''t put space before argument
parentheses
> > (erb):22: [BUG] Segmentation fault
> > ruby 1.8.2 (2004-12-25) [i386-mswin32]
> >
> > This application has requested the Runtime to terminate it in an
unusual
> > way.
> > Please contact the application''s support team for more
information.
> >
> > Is there any other information that could help you?
> >
> > --
> > Aníbal Rojas
> > anibalrojas-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
> > _______________________________________________
> > Rails mailing list
> > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> > http://lists.rubyonrails.org/mailman/listinfo/rails
> >
>
>
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
Zed,
I must apologize, we trimmed down the memory leak problem to
ActiveRecord, and got rid of it just updating our app to 0.14.2 from
0.13.1 :-(
Somenting I really don''t understand is why the memory leak only
happened under Linux not on Windows, and actually I need to do some
more testing both under LInux and Windows to check if everything is
working ok.
--
Aníbal Rojas
http://www.lacaraoscura.com/
anibalrojas-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
On 10/24/05, zedshaw-dd7LMGGEL7NBDgjK7y7TUQ@public.gmane.org
<zedshaw-dd7LMGGEL7NBDgjK7y7TUQ@public.gmane.org>
wrote:> Hey,
>
> I can''t really think of any other way that SRR could be causing a
memory
> leak. I''ve scrubbed out all possible things I could do, and
it''s pure
> ruby so no extensions or anything. Also that segfault looks like a major
> problem with your install. If anything''s being stored in
thread-local
> storage then it should get cleared now with the new thread collector.
>
> Is it possible for you to give me the code for your project (off line) so
> I can test it myself and hunt down the bug. I''m seriously stumped
at this
> point.
>
> I''ll also look at sending you a special scgi.rb file with some
extra
> debugging in it.
>
> Zed A. Shaw
> http://www.zedshaw.com/
>
>
> > Zed,
> >
> > I am testing the new version you released,and I am getting the same
> > memory leak in linux, the runner started in the normal way:
> >
> > [INF][5937] Running in production mode on 127.0.0.1:21000
> >
> > And I got a production.log (so my settings are not for development)
> > the application works but memory started in 26 and grew to over 60 in
> > just an hour.
> >
> > In windows the memory the process crashes after a few
> > minutes/requests with a:
> >
> > (erb):22: warning: don''t put space before argument
parentheses
> > (erb):12: warning: don''t put space before argument
parentheses
> > (erb):22: warning: don''t put space before argument
parentheses
> > (erb):12: warning: don''t put space before argument
parentheses
> > (erb):22: [BUG] Segmentation fault
> > ruby 1.8.2 (2004-12-25) [i386-mswin32]
> >
> > This application has requested the Runtime to terminate it in an
unusual
> > way.
> > Please contact the application''s support team for more
information.
> >
> > Is there any other information that could help you?
> >
> > --
> > Aníbal Rojas
> > anibalrojas-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
> > _______________________________________________
> > Rails mailing list
> > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> > http://lists.rubyonrails.org/mailman/listinfo/rails
> >
>
>
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
Tobias Luetke
2005-Nov-01 16:26 UTC
Re: [ANN] SCGI Rails Runner 0.4.3 (Docs and Mem Leaks)
There was a memory leak fixed by Jamis Buck between those releases. Turned out that it only appeared in development mode and using fcgi or i take it scgi, not however using webrick. I''m pretty sure thats what you were seeing. On 10/31/05, Aníbal Rojas <anibalrojas-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Zed, > > I must apologize, we trimmed down the memory leak problem to > ActiveRecord, and got rid of it just updating our app to 0.14.2 from > 0.13.1 :-( > > Somenting I really don''t understand is why the memory leak only > happened under Linux not on Windows, and actually I need to do some > more testing both under LInux and Windows to check if everything is > working ok. > > -- > Aníbal Rojas > http://www.lacaraoscura.com/ > anibalrojas-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org > > On 10/24/05, zedshaw-dd7LMGGEL7NBDgjK7y7TUQ@public.gmane.org <zedshaw-dd7LMGGEL7NBDgjK7y7TUQ@public.gmane.org> wrote: > > Hey, > > > > I can''t really think of any other way that SRR could be causing a memory > > leak. I''ve scrubbed out all possible things I could do, and it''s pure > > ruby so no extensions or anything. Also that segfault looks like a major > > problem with your install. If anything''s being stored in thread-local > > storage then it should get cleared now with the new thread collector. > > > > Is it possible for you to give me the code for your project (off line) so > > I can test it myself and hunt down the bug. I''m seriously stumped at this > > point. > > > > I''ll also look at sending you a special scgi.rb file with some extra > > debugging in it. > > > > Zed A. Shaw > > http://www.zedshaw.com/ > > > > > > > Zed, > > > > > > I am testing the new version you released,and I am getting the same > > > memory leak in linux, the runner started in the normal way: > > > > > > [INF][5937] Running in production mode on 127.0.0.1:21000 > > > > > > And I got a production.log (so my settings are not for development) > > > the application works but memory started in 26 and grew to over 60 in > > > just an hour. > > > > > > In windows the memory the process crashes after a few > > > minutes/requests with a: > > > > > > (erb):22: warning: don''t put space before argument parentheses > > > (erb):12: warning: don''t put space before argument parentheses > > > (erb):22: warning: don''t put space before argument parentheses > > > (erb):12: warning: don''t put space before argument parentheses > > > (erb):22: [BUG] Segmentation fault > > > ruby 1.8.2 (2004-12-25) [i386-mswin32] > > > > > > This application has requested the Runtime to terminate it in an unusual > > > way. > > > Please contact the application''s support team for more information. > > > > > > Is there any other information that could help you? > > > > > > -- > > > Aníbal Rojas > > > anibalrojas-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org > > > _______________________________________________ > > > Rails mailing list > > > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > > > http://lists.rubyonrails.org/mailman/listinfo/rails > > > > > > > > > _______________________________________________ > > Rails mailing list > > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > > http://lists.rubyonrails.org/mailman/listinfo/rails > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- Tobi http://jadedpixel.com - modern e-commerce software http://typo.leetsoft.com - Open source weblog engine http://blog.leetsoft.com - Technical weblog