My application runs a backgrond process and after running for a while I get this error in my logs: #<Errno::EMFILE: Too many open files Lasty, the ONLY code I have that does anything with files is: File.open(image_path, "wb") do |file| file.puts file_content end Any idea what this means? -- Posted via http://www.ruby-forum.com/.
If you are on a shared host some restrict the number of open files per user/process. TextDrive for example is 200 open files. Perhaps you have hit the limit! -- Thanks, -Steve http://www.stevelongdo.com On 8/17/06, Ben Johnson <bjohnson@mediamanifest.com> wrote:> > My application runs a backgrond process and after running for a while I > get this error in my logs: > > #<Errno::EMFILE: Too many open files > > Lasty, the ONLY code I have that does anything with files is: > > File.open(image_path, "wb") do |file| > file.puts file_content > end > > Any idea what this means? > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060818/18885610/attachment.html
Steve Longdo wrote:> If you are on a shared host some restrict the number of open files per > user/process. TextDrive for example is 200 open files. Perhaps you > have > hit the limit! > -- > Thanks, > -Steve > http://www.stevelongdo.comI will open and write close to 500 files in 10 minutes. The thing is, this is a dedicated server. There is absolutely nothing else on this server except my application. How do I make this unlimited? -- Posted via http://www.ruby-forum.com/.
On Fri Aug 18, 2006 at 06:32:15AM +0200, Ben Johnson wrote:> Steve Longdo wrote: > > If you are on a shared host some restrict the number of open files per > > user/process. TextDrive for example is 200 open files. Perhaps you > > have > > hit the limit! > > -- > > Thanks, > > -Steve > > http://www.stevelongdo.com > > I will open and write close to 500 files in 10 minutes. The thing is, > this is a dedicated server. There is absolutely nothing else on this > server except my application. How do I make this unlimited?have you tried a non-block version of your file write. eg file=File.new;file.puts("stuff"),file.close to see if it doesnt leak handles?> > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
Ben Johnson wrote:> Steve Longdo wrote: >> If you are on a shared host some restrict the number of open files per >> user/process. TextDrive for example is 200 open files. Perhaps you >> have >> hit the limit! >> -- >> Thanks, >> -Steve >> http://www.stevelongdo.com > > I will open and write close to 500 files in 10 minutes. The thing is, > this is a dedicated server. There is absolutely nothing else on this > server except my application. How do I make this unlimited?I also found these links: http://www.phpman.info/index.php/man/open/2 If you scroll down on this link you will see the EMFILE error. It says that the total number of files has been reached. http://docsrv.caldera.com:8457/cgi-bin/info2html?(libc.info.gz)Limits%2520on%2520Resources Also on this link it says that each process has a open file limit. Obviously I am exceeding this limit because this process is supposed to run for days on end. Any idea how I can change this limit? -- Posted via http://www.ruby-forum.com/.
carmen wrote:> On Fri Aug 18, 2006 at 06:32:15AM +0200, Ben Johnson wrote: >> I will open and write close to 500 files in 10 minutes. The thing is, >> this is a dedicated server. There is absolutely nothing else on this >> server except my application. How do I make this unlimited? > > have you tried a non-block version of your file write. eg > file=File.new;file.puts("stuff"),file.close to see if it doesnt leak > handles?I would prefer not to do that because File.open automatically closes the file for you, Also if there is an exception raised in the File.open block it will close the file for you and then raise the exception up. -- Posted via http://www.ruby-forum.com/.
> I would prefer not to do that because File.open automatically closes the > file for youhow do you know it is closing them? what shows in lsof?
Ben Johnson
2006-Aug-18 04:52 UTC
[Rails] Re: Re: Re: #<Errno::EMFILE: Too many open files
carmen wrote:>> I would prefer not to do that because File.open automatically closes the >> file for you > > how do you know it is closing them? what shows in lsof?I know because that''s what File.open does for you. Just like file.close closes a file, so does File.open. I also found this link: http://www.patoche.org/LTT/kernel/00000128.html Hopefully that comes in handy for future users. -- Posted via http://www.ruby-forum.com/.
On Fri Aug 18, 2006 at 06:52:19AM +0200, Ben Johnson wrote:> carmen wrote: > >> I would prefer not to do that because File.open automatically closes the > >> file for you > > > > how do you know it is closing them? what shows in lsof? > > I know because that''s what File.open does for you.so youve used lsof or similar to determine that the open files are not from this code block, but are from the web server, or something else? oconsidering the nature of the error, it seems remarkably stubborn to assume this file handle is not leaking just because ''the docs say it doesnt''... valgrind is useful as well..
Ben Johnson
2006-Aug-18 05:08 UTC
[Rails] Re: Re: Re: Re: #<Errno::EMFILE: Too many open files
carmen wrote:> On Fri Aug 18, 2006 at 06:52:19AM +0200, Ben Johnson wrote: >> carmen wrote: >> >> I would prefer not to do that because File.open automatically closes the >> >> file for you >> > >> > how do you know it is closing them? what shows in lsof? >> >> I know because that''s what File.open does for you. > > so youve used lsof or similar to determine that the open files are not > from this code block, but are from the web server, or something else? > > oconsidering the nature of the error, it seems remarkably stubborn to > assume this file handle is not leaking just because ''the docs say it > doesnt''... > > valgrind is useful as well..I agree, I just figured opening and closing a file is not a hard task wouldn''t think that would be a bug in ruby, but you may be right. What is valgrind? -- Posted via http://www.ruby-forum.com/.
On Fri Aug 18, 2006 at 04:55:40AM +0000, carmen wrote:> On Fri Aug 18, 2006 at 06:52:19AM +0200, Ben Johnson wrote: > > carmen wrote: > > >> I would prefer not to do that because File.open automatically closes the > > >> file for you > > > > > > how do you know it is closing them? what shows in lsof? > > > > I know because that''s what File.open does for you.see http://www.hezmatt.org/~mpalmer/blog/general/ruby/blocks_as_resource_management.html "in theory, you could lose the file handle if there was an exception thrown in the block" i think theres only about a 33% chance that''s what it is..
Berger, Daniel
2006-Aug-18 13:56 UTC
[Rails] Re: Re: Re: #<Errno::EMFILE: Too many open files
> -----Original Message----- > From: rails-bounces@lists.rubyonrails.org > [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Ben Johnson > Sent: Thursday, August 17, 2006 10:52 PM > To: rails@lists.rubyonrails.org > Subject: [Rails] Re: Re: Re: #<Errno::EMFILE: Too many open files > > > carmen wrote: > >> I would prefer not to do that because File.open > automatically closes > >> the > >> file for you > > > > how do you know it is closing them? what shows in lsof? > > I know because that''s what File.open does for you. Just like > file.close > closes a file, so does File.open.I''d verify that just to be sure. In 1.8.3 a bug was introduced where subclasses of IO did not automatically call .close in block form. See http://rubyforge.org/tracker/index.php?func=detail&aid=4898&group_id=426 &atid=1698 (or ruby-core:6911). However, I was under the impression that the core classes were unaffected. I ran this snippet on my Windows box with 1.8.4 just to verify: 10000.times{ File.open("test.txt","w"){ |fh| fh.puts "hello" } sleep 1 } Then I watched the handle and thread count in the process manager. They never increased. I haven''t validated this on all platforms, however. Regards, Dan This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.