Wondering what the status is on EM and Ruby 1.9--do the jive at this point? Thanks for all your work. -- -Roger Pack For God hath not given us the spirit of fear; but of power, and of love, and of a sound mind" -- 2 Timothy 1:7
On Jan 10, 2008 4:23 PM, Roger Pack <rogerpack2005 at gmail.com> wrote:> Wondering what the status is on EM and Ruby 1.9--do the jive at this point? > Thanks for all your work.It''s a trivial 2 line change to get your favorite version of EM working with 1.9. Since Base64 has been deprecated, you need to replace their usage in the SMTP client/server with the appropriate pack/unpack commands. Then, it just works. If you check out the trunk, it has this fix and some other things in it. There is a decent speed boost. Analogger, on my server, under 1.8.5/6 runs at around 78000 messages per second for short messages. When I first ran it under 1.9, that dropped to 6200/second. Yeah. Horrific. But....a little digging showed that String#slice! was the culprit (it is absurdly slow on 1.9; that slowdown was caused by a single line in the code). Rewriting that line to not use slice! resulted in a modest slowdown on 1.8 -- about 60000/second. However, when I run that code under 1.9 it runs at 135000/second. It''s a nice speedup. Kirk Haines
I also did some small benchmarks w/ Thin on 1.9 and it''s definitely faster: [~/projects/thin] ruby benchmark/simple.rb server request concurrency req/s failures ====================================================WEBrick 1000 1 389.31 0 Mongrel 1000 1 1495.70 0 EMongrel 1000 1 2070.35 0 Thin 1000 1 2219.45 0 WEBrick 1000 10 366.16 0 Mongrel 1000 10 1501.02 0 EMongrel 1000 10 2562.43 0 Thin 1000 10 3129.64 0 WEBrick 1000 100 ERROR Mongrel 1000 100 1500.82 0 EMongrel 1000 100 3031.29 0 Thin 1000 100 3372.49 0 [~/projects/thin] ruby19 benchmark/simple.rb server request concurrency req/s failures ====================================================WEBrick 1000 1 506.33 0 Mongrel 1000 1 1768.31 0 Thin 1000 1 2528.11 0 WEBrick 1000 10 461.66 0 Mongrel 1000 10 1541.35 0 Thin 1000 10 4003.19 0 WEBrick 1000 100 ERROR Mongrel 1000 100 1313.19 0 Thin 1000 100 4154.67 0 On 10-Jan-08, at 6:30 PM, Kirk Haines wrote:> On Jan 10, 2008 4:23 PM, Roger Pack <rogerpack2005 at gmail.com> wrote: >> Wondering what the status is on EM and Ruby 1.9--do the jive at >> this point? >> Thanks for all your work. > > It''s a trivial 2 line change to get your favorite version of EM > working with 1.9. > > Since Base64 has been deprecated, you need to replace their usage in > the SMTP client/server with the appropriate pack/unpack commands. > > Then, it just works. > > If you check out the trunk, it has this fix and some other things in > it. > > There is a decent speed boost. > > Analogger, on my server, under 1.8.5/6 runs at around 78000 messages > per second for short messages. > > When I first ran it under 1.9, that dropped to 6200/second. Yeah. > Horrific. But....a little digging showed that String#slice! was the > culprit (it is absurdly slow on 1.9; that slowdown was caused by a > single line in the code). > > Rewriting that line to not use slice! resulted in a modest slowdown on > 1.8 -- about 60000/second. However, when I run that code under 1.9 it > runs at 135000/second. It''s a nice speedup. > > > Kirk Haines > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk
Marc-Andr?, you''re running an 8-core Mac Pro, aren''t you? ;-) My MBP shows the same % increase in performance, but with numbers about 4 times smaller. On Jan 10, 2008 4:52 PM, Marc-Andr? Cournoyer <macournoyer at gmail.com> wrote:> I also did some small benchmarks w/ Thin on 1.9 and it''s definitely > faster: > > [~/projects/thin] ruby benchmark/simple.rb > server request concurrency req/s failures > ====================================================> WEBrick 1000 1 389.31 0 > Mongrel 1000 1 1495.70 0 > EMongrel 1000 1 2070.35 0 > Thin 1000 1 2219.45 0 > WEBrick 1000 10 366.16 0 > Mongrel 1000 10 1501.02 0 > EMongrel 1000 10 2562.43 0 > Thin 1000 10 3129.64 0 > WEBrick 1000 100 ERROR > Mongrel 1000 100 1500.82 0 > EMongrel 1000 100 3031.29 0 > Thin 1000 100 3372.49 0 > > [~/projects/thin] ruby19 benchmark/simple.rb > server request concurrency req/s failures > ====================================================> WEBrick 1000 1 506.33 0 > Mongrel 1000 1 1768.31 0 > Thin 1000 1 2528.11 0 > WEBrick 1000 10 461.66 0 > Mongrel 1000 10 1541.35 0 > Thin 1000 10 4003.19 0 > WEBrick 1000 100 ERROR > Mongrel 1000 100 1313.19 0 > Thin 1000 100 4154.67 0 > > > > On 10-Jan-08, at 6:30 PM, Kirk Haines wrote: > > > On Jan 10, 2008 4:23 PM, Roger Pack <rogerpack2005 at gmail.com> wrote: > >> Wondering what the status is on EM and Ruby 1.9--do the jive at > >> this point? > >> Thanks for all your work. > > > > It''s a trivial 2 line change to get your favorite version of EM > > working with 1.9. > > > > Since Base64 has been deprecated, you need to replace their usage in > > the SMTP client/server with the appropriate pack/unpack commands. > > > > Then, it just works. > > > > If you check out the trunk, it has this fix and some other things in > > it. > > > > There is a decent speed boost. > > > > Analogger, on my server, under 1.8.5/6 runs at around 78000 messages > > per second for short messages. > > > > When I first ran it under 1.9, that dropped to 6200/second. Yeah. > > Horrific. But....a little digging showed that String#slice! was the > > culprit (it is absurdly slow on 1.9; that slowdown was caused by a > > single line in the code). > > > > Rewriting that line to not use slice! resulted in a modest slowdown on > > 1.8 -- about 60000/second. However, when I run that code under 1.9 it > > runs at 135000/second. It''s a nice speedup. > > > > > > Kirk Haines > > _______________________________________________ > > Eventmachine-talk mailing list > > Eventmachine-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/eventmachine-talk > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-- Cheers, Kevin Williams http://www.bantamtech.com/
On Jan 10, 2008 6:23 PM, Roger Pack <rogerpack2005 at gmail.com> wrote:> Wondering what the status is on EM and Ruby 1.9--do the jive at this > point? > Thanks for all your work. >To add a bit to Kirk''s reply: if you compile the current HEAD revision (remember, it''s under trunk now, not version_0) under Ruby 1.9, you''ll automatically get the correct treatment for thread interoperability. A build of EM compiled under 1.9 will NOT run on 1.8 A build of EM compiled under 1.8 WILL run on 1.9, but it might be a bit slower. My tests should that under heavy network loads, there''s no measurable difference between the thread-interoperability mechanisms. I know that there was a lot of anticipation of this in certain quarters, but as far as I''m concerned there''s no benefit. However, you will get a speed boost on 1.9 arising from other factors. One thing I''m in the middle of is porting the EM reactor core to C from C++, which should make it more compatible with various compiler environments, as well as with core Ruby. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20080110/2060fea9/attachment.html
No, I got the latest MBP, very weird... what are your numbers? On 10-Jan-08, at 7:38 PM, Kevin Williams wrote:> Marc-Andr?, you''re running an 8-core Mac Pro, aren''t you? ;-) My MBP > shows the same % increase in performance, but with numbers about 4 > times smaller. > > On Jan 10, 2008 4:52 PM, Marc-Andr? Cournoyer > <macournoyer at gmail.com> wrote: >> I also did some small benchmarks w/ Thin on 1.9 and it''s definitely >> faster: >> >> [~/projects/thin] ruby benchmark/simple.rb >> server request concurrency req/s failures >> ====================================================>> WEBrick 1000 1 389.31 0 >> Mongrel 1000 1 1495.70 0 >> EMongrel 1000 1 2070.35 0 >> Thin 1000 1 2219.45 0 >> WEBrick 1000 10 366.16 0 >> Mongrel 1000 10 1501.02 0 >> EMongrel 1000 10 2562.43 0 >> Thin 1000 10 3129.64 0 >> WEBrick 1000 100 ERROR >> Mongrel 1000 100 1500.82 0 >> EMongrel 1000 100 3031.29 0 >> Thin 1000 100 3372.49 0 >> >> [~/projects/thin] ruby19 benchmark/simple.rb >> server request concurrency req/s failures >> ====================================================>> WEBrick 1000 1 506.33 0 >> Mongrel 1000 1 1768.31 0 >> Thin 1000 1 2528.11 0 >> WEBrick 1000 10 461.66 0 >> Mongrel 1000 10 1541.35 0 >> Thin 1000 10 4003.19 0 >> WEBrick 1000 100 ERROR >> Mongrel 1000 100 1313.19 0 >> Thin 1000 100 4154.67 0 >> >> >> >> On 10-Jan-08, at 6:30 PM, Kirk Haines wrote: >> >>> On Jan 10, 2008 4:23 PM, Roger Pack <rogerpack2005 at gmail.com> wrote: >>>> Wondering what the status is on EM and Ruby 1.9--do the jive at >>>> this point? >>>> Thanks for all your work. >>> >>> It''s a trivial 2 line change to get your favorite version of EM >>> working with 1.9. >>> >>> Since Base64 has been deprecated, you need to replace their usage in >>> the SMTP client/server with the appropriate pack/unpack commands. >>> >>> Then, it just works. >>> >>> If you check out the trunk, it has this fix and some other things in >>> it. >>> >>> There is a decent speed boost. >>> >>> Analogger, on my server, under 1.8.5/6 runs at around 78000 messages >>> per second for short messages. >>> >>> When I first ran it under 1.9, that dropped to 6200/second. Yeah. >>> Horrific. But....a little digging showed that String#slice! was the >>> culprit (it is absurdly slow on 1.9; that slowdown was caused by a >>> single line in the code). >>> >>> Rewriting that line to not use slice! resulted in a modest >>> slowdown on >>> 1.8 -- about 60000/second. However, when I run that code under >>> 1.9 it >>> runs at 135000/second. It''s a nice speedup. >>> >>> >>> Kirk Haines >>> _______________________________________________ >>> Eventmachine-talk mailing list >>> Eventmachine-talk at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/eventmachine-talk >> >> _______________________________________________ >> Eventmachine-talk mailing list >> Eventmachine-talk at rubyforge.org >> http://rubyforge.org/mailman/listinfo/eventmachine-talk >> > > > > -- > Cheers, > > Kevin Williams > http://www.bantamtech.com/ > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk
My MBP is the fastest of the old 32-bit Core Duos, probably 40% slower. I got about 1100 on a simple ramaze 0.3.0 hello world example. With webrick, it was about 120. This is for ruby1.9, I should mention. On Jan 10, 2008 7:29 PM, Marc-Andr? Cournoyer <macournoyer at gmail.com> wrote:> No, I got the latest MBP, very weird... what are your numbers? > > > On 10-Jan-08, at 7:38 PM, Kevin Williams wrote: > > > Marc-Andr?, you''re running an 8-core Mac Pro, aren''t you? ;-) My MBP > > shows the same % increase in performance, but with numbers about 4 > > times smaller. > > > > On Jan 10, 2008 4:52 PM, Marc-Andr? Cournoyer > > <macournoyer at gmail.com> wrote: > >> I also did some small benchmarks w/ Thin on 1.9 and it''s definitely > >> faster: > >> > >> [~/projects/thin] ruby benchmark/simple.rb > >> server request concurrency req/s failures > >> ====================================================> >> WEBrick 1000 1 389.31 0 > >> Mongrel 1000 1 1495.70 0 > >> EMongrel 1000 1 2070.35 0 > >> Thin 1000 1 2219.45 0 > >> WEBrick 1000 10 366.16 0 > >> Mongrel 1000 10 1501.02 0 > >> EMongrel 1000 10 2562.43 0 > >> Thin 1000 10 3129.64 0 > >> WEBrick 1000 100 ERROR > >> Mongrel 1000 100 1500.82 0 > >> EMongrel 1000 100 3031.29 0 > >> Thin 1000 100 3372.49 0 > >> > >> [~/projects/thin] ruby19 benchmark/simple.rb > >> server request concurrency req/s failures > >> ====================================================> >> WEBrick 1000 1 506.33 0 > >> Mongrel 1000 1 1768.31 0 > >> Thin 1000 1 2528.11 0 > >> WEBrick 1000 10 461.66 0 > >> Mongrel 1000 10 1541.35 0 > >> Thin 1000 10 4003.19 0 > >> WEBrick 1000 100 ERROR > >> Mongrel 1000 100 1313.19 0 > >> Thin 1000 100 4154.67 0 > >> > >> > >> > >> On 10-Jan-08, at 6:30 PM, Kirk Haines wrote: > >> > >>> On Jan 10, 2008 4:23 PM, Roger Pack <rogerpack2005 at gmail.com> wrote: > >>>> Wondering what the status is on EM and Ruby 1.9--do the jive at > >>>> this point? > >>>> Thanks for all your work. > >>> > >>> It''s a trivial 2 line change to get your favorite version of EM > >>> working with 1.9. > >>> > >>> Since Base64 has been deprecated, you need to replace their usage in > >>> the SMTP client/server with the appropriate pack/unpack commands. > >>> > >>> Then, it just works. > >>> > >>> If you check out the trunk, it has this fix and some other things in > >>> it. > >>> > >>> There is a decent speed boost. > >>> > >>> Analogger, on my server, under 1.8.5/6 runs at around 78000 messages > >>> per second for short messages. > >>> > >>> When I first ran it under 1.9, that dropped to 6200/second. Yeah. > >>> Horrific. But....a little digging showed that String#slice! was the > >>> culprit (it is absurdly slow on 1.9; that slowdown was caused by a > >>> single line in the code). > >>> > >>> Rewriting that line to not use slice! resulted in a modest > >>> slowdown on > >>> 1.8 -- about 60000/second. However, when I run that code under > >>> 1.9 it > >>> runs at 135000/second. It''s a nice speedup. > >>> > >>> > >>> Kirk Haines > >>> _______________________________________________ > >>> Eventmachine-talk mailing list > >>> Eventmachine-talk at rubyforge.org > >>> http://rubyforge.org/mailman/listinfo/eventmachine-talk > >> > >> _______________________________________________ > >> Eventmachine-talk mailing list > >> Eventmachine-talk at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/eventmachine-talk > >> > > > > > > > > -- > > Cheers, > > > > Kevin Williams > > http://www.bantamtech.com/ > > _______________________________________________ > > Eventmachine-talk mailing list > > Eventmachine-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/eventmachine-talk > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-- Cheers, Kevin Williams http://www.bantamtech.com/
> > > It''s a trivial 2 line change to get your favorite version of EM > > > working with 1.9.Ahh I notice now the line for how to accomplish this within kqueue. Thanks for making it work already with select and epoll :) To Kirk:> > > Rewriting that line to not use slice! resulted in a modest slowdown on > > > 1.8 -- about 60000/second.Maybe an if statement would avoid the slowdown? Or maybe not. :)> A build of EM compiled under 1.9 will NOT run on 1.8 > > A build of EM compiled under 1.8 WILL run on 1.9, but it might be a bit > slower.Thanks for the info!> My tests should that under heavy network loads, there''s no measurable > difference between the thread-interoperability mechanisms. I know that there > was a lot of anticipation of this in certain quarters, but as far as I''m > concerned there''s no benefit.Yeah I guess 1.9 is multi-threaded but only allows one thread to operate at a time. Maybe that''s the reason?> It''s possible to accept a socket that''s too high-numbered to work in a(c connect''s can yield the same difficulty)> select set, if the process is permitted to create a larger number of > descriptors than FD_SETSIZE. You''re really not likely to see this on > Windows, however. (On some versions of Windows, file descriptors are > pointers rather than index numbers, anyway, so there''s no meaningful numeric > comparison.) On Unix you can make it happen if you go out of your way.It''s actually quite easy on windows--windows allows at least 256 descriptors/process, and select only accepts 64. You wouldn''t think it would be possible. Go Microsoft.> I think that our future direction really should be away from select and > toward the native high-performance polling mechanism on all platforms. We > have epoll and kqueue now. We should add /dev/poll for Solaris and IOCP for > Windows. Then the only platforms left on select will be be back versions of > Linux and less-common Unixes.I write a lot of Ruby on windows. Might be worth considering supporting it (as well as supporting he newbies out there that don''t use epoll or kqueue by default). I posted a copy of the patches for this to https://rubyforge.org/tracker/?atid=6060&group_id=1555&func=browse for easy access :) As a note on windows, it might be possible to overcome the 64 descriptor issue by recompiling all of ruby setting FD_SETSIZE=2048 (which is possible using a mingw build), or maybe by having ruby link against a different version of msvcrtXX.dll (also possible using mingw, as well as by using VC8 instead of VC6). "Just" setting FD_SETSIZE to a higher number when compiling the extension using mingw doesn''t seem to work. Of course, linking against a different .dll would mean you''d need to recompile Ruby and all its extensions against it, too, but it would be possible. For now I''d just leave the limit at 64 :) Thanks for your help! -Roger -- Wherefore if ye shall press forward, feasting upon the word of Christ, and endure to the end, behold thus saith the Father: ye shall have eternal life. - 2 Nephi 31:20
> > It''s possible to accept a socket that''s too high-numbered to work in a > (c connect''s can yield the same difficulty)as a note, on win32 I read that you can use HANDLE''s instead of descriptors, to overcome the 256 file (or 2048 file) limit. I''m not sure if that would help overcome select''s limitations, though, as Tony pointed out. I also wonder if running out of descriptors shows itself as a problem for the ruby version or the jruby version. Might be worth running the tests against them. Take care. Go EM. -Roger
On Jan 14, 2008 11:57 AM, Roger Pack <rogerpack2005 at gmail.com> wrote:> > > It''s possible to accept a socket that''s too high-numbered to work in a > > (c connect''s can yield the same difficulty) > > as a note, on win32 I read that you can use HANDLE''s instead of > descriptors, to overcome the 256 file (or 2048 file) limit. I''m not > sure if that would help overcome select''s limitations, though, as Tony > pointed out. >One of these days, we''ll just do it right and use IOCP on Windows. It''s a rewrite of the reactor core but there are worse things in the world. The reason I didn''t complete it the last time I tried is because the Win32 API for nonblocking IO isn''t complete on all versions of Windows. You don''t get all the features unless you''re on XP or Vista. This is quite a problem if you use W2K or Server 2003, as many shops still do.> I also wonder if running out of descriptors shows itself as a problem > for the ruby version or the jruby version. Might be worth running the > tests against them.Probably does on pure Ruby. Probably doesn''t on jruby. But I haven''t tested. Anyone care to give it a try?> >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20080115/7fab592a/attachment-0001.html
On Jan 12, 2008 11:35 AM, Roger Pack <rogerpack2005 at gmail.com> wrote:> > My tests should that under heavy network loads, there''s no measurable > > difference between the thread-interoperability mechanisms. I know that > there > > was a lot of anticipation of this in certain quarters, but as far as I''m > > concerned there''s no benefit. > > Yeah I guess 1.9 is multi-threaded but only allows one thread to > operate at a time. Maybe that''s the reason? >These tests are all single-threaded to begin with. I can construct a stress test that uses non-network events, for example, a large number of EM''s Erlang-like spawned processes. These tests run far faster on 1.9 than on 1.8. As soon as large numbers of network events come into the picture, the improvement is much less noticeable. Since EM doesn''t rely on Ruby to do any of its network handling, I think the difference is just because the performance profile becomes dominated by the kernel. But that''s just a hypothesis.> > It''s possible to accept a socket that''s too high-numbered to work in a > (c connect''s can yield the same difficulty) > > select set, if the process is permitted to create a larger number of > > descriptors than FD_SETSIZE. You''re really not likely to see this on > > Windows, however. (On some versions of Windows, file descriptors are > > pointers rather than index numbers, anyway, so there''s no meaningful > numeric > > comparison.) On Unix you can make it happen if you go out of your way. > It''s actually quite easy on windows--windows allows at least 256 > descriptors/process, and select only accepts 64. You wouldn''t think > it would be possible. Go Microsoft. >Are you sure? I''ve used select loops in Microsoft for years and I don''t remember a 64-descriptor limit. Unless you''re using a desktop version of Windows, like XP or Vista. As a note on windows, it might be possible to overcome the 64> descriptor issue by recompiling all of ruby setting FD_SETSIZE=2048 > (which is possible using a mingw build), or maybe by having ruby link > against a different version of msvcrtXX.dll (also possible using > mingw, as well as by using VC8 instead of VC6). "Just" setting > FD_SETSIZE to a higher number when compiling the extension using mingw > doesn''t seem to work. > Of course, linking against a different .dll would mean you''d need to > recompile Ruby and all its extensions against it, too, but it would be > possible. For now I''d just leave the limit at 64 :) >Interesting experiment, but it''s bad enough that we have an extension in the first place. And the whole question of compiling Ruby on Windows is so cursed (see the humongous literature on the subject in the Ruby ML archives) that I''d rather not touch it! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20080115/3db8fee3/attachment.html
On Jan 15, 2008 6:11 AM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> On Jan 12, 2008 11:35 AM, Roger Pack <rogerpack2005 at gmail.com> wrote: > > > > My tests should that under heavy network loads, there''s no measurable > > > difference between the thread-interoperability mechanisms. I know that > > there > > > was a lot of anticipation of this in certain quarters, but as far as > > I''m > > > concerned there''s no benefit. > > > > Yeah I guess 1.9 is multi-threaded but only allows one thread to > > operate at a time. Maybe that''s the reason? > > > > These tests are all single-threaded to begin with. I can construct a > stress test that uses non-network events, for example, a large number of > EM''s Erlang-like spawned processes. These tests run far faster on 1.9 than > on 1.8. As soon as large numbers of network events come into the picture, > the improvement is much less noticeable. Since EM doesn''t rely on Ruby to do > any of its network handling, I think the difference is just because the > performance profile becomes dominated by the kernel. But that''s just a > hypothesis. > >Let me add a clarification to this: I did all these tests with only a single thread of actual EM code running. But all of them had a simple Ruby thread running that just looped and did some trivial work. That way I knew the interoperability mechanisms were actually being exercised. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20080115/378c53b4/attachment.html