Eventmachine talk - Mar 2007 - Ruby I/O

This thread primarily addresses Francis, although I''d like to hear from
anyone interested in the subject.  Recently I''ve been asking around
with
various people, namely Koichi Sasada (author of the new Ruby VM) and
ruby-core about how to address the issue of Ruby''s I/O performance.

I''ve seen a number of interesting things in gprof, including a number
of
platform-specific oddities.  On Linux, during an I/O handling event loop,
Ruby spends a relatively large amount of time calling sigprocmask() (this
doesn''t occur on MacOS X).  When running EventMachine, there are a
large
number of calls to gettimeofday() (which I assume EventMachine is using for
its timer scheduling).  However, the overwhelming number of calls are to
select(), although this number decreased with an increasing number of
connections.

When there aren''t many connections being monitored, calls to select()
are
quick.  However, as the number of descriptors select() is monitoring
increases, it takes an increasing amount of system time.  This results in a
shorter amount of time in which Ruby is running and more system time spent.

This is, of course, in the context of synthetic benchmarks, and I''d
like to
do some more tests before I can really say how Ruby scales in terms of I/O.
In my initial tests with a higher number of connections select() quickly
popped up as the primary system time consumer.

Has anyone else gprofed Ruby I/O, and if so, what were your results?

My tests were done with Ruby 1.8.5, but I also plan to do a set against Ruby
1.9, and a more extensive set of benchmarks I can graph and actually get a
sense of the scalability.

I''d also be extremely interested in benchmarking the epoll-based
kqueue.
While other synthetic benchmarks of epoll I''ve seen still show O(n)
scalability with epoll, I''d be interested to see if epoll_wait() calls
so
quickly jump to the #1 system time consumer.

--
Tony Arcieri
ClickCaster, Inc.
tony at clickcaster.com
(970) 232-4208
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/8b8a376e/attachment.html

On 3/13/07, Tony Arcieri <tony at clickcaster.com>
wrote:
>
>  I''ve seen a number of interesting things in gprof, including a
number of
> platform-specific oddities.  On Linux, during an I/O handling event loop,
> Ruby spends a relatively large amount of time calling sigprocmask() (this
> doesn''t occur on MacOS X).  When running EventMachine, there are a
large
> number of calls to gettimeofday() (which I assume EventMachine is using for
> its timer scheduling).  However, the overwhelming number of calls are to
> select(), although this number decreased with an increasing number of
> connections.


how expensive are the gettimeofday calls? Ruby calls sigprocmask because it
uses a signal to schedule its thread scheduler.

I''m running out of excuses to do an epoll implementation of EM.
Incidentally, did you guys get a chance to look at the HTTP server I sent
you?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/3e3091ca/attachment.html

I don''t want to sound pendantic, or, worse, dense, but select() is
USUALLY at the top of the profile for an event-driven program. When
something else (like write, or malloc) is instead, you know
something''s wrong.
>From reading this message, the only sense I got was that select
occupies relatively more of the profile than everything else, which
may be as it should be.

The gettimeofday stuff is a problem. Every high-speed project I''ve
worked on has replaced gettimeofday; it''s a drag on performance.
I''m
pretty sure the "correct" answer now is to calibrate and use rdtsc,
the cycle counter, which is essentially free (costs about as much as a
non-cached DRAM access). If you calibrate, you can convert from TSC
cycles to a struct timeval to hand select().

On 3/13/07, Tony Arcieri <tony at clickcaster.com>
wrote:
> When there aren''t many connections being monitored, calls to
select() are
> quick.  However, as the number of descriptors select() is monitoring
> increases, it takes an increasing amount of system time.  This results in a
> shorter amount of time in which Ruby is running and more system time spent.
>
> This is, of course, in the context of synthetic benchmarks, and
I''d like to
> do some more tests before I can really say how Ruby scales in terms of I/O.
> In my initial tests with a higher number of connections select() quickly
> popped up as the primary system time consumer.
>
> Has anyone else gprofed Ruby I/O, and if so, what were your results?
>
> My tests were done with Ruby 1.8.5, but I also plan to do a set against
Ruby
> 1.9, and a more extensive set of benchmarks I can graph and actually get a
> sense of the scalability.
>
> I''d also be extremely interested in benchmarking the epoll-based
kqueue.
> While other synthetic benchmarks of epoll I''ve seen still show
O(n)
> scalability with epoll, I''d be interested to see if epoll_wait()
calls so
> quickly jump to the #1 system time consumer.
>
> --
> Tony Arcieri
> ClickCaster, Inc.
> tony at clickcaster.com
> (970) 232-4208
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>

On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
> I''m running out of excuses to do an epoll implementation of EM.
> Incidentally, did you guys get a chance to look at the HTTP server I sent
> you?
I did.  I assume that you received the patches that I sent to you for
the changes I made?

I have found that as it sits right now, it appears to be a little bit
slower than the Mongrel parser.  Just FYI.


Kirk

On 3/13/07, Kirk Haines <wyhaines at gmail.com>
wrote:
>
> On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
>
> > I''m running out of excuses to do an epoll implementation of
EM.
> > Incidentally, did you guys get a chance to look at the HTTP server I
> sent
> > you?
>
> I did.  I assume that you received the patches that I sent to you for
> the changes I made?
>
> I have found that as it sits right now, it appears to be a little bit
> slower than the Mongrel parser.  Just FYI.

Yes, I got the patches. Did you try the parser with environment-strings
turned off? That''s not the default, but it''s a lot faster.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/64ee6380/attachment.html

On 3/13/07, Thomas Ptacek <tqbf at matasano.com>
wrote:
>
>
> The gettimeofday stuff is a problem. Every high-speed project I''ve
> worked on has replaced gettimeofday; it''s a drag on performance.
I''m
> pretty sure the "correct" answer now is to calibrate and use
rdtsc,
> the cycle counter, which is essentially free (costs about as much as a
> non-cached DRAM access). If you calibrate, you can convert from TSC
> cycles to a struct timeval to hand select().


You''re cordially invited to send me a patch ;-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/3af54ee2/attachment-0001.html

On 3/13/07, Thomas Ptacek <tqbf at matasano.com>
wrote:
>
>  The gettimeofday stuff is a problem. Every high-speed project
I''ve
> worked on has replaced gettimeofday; it''s a drag on performance.
I''m
> pretty sure the "correct" answer now is to calibrate and use
rdtsc,
> the cycle counter, which is essentially free (costs about as much as a
> non-cached DRAM access). If you calibrate, you can convert from TSC
> cycles to a struct timeval to hand select().


gettimeofday is there in the Unix versions to support programmable timers.
The implementation originally used time but I changed it when the need for
subsecond resolution came up. There is probably a range of approaches to
dealing with this. At least we could detect when subsecond timers are in use
(the exception rather than the rule) and behave differently.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/8819aba8/attachment.html

On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
>
> On 3/13/07, Thomas Ptacek <tqbf at matasano.com> wrote:
> >
> >  The gettimeofday stuff is a problem. Every high-speed project
I''ve
> > worked on has replaced gettimeofday; it''s a drag on
performance. I''m
> > pretty sure the "correct" answer now is to calibrate and use
rdtsc,
> > the cycle counter, which is essentially free (costs about as much as a
> > non-cached DRAM access). If you calibrate, you can convert from TSC
> > cycles to a struct timeval to hand select().
>
>
>
> gettimeofday is there in the Unix versions to support programmable timers.
> The implementation originally used time but I changed it when the need for
> subsecond resolution came up. There is probably a range of approaches to
> dealing with this. At least we could detect when subsecond timers are in
use
> (the exception rather than the rule) and behave differently.
>
>
Well, on Linux, time() seems just about as costly as gettimeofday.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/d0c50e38/attachment.html

On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
> Yes, I got the patches. Did you try the parser with environment-strings
> turned off? That''s not the default, but it''s a lot
faster.
Well, poop.  With all that hacking you would think I would have
noticed that they are ON by default, but I didn''t.  I''ll
retest.  :)


Kirk

On 3/13/07, Kirk Haines <wyhaines at gmail.com>
wrote:
>
> On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
>
> > Yes, I got the patches. Did you try the parser with
environment-strings
> > turned off? That''s not the default, but it''s a lot
faster.
>
> Well, poop.  With all that hacking you would think I would have
> noticed that they are ON by default, but I didn''t.  I''ll
retest.  :)


Heh. On by default would make sense, but i hate to add a new behavior and
change the previous defaults. My raw testing indicates the parser is at
least three times faster without the environment strings, which are only
there in case you want to use cgi.rb.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/1214d1a5/attachment.html

Yes, it didn''t build.  It looks like it''s trying to cast a
pointer to an
int, and since I''m building it on 64-bit machines pointers are 64-bit
and
int 32-bit:

g++ -I. -I. -I/usr/local/lib/ruby/1.8/x86_64-linux -I. -DHAVE_OPENSSL_SSL_H
-DHAVE_OPENSSL_ERR_H  -DOS_UNIX -DBUILD_FOR_RUBY -DWITH_SSL -fPIC -g -O2
-c rubyhttp.cpp
rubyhttp.cpp: In function ''VALUE t_post_init(VALUE)'':
rubyhttp.cpp:166: error: cast from ''RubyHttpConnection_t*'' to
''int'' loses
precision
make: *** [rubyhttp.o] Error 1

- Tony

On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
>
> On 3/13/07, Tony Arcieri <tony at clickcaster.com> wrote:
> >
> >  I''ve seen a number of interesting things in gprof, including
a number
> > of platform-specific oddities.  On Linux, during an I/O handling event
loop,
> > Ruby spends a relatively large amount of time calling sigprocmask()
(this
> > doesn''t occur on MacOS X).  When running EventMachine, there
are a large
> > number of calls to gettimeofday() (which I assume EventMachine is
using for
> > its timer scheduling).  However, the overwhelming number of calls are
to
> > select(), although this number decreased with an increasing number of
> > connections.
>
>
>
> how expensive are the gettimeofday calls? Ruby calls sigprocmask because
> it uses a signal to schedule its thread scheduler.
>
> I''m running out of excuses to do an epoll implementation of EM.
> Incidentally, did you guys get a chance to look at the HTTP server I sent
> you?
>
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>


--
Tony Arcieri
ClickCaster, Inc.
tony at clickcaster.com
(970) 232-4208
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/555b27bb/attachment.html

Well, the main problem I''m seeing here is that while select() is called
the
most, it runs relatively quick until you start adding descriptors.  As more
descriptors are added the amount of time each call takes increases, even
with no I/O activity, and with it the amount of system time incurred by the
process.

- Tony

On 3/13/07, Thomas Ptacek <tqbf at matasano.com>
wrote:
>
> I don''t want to sound pendantic, or, worse, dense, but select() is
> USUALLY at the top of the profile for an event-driven program. When
> something else (like write, or malloc) is instead, you know
> something''s wrong.
>
> >From reading this message, the only sense I got was that select
> occupies relatively more of the profile than everything else, which
> may be as it should be.
>
> The gettimeofday stuff is a problem. Every high-speed project I''ve
> worked on has replaced gettimeofday; it''s a drag on performance.
I''m
> pretty sure the "correct" answer now is to calibrate and use
rdtsc,
> the cycle counter, which is essentially free (costs about as much as a
> non-cached DRAM access). If you calibrate, you can convert from TSC
> cycles to a struct timeval to hand select().
>
> On 3/13/07, Tony Arcieri <tony at clickcaster.com> wrote:
> > When there aren''t many connections being monitored, calls to
select()
> are
> > quick.  However, as the number of descriptors select() is monitoring
> > increases, it takes an increasing amount of system time.  This results
> in a
> > shorter amount of time in which Ruby is running and more system time
> spent.
> >
> > This is, of course, in the context of synthetic benchmarks, and
I''d like
> to
> > do some more tests before I can really say how Ruby scales in terms of
> I/O.
> > In my initial tests with a higher number of connections select()
quickly
> > popped up as the primary system time consumer.
> >
> > Has anyone else gprofed Ruby I/O, and if so, what were your results?
> >
> > My tests were done with Ruby 1.8.5, but I also plan to do a set
against
> Ruby
> > 1.9, and a more extensive set of benchmarks I can graph and actually
get
> a
> > sense of the scalability.
> >
> > I''d also be extremely interested in benchmarking the
epoll-based kqueue.
> > While other synthetic benchmarks of epoll I''ve seen still
show O(n)
> > scalability with epoll, I''d be interested to see if
epoll_wait() calls
> so
> > quickly jump to the #1 system time consumer.
> >
> > --
> > Tony Arcieri
> > ClickCaster, Inc.
> > tony at clickcaster.com
> > (970) 232-4208
> > _______________________________________________
> > Eventmachine-talk mailing list
> > Eventmachine-talk at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/eventmachine-talk
> >
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>


--
Tony Arcieri
ClickCaster, Inc.
tony at clickcaster.com
(970) 232-4208
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/a6584847/attachment-0001.html

I''d really like to profile your epoll implementation.  While epoll_wait
should theoretically scale O(1) with the number of descriptors, I''d be
interested to see how the system call footprint changes.

- Tony

On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
>
> On 3/13/07, Tony Arcieri <tony at clickcaster.com> wrote:
> >
> >  I''ve seen a number of interesting things in gprof, including
a number
> > of platform-specific oddities.  On Linux, during an I/O handling event
loop,
> > Ruby spends a relatively large amount of time calling sigprocmask()
(this
> > doesn''t occur on MacOS X).  When running EventMachine, there
are a large
> > number of calls to gettimeofday() (which I assume EventMachine is
using for
> > its timer scheduling).  However, the overwhelming number of calls are
to
> > select(), although this number decreased with an increasing number of
> > connections.
>
>
>
> how expensive are the gettimeofday calls? Ruby calls sigprocmask because
> it uses a signal to schedule its thread scheduler.
>
> I''m running out of excuses to do an epoll implementation of EM.
> Incidentally, did you guys get a chance to look at the HTTP server I sent
> you?
>
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>


--
Tony Arcieri
ClickCaster, Inc.
tony at clickcaster.com
(970) 232-4208
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/9ef51367/attachment.html

On 3/13/07, Tony Arcieri <tony at clickcaster.com>
wrote:
>
> Yes, it didn''t build.  It looks like it''s trying to cast
a pointer to an
> int, and since I''m building it on 64-bit machines pointers are
64-bit and
> int 32-bit:
>
> g++ -I. -I. -I/usr/local/lib/ruby/1.8/x86 _64-linux -I.
> -DHAVE_OPENSSL_SSL_H -DHAVE_OPENSSL_ERR_H  -DOS_UNIX -DBUILD_FOR_RUBY
> -DWITH_SSL -fPIC -g -O2   -c rubyhttp.cpp
> rubyhttp.cpp: In function ''VALUE t_post_init(VALUE)'':
> rubyhttp.cpp:166: error: cast from
''RubyHttpConnection_t*'' to ''int'' loses
> precision
> make: *** [rubyhttp.o ] Error 1
>

Looks like you should be able to change  line 166 to the following:

 rb_ivar_set (self, rb_intern ("@http______conn"), INT2NUM
((long)hc));

looking in ruby.h, INT2NUM takes a long, so this should silence the compiler
error. Let me know if you get a chance to try it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/bddc5e01/attachment.html

A SWAG, created exclusively in the service of not cluttering the list
with another 500 word message without writing a line of code:

http://pastie.caboo.se/46697

tsc_gettimeofday() from this link is thus a drop-in replacement for
g.t.o.d(). I haven''t profiled; I recalibrate every 20 calls
(calibrating the TSC is a pain). I make no claims to performance,
portability, or correctness.

On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
> On 3/13/07, Thomas Ptacek <tqbf at matasano.com> wrote:
> >
> > The gettimeofday stuff is a problem. Every high-speed project
I''ve
> > worked on has replaced gettimeofday; it''s a drag on
performance. I''m
> > pretty sure the "correct" answer now is to calibrate and use
rdtsc,
> > the cycle counter, which is essentially free (costs about as much as a
> > non-cached DRAM access). If you calibrate, you can convert from TSC
> > cycles to a struct timeval to hand select().
>
>
> You''re cordially invited to send me a patch ;-)
>
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>

BTW, I don''t know if the comment in ed.cpp:463 (0.7.0) is still
germane, but I thought getpeername() did the sane thing in winsock,
returning error on unconnected sockets that select writeable.

On 3/13/07, Thomas Ptacek <tqbf at matasano.com>
wrote:
> A SWAG, created exclusively in the service of not cluttering the list
> with another 500 word message without writing a line of code:
>
> http://pastie.caboo.se/46697
>
> tsc_gettimeofday() from this link is thus a drop-in replacement for
> g.t.o.d(). I haven''t profiled; I recalibrate every 20 calls
> (calibrating the TSC is a pain). I make no claims to performance,
> portability, or correctness.
>
> On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
> > On 3/13/07, Thomas Ptacek <tqbf at matasano.com> wrote:
> > >
> > > The gettimeofday stuff is a problem. Every high-speed project
I''ve
> > > worked on has replaced gettimeofday; it''s a drag on
performance. I''m
> > > pretty sure the "correct" answer now is to calibrate
and use rdtsc,
> > > the cycle counter, which is essentially free (costs about as much
as a
> > > non-cached DRAM access). If you calibrate, you can convert from
TSC
> > > cycles to a struct timeval to hand select().
> >
> >
> > You''re cordially invited to send me a patch ;-)
> >
> >
> > _______________________________________________
> > Eventmachine-talk mailing list
> > Eventmachine-talk at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/eventmachine-talk
> >
>

On 3/13/07, Thomas Ptacek <tqbf at matasano.com>
wrote:
>
> A SWAG, created exclusively in the service of not cluttering the list
> with another 500 word message without writing a line of code:
>
> http://pastie.caboo.se/46697
>
> tsc_gettimeofday() from this link is thus a drop-in replacement for
> g.t.o.d(). I haven''t profiled; I recalibrate every 20 calls
> (calibrating the TSC is a pain). I make no claims to performance,
> portability, or correctness.


What''s the strategy here? You calculate an average delta value between
calls
and re-set against the real time every twenty times? So the real timesaver
is that you avoiding the kernel crossing?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/b68ff4d2/attachment.html

The kernel crossing is the entire expense of gettimeofday. There''s no
strategy here other than a recognition that converting TSC deltas to
microseconds is error-prone and causes drift.

On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
> On 3/13/07, Thomas Ptacek <tqbf at matasano.com> wrote:
> > A SWAG, created exclusively in the service of not cluttering the list
> > with another 500 word message without writing a line of code:
> >
> > http://pastie.caboo.se/46697
> >
> > tsc_gettimeofday() from this link is thus a drop-in replacement for
> > g.t.o.d(). I haven''t profiled; I recalibrate every 20 calls
> > (calibrating the TSC is a pain). I make no claims to performance,
> > portability, or correctness.
>
>
> What''s the strategy here? You calculate an average delta value
between calls
> and re-set against the real time every twenty times? So the real timesaver
> is that you avoiding the kernel crossing?
>
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>

On 3/13/07, Thomas Ptacek <tqbf at matasano.com>
wrote:
>
> BTW, I don''t know if the comment in ed.cpp:463 (0.7.0) is still
> germane, but I thought getpeername() did the sane thing in winsock,
> returning error on unconnected sockets that select writeable.


so you think a getpeername call would return a recognizable error on
Windows? The real problem with that code path is that I don''t think
Windows
reliably selects a socket as selectable when it has a connect error, so
there''s no opportunity to call getpeername or anything else. (This is
only
germane to the case in which you''re initiating a TCP connection to a
remote
peer.) Perhaps the socket selects in an error state on Windows? I never
tried that because it''s so unlike all the Unix flavors.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/4c22aa78/attachment.html

On 3/13/07, Thomas Ptacek <tqbf at matasano.com>
wrote:
>
> BTW, I don''t know if the comment in ed.cpp:463 (0.7.0) is still
> germane, but I thought getpeername() did the sane thing in winsock,
> returning error on unconnected sockets that select writeable.


Correction, I meant to say that I don''t think Windows reliably selects
a
socket as *writable* on a connect error. When I was writing that code, I
couldn''t get it to do particular thing consistently.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/c31fee43/attachment-0001.html

Have you checked out how ACE handles this? (Their answer might just be
to use WMFO instead of select).

On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
> On 3/13/07, Thomas Ptacek <tqbf at matasano.com> wrote:
> > BTW, I don''t know if the comment in ed.cpp:463 (0.7.0) is
still
> > germane, but I thought getpeername() did the sane thing in winsock,
> > returning error on unconnected sockets that select writeable.
>
>
> so you think a getpeername call would return a recognizable error on
> Windows? The real problem with that code path is that I don''t
think Windows
> reliably selects a socket as selectable when it has a connect error, so
> there''s no opportunity to call getpeername or anything else. (This
is only
> germane to the case in which you''re initiating a TCP connection to
a remote
> peer.) Perhaps the socket selects in an error state on Windows? I never
> tried that because it''s so unlike all the Unix flavors.
>
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>

On 3/13/07, Thomas Ptacek <tqbf at matasano.com>
wrote:
>
> Have you checked out how ACE handles this? (Their answer might just be
> to use WMFO instead of select).


To my knowledge, ACE uses select. But I don''t really know that much
about
it. I think they hacked select a little bit too. I worked up an epoll-based
implementation today and sent it off to Tony. Let''s see if it works any
better with large connection sets.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/444bdceb/attachment.html

I''m still talking about the Win32 nonblocking connect issue. =)

On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
> On 3/13/07, Thomas Ptacek <tqbf at matasano.com> wrote:
> > Have you checked out how ACE handles this? (Their answer might just be
> > to use WMFO instead of select).
>
>
> To my knowledge, ACE uses select. But I don''t really know that
much about
> it. I think they hacked select a little bit too. I worked up an epoll-based
> implementation today and sent it off to Tony. Let''s see if it
works any
> better with large connection sets.
>
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>

On 3/13/07, Thomas Ptacek <tqbf at matasano.com>
wrote:
>
> I''m still talking about the Win32 nonblocking connect issue. =)

Ah, sorry, my bad.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/245dc295/attachment.html

I''d like to put together a simple test harness for running gprof with a
program that creates large numbers of connections.  Maybe that could go into
the benchmarks directory?

Maybe we should get in touch with this guy as well:

http://t-a-w.blogspot.com/2007/02/making-ruby-faster.html

- Tony

On 3/13/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
>
> On 3/13/07, Thomas Ptacek <tqbf at matasano.com> wrote:
> >
> > I''m still talking about the Win32 nonblocking connect issue.
=)
>
>
> Ah, sorry, my bad.
>
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>


--
Tony Arcieri
ClickCaster, Inc.
tony at clickcaster.com
(970) 232-4208
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/e606bee1/attachment.html

On 3/13/07, Tony Arcieri <tony at clickcaster.com>
wrote:
>
> I''d like to put together a simple test harness for running gprof
with a
> program that creates large numbers of connections.  Maybe that could go
into
> the benchmarks directory?


That sounds great, Tony.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070313/da588beb/attachment.html

In winsock, getpeername will return a SOCKET_ERROR when it fails but it
supposed to be called on a connected socket only (you can of course get the
error through the WSAGetLastError). The issue is when the socket is not
connected and you call getpeername anyway.  I this is the case, it may NOT
return SOCKET_ERROR, since it''s the wacky windows kernel, but I am
certain
it will return something other than 0, which is bad.



From: eventmachine-talk-bounces at rubyforge.org
[mailto:eventmachine-talk-bounces at rubyforge.org] On Behalf Of Francis
Cianfrocca
Sent: Tuesday, March 13, 2007 8:16 PM
To: eventmachine-talk at rubyforge.org
Subject: Re: [Eventmachine-talk] Ruby I/O



On 3/13/07, Thomas Ptacek <tqbf at matasano.com> wrote:

BTW, I don''t know if the comment in ed.cpp:463 (0.7.0) is still
germane, but I thought getpeername() did the sane thing in winsock,
returning error on unconnected sockets that select writeable.



so you think a getpeername call would return a recognizable error on
Windows? The real problem with that code path is that I don''t think
Windows
reliably selects a socket as selectable when it has a connect error, so
there''s no opportunity to call getpeername or anything else. (This is
only
germane to the case in which you''re initiating a TCP connection to a
remote
peer.) Perhaps the socket selects in an error state on Windows? I never
tried that because it''s so unlike all the Unix flavors.



-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070314/8e93dc8b/attachment-0001.html