[Eventmachine-talk] Erlang-like Processes

hemant
Tue, 28 Aug 2007 08:11:05 -0000

On 8/28/07, Kirk Haines <wyhaines at gmail.com>
wrote:
> On 8/28/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
>
> > If you really want to use that style, however, EM has another
> > lightweight-concurrency feature called "Deferrable." This is
inspired
> > directly by the "deferred" object in Python''s
Twisted. EM Deferrable is very
> > useful and very graceful, but rather weird conceptually, so it needs
its own
> > documentation (which hasn''t been written yet). It ultimately
provides the
> > same functionality as spawned processes, but packaged in a way that
makes it
> > easier to "return values" to code that invokes lengthy
operations. And of
> > course, it also runs without threads.
>
> If you want to see an example of a Deferrable in action, they figure
> prominently in the Swiftiply code, which Francis contributed, to
> handle delivering a file via chunked encoding.  It''s pretty neat.
>
>

No no, I fully understand deferable pattern and have seen and used
Twisted deferable as well. I am using EM more than long enough for
these sort of things. In fact, sometime back i needed my own callback
pattern and i implemented it similar to the way, deferables are
implemented in EventMachine.

But Lets say, one server has to truely handle 10,000 concurrent
connections ( there could be several use cases of this, and i have my
own like a Comet implementation). So what options one really have?

Me and my collegues experience says that, EM is really nice, but as
number of connections gets increased ( or lets say number of
connections is small, but number of requests from them increases
exponentially), EM loop gets clogged and request processing slows
down, So what options, does one have?

We are facing that situation right now. We are hunting down options
and will surely find a solution.

Also, #defer is just a Ruby green thread and hence doesn''t guarantee
any concurrency at all, apart from different execution path.
Obviously, multiple processes is one solution, but I am just worried
about IPC overhead. Thats where erlang processes come into picture,
because they are as independent as a system process and yet creating
them and communicating between them is not very costly. We could have
used Erlang, but our code is full of String handling functions and
Strings are well just one sort of things, erlang doesn''t do too well.

I am still very excited about interprocess mechanism, that Francis has
talked about.