Eventmachine talk - Dec 2007 - HTTP server, UNIX client

Hello,

I''m implementing an HTTP server that upon receipt of a request, will
itself issue a request to one of a pool of server processes using unix
sockets. What is the best way to structure something like this?

Is it something like this?

class MyHttpServer
  def process_http_request
    ...
    EM.connect_unix_domain(socket, MyUnixClient)
    ...
  end
end

class MyUnixClient < EM::Connection
  def post_init
    ...
  end

  def receive_data(data)
    ...
  end
end

If so, how can I send data back to the HTTP client once the unix client
receives it? Should I be using EM::UNIXClient instead?

Thanks in advance,
Andre

On Dec 12, 2007 8:24 AM, Andre Nathan <andre at digirati.com.br>
wrote:
> Hello,
>
> I''m implementing an HTTP server that upon receipt of a request,
will
> itself issue a request to one of a pool of server processes using unix
> sockets. What is the best way to structure something like this?
I''d suggest you use an existing HTTP parser/server instead of
reinventing it.  It''s easy to use Mongrel''s HTTP parser with
EM.
There is a dedicated EM based HTTP server with a good performance
profile.  I have a more fleshed out HTTP server based upon a modified
(i.e. faster) version of Francis'' one that I am releasing as part of
my Wisteria micro framework soon, too (after which, I imagine that the
performance related changes will find their way back into the
em_httpserver).
> Is it something like this?
>
> class MyHttpServer
>  def process_http_request
>    ...
>    EM.connect_unix_domain(socket, MyUnixClient)
>    ...
>  end
> end
>
> class MyUnixClient < EM::Connection
>  def post_init
>    ...
>  end
>
>  def receive_data(data)
>    ...
>  end
> end
>
> If so, how can I send data back to the HTTP client once the unix client
> receives it? Should I be using EM::UNIXClient instead?
That''s the basic idea.

You have a protocol for communicating with the HTTP clients.  You have
a separate protocol for communicating with your backends.

When your client_protocol receives a request, it calls a method in
your backend_client with will send_data the info to your back end.
The receive_data method in your backend_client calls a method in your
client_protocol to send_data that data back to your HTTP client.


Kirk Haines

On Wed, 2007-12-12 at 09:37 -0700, Kirk Haines wrote:
> I''d suggest you use an existing HTTP parser/server instead of
> reinventing it.
I''m actually using the em_httpserver gem. Maybe I''ll try
mongrel''s
parser later to see how they compare to each other.
> The receive_data method in your backend_client calls a method in your
> client_protocol to send_data that data back to your HTTP client.
Yeah, my question is how to make the backend_client call the
client_protocol method. I guess I could do something like this in the
HTTP server:

  MyUnixClient.http_server = self
  EM.connect_unix_domain(socket, MyUnixClient)

which would set a class variable in MyUnixClient, so that in
MyUnixClient#receive_data I could call @@http_server.send_data to send
the response back to the HTTP client. Is that how it''s supposed to be
done?

Best,
Andre

On Dec 12, 2007 12:18 PM, Andre Nathan <andre at digirati.com.br> wrote:
> On Wed, 2007-12-12 at 09:37 -0700, Kirk Haines wrote:
>
> > I''d suggest you use an existing HTTP parser/server instead of
> > reinventing it.
>
> I''m actually using the em_httpserver gem. Maybe I''ll try
mongrel''s
> parser later to see how they compare to each other.
>
> > The receive_data method in your backend_client calls a method in your
> > client_protocol to send_data that data back to your HTTP client.
>
> Yeah, my question is how to make the backend_client call the
> client_protocol method. I guess I could do something like this in the
> HTTP server:
>
>  MyUnixClient.http_server = self
>  EM.connect_unix_domain(socket, MyUnixClient)
>
> which would set a class variable in MyUnixClient, so that in
> MyUnixClient#receive_data I could call @@http_server.send_data to send
> the response back to the HTTP client. Is that how it''s supposed to
be
> done?
>

Can you use a deferrable instead? Using em_httpserver, you could have
something like this (I''m doing the class names from memory so they may
be
slightly wrong):


class MyHandler
  include EM::HttpServer

  def process_http_request
    d = MyUnixClient.dispatch_request( cooked_data_from_the_http_request )
    d.callback {|response|
      send_response( data_taken_from_response )
    }
  end

end

You would then have a class MyUnixClient, derived from EM::Connection, with
a class method that would invoke a new connection and return a Deferrable.
Make any sense?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071212/64fd92e6/attachment.html

On Wed, 2007-12-12 at 13:29 -0500, Francis Cianfrocca
wrote:
> You would then have a class MyUnixClient, derived from EM::Connection,
> with a class method that would invoke a new connection and return a
> Deferrable. Make any sense?
Thanks Francis, that worked. Currently the HTTP server is not using
deferrables (only the Unix socket communication is), and I was going to
start changing the code to use deferrables everywhere, so I figured I''d
ask this: considering what you see for the future of EM, is that the way
to go? I''m just wondering if you''re going to focus on, for
example,
adding features to the spawned process style of concurrency or
something. I''m just trying to avoid future rewrites when this code
becomes too complex =)

Best,
Andre

On Dec 12, 2007 5:35 PM, Andre Nathan <andre at digirati.com.br> wrote:
> On Wed, 2007-12-12 at 13:29 -0500, Francis Cianfrocca wrote:
> > You would then have a class MyUnixClient, derived from EM::Connection,
> > with a class method that would invoke a new connection and return a
> > Deferrable. Make any sense?
>
> Thanks Francis, that worked. Currently the HTTP server is not using
> deferrables (only the Unix socket communication is), and I was going to
> start changing the code to use deferrables everywhere, so I figured
I''d
> ask this: considering what you see for the future of EM, is that the way
> to go? I''m just wondering if you''re going to focus on,
for example,
> adding features to the spawned process style of concurrency or
> something. I''m just trying to avoid future rewrites when this code
> becomes too complex =)
>
>
To me, Deferrables and Spawned Processes are two sides of the same  coin.
The only thing that Spawned Processes will be able to do that Deferrables
can''t, is to work across processes (as they do in Erlang). Both will be
supported indefinitely.

I use Deferrables all over my own code, so you can depend on them never
going away.  ;-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071213/c565a31b/attachment.html