Eventmachine talk - May 2006 - Protocol State Machine Parser

I''ve finally got enough of an example that I think people could see
what I''m talking about, so I''m sending in what I made.  I have
not
had time to document the methods or actually integrate this with
EventMachine (not hard), but looking through tc_protocol.rb should
give you ideas about intended usage.  Especially see test_push_and_pop.

I''ve not followed recent EM development closely and from just
passively reading this list, I''m guessing this concept is pretty far
from where you are going now.  However, in my early playing with EM,
this is the interface that popped into my head.  If nothing else, I
hope it gives someone ideas.

Thank you for the very nice library.

James Edward Gray II


-------------- next part --------------
A non-text attachment was scrubbed...
Name: em_protocol.zip
Type: application/zip
Size: 2887 bytes
Desc: not available
Url :
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060530/444b6c9b/attachment-0001.zip

Many thanks James, I''mm be reading this with great interest and I
expect
Jeff will too. It''s timely because we''ve been struggling with
how to
implement protocol stacks right.

On 5/30/06, James Edward Gray II <james at grayproductions.net>
wrote:
>
> I''ve finally got enough of an example that I think people could
see
> what I''m talking about, so I''m sending in what I made.  I
have not
> had time to document the methods or actually integrate this with
> EventMachine (not hard), but looking through tc_protocol.rb should
> give you ideas about intended usage.  Especially see test_push_and_pop.
>
> I''ve not followed recent EM development closely and from just
> passively reading this list, I''m guessing this concept is pretty
far
> from where you are going now.  However, in my early playing with EM,
> this is the interface that popped into my head.  If nothing else, I
> hope it gives someone ideas.
>
> Thank you for the very nice library.
>
> James Edward Gray II
>
>
>
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060531/085c702c/attachment.htm

James, I''m reading through your code. I think you''re taking a
computer-language view of an I/O stream which can be really interesting for
people writing DSLs etc. Can you write me a very quick (maybe one or two
paragraphs) description of how this would be integrated with the main event
machine, and also what steps would be required to turn this into a working
handler for (let''s say) HTTP? (If that''s a bad example, then
pick another
protocol or a DSL.) That would help me understand the philosophy of this
better so the code will make more sense to me.
Thanks again for all this good effort.

On 5/30/06, James Edward Gray II <james at grayproductions.net>
wrote:
>
> I''ve finally got enough of an example that I think people could
see
> what I''m talking about, so I''m sending in what I made.  I
have not
> had time to document the methods or actually integrate this with
> EventMachine (not hard), but looking through tc_protocol.rb should
> give you ideas about intended usage.  Especially see test_push_and_pop.
>
> I''ve not followed recent EM development closely and from just
> passively reading this list, I''m guessing this concept is pretty
far
> from where you are going now.  However, in my early playing with EM,
> this is the interface that popped into my head.  If nothing else, I
> hope it gives someone ideas.
>
> Thank you for the very nice library.
>
> James Edward Gray II
>
>
>
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060531/50350b3e/attachment.htm

On May 31, 2006, at 5:11 AM, Francis Cianfrocca wrote:
> James, I''m reading through your code. I think you''re
taking a
> computer-language view of an I/O stream which can be really
> interesting for people writing DSLs etc. Can you write me a very
> quick (maybe one or two paragraphs) description of how this would
> be integrated with the main event machine
Well, I''m not sure what EventMachine::Connection#receive_data
currently does, but perhaps we could change its body to something like:

   @protocol_parser << data unless @protocol_parser.nil?

Then we could add a method #install_protocol_parser that takes a
block to configure the parser and just assigns the result to the
instance variable.

With those two pieces, you could just build the parsers in the
#post_init method and everything should be handled for you.

(This is may not be ideal for a million reasons, it''s just the first
thing that popped into my head.)
> and also what steps would be required to turn this into a working
> handler for (let''s say) HTTP? (If that''s a bad example,
then pick
> another protocol or a DSL.) That would help me understand the
> philosophy of this better so the code will make more sense to me.
Here''s my crack at a usage example to build a MUD server, assuming
the integration described above:

require "rubygems"
require "eventmachine"

module MyGreatMUD
   def post_init
     # send welcome message and ask for name here...

     install_protocol_parser do |mud|
       # lex into MUD commands
       mud.lexer do |input|
         @pro.parse($1.strip.split) while input.sub!(/\A(.*?)\r?\n/,
"")
       end

       # simple command parser--call proper methods on @character
       mud.add_parser(:command) do |command, *args|
         @character.send(command, *args)
       end

       # not used in this example, but commands could install their own
       # sub protocol parsers, here''s an example:
       mud.add_parser(:mail) do
         # install the mail protocol parser
         mud.push do |mail_parser|
           # ...

           mail_parser.add_parser(:quit) do
             mud.pop  # mail_parser.pop is the same thing
           end
         end
       end

       # but first we need a login, to find a character
       mud.push do |login_parser|
         login_parser.add_parser(:login) do |name|
           if @character = Character.find(name)
             # ask for password here...
             login_parser.parser = :password
           else
             # handle new character here...
           end
         end

         login_parser.add_parser(:password) do |pass|
           if @character.login? pass
             # send successful login messade here...
             mud.pop  # login_parser.pop is the same thing
           else
             # handle attempt counter, disconnects for too many tries,
             # and failure messages here...
           end
         end
       end
     end
   end
end

EventMachine::run do
   EventMachine::start_server("127.0.0.1", 61676, MyGreatMUD)
end

__END__

Hope that helps.

James Edward Gray II

Reading now, thanks......

On 5/31/06, James Edward Gray II <james at grayproductions.net>
wrote:
>
> On May 31, 2006, at 5:11 AM, Francis Cianfrocca wrote:
>
> > James, I''m reading through your code. I think you''re
taking a
> > computer-language view of an I/O stream which can be really
> > interesting for people writing DSLs etc. Can you write me a very
> > quick (maybe one or two paragraphs) description of how this would
> > be integrated with the main event machine
>
> Well, I''m not sure what EventMachine::Connection#receive_data
> currently does, but perhaps we could change its body to something like:
>
>    @protocol_parser << data unless @protocol_parser.nil?
>
> Then we could add a method #install_protocol_parser that takes a
> block to configure the parser and just assigns the result to the
> instance variable.
>
> With those two pieces, you could just build the parsers in the
> #post_init method and everything should be handled for you.
>
> (This is may not be ideal for a million reasons, it''s just the
first
> thing that popped into my head.)
>
> > and also what steps would be required to turn this into a working
> > handler for (let''s say) HTTP? (If that''s a bad
example, then pick
> > another protocol or a DSL.) That would help me understand the
> > philosophy of this better so the code will make more sense to me.
>
> Here''s my crack at a usage example to build a MUD server, assuming
> the integration described above:
>
> require "rubygems"
> require "eventmachine"
>
> module MyGreatMUD
>    def post_init
>      # send welcome message and ask for name here...
>
>      install_protocol_parser do |mud|
>        # lex into MUD commands
>        mud.lexer do |input|
>          @pro.parse($1.strip.split) while input.sub!(/\A(.*?)\r?\n/,
"")
>        end
>
>        # simple command parser--call proper methods on @character
>        mud.add_parser(:command) do |command, *args|
>          @character.send(command, *args)
>        end
>
>        # not used in this example, but commands could install their own
>        # sub protocol parsers, here''s an example:
>        mud.add_parser(:mail) do
>          # install the mail protocol parser
>          mud.push do |mail_parser|
>            # ...
>
>            mail_parser.add_parser(:quit) do
>              mud.pop  # mail_parser.pop is the same thing
>            end
>          end
>        end
>
>        # but first we need a login, to find a character
>        mud.push do |login_parser|
>          login_parser.add_parser(:login) do |name|
>            if @character = Character.find(name)
>              # ask for password here...
>              login_parser.parser = :password
>            else
>              # handle new character here...
>            end
>          end
>
>          login_parser.add_parser(:password) do |pass|
>            if @character.login? pass
>              # send successful login messade here...
>              mud.pop  # login_parser.pop is the same thing
>            else
>              # handle attempt counter, disconnects for too many tries,
>              # and failure messages here...
>            end
>          end
>        end
>      end
>    end
> end
>
> EventMachine::run do
>    EventMachine::start_server("127.0.0.1", 61676, MyGreatMUD)
> end
>
> __END__
>
> Hope that helps.
>
> James Edward Gray II
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060531/4a9b6c52/attachment.htm

On May 31, 2006, at 12:25 PM, Francis Cianfrocca wrote:
> Reading now, thanks......
Did you get a chance to play with this a bit?  If so, what did you
think?  Any good ideas hiding in there?

I''m just curious.

James Edward Gray II

I think there is a lot of good stuff there, James, and I was meaning to
write to you today. We still have to work out exactly how to present the
protocol stack, but I think what you''ve done fits into the middle
layer.
I''ll have some more specific questions soon, about how to specify
parsers
and scanners for your state machine.

On 6/5/06, James Edward Gray II <james at grayproductions.net>
wrote:
>
> On May 31, 2006, at 12:25 PM, Francis Cianfrocca wrote:
>
> > Reading now, thanks......
>
> Did you get a chance to play with this a bit?  If so, what did you
> think?  Any good ideas hiding in there?
>
> I''m just curious.
>
> James Edward Gray II
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060605/72187277/attachment.htm

On Jun 5, 2006, at 3:47 PM, Francis Cianfrocca wrote:
> I think there is a lot of good stuff there, James, and I was
> meaning to write to you today. We still have to work out exactly
> how to present the protocol stack, but I think what you''ve done
> fits into the middle layer. I''ll have some more specific questions
> soon, about how to specify parsers and scanners for your state
> machine.
That''s fine.  Been very busy myself, so I understand.  No rush.  Just
had a moment or two today and thought of you guys.

My code was really just intended as a proof of concept, but if any of
it is remotely valuable, it''s all yours.  If I can help tie it in to
anything I''m happy to do so.

James Edward Gray II