I''ve added experiments/core-model-2 to SVN. I think this is pretty clean and satisfactory. Jeff, please have a look and see if you like the placement of timeouts (which are currently unimplemented). Any event now has the possibility of a timeout. To get an application-level timeout, you''d just post an event with a timeout to a nil target. When the timeout expires, the errback will get called. Callbacks and errbacks are unimplemented in core-model-2. (That''s how far I stripped this down!) A callback-chain should be invoked by posting an event, but I''m not sure exactly when and how it should be done. (See the comment in Eventable#receive_data.) Sorry to jerk you guys around with all these straw-man implementations. But again, I think this one is close. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060522/da1895dc/attachment.htm
Francis Cianfrocca wrote:> I''ve added experiments/core-model-2 to SVN. I think this is pretty clean > and satisfactory. Jeff, please have a look and see if you like the > placement of timeouts (which are currently unimplemented). Any event now > has the possibility of a timeout. To get an application-level timeout, > you''d just post an event with a timeout to a nil target. When the > timeout expires, the errback will get called.In general the event system is looking pretty clean. One major thing I''m not sure about is the whole idea of having a single event queue for the whole system. I would like to enable systems where you want various components to have their own event queues. This allows you to have a pool of threads processing events from one queue, and then when completed they can be sent to another. (That model lends itself well to distributed processing as well.) At some point you have to be able to handle the events though, and I really think that the most basic instantiation of a Machined application should be able to run as a single threaded process. Here is an idea. Each Eventable class would have its own queue, which stores events that need to be handled. By default these objects would register themselves with the active reactor to be handled by the main loop. However, in the future, it will be very straight forward to delegate processing to a separate thread or a pool of threads. It also keeps all of the event processing code contained in Eventable, rather than being spread into the reactor. A couple topics... 1) Timeouts. What''s the use case or goal with having timeouts for every event? Although you could implement timeouts as an event to a nil target, I don''t think it should feel like that. You really can''t beat this for ease of use and clarity, can you? my_timeout = Timeout.new(10) { run_timeout_code } which also lets you do: my_timeout.cancel if something happens in the 10 seconds which makes the timeout unnecessary. In the end, a timeout should really generate an event, rather than being an event. I think the Timeout class could include Eventable, which would let it use the event handling guts for free, but it should feel like exactly what it is, right? 2) Broadcast. I typically think of broadcast as sending something to all listeners, but that is what you want to do with any event being sent. For some event based API''s the broadcast_event method is the only way you send an event into the system, which makes sense since every listener would receive it. I guess what you mean here is that the event persists, even if there are no listeners for that event type? Is this something typical? Either way I don''t think having send, post and broadcast is very clear. Why not just have send_event? It''s clear, and any options can be set to defaults. 3) Threading. I totally agree that we will want to support event processing in threads, but I''m pretty sure I''d want to run anything I was doing in a single thread unless it required it. Dealing with concurrency is a major pain, and requiring that for simple stuff is not nice. We are probably going to have to put a lot of energy into making the thread model work well, but for now I''d rather focus on single threaded so we can get the event and IO stuff figured out first. 4) Target. When you say target do you mean event type? Whoever you are adding the handler to is the one who will be receiving or generating the events right? I would call that the target and the type the type... 5) Errbacks. Do we need to have errbacks in this type of model? An error is really just another event type isn''t it? Could help to simplify things by not dealing with two styles of callbacks, unless we need to. 5) Handlers. This general model seems cleaner than the deferred style or the typed listener interface style, but we still need to figure out a few things. We agree that passing blocks as handlers has to be very simple, but how about multiple handlers, and using methods as handlers? Using this we could support a lot of options: def add_event_handler type, callback=nil, &block ... end This could be really flexible if we wanted. Here are some possibilities for an object ''proto'' that generates events (e.g. a lower layer protocol): # Simple case with a block for the handler and no error handler. # What is an error for an event handler anyway? proto.add_event_handler :data_received {|data| process_data(data) } # Simple case using a method rather than a block proto.add_event_handler :data_received, :process_data # Passing an array of callbacks that will be pipelined (output passed # as input to the next handler, on up the chain.) proto.add_event_handler :data_received, [:stage_one, :stage_two] proto.add_event_handler :data_received, [ proc {|data| stage_one(data)}, proc {|data| stage_two(data)} ] Sorry it took me a while to get back... I''ve gotten engrossed in a book :-) Ever read ''Godel, Escher, Bach''? -Jeff> Callbacks and errbacks are unimplemented in core-model-2. (That''s how > far I stripped this down!) A callback-chain should be invoked by posting > an event, but I''m not sure exactly when and how it should be done. (See > the comment in Eventable#receive_data.) > > Sorry to jerk you guys around with all these straw-man implementations. > But again, I think this one is close. > > > ------------------------------------------------------------------------ > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk
How do you like the following, for a thread pool implementation:
class X
include Machine::Eventable
def a_handler evt
# do some stuff
end
end
10.times {X.new.listen_events( :evt_type, :a_handler )}
In this code, the method Eventable#listen_events will set up a thread
and subscribe for the specified event. Instantiating 10 objects give
you 10 threads. I think we should implement both styles for receiving
events in this case (caller''s option): either the thread subscribes
for one event at a time, or it gathers all events on an internal queue
and consumes them at its own pace. The former is more appropriate for
workpiles (such as database connections), the latter for broadcast
events.
Single thread: you''re quite right, of course. I currently have three
threads in addition to the main thread! The difficulty is to integrate
events that do and don''t come from select, and timer events. But your
comment made me think of an optimization that should make it possible
to unify them. More later.
Timeout API: your suggestion can (and will) be implemented on top of
an event-creation :-).
Here''s the thinking with respect to event-sending models: broadcasts
are relatively rare (I think), primarily stuff like keyboard and GUI
events. Pure application-generated events and network-generated events
are likely only to want a single consumer, as these often are events
that elicit a response. (Of course broadcast events sometimes create
(many) responses.) So I thought to support three models: 1) post an
event, to be consumed by at most one subscriber. The event generates
an error callback and is removed from the event queue if its timout
expires. 2) send an event, which is equivalent to 1) but there is no
timeout, so a subscriber must be ready to consume the message when
it''s posted. 3) broadcast, which means to dispatch the event to all
subscribers who are present at the time the event is broadcast. It''s
possible to add a timeout to broadcast (meaning it will be sent to all
subscribers who appear in a particular interval of time, but then we
have to keep a list so we don''t double-send it to the same recipient.
Target: I''m glad you asked that question. I''m conflating the
notion of
an event "type" (which doesn''t appear in my nomenclature)
with a
specific recipient. In theory, ANY object can be an event target
(including, eventually, queues in other process spaces). If a target
responds to receive_event, you just call the method and you''re done.
If the target is a symbol, it specifies a named internal queue. We can
add other semantics for other types later if we wish. Maybe this is
too precious. Maybe we should keep target distinct from type. Still
thinking about that.
Errbacks: yeah, they''re damned ugly. If we have only one callback,
then we''ll want to add a parameter (or a property to the event) to
specify that an error occurred, and then user code will have to deal
with it. What do you think?
I like your suggested add_handler APIs.
Ok, I''m going to let a day go by and work on some other stuff while
this percolates. Feel free to work on the code. I may try to write a
single-threaded reactor. If I do, it will go into core-model-3. Do you
want to take a crack at integrating keyboard and GUI events?
On 5/23/06, Jeff Rose <rosejn at gmail.com> wrote:> Francis Cianfrocca wrote:
> > I''ve added experiments/core-model-2 to SVN. I think this is
pretty clean
> > and satisfactory. Jeff, please have a look and see if you like the
> > placement of timeouts (which are currently unimplemented). Any event
now
> > has the possibility of a timeout. To get an application-level timeout,
> > you''d just post an event with a timeout to a nil target. When
the
> > timeout expires, the errback will get called.
>
> In general the event system is looking pretty clean. One major thing
> I''m not sure about is the whole idea of having a single event
queue for
> the whole system. I would like to enable systems where you want various
> components to have their own event queues. This allows you to have a
> pool of threads processing events from one queue, and then when
> completed they can be sent to another. (That model lends itself well to
> distributed processing as well.)
>
> At some point you have to be able to handle the events though, and I
> really think that the most basic instantiation of a Machined application
> should be able to run as a single threaded process. Here is an idea.
> Each Eventable class would have its own queue, which stores events that
> need to be handled. By default these objects would register themselves
> with the active reactor to be handled by the main loop. However, in the
> future, it will be very straight forward to delegate processing to a
> separate thread or a pool of threads. It also keeps all of the event
> processing code contained in Eventable, rather than being spread into
> the reactor.
>
> A couple topics...
>
> 1) Timeouts. What''s the use case or goal with having timeouts for
every
> event? Although you could implement timeouts as an event to a nil
> target, I don''t think it should feel like that. You really
can''t beat
> this for ease of use and clarity, can you?
>
> my_timeout = Timeout.new(10) { run_timeout_code }
>
> which also lets you do:
>
> my_timeout.cancel
>
> if something happens in the 10 seconds which makes the timeout
> unnecessary. In the end, a timeout should really generate an event,
> rather than being an event. I think the Timeout class could include
> Eventable, which would let it use the event handling guts for free, but
> it should feel like exactly what it is, right?
>
> 2) Broadcast. I typically think of broadcast as sending something to
> all listeners, but that is what you want to do with any event being
> sent. For some event based API''s the broadcast_event method is
the only
> way you send an event into the system, which makes sense since every
> listener would receive it. I guess what you mean here is that the event
> persists, even if there are no listeners for that event type? Is this
> something typical? Either way I don''t think having send, post and
> broadcast is very clear. Why not just have send_event? It''s
clear, and
> any options can be set to defaults.
>
> 3) Threading. I totally agree that we will want to support event
> processing in threads, but I''m pretty sure I''d want to
run anything I
> was doing in a single thread unless it required it. Dealing with
> concurrency is a major pain, and requiring that for simple stuff is not
> nice. We are probably going to have to put a lot of energy into making
> the thread model work well, but for now I''d rather focus on single
> threaded so we can get the event and IO stuff figured out first.
>
> 4) Target. When you say target do you mean event type? Whoever you are
> adding the handler to is the one who will be receiving or generating the
> events right? I would call that the target and the type the type...
>
> 5) Errbacks. Do we need to have errbacks in this type of model? An
> error is really just another event type isn''t it? Could help to
> simplify things by not dealing with two styles of callbacks, unless we
> need to.
>
> 5) Handlers. This general model seems cleaner than the deferred style
> or the typed listener interface style, but we still need to figure out a
> few things. We agree that passing blocks as handlers has to be very
> simple, but how about multiple handlers, and using methods as handlers?
> Using this we could support a lot of options:
>
> def add_event_handler type, callback=nil, &block
> ...
> end
>
> This could be really flexible if we wanted. Here are some possibilities
> for an object ''proto'' that generates events (e.g. a lower
layer protocol):
>
> # Simple case with a block for the handler and no error handler.
> # What is an error for an event handler anyway?
> proto.add_event_handler :data_received {|data| process_data(data) }
>
> # Simple case using a method rather than a block
> proto.add_event_handler :data_received, :process_data
>
> # Passing an array of callbacks that will be pipelined (output passed
>
> # as input to the next handler, on up the chain.)
> proto.add_event_handler :data_received, [:stage_one, :stage_two]
>
> proto.add_event_handler :data_received, [
> proc {|data| stage_one(data)},
> proc {|data| stage_two(data)}
> ]
>
> Sorry it took me a while to get back... I''ve gotten engrossed in
a book
> :-) Ever read ''Godel, Escher, Bach''?
>
> -Jeff
>
> > Callbacks and errbacks are unimplemented in core-model-2.
(That''s how
> > far I stripped this down!) A callback-chain should be invoked by
posting
> > an event, but I''m not sure exactly when and how it should be
done. (See
> > the comment in Eventable#receive_data.)
> >
> > Sorry to jerk you guys around with all these straw-man
implementations.
> > But again, I think this one is close.
> >
> >
> >
------------------------------------------------------------------------
> >
> > _______________________________________________
> > Eventmachine-talk mailing list
> > Eventmachine-talk at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
An important API point: if the reactor is running on the main thread, then
the user MUST call a dispatch_events method before anything will happen.
That''s another thing I was hoping to avoid with the internal thread,
but
perhaps it''s not the worst thing in the world. The current production
C++
EventMachine does this too, requiring the following idiom:
EventMachine::run {
# set up servers and other stuff here.
}
The action of the block passed to #run is to create a one-shot timer with an
expiration time of zero so it executes immediately after the main loop
starts running. I guess we can do the same thing but sugar it up a litte.
Were you thinking about the possibility of having more than one Reactor in
the system?
On 5/23/06, Jeff Rose <rosejn at gmail.com> wrote:>
> Francis Cianfrocca wrote:
> > I''ve added experiments/core-model-2 to SVN. I think this is
pretty clean
> > and satisfactory. Jeff, please have a look and see if you like the
> > placement of timeouts (which are currently unimplemented). Any event
now
> > has the possibility of a timeout. To get an application-level timeout,
> > you''d just post an event with a timeout to a nil target. When
the
> > timeout expires, the errback will get called.
>
> In general the event system is looking pretty clean. One major thing
> I''m not sure about is the whole idea of having a single event
queue for
> the whole system. I would like to enable systems where you want various
> components to have their own event queues. This allows you to have a
> pool of threads processing events from one queue, and then when
> completed they can be sent to another. (That model lends itself well to
> distributed processing as well.)
>
> At some point you have to be able to handle the events though, and I
> really think that the most basic instantiation of a Machined application
> should be able to run as a single threaded process. Here is an idea.
> Each Eventable class would have its own queue, which stores events that
> need to be handled. By default these objects would register themselves
> with the active reactor to be handled by the main loop. However, in the
> future, it will be very straight forward to delegate processing to a
> separate thread or a pool of threads. It also keeps all of the event
> processing code contained in Eventable, rather than being spread into
> the reactor.
>
> A couple topics...
>
> 1) Timeouts. What''s the use case or goal with having timeouts for
every
> event? Although you could implement timeouts as an event to a nil
> target, I don''t think it should feel like that. You really
can''t beat
> this for ease of use and clarity, can you?
>
> my_timeout = Timeout.new(10) { run_timeout_code }
>
> which also lets you do:
>
> my_timeout.cancel
>
> if something happens in the 10 seconds which makes the timeout
> unnecessary. In the end, a timeout should really generate an event,
> rather than being an event. I think the Timeout class could include
> Eventable, which would let it use the event handling guts for free, but
> it should feel like exactly what it is, right?
>
> 2) Broadcast. I typically think of broadcast as sending something to
> all listeners, but that is what you want to do with any event being
> sent. For some event based API''s the broadcast_event method is
the only
> way you send an event into the system, which makes sense since every
> listener would receive it. I guess what you mean here is that the event
> persists, even if there are no listeners for that event type? Is this
> something typical? Either way I don''t think having send, post and
> broadcast is very clear. Why not just have send_event? It''s
clear, and
> any options can be set to defaults.
>
> 3) Threading. I totally agree that we will want to support event
> processing in threads, but I''m pretty sure I''d want to
run anything I
> was doing in a single thread unless it required it. Dealing with
> concurrency is a major pain, and requiring that for simple stuff is not
> nice. We are probably going to have to put a lot of energy into making
> the thread model work well, but for now I''d rather focus on single
> threaded so we can get the event and IO stuff figured out first.
>
> 4) Target. When you say target do you mean event type? Whoever you are
> adding the handler to is the one who will be receiving or generating the
> events right? I would call that the target and the type the type...
>
> 5) Errbacks. Do we need to have errbacks in this type of model? An
> error is really just another event type isn''t it? Could help to
> simplify things by not dealing with two styles of callbacks, unless we
> need to.
>
> 5) Handlers. This general model seems cleaner than the deferred style
> or the typed listener interface style, but we still need to figure out a
> few things. We agree that passing blocks as handlers has to be very
> simple, but how about multiple handlers, and using methods as handlers?
> Using this we could support a lot of options:
>
> def add_event_handler type, callback=nil, &block
> ...
> end
>
> This could be really flexible if we wanted. Here are some possibilities
> for an object ''proto'' that generates events (e.g. a lower
layer protocol):
>
> # Simple case with a block for the handler and no error handler.
> # What is an error for an event handler anyway?
> proto.add_event_handler :data_received {|data| process_data(data) }
>
> # Simple case using a method rather than a block
> proto.add_event_handler :data_received, :process_data
>
> # Passing an array of callbacks that will be pipelined (output passed
>
> # as input to the next handler, on up the chain.)
> proto.add_event_handler :data_received, [:stage_one, :stage_two]
>
> proto.add_event_handler :data_received, [
> proc {|data| stage_one(data)},
> proc {|data| stage_two(data)}
> ]
>
> Sorry it took me a while to get back... I''ve gotten engrossed in
a book
> :-) Ever read ''Godel, Escher, Bach''?
>
> -Jeff
>
> > Callbacks and errbacks are unimplemented in core-model-2.
(That''s how
> > far I stripped this down!) A callback-chain should be invoked by
posting
> > an event, but I''m not sure exactly when and how it should be
done. (See
> > the comment in Eventable#receive_data.)
> >
> > Sorry to jerk you guys around with all these straw-man
implementations.
> > But again, I think this one is close.
> >
> >
> >
------------------------------------------------------------------------
> >
> > _______________________________________________
> > Eventmachine-talk mailing list
> > Eventmachine-talk at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060523/65f7cc8f/attachment.htm