I''m doing some work on Patrick''s stompserver, mainly trying to create fast, reliable queue storage backends that will work on both windows and unix. For unix I have a berkeleydb storage that uses the bdb queue structure and a hash structure for storing the actual frames. Since calls to bdb block and with the way the queue structure works, there is no possibility for conflict. The second queue storage I created is a flat file queue. It''s simple and fairly fast, but I think there is a possibility for conflict so I think I need a way to create a lock whenever I enqueue or dequeue. File based queues are just files stored in a directory and named sequentially starting from 1. When a queue is opened it reads the filenames into an array. When I enqueue a frame I take the last element of the array, increment it by one, write the file, then push the filename onto the array. This all happens very quickly of course, but isn''t there a chance of another consumer reading the last element of the array before the first consumer can write the file and push the next element onto the array? Here is basically how the code works now: id = array.last id += 1 write file array.push(id) I know it''s probably a very small chance of a collision, but it does exist right? I''m wrapping the whole thing in a Lockfile block right now, which doesn''t seem to effect performance all that much. Haven''t tried using a semaphore, would that be faster then Lockfile? Chris
Am confused; are you threading, and if so doesn''t that sort of defeat the purpose of using eventmachine? If you''re not threading, and no other process is accessing the queue, you have serialized access and don''t need to lock it. If other processes need to access them, you need to use lockf/flock/fcntl-style locking. Note also that using normal IO API, your writes aren''t event-safe (they can take more than N milliseconds to complete and will block). On 10/21/06, snacktime <snacktime at gmail.com> wrote:> I''m doing some work on Patrick''s stompserver, mainly trying to create > fast, reliable queue storage backends that will work on both windows > and unix. For unix I have a berkeleydb storage that uses the bdb...> created is a flat file queue. It''s simple and fairly fast, but I > think there is a possibility for conflict so I think I need a way to > create a lock whenever I enqueue or dequeue....> but isn''t there a chance of another consumer reading the last element > of the array before the first consumer can write the file and push the > next element onto the array? Here is basically how the code works > now: > > id = array.last > id += 1 > write file > array.push(id) > > I know it''s probably a very small chance of a collision, but it does > exist right? I''m wrapping the whole thing in a Lockfile block right > now, which doesn''t seem to effect performance all that much. Haven''t > tried using a semaphore, would that be faster then Lockfile?
On 10/21/06, Thomas Ptacek <thomasptacek at gmail.com> wrote:> Am confused; are you threading, and if so doesn''t that sort of defeat > the purpose of using eventmachine? > > If you''re not threading, and no other process is accessing the queue, > you have serialized access and don''t need to lock it. > > If other processes need to access them, you need to use > lockf/flock/fcntl-style locking. > > Note also that using normal IO API, your writes aren''t event-safe > (they can take more than N milliseconds to complete and will block).I''m not using threads, and there can be several concurrent consumers on a single queue. Actually I just tried using a semaphore lock instead of a file lock and the performance is better, and probably more reliable. And blocking file IO isn''t really an issue since I need to block other readers/writers anyways while a message is being enqueued/dequeued. Writing would probably work if the IO was non blocking, since there really isn''t a guarantee that messages received in receive_data are going to be in the same order once they get to the enqueue method. But messages coming off the queue have to be in order, so the dequeue method has to block. Chris Chris
On 10/21/06, snacktime <snacktime at gmail.com> wrote:> > > I''m not using threads, and there can be several concurrent consumers > on a single queue. Actually I just tried using a semaphore lock > instead of a file lock and the performance is better, and probably > more reliable. And blocking file IO isn''t really an issue since I > need to block other readers/writers anyways while a message is being > enqueued/dequeued. Writing would probably work if the IO was non > blocking, since there really isn''t a guarantee that messages received > in receive_data are going to be in the same order once they get to the > enqueue method. But messages coming off the queue have to be in > order, so the dequeue method has to block.Sounds like you have a good handle on this, but just for my own curiosity: Do you have multiple processes reading and writing this directory with numbered files in it? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061022/a00ea623/attachment.html
> Sounds like you have a good handle on this, but just for my own curiosity: > Do you have multiple processes reading and writing this directory with > numbered files in it?No just one.
On 10/22/06, snacktime <snacktime at gmail.com> wrote:> > > Sounds like you have a good handle on this, but just for my own > curiosity: > > Do you have multiple processes reading and writing this directory with > > numbered files in it? > > No just one.Then I''m with Thomas. I''m not seeing the concurrency issue. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061022/f2adb447/attachment.html
On 10/21/06, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> On 10/22/06, snacktime <snacktime at gmail.com> wrote: > > > > Sounds like you have a good handle on this, but just for my own > curiosity: > > > Do you have multiple processes reading and writing this directory with > > > numbered files in it? > > > > No just one. > > > > Then I''m with Thomas. I''m not seeing the concurrency issue.I guess I''m not completely clear on how the time slicing works. I understand IO multiplexing, but how the code itself is run I''m fuzzy on. when EM enters a block of code, does it run in serial until it''s finished? I''m assuming so, it''s the only thing that makes sense to me given what I know. So objects are protected because only one instance of any object can exist at any one point in time? Please correct where I''m wrong, I''d like to get clear on this once and for all.
I think what I''m missing is how context/task switching works in ruby. Say I have the following code, what synchronizes the variables so that array1 doesn''t change between before and after the file is written by another task/thread/whatever? enqueue would be a method in a class that is already instantiated before EM is run. def enqueue id = array1.first File.write(id) array1.shift end
On 10/22/06, snacktime <snacktime at gmail.com> wrote:> > I guess I''m not completely clear on how the time slicing works. I > understand IO multiplexing, but how the code itself is run I''m fuzzy > on. when EM enters a block of code, does it run in serial until it''s > finished? I''m assuming so, it''s the only thing that makes sense to me > given what I know. So objects are protected because only one instance > of any object can exist at any one point in time? Please correct > where I''m wrong, I''d like to get clear on this once and for all. > _______________________________________________An EM-based program that has no other threads does no time-slicing whatsoever. An EM program that has other Ruby threads will do time-slicing based on Ruby''s green-thread scheduler, but EM is not otherwise affected. EventMachine#run is a loop that continuously scans the system for events that have been registered by user code (and some system-generated events like signals). It uses multiplexed I/O, but in such a way that Ruby''s green threads get a chance to run while EM is waiting for multiplexed I/O. With EM, you''re essentially writing blocks of code that will be executed whenever specific events occur (such as the passage of a specific amount of time, a network socket becoming readable, a signal, etc). When these events occur, EM#run calls your code block, and *never interrupts it.* EM doesn''t go back to its own event loop until your block returns. You may be thinking "aha, nonpreemptive multitasking," but it''s not really multitasking since there''s only one task. The events that pile up in the system will be processed when your code block completes. Your program''s control flow is still serial and single-threaded. It''s just that the order of execution is determined by when the events occur. That''s also why proper EM style calls for your code blocks to execute quickly without blocking. And of course there are times when that''s impossible (like, you''re calling a standard database client library). That''s what EventMachine#defer is for. EM has its own internal thread pool managed by #defer. If you have to do something in an event-handler that will block locally, then pass it in a block to #defer, which will run it on another thread and then signal the main thread when the deferred operation completes. In this case, you have a concurrency issue with respect to all the code that runs inside of #defer blocks, but nothing else. Aside: one thing EM doesn''t currently have is user-defined events. Meaning, you can fire an event internal to your own program, and something else picks up the event and acts on it. I''ve found this pattern to be incredibly useful in writing Ajax programs, where there can be a lot of different active bits all mashed up on a single HTML page. Since they all came from separate Ajax calls, they know nothing about each other. I wrote a little event-library for these and I now can''t imagine writing Ajax any other way. But with server-side programs, I don''t imagine you''d have such a situation intra-process, and you''d use message-queueing (a different kind of event-driven programming) to do it across multiple processes. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061022/c9d8889f/attachment-0001.html
On 10/22/06, snacktime <snacktime at gmail.com> wrote:> > I think what I''m missing is how context/task switching works in ruby. > Say I have the following code, what synchronizes the variables so that > array1 doesn''t change between before and after the file is written by > another task/thread/whatever? enqueue would be a method in a class > that is already instantiated before EM is run. > > def enqueue > id = array1.first > File.write(id) > array1.shift > endThe only way to have a concurrency issue here is if you write a thread that is entirely separate from your EM event handlers, and the separate thread *also* writes the array and the file. Another possibility is if this code is passed in an EventMachine#defer block. In either of these cases, you will need to synchronize this block. (Which opens a can of worms because there really aren''t any good ways to thread-synchronize file i/o, but that''s another story.) If this code only appears in an EM event handler and there are no other threads in your program and no #defer calls, then the code will *never* be interrupted. (Except by your kernel''s process scheduler of course, but that doesn''t create a concurrency issue in this code unless some other process is reading or writing the file simultaneously.) It may be that this seems like black magic to you, that there''s something going on that you''re missing. EM is actually very simple, it''s just a loop that scans for events and calls their respective handlers in sequence. With programs that interact with the outside world, that''s just exactly how they most naturally work: your program is impinged upon by actions that are exogenous to it, like the passage of time, or users connecting to it. The event-driven model is completely natural here, because it models the behavior of the real-world, which is basically synchronous but not deterministic as regards the order of events. What is really unnatural is the standard approach of using threads to model an event-driven world. With threads, your program runs in a deterministic order you have little control over (it''s created by the thread scheduler), but you have to simulate the nondeterministic event-order of the real world by allowing all of your code to be interruptible at arbitrary times. That is the real black magic. EM, like many other event-driven frameworks, is actually simpler and easier for fundamental reasons. Many people (including me) are so used to doing battle with threads that it seems like they must be missing something when an easier approach appears. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061022/266a9c77/attachment.html
Sorry for making you write all that Franics, I must have had a brain lock yesterday because I woke up this morning wondering what the hell was I thinking when I wrote that. Need to get away from the computer...
On 10/22/06, snacktime <snacktime at gmail.com> wrote:> > Sorry for making you write all that Franics, I must have had a brain > lock yesterday because I woke up this morning wondering what the hell > was I thinking when I wrote that. Need to get away from the > computer... > _______________________________________________Does that mean your concurrency issue is solved? Now just imagine how much harder it would be to track down a race-condition in a thread-hot program running on SMP hardware. You know the drill, it''s when your client says "This crappy software you sold me works perfectly for two weeks, then it crashes twice in one night!" :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061022/1ef925cd/attachment.html
On 10/22/06, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> On 10/22/06, snacktime <snacktime at gmail.com> wrote: > > > > Sorry for making you write all that Franics, I must have had a brain > > lock yesterday because I woke up this morning wondering what the hell > > was I thinking when I wrote that. Need to get away from the > > computer... > > _______________________________________________ > > > Does that mean your concurrency issue is solved? >Yes, there is no concurrency issue.> Now just imagine how much harder it would be to track down a race-condition > in a thread-hot program running on SMP hardware. You know the drill, it''s > when your client says "This crappy software you sold me works perfectly for > two weeks, then it crashes twice in one night!"Ya no thanks, I have enough to deal with:)
From: Francis Cianfrocca Sent: Sunday, October 22, 2006 6:09 AM Aside: one thing EM doesn''t currently have is user-defined events. Meaning, you can fire an event internal to your own program, and something else picks up the event and acts on it. I''ve found this pattern to be incredibly useful in writing Ajax programs, where there can be a lot of different active bits all mashed up on a single HTML page. Since they all came from separate Ajax calls, they know nothing about each other. I wrote a little event-library for these and I now can''t imagine writing Ajax any other way. But with server-side programs, I don''t imagine you''d have such a situation intra-process, and you''d use message-queueing (a different kind of event-driven programming) to do it across multiple processes. Currently, client-side, I''m puzzling over how to use EventMachine with FreeGLUT (or possibly SDL), as I''d like to write an OpenGL window server. Unfortuantely, it seems FreeGLUT''s GLUT.MainLoop is not green-threads friendly. Seems to be an ugly problem. I suppose if I turned FreeGLUT''s MainLoop inside out, so I could just poll it to handle events, then I could use a timer within EV to poll the FreeGLUT event loop at 100Hz or something. Weird. An integration of EM and operating system GUI events, somehow, would be awesome. Regards, Bill -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061022/d383dee5/attachment.html
On 10/22/06, Bill Kelly <billk at cts.com> wrote:> > > Weird. An integration of EM and operating system GUI events, somehow, > would be awesome. >This has been talked about a lot, and Twisted of course does it. We should have it in EM. Anybody knowledgeable about how to go about it? Alternatively, does anyone know how Twisted does it? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061022/1837ed64/attachment.html
From: Francis Cianfrocca> > On 10/22/06, Bill Kelly <billk at cts.com> wrote: > > > Weird. An integration of EM and operating system GUI events, > > somehow, would be awesome. > > This has been talked about a lot, and Twisted of course does it. > We should have it in EM. Anybody knowledgeable about how to go > about it? Alternatively, does anyone know how Twisted does it?Suppose I was receiving the GUI events in a separate native thread, and I wanted to forward these events to EM, injecting them into EM as (as-yet-hypothetical) user-defined events? So in this scenario both ruby and EM share a single native OS thread, while my GUI events are received in a separate native thread. Is it theoretically possible to have the GUI thread inject events into the EM thread in a way that allows EM to receive such events in a ruby-green-threads compatbile way? This has always stymied me on Windows: trying to wake up a ruby green thread based on a non-socket-related external event, in a green-threads friendly way. (Apart from using a UDP socket as a means of signaling the ruby thread, which should work, but seems clumsy.) I guess what I''m asking is, if EM *did* support user-defined events, should I expect to be able to inject events into EM from a separate native thread? (Without some timer-based polling hack...) Thanks, Bill
On 10/24/06, Bill Kelly <billk at cts.com> wrote:> > > From: Francis Cianfrocca > > > > On 10/22/06, Bill Kelly <billk at cts.com> wrote: > > > > > Weird. An integration of EM and operating system GUI events, > > > somehow, would be awesome. > > > > This has been talked about a lot, and Twisted of course does it. > > We should have it in EM. Anybody knowledgeable about how to go > > about it? Alternatively, does anyone know how Twisted does it? > > Suppose I was receiving the GUI events in a separate native > thread, and I wanted to forward these events to EM, injecting > them into EM as (as-yet-hypothetical) user-defined events? > > So in this scenario both ruby and EM share a single native > OS thread, while my GUI events are received in a separate > native thread. Is it theoretically possible to have the GUI > thread inject events into the EM thread in a way that allows > EM to receive such events in a ruby-green-threads compatbile > way? > > This has always stymied me on Windows: trying to wake up a > ruby green thread based on a non-socket-related external event, > in a green-threads friendly way. (Apart from using a UDP socket > as a means of signaling the ruby thread, which should work, but > seems clumsy.) > > I guess what I''m asking is, if EM *did* support user-defined > events, should I expect to be able to inject events into EM > from a separate native thread? (Without some timer-based polling > hack...)The native version of the EM core is designed to be completely independent of Ruby, and it''s no big deal to send EM events from native threads. My first reaction to your question, however, was that you can''t allow a native thread to block. On second thought, if your native thread doesn''t know anything at all about Ruby, it just might work. Are you using Windows or GTK or something else? What you''re proposing is awfully intriguing and now I have to try it out. But Twisted incorporates native-platform GUI events (and keyboard events) directly into its event loop, and that would be the better long term solution for us as well. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061024/f5efb52b/attachment-0001.html
From: "Bill Kelly" <billk at cts.com>> > This has always stymied me on Windows: trying to wake up a > ruby green thread based on a non-socket-related external event, > in a green-threads friendly way. (Apart from using a UDP socket > as a means of signaling the ruby thread, which should work, but > seems clumsy.)Oh... lol... just read the comments for LoopBreakerReader in em.cpp. :-) Well since EM already has a loop breaker mechanism, I''ll withdraw my previous questions.... Should have read the source first... Regards, Bill
On 10/24/06, Bill Kelly <billk at cts.com> wrote:> > From: "Bill Kelly" <billk at cts.com> > > > > This has always stymied me on Windows: trying to wake up a > > ruby green thread based on a non-socket-related external event, > > in a green-threads friendly way. (Apart from using a UDP socket > > as a means of signaling the ruby thread, which should work, but > > seems clumsy.) > > Oh... lol... just read the comments for LoopBreakerReader in > em.cpp. :-) > > Well since EM already has a loop breaker mechanism, I''ll withdraw > my previous questions.... Should have read the source first...And did you notice how it uses a UDP socket to signal the Ruby thread (on Windows only), which does work, but is most definitely clumsy? :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061024/0623f7c8/attachment.html
From: Francis Cianfrocca> > The native version of the EM core is designed to be completely > independent of Ruby, and it''s no big deal to send EM events from > native threads. My first reaction to your question, however, was > that you can''t allow a native thread to block. On second thought, > if your native thread doesn''t know anything at all about Ruby, it > just might work.Yeah; actually in a different C++ app, I have ruby in its own thread, and have written a bridge so that any C++ thread can (for ex.) : RubyContext r; // creates a bridge between this C++ thread and a new ruby thread r("some_ruby_code"); // async eval... r("more_ruby_code"); // async eval, queued... return r.value(); // wait for last eval to finish and return result And it bridges exceptions between Ruby/C++ as well. Now if the Ruby interpreter were thread safe (or rather, used native threads instead of green threads), I would have preferred to just call into Ruby directly from any C++ thread. But anyway, I can report that I''ve had no trouble as yet running Ruby (with multiple green threads) as a single native thread among many native threads in a C++ app. (App runs on Win32 and OS X.) (Note: well one caveat is, if Ruby isn''t running on the main thread, the thread stack size can be very small, so we had to crank that up.)> Are you using Windows or GTK or something else? What you''re proposing > is awfully intriguing and now I have to try it out. But Twisted > incorporates native-platform GUI events (and keyboard events) directly > into its event loop, and that would be the better long term solution for > us as well.This new application is windowed but OpenGL-based. (I want it to run on Win32, OS X, and Linux.) I''d like to use SDL; but SDL currently supports only one window per process. I could use a more heavyweight toolkit like FOX or wxWidgets, or GTK, but I''ve avoided that so far as I''d prefer not to pull in a whole toolkit for just an "OpenGL pane". So I''m probably going to go with FreeGLUT (supports multiple OpenGL windows per process, but sort of quirky), or my own custom "open OpenGL window on {win32,os_x,linux}" module. I''ve already written the latter custom module for win32; so if EventMachine were like Twisted where native GUI and keyboard events were natively supported, I''d probably just adapt that code to work with EM, as it''s a pretty small module. (And take the hit of having to also implement that functionality for Linux and OS X.) But anyway, per my earlier post, I was thinking, well, I''ll just run FreeGLUT in its own thread, and just inject all the GUI/kbd events it gives me into the EM thread. :) Regards, Bill
From: Francis Cianfrocca> > And did you notice how it uses a UDP socket to signal the > Ruby thread (on Windows only), which does work, but is most > definitely clumsy? > > :-)Haha, yes. :) Regards, Bill
On 10/24/06, Bill Kelly <billk at cts.com> wrote:> > From: Francis Cianfrocca > > > > The native version of the EM core is designed to be completely > > independent of Ruby, and it''s no big deal to send EM events from > > native threads. My first reaction to your question, however, was > > that you can''t allow a native thread to block. On second thought, > > if your native thread doesn''t know anything at all about Ruby, it > > just might work. > > Yeah; actually in a different C++ app, I have ruby in its own > thread, and have written a bridge so that any C++ thread can > (for ex.) : > > RubyContext r; // creates a bridge between this C++ thread and a > new ruby thread > r("some_ruby_code"); // async eval... > r("more_ruby_code"); // async eval, queued... > return r.value(); // wait for last eval to finish and return result > > And it bridges exceptions between Ruby/C++ as well. > > Now if the Ruby interpreter were thread safe (or rather, used > native threads instead of green threads), I would have preferred > to just call into Ruby directly from any C++ thread. But anyway, > I can report that I''ve had no trouble as yet running Ruby (with > multiple green threads) as a single native thread among many native > threads in a C++ app. (App runs on Win32 and OS X.) (Note: well > one caveat is, if Ruby isn''t running on the main thread, the > thread stack size can be very small, so we had to crank that up.) > > > > Are you using Windows or GTK or something else? What you''re proposing > > is awfully intriguing and now I have to try it out. But Twisted > > incorporates native-platform GUI events (and keyboard events) directly > > into its event loop, and that would be the better long term solution for > > us as well. > > This new application is windowed but OpenGL-based. (I want it > to run on Win32, OS X, and Linux.) I''d like to use SDL; but SDL > currently supports only one window per process. > I could use a more heavyweight toolkit like FOX or wxWidgets, > or GTK, but I''ve avoided that so far as I''d prefer not to > pull in a whole toolkit for just an "OpenGL pane". So I''m > probably going to go with FreeGLUT (supports multiple OpenGL > windows per process, but sort of quirky), or my own custom > "open OpenGL window on {win32,os_x,linux}" module. > > I''ve already written the latter custom module for win32; so if > EventMachine were like Twisted where native GUI and keyboard events > were natively supported, I''d probably just adapt that code to work > with EM, as it''s a pretty small module. (And take the hit of > having to also implement that functionality for Linux and OS X.) > > But anyway, per my earlier post, I was thinking, well, I''ll just > run FreeGLUT in its own thread, and just inject all the GUI/kbd > events it gives me into the EM thread. :)Ok, so you know I''ll hit you up to integrate all these different windowing engines into EM someday :-). Until then, would it be enough if you could call a C function that would inject an event into EM that would get funnelled into a Ruby event handler in the normal way? Obviously you''d have to give your events a globally-unique name (so Ruby code can register to receive them) and of course you can put whatever data you want into each event. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061024/b6e4ddb8/attachment.html
From: Francis Cianfrocca> > Ok, so you know I''ll hit you up to integrate all these different > windowing engines into EM someday :-). Until then, would it be > enough if you could call a C function that would inject an event > into EM that would get funnelled into a Ruby event handler in the > normal way? Obviously you''d have to give your events a globally- > unique name (so Ruby code can register to receive them) and of > course you can put whatever data you want into each event.I think that would be fine. Would this also work with the pure ''C'' interface to EM? It''s possible my thin client may end up being pure C++, as much as I''d prefer to use Ruby. Regards, Bill
From: "Bill Kelly" <billk at cts.com>> > From: Francis Cianfrocca >> >> Ok, so you know I''ll hit you up to integrate all these different >> windowing engines into EM someday :-). Until then, would it be >> enough if you could call a C function that would inject an event >> into EM that would get funnelled into a Ruby event handler in the >> normal way? Obviously you''d have to give your events a globally- >> unique name (so Ruby code can register to receive them) and of >> course you can put whatever data you want into each event. > > I think that would be fine. Would this also work with the > pure ''C'' interface to EM? It''s possible my thin client may > end up being pure C++, as much as I''d prefer to use Ruby.I''m still agonizing over this a bit. I don''t want to give in to Premature Optimization - so I definitely want the GUI events to ultimately get funneled into a Ruby event handler. But I''d like to write the GUI event bridge in such a way, that it will also work fine with EM in a pure ''C'' environment. N.B. It turns out, after some discussion with the FreeGLUT maintainers, that FreeGLUT doesn''t quite meet my needs, either. So I''m looking into FLTK now. ( http://ruby-fltk.sourceforge.net/ ) Argh... of course, the catch is, I can''t run RUBY-fltk in its own native thread, and run RUBY-EM in its own native thread (because that would require parallel ruby interpereters in native threads... which I guess is on the horizon for YARV, but...) What a weird problem. I am constantly fighting green threads vs. native threads on this project... Guess I really should peek at Twisted. (Although, Python does use native threads, if I recall correctly...) Regards, Bill
On 10/26/06, Bill Kelly <billk at cts.com> wrote:> > > I''m still agonizing over this a bit. I don''t want to > give in to Premature Optimization - so I definitely want > the GUI events to ultimately get funneled into a Ruby > event handler. But I''d like to write the GUI event > bridge in such a way, that it will also work fine with > EM in a pure ''C'' environment.I''ve been working on this. At this point, I have two new APIs: EventMachine#post_event( event_name, method_name, data ) EventMachine#add_event_handler( event_name, handler ) You would set up an event handler like so: module MyHandler def function_1 data # your code here end def function_2 data # more code end end EventMachine.run { EventMachine.add_event_handler "xyz", MyHandler EventMachine.post_event "xyz", :function_1, "the data" } Make any sense? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061026/0d048935/attachment-0001.html
From: Francis Cianfrocca> > I''ve been working on this. At this point, I have two new APIs: > > EventMachine#post_event( event_name, method_name, data ) > EventMachine#add_event_handler( event_name, handler ) > > > You would set up an event handler like so: > > module MyHandler > def function_1 data > # your code here > end > def function_2 data > # more code > end > end > > > EventMachine.run { > EventMachine.add_event_handler "xyz", MyHandler > EventMachine.post_event "xyz", :function_1, "the data" > } > > > Make any sense?Cool, interesting. So I''m guessing in this system we wouldn''t be chaining event handlers? Or, if I wanted chaining, I guess I could implement it on top of the basic callback functionality. (Not sure I''ll need it, I''m just used to thinking of event handling in terms of chaining... may not apply here.) At its simplest (I''m not considering nested views and components and such, just windows) I guess I''ll be posting events somewhat like this: EventMachine.post_event "mouse", :click, "the data (window_id,x,y)" EventMachine.post_event "keyboard", :key_down, "the data (window_id,x,y,key)" Something like that . . . Seems to me it should work! I''ll be interested to see what the callback mechanism looks like on the ''C'' side. Would there be any ''C''-level hook where one could simply receive the event at a raw level, like: void handle_event (const char * ev_handler_classname, const char * ev_name, const unsigned char * ev_data, unsigned int ev_data_len) ? Regards, Bill
On 10/27/06, Bill Kelly <billk at cts.com> wrote:> > So I''m guessing in this system we wouldn''t be chaining event > handlers? > > Or, if I wanted chaining, I guess I could implement it on > top of the basic callback functionality. (Not sure I''ll > need it, I''m just used to thinking of event handling in > terms of chaining... may not apply here.)This is a terrific question, and my initial thinking is to avoid chaining because that''s not the pattern we''re using with network events. But the real answer to the question might be eventually to conform how we do network i/o to the more general model represented by these user-defined events. At its simplest (I''m not considering nested views and> components and such, just windows) I guess I''ll be posting > events somewhat like this: > > EventMachine.post_event "mouse", :click, "the data (window_id,x,y)" > EventMachine.post_event "keyboard", :key_down, "the data > (window_id,x,y,key)" > > Something like that . . . > > Seems to me it should work!It would probably be better to define the #post_event parameters as: name, function, *args, wouldn''t it? Then you could say: EventMachine.post_event "keyboard", :key_down, window_id, x, y, key. Another thing I had in mind was anonymous rather than well-known event handlers. This matches the pattern for network I/O. In your example it might be: my_window_object.post_event( :gui_event, :key_down, x, y, key ) I''ll be interested to see what the callback mechanism looks> like on the ''C'' side. Would there be any ''C''-level hook where > one could simply receive the event at a raw level, like: > > void handle_event (const char * ev_handler_classname, > const char * ev_name, > const unsigned char * ev_data, > unsigned int ev_data_len) > > ?Haven''t thought that through but your suggestion looks pretty good. I''m hoping to have this released over the weekend so you can try it out. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061027/696fb6cf/attachment-0001.html
Wouldn''t the answer to chaining really be another module you mix in to the Connection class which provides the functionality? On 10/27/06, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> > On 10/27/06, Bill Kelly <billk at cts.com> wrote: > > > > So I''m guessing in this system we wouldn''t be chaining event > > handlers? > > > > Or, if I wanted chaining, I guess I could implement it on > > top of the basic callback functionality. (Not sure I''ll > > need it, I''m just used to thinking of event handling in > > terms of chaining... may not apply here.) > > > > This is a terrific question, and my initial thinking is to avoid chaining > because that''s not the pattern we''re using with network events. But the real > answer to the question might be eventually to conform how we do network i/o > to the more general model represented by these user-defined events. > > > At its simplest (I''m not considering nested views and > > components and such, just windows) I guess I''ll be posting > > events somewhat like this: > > > > EventMachine.post_event "mouse", :click, "the data (window_id,x,y)" > > EventMachine.post_event "keyboard", :key_down, "the data > > (window_id,x,y,key)" > > > > Something like that . . . > > > > Seems to me it should work! > > > It would probably be better to define the #post_event parameters as: name, > function, *args, wouldn''t it? Then you could say: > EventMachine.post_event "keyboard", :key_down, window_id, x, y, key. > > Another thing I had in mind was anonymous rather than well-known event > handlers. This matches the pattern for network I/O. In your example it might > be: > > my_window_object.post_event( :gui_event, :key_down, x, y, key ) > > > I''ll be interested to see what the callback mechanism looks > > like on the ''C'' side. Would there be any ''C''-level hook where > > one could simply receive the event at a raw level, like: > > > > void handle_event (const char * ev_handler_classname, > > const char * ev_name, > > const unsigned char * ev_data, > > unsigned int ev_data_len) > > > > ? > > > Haven''t thought that through but your suggestion looks pretty good. > > I''m hoping to have this released over the weekend so you can try it out. > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk > >-- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com (970) 232-4208 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061027/55e5332f/attachment.html
On 10/28/06, Tony Arcieri <tony at clickcaster.com> wrote:> > Wouldn''t the answer to chaining really be another module you mix in to the > Connection class which provides the functionality?Can you give a pseudo-code sample to illustrate what you mean? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061028/b7b4c806/attachment.html
Sounds... non-trivial. Heh. - Tony On 10/27/06, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> > On 10/28/06, Tony Arcieri <tony at clickcaster.com> wrote: > > > > Wouldn''t the answer to chaining really be another module you mix in to > > the Connection class which provides the functionality? > > > > Can you give a pseudo-code sample to illustrate what you mean? > > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk > >-- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com (970) 232-4208 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061027/ec2e84d2/attachment.html
From: Francis Cianfrocca> > On 10/27/06, Bill Kelly <billk at cts.com> wrote: > > EventMachine.post_event "mouse", :click, "the data (window_id,x,y)" > EventMachine.post_event "keyboard", :key_down, "the data (window_id,x,y,key)" > > Something like that . . . > > Seems to me it should work! > > It would probably be better to define the #post_event parameters > as: name, function, *args, wouldn''t it? Then you could say: > EventMachine.post_event "keyboard", :key_down, window_id, x, y, key.Oh, cool - I thought "the data" was a binary string (to be #unpack''d) name, function, *args name, function, *args = "mouse", "click", window_id, x, y name, function, *args = "keyboard", "key_down", window_id, x, y, key_code Yeah, name, function, *args is what I thought you mean, except I guessed "the data" was binary. (Or I might have misunderstood name,function :) Regards, Bill
From: Francis Cianfrocca> > Another thing I had in mind was anonymous rather than well-known > event handlers. This matches the pattern for network I/O. In your > example it might be: > > my_window_object.post_event( :gui_event, :key_down, x, y, key )Trying to figure this out ... Does ''my_window_object'' OK, so that shifts the second parameter to being more categorical of any GUI event. But does ''my_window_object'' get included in the event somehow? I.e. when event is received, is ''my_window_object.object_id'' (or equivalent) available to receiver of event? Sorry for confusion, just wondering if the window object sending the message is part of the event structure? Regards, Bill
On 10/28/06, Bill Kelly <billk at cts.com> wrote:> > > From: Francis Cianfrocca > > > > On 10/27/06, Bill Kelly <billk at cts.com> wrote: > > > > EventMachine.post_event "mouse", :click, "the data (window_id,x,y)" > > EventMachine.post_event "keyboard", :key_down, "the data > (window_id,x,y,key)" > > > > Something like that . . . > > > > Seems to me it should work! > > > > It would probably be better to define the #post_event parameters > > as: name, function, *args, wouldn''t it? Then you could say: > > EventMachine.post_event "keyboard", :key_down, window_id, x, y, key. > > Oh, cool - I thought "the data" was a binary string (to be #unpack''d) > > name, function, *args > name, function, *args = "mouse", "click", window_id, x, y > name, function, *args = "keyboard", "key_down", window_id, x, y, key_code > > Yeah, name, function, *args is what I thought you mean, except > I guessed "the data" was binary. (Or I might have misunderstood > name,function :)Well, the idea behind "name" is the event source. With the current (system-defined) event sources, it would be something like "keyboard" or "signal" or "TCP" or "UDP." And :function would be the method that would be called inside the handler. Analogous to today''s "post_init" and "receive_data" and "close_connection." I''m hoping to come up with a uniform framework for handling both system events and user-defined ones. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20061028/6673001d/attachment.html
From: Francis Cianfrocca> > Well, the idea behind "name" is the event source. With the > current (system-defined) event sources, it would be something > like "keyboard" or "signal" or "TCP" or "UDP." And :function > would be the method that would be called inside the handler. > Analogous to today''s "post_init" and "receive_data" and > "close_connection." > > I''m hoping to come up with a uniform framework for handling > both system events and user-defined ones.Cool. I was looking at possibilities for handling both the socket and GUI events from a single thread on Windows. Unfortunately, it looks like MsgWaitForMultipleObjects takes every kind of object known to man except sockets or file handles. (And why sockets and file handles aren''t the same is apparently a different discussion entirely....) One possibility seems to be to use WSAAsyncSelect. This causes socket events to be posted to the windows event queue. One could then use MsgWaitForMultipleObjects to receive both GUI and socket events. However, WSAAsyncSelect seems bizarre to me, in that one has to specify a window handle to which the socket events are posted. This would seem to imply, any application which opened and closed its windows at will (or, which didn''t want to open any windows at all), would still have to open some invisible/fake window, just to receive the socket events. If that''s true--I don''t know windows very well (so far it seems like, "You are lost in a twisty little maze of APIs, all different")--opening a little fake window just seems crazy to me. So I think I''ll go ahead with receiving the GUI messages on a separate thread, after all... (And inject them into EventMachine.) Regards, Bill