Hi, First of all I''d like to say I really love this Rails community. Everyone is very helpful and I''ve learned so much that I simply couldn''t have learned just by reading books. I was wondering if you could help me with the following as well. My general question is: will Rails be the way to go with this? I need an application that does server monitoring. But not general server monitoring (as in ping specific ports), it actually queries servers for information and based on that information it will act. There will be a list of servers and generally, when the end of the list is reached, it will start all over again. The list will be updated by the application, but also by user input. I need to be able to set an interval between the requests, but the application also needs to know about certain limitations. For example, there will be a maximum of queries to a specific server in a single hour as some database entries point to the same server, but with a different query. At this point I ask you - is this all possible? The application needs to run on itself for months, maybe even years in a row and be very low- maintenance. Well, if that''s possible with Rails, there''s another part of the application that needs to act on specific results. For example, when something''s not the way it should be, it needs to send an e-mail. The thing is, this e-mail needs to be sent -fast- ! I''m talking about milliseconds here. This is because the e-mail will be processed by another server automatically. Every millisecond counts with this. Again, I ask you: is Rails the way to go? I''m quite sure I want to develop the front-end in Rails, but I''m unsure as to whether the backend will be just as fast in Rails as in any other programming language. Are there things to think about here? For example, I don''t know how I would best send these e-mails. SMTP? Is this fast in Rails? Any faster or slower than in any other programming language? And why? I''m very inexperienced with the properties of various available programming languages and wouldn''t know where to start exploring this. And how can I make this an ongoing process? I don''t want a cronjob- powered script, for example. Later on, when there''s a lot to query, we might move from sending a query once every five seconds to a hundred times per second. How to do this? It could also be also time-specific. Some database entries only need to be monitored from 8AM to 5PM for example. But not all. I hope you can help me with this, as I''m not sure where to start. If you tell me to go and learn another programming language that''s faster, I will. Just be honest. Thank you very much.
www.spiceworks.com ? -- Posted via http://www.ruby-forum.com/.
> www.spiceworks.com?I love Spiceworks, but it''s not what I''m looking for here. Sorry!
On Sep 2, 7:13 pm, jhaagmans <jaap.haagm...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > And how can I make this an ongoing process? I don''t want a cronjob- > powered script, for example. Later on, when there''s a lot to query, we > might move from sending a query once every five seconds to a hundred > times per second. How to do this? It could also be also time-specific. > Some database entries only need to be monitored from 8AM to 5PM for > example. But not all. > > I hope you can help me with this, as I''m not sure where to start. If > you tell me to go and learn another programming language that''s > faster, I will. Just be honest. >To be honest, if you''re sending emails, querying remote servers etc. then you shouldn''t be worrying about milliseconds - either of those two actions could take far longer than that. The smtp server you talk too could easily sit on your message for seconds or even minutes - email isn''t an instant delivery protocol. Your server-monitory thing is going to be some sort of background process, you may find a single instance of that gets more done if you''re using something like jruby that has better threading than normal ruby. Stuff like eventmachine is pretty cool for having a single process multiplex a lot of IO. Fred> Thank you very much.
In a prior life I wrote a control process for the FAA that managed a distributed suite. The suite was spread over three sites, 45+ pieces of hardware, 25 communicating apps, 165+ command line arguments and provided near realtime advice to air traffic controllers. Features included near real time sensing of failed hardware and software with automated failover to redundant components. The system needed to be site configurable to support (initially) four regions. I started by developing a language specific to the problem at hand and implemented this using lex and yacc. For infrastructure I used the Parallel Virtual Machine (PVM) from Oak Ridge National Laboratory, all coding was done in C with user application libraries available in C and C++. Ruby and Rails are fine applications but I would not consider them suitable for any time sensitive application. You could build useful support and analysis tools and you might even be able to build a simulator, but... My guess is that in a distributed monitor and control situation you''ld be looking at sense and respond times in the multi tens of seconds. On Sep 2, 5:52 pm, Frederick Cheung <frederick.che...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On Sep 2, 7:13 pm, jhaagmans <jaap.haagm...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > > And how can I make this an ongoing process? I don''t want a cronjob- > > powered script, for example. Later on, when there''s a lot to query, we > > might move from sending a query once every five seconds to a hundred > > times per second. How to do this? It could also be also time-specific. > > Some database entries only need to be monitored from 8AM to 5PM for > > example. But not all. > > > I hope you can help me with this, as I''m not sure where to start. If > > you tell me to go and learn another programming language that''s > > faster, I will. Just be honest. > > To be honest, if you''re sending emails, querying remote servers etc. > then you shouldn''t be worrying about milliseconds - either of those > two actions could take far longer than that. The smtp server you talk > too could easily sit on your message for seconds or even minutes - > email isn''t an instant delivery protocol. Your server-monitory thing > is going to be some sort of background process, you may find a single > instance of that gets more done if you''re using something like jruby > that has better threading than normal ruby. Stuff like eventmachine is > pretty cool for having a single process multiplex a lot of IO. > > Fred > > > Thank you very much.
On 2 sep, 23:52, Frederick Cheung <frederick.che...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On Sep 2, 7:13 pm, jhaagmans <jaap.haagm...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > > And how can I make this an ongoing process? I don''t want a cronjob- > > powered script, for example. Later on, when there''s a lot to query, we > > might move from sending a query once every five seconds to a hundred > > times per second. How to do this? It could also be also time-specific. > > Some database entries only need to be monitored from 8AM to 5PM for > > example. But not all. > > > I hope you can help me with this, as I''m not sure where to start. If > > you tell me to go and learn another programming language that''s > > faster, I will. Just be honest. > > To be honest, if you''re sending emails, querying remote servers etc. > then you shouldn''t be worrying about milliseconds - either of those > two actions could take far longer than that. The smtp server you talk > too could easily sit on your message for seconds or even minutes - > email isn''t an instant delivery protocol. Your server-monitory thing > is going to be some sort of background process, you may find a single > instance of that gets more done if you''re using something like jruby > that has better threading than normal ruby. Stuff like eventmachine is > pretty cool for having a single process multiplex a lot of IO. > > Fred >Thanks Frederick, The thing is, I can''t choose the tools I need to work with. I''m not worrying about server response time from the servers that I will query, that''s just a given and I cannot do anything about it. I also cannot worry about e-mail delivery time -after it left my machine- because that also is just a given. The e-mails will be sent to servers physically very close to the server sending out the e-mails. Just believe me, any millisecond counts. That''s also the reason I will be creating this application: I will need to be able to fine-tune it later on so that it fits perfectly. To Rick: Do you think C and/or C++ are more capable of doing this? I have no prior experience with C, so I''ll have to look into the possibilities. To all: Is Rails, or at least Ruby, able to do server monitoring on pre-set intervals? Forget about the speed issue for a moment, but would it be able to query a server, wait for its response (we''re talking about normal HTTP-requests), store this response and go on? There will be three applications running on three servers. One is just querying -a lot- of servers and it doesn''t matter whether it has to wait for the previous request to finish or not (so I guess multi-threading will not be needed, right?). It only needs to store the last response and the time of that last response. The second is querying only a few servers and there are limitations to how often it can query a specific server, but it should do a request every second (meaning that when a request takes over 1 second, it will need to start a new thread). "Normal" Ruby will probably not be able to do this, right? The last one will do many requests per second to one or two servers, also on a pre-set interval, but it also needs to send out an e-mail when the response differs from what is expected. I guess, especially for this last application, Ruby won''t be able to help me. Am I right? I''m going to buy some C books, at least for app 2 and 3, I guess ;) Thank you both!
On Sep 3, 11:03 am, jhaagmans <jaap.haagm...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On 2 sep, 23:52, Frederick Cheung <frederick.che...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > Thanks Frederick, > > The thing is, I can''t choose the tools I need to work with. I''m not > worrying about server response time from the servers that I will > query, that''s just a given and I cannot do anything about it. I also > cannot worry about e-mail delivery time -after it left my machine- > because that also is just a given. The e-mails will be sent to servers > physically very close to the server sending out the e-mails. Just > believe me, any millisecond counts. That''s also the reason I will be > creating this application: I will need to be able to fine-tune it > later on so that it fits perfectly.Just seems highly likely that any time you save will actually be lost in the noise.> > Is Rails, or at least Ruby, able to do server monitoring on pre-set > intervals? Forget about the speed issue for a moment, but would it be > able to query a server, wait for its response (we''re talking about > normal HTTP-requests), store this response and go on?Don''t see why not. As you suspect this bit wouldn''t actually use rails at all (maybe activerecord if you want to store the results in a database)> There will be > three applications running on three servers. One is just querying -a > lot- of servers and it doesn''t matter whether it has to wait for the > previous request to finish or not (so I guess multi-threading will not > be needed, right?). It only needs to store the last response and the > time of that last response. The second is querying only a few servers > and there are limitations to how often it can query a specific server, > but it should do a request every second (meaning that when a request > takes over 1 second, it will need to start a new thread). "Normal" > Ruby will probably not be able to do this, right? The last one will do > many requests per second to one or two servers, also on a pre-set > interval, but it also needs to send out an e-mail when the response > differs from what is expected. I guess, especially for this last > application, Ruby won''t be able to help me. Am I right? >stuff like eventmachine does all the gnarly IO bits in C so can be pretty effective for this sort of stuff if you''re worried about the performance side Fred> I''m going to buy some C books, at least for app 2 and 3, I guess ;) > > Thank you both!
Thanks Fred :)> Just seems highly likely that any time you save will actually be lost > in the noise.If it does, it does. However, on average, an e-mail sent 1 millisecond faster will also be received 1 ms faster. I know you think it doesn''t matter, but for this application, it matters. Trust me on this. Even server load isn''t as important as speed. And thanks for pointing me towards the EventMachine, I knew it was there, but I just forgot.
My point in bringing in C was that the tools you choose are those that can do the job. If you have a real time requirement (something with msecs response) you won''t be happy with interpreted code exchanging messages via email. However, if what you''ve been given is more on the order of "use Ruby on Rails and make this as fast as possible" then you can certainly get the job done. The real tradeoff here is that to achieve real speed you must invariably give up flexibility. Even though you could write a hard drive controller in Ruby I doubt you would be satisfied with the result. I would recommend that you spend some time defining what you are trying to do and get requirements set for the areas of your application in which performance is critical. You should have real numbers here - or agreement with your user that "as fast as possible" is adequate. Take a look at SNMP as a mechanism for collecting data from remote systems - you might look at MRTG for example. Quite often you can partition the time sensitive portion out from the reporting portion. You could, for example, build status logs to report out when all is running in a pre-defined "good" way and make a nice web view that''s driven from the logs. You obviously need to develop a strategy for reporting and responding to the alert and alarm states that populate the "not good" space. This is the place where time can be critical. So go for it - sounds like fun. On Sep 3, 6:03 am, jhaagmans <jaap.haagm...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On 2 sep, 23:52, Frederick Cheung <frederick.che...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > > On Sep 2, 7:13 pm, jhaagmans <jaap.haagm...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > And how can I make this an ongoing process? I don''t want a cronjob- > > > powered script, for example. Later on, when there''s a lot to query, we > > > might move from sending a query once every five seconds to a hundred > > > times per second. How to do this? It could also be also time-specific. > > > Some database entries only need to be monitored from 8AM to 5PM for > > > example. But not all. > > > > I hope you can help me with this, as I''m not sure where to start. If > > > you tell me to go and learn another programming language that''s > > > faster, I will. Just be honest. > > > To be honest, if you''re sending emails, querying remote servers etc. > > then you shouldn''t be worrying about milliseconds - either of those > > two actions could take far longer than that. The smtp server you talk > > too could easily sit on your message for seconds or even minutes - > > email isn''t an instant delivery protocol. Your server-monitory thing > > is going to be some sort of background process, you may find a single > > instance of that gets more done if you''re using something like jruby > > that has better threading than normal ruby. Stuff like eventmachine is > > pretty cool for having a single process multiplex a lot of IO. > > > Fred > > Thanks Frederick, > > The thing is, I can''t choose the tools I need to work with. I''m not > worrying about server response time from the servers that I will > query, that''s just a given and I cannot do anything about it. I also > cannot worry about e-mail delivery time -after it left my machine- > because that also is just a given. The e-mails will be sent to servers > physically very close to the server sending out the e-mails. Just > believe me, any millisecond counts. That''s also the reason I will be > creating this application: I will need to be able to fine-tune it > later on so that it fits perfectly. > > To Rick: > > Do you think C and/or C++ are more capable of doing this? I have no > prior experience with C, so I''ll have to look into the possibilities. > > To all: > > Is Rails, or at least Ruby, able to do server monitoring on pre-set > intervals? Forget about the speed issue for a moment, but would it be > able to query a server, wait for its response (we''re talking about > normal HTTP-requests), store this response and go on? There will be > three applications running on three servers. One is just querying -a > lot- of servers and it doesn''t matter whether it has to wait for the > previous request to finish or not (so I guess multi-threading will not > be needed, right?). It only needs to store the last response and the > time of that last response. The second is querying only a few servers > and there are limitations to how often it can query a specific server, > but it should do a request every second (meaning that when a request > takes over 1 second, it will need to start a new thread). "Normal" > Ruby will probably not be able to do this, right? The last one will do > many requests per second to one or two servers, also on a pre-set > interval, but it also needs to send out an e-mail when the response > differs from what is expected. I guess, especially for this last > application, Ruby won''t be able to help me. Am I right? > > I''m going to buy some C books, at least for app 2 and 3, I guess ;) > > Thank you both!
On Sep 3, 12:00 pm, jhaagmans <jaap.haagm...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Thanks Fred :) > > > Just seems highly likely that any time you save will actually be lost > > in the noise. > > If it does, it does. However, on average, an e-mail sent 1 millisecond > faster will also be received 1 ms faster. I know you think it doesn''t > matter, but for this application, it matters. Trust me on this. Even > server load isn''t as important as speed.I have to agree with Fred - trying to optimize things to this level is almost certainly a mistake. There are far too many other factors that can randomly devour >10ms at a pass: the size of the OS timeslice (typically around 10ms), swapping pages in from disk (50ms to ?), network contention, etc. On the other hand, if you''re really serious about being fast, SMTP is doing it completely wrong. Write a real TCP client and you''ll clear out a massive amount of overhead. Unless this is for one of the high-frequency stock trading operations, in which case I encourage you to spend as much time and money on it as possible - tell them they need a custom OS or something... ;) --Matt Jones
On 3 sep, 18:08, Rick <richard.t.ll...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> My point in bringing in C was that the tools you choose are those that > can do the job. If you have a real time requirement (something with > msecs response) you won''t be happy with interpreted code exchanging > messages via email. However, if what you''ve been given is more on the > order of "use Ruby on Rails and make this as fast as possible" then > you can certainly get the job done.What I was given was: - I''d like you to develop an app in PHP - When x happens, an e-mail needs to be sent out as fast as possible and every millisecond counts. At that moment I mentioned Rails because I had no clue on what this app needed to do and I just love Rails and want to limit the time I spend coding PHP scripts. However, now I know the details, I think it might be important to look into other programming languages and/or frameworks. Of course, cost is an issue and it''s an application for which I could probably have a v0.1 ready in under 20 hours. If I need to learn a new language, though, I might be taking alot longer. I know, that''s his problem, but I want to offer a set of possibilities and it''s for him to decide what''s most important.> > The real tradeoff here is that to achieve real speed you must > invariably give up flexibility. Even though you could write a hard > drive controller in Ruby I doubt you would be satisfied with the > result. > > I would recommend that you spend some time defining what you are > trying to do and get requirements set for the areas of your > application in which performance is critical. You should have real > numbers here - or agreement with your user that "as fast as possible" > is adequate.The numbers aren''t disclosed. We will need to develop the application, cross our fingers and hope it''s fast enough. It could easily be that it''s not possible and our e-mail messages arrive too late every time. That''s why I need to be sure that what we develop is the best we can come up with within the budget. If C is the way to go, I will offer that possibility. However, I''m quite sure I will need twice as long in C and might have less time left for fine-tuning.> > Take a look at SNMP as a mechanism for collecting data from remote > systems - you might look at MRTG for example. Quite often you can > partition the time sensitive portion out from the reporting portion. > You could, for example, build status logs to report out when all is > running in a pre-defined "good" way and make a nice web view that''s > driven from the logs. You obviously need to develop a strategy for > reporting and responding to the alert and alarm states that populate > the "not good" space. This is the place where time can be critical.What you''re saying might be very important. I know the kind of information I''m looking for, so if I can assign the highest priority to the incoming data that contains this information, that could help. The third time-critical application will actually be designed so that when the time-critical stuff comes up, it drops everything it does and sends out the e-mail.> > So go for it - sounds like fun.At least it''s very interesting! It will be fun when I figure it out, - if- I figure it out ;)
Hi, Thanks Matt. Really appreciate your help :) On 3 sep, 18:29, Matt Jones <al2o...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On Sep 3, 12:00 pm, jhaagmans <jaap.haagm...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > Thanks Fred :) > > > > Just seems highly likely that any time you save will actually be lost > > > in the noise. > > > If it does, it does. However, on average, an e-mail sent 1 millisecond > > faster will also be received 1 ms faster. I know you think it doesn''t > > matter, but for this application, it matters. Trust me on this. Even > > server load isn''t as important as speed. > > I have to agree with Fred - trying to optimize things to this level is > almost certainly a mistake. There are far too many other factors that > can randomly devour >10ms at a pass: the size of the OS timeslice > (typically around 10ms), swapping pages in from disk (50ms to ?), > network contention, etc.That doesn''t take away my first point. On average, every millisecond that I can shave off, will result in the e-mail getting there faster. And again, trust me, every millisecond counts. I don''t want to debate on the need of shaving these milliseconds off; it''s the task I got. My question is where to start shaving.> > On the other hand, if you''re really serious about being fast, SMTP is > doing it completely wrong. Write a real TCP client and you''ll clear > out a massive amount of overhead.Great, that''s useful. I''ll look into that.> > Unless this is for one of the high-frequency stock trading operations, > in which case I encourage you to spend as much time and money on it as > possible - tell them they need a custom OS or something... ;)You''re actually on the right track, but also way off ;) The thing is, I -will- start work on this at some point and I -will- need to make sure it''s my best effort. The budget also has some weight of course, which is why I can''t get too fancy, developing a custom OS for example ;)
Well, just remember that Rails is a web framework, and Ruby is (as has been mentioned) an interpreted language. If speed matters, then you need to go as low in the language hierarchy as possible. And usually when you talk about going low, you mean C... As I was reminded here, though, bits of Ruby gems are written in C, precisely for speed reasons :) And if every millisecond counts, then yes, you probably want to control everything, and that means email is probably also the wrong way to go. You''ll probably want to send your own messages through your own interface. I did not test this and I can be wrong, but it may be faster to just send one TCP packet with a code which gets translated to a real message when it gets to the recipient. Again .. Speed vs. Flexibility :) Good luck, and remember to share what you learned.. If you can''t share the code ;-) -- Posted via http://www.ruby-forum.com/.
On 3 sep, 21:01, Aldric Giacomoni <rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:> Well, just remember that Rails is a web framework, and Ruby is (as has > been mentioned) an interpreted language. > If speed matters, then you need to go as low in the language hierarchy > as possible. And usually when you talk about going low, you mean C... As > I was reminded here, though, bits of Ruby gems are written in C, > precisely for speed reasons :)That helps :)> > And if every millisecond counts, then yes, you probably want to control > everything, and that means email is probably also the wrong way to go. > You''ll probably want to send your own messages through your own > interface. I did not test this and I can be wrong, but it may be faster > to just send one TCP packet with a code which gets translated to a real > message when it gets to the recipient. Again .. Speed vs. Flexibility :)We don''t control the recipient and we have to send an e-mail. It might change, but it won''t for the forseeable future.> > Good luck, and remember to share what you learned.. If you can''t share > the code ;-)I will share what I can share, but if I don''t use Rails, I probably won''t be sharing it here ;)
You keep saying "every millisecond counts", yet your requirement was given basically as "I need an F1 racecar, here take this bicycle and build me one" to which you responded "I''m more familiar with mopeds, I''ll use that instead".
jhaagmans wrote:> I will share what I can share, but if I don''t use Rails, I probably > won''t be sharing it here ;)Well, you''ve got a lot of people curious now, I hope you leave us with a way to follow up, at least! "And in other news, a crazed developer was seen hanging from an F-16. He had apparently painted the word ''Jets on Rails'' on all the planes while foaming at the mouth. Investigations continue as to how he circumvented security." "Hey, I remember him!" -- Posted via http://www.ruby-forum.com/.
On 3 sep, 21:07, jemminger <jemmin...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> You keep saying "every millisecond counts", yet your requirement was > given basically as "I need an F1 racecar, here take this bicycle and > build me one" to which you responded "I''m more familiar with mopeds, > I''ll use that instead".Well, I''m not more familiar with Rails. I''ve been using PHP for over eight years now and I''m still learning Rails. However, when he asked me whether I could do "something" in PHP, I stated that I preferred using Rails. When it got clear what needed to be done, I wasn''t so sure what I would need, which is why I started this topic. What''s wrong about that?
jhaagmans wrote:> On 3 sep, 18:08, Rick <richard.t.ll...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >> My point in bringing in C was that the tools you choose are those that >> can do the job. �If you have a real time requirement (something with >> msecs response) you won''t be happy with interpreted code exchanging >> messages via email. �However, if what you''ve been given is more on the >> order of "use Ruby on Rails and make this as fast as possible" then >> you can certainly get the job done. > > What I was given was: > - I''d like you to develop an app in PHP > - When x happens, an e-mail needs to be sent out as fast as possible > and every millisecond counts. >Why not just set up Nagios and have done with it? SNMP might be useful too... Anyway, if you''ve been told every millisecond counts, you need to know *why*. Do you? Best, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org -- Posted via http://www.ruby-forum.com/.
On 3 sep, 21:18, Aldric Giacomoni <rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:> jhaagmans wrote: > > I will share what I can share, but if I don''t use Rails, I probably > > won''t be sharing it here ;) > > Well, you''ve got a lot of people curious now, I hope you leave us with a > way to follow up, at least!I''m working on a blog to share these kinds of things with the rest of the world. I have a feeling everyone having a crazy idea starts knocking on my door lately. I''ve never done the things I''m trying to do now when I was using PHP. I learn so much so fast! Lol @ news update xD
Thanks Marnen :) On 3 sep, 21:22, Marnen Laibow-Koser <rails-mailing-l...@andreas- s.net> wrote:> jhaagmans wrote: > > On 3 sep, 18:08, Rick <richard.t.ll...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > >> My point in bringing in C was that the tools you choose are those that > >> can do the job. If you have a real time requirement (something with > >> msecs response) you won''t be happy with interpreted code exchanging > >> messages via email. However, if what you''ve been given is more on the > >> order of "use Ruby on Rails and make this as fast as possible" then > >> you can certainly get the job done. > > > What I was given was: > > - I''d like you to develop an app in PHP > > - When x happens, an e-mail needs to be sent out as fast as possible > > and every millisecond counts. > > Why not just set up Nagios and have done with it? SNMP might be useful > too...Because I will not be doing infrastructure monitoring. I''m querying servers and applications running on servers (could even be PHP scripts) for actual information which I will use in the applications.> > Anyway, if you''ve been told every millisecond counts, you need to know > *why*. Do you?I do. And he''s not exaggerating. But again, I can''t go into the details. I''m absolutely 100% sure that anyone here would agree with me that every millisecond counts. But I can''t say any more about it and it''s useless to discuss whether it''s actually needed when I can''t explain why. Just believe me. Thanks again to all. By monday I will get back in touch with my client and I will present him with options I think are usable. If anyone has any comments on how to do this, please share them.
jhaagmans wrote:> Because I will not be doing infrastructure monitoring. I''m querying > servers and applications running on servers (could even be PHP > scripts) for actual information which I will use in the applications. > >> >> Anyway, if you''ve been told every millisecond counts, you need to know >> *why*. �Do you? > > I do. And he''s not exaggerating.And the plot thickens. One more time, then - you are being asked to develop a real-time application which depends on : a) a network b) other computers c) applications on other computers d) at least one email server And every millisecond counts? [ Hey, can I have fries with that? :) ] -- Posted via http://www.ruby-forum.com/.
> And the plot thickens. One more time, then - you are being asked to > develop a real-time application which depends on : > a) a network > b) other computers > c) applications on other computers > d) at least one email server > > And every millisecond counts? [ Hey, can I have fries with that? :) ]Just remember to forget about the why. The how, that''s what I need help with. I''ll be looking at the possibilities tomorrow and decide what I think is best. "just remember to forget" ... I should write that one down !
jhaagmans wrote:>> And the plot thickens. One more time, then - you are being asked to >> develop a real-time application which depends on : >> a) a network >> b) other computers >> c) applications on other computers >> d) at least one email server >> >> And every millisecond counts? [ Hey, can I have fries with that? :) ] > > Just remember to forget about the why.Um...no. I''m not trying to pry into your requirements, but I *am* telling you that your requirements must make sense, or else your project is doomed. You are being asked to build an F-16 with your choice of a bicycle or a moped as engine. Oh, and it has to have a fuel efficiency of 30 MPG, and don''t spend too much time developing it. Your requirements do not make sense.> The how, that''s what I need > help with. I''ll be looking at the possibilities tomorrow and decide > what I think is best.You''ve been given a self-inconsistent set of requirements. Don''t even bother until you can get a more feasible set of requirements.> > "just remember to forget" ... I should write that one down !I should write that down too -- as a sign that a project is in trouble. Best, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org -- Posted via http://www.ruby-forum.com/.
> Um...no. I''m not trying to pry into your requirements, but I *am* > telling you that your requirements must make sense, or else your project > is doomed. > > You are being asked to build an F-16 with your choice of a bicycle or a > moped as engine. Oh, and it has to have a fuel efficiency of 30 MPG, > and don''t spend too much time developing it. Your requirements do not > make sense.I''m not bound to using PHP, Ruby or anything. I''m just bound to a certain budget which limits my possibilities. And that will limit the speed of the application. It will be a matter of as fast as possible though, which is why I want to do this using the right tool. The requirements make sense. At least, to me they do. He also has his reasons to believe this can be done on a low budget. And the thing is, I don''t doubt it. The goal of the application is to increase the success rate of my client''s business. Say it''s at 1:1000 right now. If I can get it up to 1:500 (which is a matter of speed in this case) I will have myself one happy client. However, if I can get it up to 1:400, I will have an even happier client. If I exceed the budget by more than say, 10%, I will have an angry client. So I need to increase the speed by as much as possible within the budget. So again I ask you to stop questioning the why. If it''s not possible, I will find that out and so will he. But I do want to give this an honest chance.
jhaagmans wrote:>> Um...no. �I''m not trying to pry into your requirements, but I *am* >> telling you that your requirements must make sense, or else your project >> is doomed. >> >> You are being asked to build an F-16 with your choice of a bicycle or a >> moped as engine. �Oh, and it has to have a fuel efficiency of 30 MPG, >> and don''t spend too much time developing it. �Your requirements do not >> make sense. > > I''m not bound to using PHP, Ruby or anything. I''m just bound to a > certain budget which limits my possibilities. And that will limit the > speed of the application. It will be a matter of as fast as possible > though, which is why I want to do this using the right tool.That makes a lot more sense.> > The requirements make sense. At least, to me they do. He also has his > reasons to believe this can be done on a low budget. And the thing is, > I don''t doubt it.I do. At least, I doubt that it can be done on a low budget by *you*. I don''t mean that as an insult, just that based on the questions you''ve asked in this thread, your expertise seems to be elsewhere, so it will take you more time and effort for this project than it would someone who does this kind of stuff all the time.> > The goal of the application is to increase the success rate of my > client''s business. Say it''s at 1:1000 right now. If I can get it up to > 1:500 (which is a matter of speed in this case) I will have myself one > happy client. However, if I can get it up to 1:400, I will have an > even happier client. If I exceed the budget by more than say, 10%, I > will have an angry client. So I need to increase the speed by as much > as possible within the budget.In other words, your client will not be willing to pay that extra 10% even if he gets an extra 20% beyond what he was promised in success increase. Right? If that''s so, dump the client. He wants the world on a shoestring.> > So again I ask you to stop questioning the why.I hear this from clients too from time to time. It is almost always bad advice. If you can''t say "*why* do you need this" to your client, then you cannot help your client optimally. Requirements are not articles of faith.> If it''s not possible, > I will find that out and so will he. But I do want to give this an > honest chance.Then ask questions and listen to the advice you''ve been getting in this thread. Better yet, find a TCP/IP/real-time systems weenie to work with. Right now, this project has "Daily WTF" written all over it -- and that will result in a *very* angry client. Best, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org -- Posted via http://www.ruby-forum.com/.
Thanks Marnen. But again, I know about the why. And I can ask my client those questions. I''m just saying -you- shouldn''t, simply because I can''t tell you. Everyone has been very helpful as to how I might be able to pull this off and that''s what I wanted from this. None of my projects have thusfar resulted in a big traincrash and I highly doubt this one will.
2009/9/4 jhaagmans <jaap.haagmans-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:> > Thanks Marnen. > > But again, I know about the why. And I can ask my client those > questions. I''m just saying -you- shouldn''t, simply because I can''t > tell you. Everyone has been very helpful as to how I might be able to > pull this off and that''s what I wanted from this. None of my projects > have thusfar resulted in a big traincrash and I highly doubt this one > will. > --~--~---------~--~----~------------~-------~--~----~I wonder if it is the _sending_ of the email that is important here. Perhaps the time at which it is received is not important. Colin
Thanks Colin,> I wonder if it is the _sending_ of the email that is important here. > Perhaps the time at which it is received is not important.It''s actually something in between. The moment at which the e-mail leaves our machine is important to me. After that we will have to accept that it might get caught up somewhere. At least, I think we will... If the remote machine queues it up, we don''t know what happens, we''re not sure how the remote machine handles messages that were delayed by its own infrastructure. But I will have to do some trial and error there.
jhaagmans wrote:> Thanks Colin, > >> I wonder if it is the _sending_ of the email that is important here. >> Perhaps the time at which it is received is not important. > > It''s actually something in between. The moment at which the e-mail > leaves our machine is important to me. After that we will have to > accept that it might get caught up somewhere. At least, I think we > will... If the remote machine queues it up, we don''t know what > happens, we''re not sure how the remote machine handles messages that > were delayed by its own infrastructure. But I will have to do some > trial and error there.You know that just about everything email-wise can be falsified, including timestamps, right? -- Posted via http://www.ruby-forum.com/.
> You know that just about everything email-wise can be falsified, > including timestamps, right?I do :) However, I doubt that timestamps matter for the remote machine. I think everything is processed in the order it got in. But again, we''ll need to find that out. I can''t be sure about it.
jhaagmans wrote:>> You know that just about everything email-wise can be falsified, >> including timestamps, right? > > I do :) > > However, I doubt that timestamps matter for the remote machine. I > think everything is processed in the order it got in. But again, we''ll > need to find that out. I can''t be sure about it.Can you ask the folks who control the recipient machine? The less guessing you have to do, the better. Best, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org -- Posted via http://www.ruby-forum.com/.
jhaagmans wrote:> Thanks Marnen. > > But again, I know about the why. And I can ask my client those > questions.Good> I''m just saying -you- shouldn''t, simply because I can''t > tell you.You don''t get it, then. I''m not asking because I want the answers; I''m asking to make sure *you* can formulate the answers. Apparently you can; if so, great.> Everyone has been very helpful as to how I might be able to > pull this off and that''s what I wanted from this. None of my projects > have thusfar resulted in a big traincrash and I highly doubt this one > will.I hate to be a pessimist, but I cannot see how it will be anything but. You''re trying to implement a real-time low-latency messaging queue over e-mail, and you''re trying to do it on a low budget. Moreover, your expertise seems to be in a different area, so you''ll be spending lots of that low budget on research. I suppose it''s possible to pull these chestnuts out of the fire, but suffice it to say that I''m really worried. BTW, is the machine on the receiving end actually checking e-mail often enough that every millisecond is significant? I would doubt it, and if not, then this whole project is a waste of time. Best, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org -- Posted via http://www.ruby-forum.com/.
> > BTW, is the machine on the receiving end actually checking e-mail often > enough that every millisecond is significant? I would doubt it, and if > not, then this whole project is a waste of time. >Actually, Marnen, the smart thing to do here is not to rely on a client but to set up a forwarding rule on the mail server itself so that the email ends up someplace which is actively waiting for it. Implementation to be determined ;) -- Posted via http://www.ruby-forum.com/.
Hi, Sorry I am realy new in rails (and not good in active English, too *g*). I need to administrate users with two roles. I have seen Aegis and it seems to be very good for my needs - but as long as I haven''t found the clue, I can give there every user only one role. Do you know another plugin, where I can static (and centralized) define two types of roles (I don''t want to lose performance in having an additional table in the database with many accesses) or are the only two possibilities to write it myselft or to customize Aegis and will need to make this anytime they will release an update of Aegis? Thank you very much, Bente
Aldric Giacomoni wrote:>> >> BTW, is the machine on the receiving end actually checking e-mail often >> enough that every millisecond is significant? I would doubt it, and if >> not, then this whole project is a waste of time. >> > > Actually, Marnen, the smart thing to do here is not to rely on a client > but to set up a forwarding rule on the mail server itself so that the > email ends up someplace which is actively waiting for it. > Implementation to be determined ;)Except that Jaap already said he doesn''t control the recipient. Best, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org -- Posted via http://www.ruby-forum.com/.
> Can you ask the folks who control the recipient machine? The less > guessing you have to do, the better.That''s obviously not possible, I''m not guessing when I could easily ask.> I hate to be a pessimist, but I cannot see how it will be anything but. > You''re trying to implement a real-time low-latency messaging queue over > e-mail, and you''re trying to do it on a low budget. Moreover, your > expertise seems to be in a different area, so you''ll be spending lots of > that low budget on research. I suppose it''s possible to pull these > chestnuts out of the fire, but suffice it to say that I''m really > worried.You shouldn''t worry about my project as much as you do. The app is not as complicated as it may seem and my only worry at this point is to get the e-mail out as fast as possible. But that can also be done at a later time, because the automation of this process will increase business success dramatically. However, it would be nice to do it right from the start.> BTW, is the machine on the receiving end actually checking e-mail often > enough that every millisecond is significant? I would doubt it, and if > not, then this whole project is a waste of time.Again, I can''t be sure about this, but I think e-mails received at this address are piped through to an application. I know, it''s lame and it sure as hell isn''t efficient, but I can''t change this. At least I know our position in the queue is more important than the actual handling time on the remote server.
2009/9/4 jhaagmans <jaap.haagmans-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:> >> Can you ask the folks who control the recipient machine? The less >> guessing you have to do, the better. > > That''s obviously not possible, I''m not guessing when I could easily > ask. > >> I hate to be a pessimist, but I cannot see how it will be anything but. >> You''re trying to implement a real-time low-latency messaging queue over >> e-mail, and you''re trying to do it on a low budget. Moreover, your >> expertise seems to be in a different area, so you''ll be spending lots of >> that low budget on research. I suppose it''s possible to pull these >> chestnuts out of the fire, but suffice it to say that I''m really >> worried. > > You shouldn''t worry about my project as much as you do. The app is not > as complicated as it may seem and my only worry at this point is to > get the e-mail out as fast as possible. But that can also be done at a > later time, because the automation of this process will increase > business success dramatically. However, it would be nice to do it > right from the start. > >> BTW, is the machine on the receiving end actually checking e-mail often >> enough that every millisecond is significant? I would doubt it, and if >> not, then this whole project is a waste of time. > > Again, I can''t be sure about this, but I think e-mails received at > this address are piped through to an application. I know, it''s lame > and it sure as hell isn''t efficient, but I can''t change this. At least > I know our position in the queue is more important than the actual > handling time on the remote server.Ah, right, it is where your email appears in the receivers queue relative to everybody else''s emails that matters. That is why the processing time at the receiver is not important. It is a matter of getting your bid in before others that matters. Colin
> Ah, right, it is where your email appears in the receivers queue > relative to everybody else''s emails that matters. That is why the > processing time at the receiver is not important. It is a matter of > getting your bid in before others that matters.Something like that, yes :) Even though there are no real others. Cryptic, ey? ;) Anyway, thanks for all the help, I''m writing my recommendations as we speak and probably start work next week. I really appreciate you all taking the time to help me :)
Keep us in the loop. Whatever happens, I smell a useful article here on whatever tips/tricks/pitfalls you discover. You are dealing with various third parties and various protocols in a time sensitive application, so whatever you find will be interesting, even if you have to mock up the actual material (since you can''t seem to talk about specifics this is understandable) I think it would still be interesting to read. jhaagmans wrote:> > Ah, right, it is where your email appears in the receivers queue > > relative to everybody else''s emails that matters. That is why the > > processing time at the receiver is not important. It is a matter of > > getting your bid in before others that matters. > > Something like that, yes :) Even though there are no real others. > Cryptic, ey? ;) > > Anyway, thanks for all the help, I''m writing my recommendations as we > speak and probably start work next week. I really appreciate you all > taking the time to help me :)
Carl wrote:> Keep us in the loop. Whatever happens, I smell a useful article here > on whatever tips/tricks/pitfalls you discover. You are dealing with > various third parties and various protocols in a time sensitive > application, so whatever you find will be interesting, even if you > have to mock up the actual material (since you can''t seem to talk > about specifics this is understandable) I think it would still be > interesting to read.I don''t know, thinking about this, it sounds like: a) build a bunch of observers b) gather data c) put data in email d) fire email 5) get coffee (hey, it''s Friday, I''m allowed to change bulleting style). So who do you think is better at watching? I think the gem that was suggested earlier on may really be all he needs.. And a way to manage it all. I wonder, if you really want to shave off milliseconds, string manipulation is probably too high level for you, but I am guessing by now that you are dealing with human-readable data. -- Posted via http://www.ruby-forum.com/.