So questions on start_server
observation:
if you run start_server(''192.168.whatever''..)
then calls to connect to 127.0.0.1 don''t connect to it. Kind of a
gotcha. Kind of not.
2) Is there a way to say ''bind me to all address, on such and such a
port''?
3) Is there a way to determine the current host''s default IP address,
or should I just use the ''old'' Socket.gethostname way? Or
look up all
of its IP addresses?
4) Also for curiosity: am I correct in operating under the assumption
that EM calls will not block while waiting for DNS resolution to occur
for certain connections?
Thanks for your help.
-Roger
On Nov 16, 2007 12:17 AM, Roger Pack <rogerpack2005 at gmail.com> wrote:> So questions on start_server > observation: > if you run start_server(''192.168.whatever''..) > then calls to connect to 127.0.0.1 don''t connect to it. Kind of a > gotcha. Kind of not.That''s a feature, not a bug :-)> > 2) Is there a way to say ''bind me to all address, on such and such a port''? >EM.start_server( "0.0.0.0", port )> 3) Is there a way to determine the current host''s default IP address, > or should I just use the ''old'' Socket.gethostname way? Or look up all > of its IP addresses?There''s a lot of non-EM-specific literature that you google for. Short answer, it depends on your platform, and it can be a real pain in the neck.> > 4) Also for curiosity: am I correct in operating under the assumption > that EM calls will not block while waiting for DNS resolution to occur > for certain connections? >That''s a very good question. If you do an EM#connect call to a hostname rather than an IP address, then EM will do the host resolution using the underlying system libraries, and the resolution will block. However, it''s not nearly as bad as DNS using Ruby''s libraries, which spin threads and do other kinky things. Having said that, DNS is enough of a problem that I recently wrote an EM-based DNS resolver. It handles A records and MX records, caches results, and allows you to statically define associations. This resolver does NOT block. If anyone is interested, I can probably release it into the EM distro. Additionally there is a gentleman in England whose name I unfortunately don''t remember, who''s been working on a full replacement for Ruby''s DNS.
Thanks for the prompt informative replies.> > > > 4) Also for curiosity: am I correct in operating under the assumption > > that EM calls will not block while waiting for DNS resolution to occur > > for certain connections? > > > > That''s a very good question. If you do an EM#connect call to a > hostname rather than an IP address, then EM will do the host > resolution using the underlying system libraries, and the resolution > will block. However, it''s not nearly as bad as DNS using Ruby''s > libraries, which spin threads and do other kinky things.Ahh I assume you mean that resolv-replace spins threads and stuff--I agree it seems quirky. I think I ran into a case of calling connection.peername blocks, even if you use resolv-replace. Weird.> Having said that, DNS is enough of a problem that I recently wrote an > EM-based DNS resolver. It handles A records and MX records, caches > results, and allows you to statically define associations. This > resolver does NOT block. If anyone is interested, I can probably > release it into the EM distro. > > Additionally there is a gentleman in England whose name I > unfortunately don''t remember, who''s been working on a full replacement > for Ruby''s DNS.I heard of a ''ruby fast dns resolver'' at http://rubyforge.org/frs/?group_id=2387 I''m not sure if it''s the same one. A nice future feature would be ''EM::set_non_blocking_dns'' (or just make it the default) or something :) Take care! -Roger
Hi -> > Additionally there is a gentleman in England whose name I > > unfortunately don''t remember, who''s been working on a full replacement > > for Ruby''s DNS. > > I heard of a ''ruby fast dns resolver'' at > http://rubyforge.org/frs/?group_id=2387As it happens, I just released version 1.0 of Dnsruby today. As well as the pure Ruby inbuilt event loop, it can also use EventMachine to do the I/O , e.g. : EventMachine::run { df = res.send_async(Dnsruby::Message.new("example.com")) df.callback {|msg| puts "Response : #{msg}" EM.stop} df.errback {|msg, err| puts "Response : #{msg}" puts "Error: #{err}" EM.stop} } (You need to set a couple of things up first, to tell Dnsruby to use EventMachine - check out the EVENTMACHINE file in the root of the distribution) Alex.
On Fri, 2007-11-16 at 08:20 -0700, Roger Pack wrote:> I heard of a ''ruby fast dns resolver'' at > http://rubyforge.org/frs/?group_id=2387 > I''m not sure if it''s the same one.There''s also my wrapper for c-ares, which is what curl uses for asynchronous name resolves. I did it for a project that never took off, but the lib itself works fine. http://rubyforge.org/projects/cares/ Andre
On Nov 16, 2007 10:35 AM, <alexd at nominet.org.uk> wrote:> Hi - > > > > Additionally there is a gentleman in England whose name I > > > unfortunately don''t remember, who''s been working on a full replacement > > > for Ruby''s DNS. > > > > I heard of a ''ruby fast dns resolver'' at > > http://rubyforge.org/frs/?group_id=2387 > > As it happens, I just released version 1.0 of Dnsruby today. As well as > the pure Ruby inbuilt event loop, it can also use EventMachine to do the > I/O , e.g. : > > EventMachine::run { > df = res.send_async(Dnsruby::Message.new("example.com")) > df.callback {|msg| > puts "Response : #{msg}" > EM.stop} > df.errback {|msg, err| > puts "Response : #{msg}" > puts "Error: #{err}" > EM.stop} > } > > (You need to set a couple of things up first, to tell Dnsruby to use > EventMachine - check out the EVENTMACHINE file in the root of the > distribution) > > > Alex.Thanks, Alex, and I apologize for forgetting your name. :-(
Would you say one is faster than the other? On Nov 16, 2007 8:35 AM, <alexd at nominet.org.uk> wrote:> Hi - > > > > Additionally there is a gentleman in England whose name I > > > unfortunately don''t remember, who''s been working on a full replacement > > > for Ruby''s DNS. > > > > I heard of a ''ruby fast dns resolver'' at > > http://rubyforge.org/frs/?group_id=2387 > > As it happens, I just released version 1.0 of Dnsruby today. As well as > the pure Ruby inbuilt event loop, it can also use EventMachine to do the > I/O , e.g. : > > EventMachine::run { > df = res.send_async(Dnsruby::Message.new("example.com")) > df.callback {|msg| > puts "Response : #{msg}" > EM.stop} > df.errback {|msg, err| > puts "Response : #{msg}" > puts "Error: #{err}" > EM.stop} > } > > (You need to set a couple of things up first, to tell Dnsruby to use > EventMachine - check out the EVENTMACHINE file in the root of the > distribution) > > > Alex. > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-- -Roger Pack For God hath not given us the spirit of fear; but of power, and of love, and of a sound mind" -- 2 Timothy 1:7
On Nov 16, 2007 10:59 AM, Roger Pack <rogerpack2005 at gmail.com> wrote:> Would you say one is faster than the other? > >I haven''t done any comparing. Perhaps Alex has. Informally testing my evented library, I got about 250 resolutions per second against a typically-slow DNS server in the internet run by a major telecom provider. But my stuff is just intended for simple host and email lookups. Alex is doing something with a lot more features.
> > As it happens, I just released version 1.0 of Dnsruby today. As wellas> > the pure Ruby inbuilt event loop, it can also use EventMachine to dothe> > I/O , e.g. :> Would you say one is faster than the other?I wrote the Ruby event loop so that Dnsruby could work anywhere that Ruby would work, with no dependencies. If EventMachine is available on a given platform that I was coding for, I would definitely use EventMachine to do the I/O. Ruby''s networking is a PITA... Alex. P.S. I hope this is what you were asking?
On Nov 16, 2007 11:13 AM, <alexd at nominet.org.uk> wrote:> > > As it happens, I just released version 1.0 of Dnsruby today. As well > as > > > the pure Ruby inbuilt event loop, it can also use EventMachine to do > the > > > I/O , e.g. : > > > Would you say one is faster than the other? > > I wrote the Ruby event loop so that Dnsruby could work anywhere that Ruby > would work, with no dependencies. > > If EventMachine is available on a given platform that I was coding for, I > would definitely use EventMachine to do the I/O. Ruby''s networking is a > PITA... >Alex, I think people would find it helpful if you wrote a HOWTO document with a working example, on the simplest way to integrate Dnsruby with EM. Like a cookbook recipe :-). The EM Wiki on rubyforge would be a good place to put it, as well as posting here. Thanks.
> Alex, I think people would find it helpful if you wrote a HOWTO > document with a working example, on the simplest way to integrate > Dnsruby with EM. Like a cookbook recipe :-). The EM Wiki on rubyforge > would be a good place to put it, as well as posting here. Thanks.Sure - I''m also thinking a wrapper file (which could live with EventMachine) could be a good idea. Then you could make DNS calls without even knowing about Dnsruby. Any interest? Alex. P.S. It''s the start of a busy weekend here - I''ll post an example Monday morning
alexd at nominet.org.uk
2007-Nov-20 03:56 UTC
[Eventmachine-talk] EventMachine and Dnsruby
eventmachine-talk-bounces at rubyforge.org wrote on 16/11/2007 16:21:17:> Alex, I think people would find it helpful if you wrote a HOWTO > document with a working example, on the simplest way to integrate > Dnsruby with EM. Like a cookbook recipe :-). The EM Wiki on rubyforge > would be a good place to put it, as well as posting here. Thanks.Hi - I''ve knocked up some text describing how to use Dnsruby with EventMachine. I''m including it here in the hope that I can fix any criticisms before posting it to the EM Wiki. Please let me know if I can improve it! Alex. ----- EventMachine and Dnsruby ======================= Dnsruby provides a full implementation of a DNS stub resolver (with the exception of DNSSEC support, which will be added soon). It is designed for the use of clients which require richer DNS functionality than the basic implementation provided in Ruby''s resolv.rb. The RDoc for Dnsruby can be found here : http://dnsruby.rubyforge.org/ Although Dnsruby can be run with an inbuilt pure Ruby event loop, it is also possible to use EventMachine to perform the I/O. This document provides a summary of how to use Dnsruby asynchronously from an EventMachine context. Configuring Dnsruby to use EventMachine --------------------------------------- First of all, we need to load Dnsruby and configure it to use EventMachine rather than the native event loop : require ''Dnsruby'' Dnsruby::Resolver.use_eventmachine With older versions of EventMachine (before 0.9.0), it''s not possible to tell whether the EM reactor is running or not. If a call is made to start EventMachine whilst EventMachine is already running, then Ruby will crash. When running with versions of EventMachine which do not implement EventMachine::reactor_running?, Dnsruby will try to start EventMachine unless it is instructed not to do so : Dnsruby::Resolver.start_eventmachine_loop(false) I''m assuming that your client code will at some point call EventMachine::run to start EventMachine. If not, then you don''t need the above line. We''re now ready to run DNS queries in EventMachine. Instantiating a Dnsruby::Resolver object ---------------------------------------- The Dnsruby::Resolver class exposes an asynchronous send call. It is possible to instantiate a Resolver object with a hash of specific characteristics (list of nameservers to query, retry behaviour, TSIG key, etc.). If no nameservers are explicitly provided then Dnsruby will use the system defaults (for example /etc/resolv.conf). res = DnsRuby::Resolver.new({:nameserver=>["ns1.example.com", "ns2.example.com"], :retry_delay=>5}) # etc. or res = Dnsruby::Resolver.new # use system defaults Note that the Resolver class will not append the default domain or search list. [The Dnsruby Dnsruby::DNS classes will use the system default domain and search list. I could add an asynchronous send call to Dnsruby and Dnsruby::DNS if there was demand for it - please let me know] The Dnsruby::Resolver class performs UDP retries across the list of configured nameservers. See the RDoc for more details on configuring the retry policy. Creating query messages with Dnsruby ------------------------------------ First, we need to tell Dnsruby what we want to query for. To do this asynchronously, we need to construct a query message. msg = Dnsruby::Message.new("example.com") # defaults to A msg = Dnsruby::Message.new("example.com", ''MX'') msg = Dnsruby::Message.new("example.com", ''ANY'') It is also possible to use Dnsruby with EventMachine in a synchronous fashion - I haven''t documented this here, as I did not feel it would be of interest to EventMachine users. However, I have tended to assume that asynchronous users want more control of their queries, so some of the syntactic sugar that is present in the synchronous interface is not (yet) in the asynchronous one. In particular, it is necessary to construct query messages to be sent, rather than simply querying for a resource. I''d be happy to add any sugar that anyone felt would be useful. Making asynchronous DNS calls with Dnsruby ------------------------------------------ The asynchronous send call is as follows : deferrable = res.send_async(msg, use_tcp=false) The call (immediately) returns an EventMachine::Deferrable object, which can be programmed with success/failure behaviour as follows : df.callback {|msg| puts "Response : #{msg}" } df.errback {|msg, err| # puts "Returned message : #{msg}" puts "Error: #{err}" } The callbacks are only called once the response is known - this could be some time later (depending on the local retry config). If a DNS message is returned from the query then the deferrable''s callback method is called with the response message as the sole parameter. If an error is returned, then the errback method is called with both the error, and any returned DNS message (e.g. SERVFAIL). Of course, if an error such as a timeout is returned, then nil will be returned for msg. Errors include : ResolvError < StandardError ResolvTimeout < TimeoutError NXDomain < ResolvError FormErr < ResolvError ServFail < ResolvError NotImp < ResolvError Refused < ResolvError OtherResolvError < ResolvError [There is another form of asynchronous send that uses a Ruby Queue for the response, but the EventMachine::Deferrable is, IMHO, much nicer] Inspecting the response message ------------------------------- If a message is returned from a query, it will be a Dnsruby::Message. A Dnsruby::Message consists of five sections : * The header section, a Dnsruby::Header object. header = msg.header * The question section, an array of Dnsruby::Question objects. msg.each_question do |question| .... end * The answer section, an array of Dnsruby::RR objects. answer_rrs = msg.answer msg.each_answer {|answer| ... } * The authority section, an array of Dnsruby::RR objects. authority_rrs = msg.authority msg.each_authority {|rr| ... } * The additional section, an array of Dnsruby::RR objects. additional_rrs = msg.additional msg.each_additional {|rr| ... } In addition, each_resource iterates the answer, additional and authority sections : msg.each_resource {|rr| ..} A complete EventMachine program to query DNS -------------------------------------------- Putting this all together, we can write a very basic program to query the DNS, for MX records for example.com, in EventMachine : require ''Dnsruby'' require ''eventmachine'' res = Dnsruby::Resolver.new Dnsruby::Resolver.use_eventmachine Dnsruby::Resolver.start_eventmachine_loop(false) EventMachine::run { df = res.send_async(Dnsruby::Message.new("example.com"), ''MX'') df.callback {|msg| msg.each_answer do |ans_rr| # only print the answer section puts "Answer RR : #{ans_rr}" end EM.stop} df.errback {|msg, err| puts "Error: #{err}" EM.stop} }
Looks nice. One ''code snippet'' that would be useful for me would be "here''s how to use it with normal EM connections so that they don''t block during DNS resolution"
alexd at nominet.org.uk
2007-Nov-20 08:02 UTC
[Eventmachine-talk] EventMachine and Dnsruby
eventmachine-talk-bounces at rubyforge.org wrote on 20/11/2007 15:49:30:> Looks nice. > One ''code snippet'' that would be useful for me would be > > "here''s how to use it with normal EM connections so that they don''t > block during DNS resolution"Could you please give an example snippet of code which shows the problem? Thanks, Alex.
EM.connect(''dns_that_takes_a_long_time_to_resolve.com'', 8289,
MyClass) {}
Sorry I can''t think of a real domain off the top of my head, but you
get the idea.
Also I assume dnsruby can be made to ''be its own dns server''
(by-pass
intermediate dns''es and go to the root servers)--is such the case?
Thank you!
--
-Roger Pack
For God hath not given us the spirit of fear; but of power, and of
love, and of a sound mind" -- 2 Timothy 1:7
alexd at nominet.org.uk
2007-Nov-20 08:17 UTC
[Eventmachine-talk] EventMachine and Dnsruby
eventmachine-talk-bounces at rubyforge.org wrote on 20/11/2007 16:11:56:> EM.connect(''dns_that_takes_a_long_time_to_resolve.com'', 8289, MyClass){}> > Sorry I can''t think of a real domain off the top of my head, but you > get the idea.Thanks - I''ll have a look at this.> Also I assume dnsruby can be made to ''be its own dns server'' (by-pass > intermediate dns''es and go to the root servers)--is such the case?Do you mean acting as a recursive resolver rather than as a stub resolver? I''m presuming you don''t mean acting as a nameserver for other DNS clients. Dnsruby doesn''t currently have a recursive implementation - it expects to speak to a recursive resolver to get its replies. There''s no reason why it can''t be a recursive resolver - I''m thinking of writing an implementation once I get DNSSEC fully implemented. [I have considered writing a nameserver implementation using Dnsruby - but I''m not convinced that it would be useful as anything other than demonstration code] Alex.
Thanks for the reply. On Nov 20, 2007 9:17 AM, <alexd at nominet.org.uk> wrote:> eventmachine-talk-bounces at rubyforge.org wrote on 20/11/2007 16:11:56: > > > EM.connect(''dns_that_takes_a_long_time_to_resolve.com'', 8289, MyClass) > {} > > > > Sorry I can''t think of a real domain off the top of my head, but you > > get the idea. > > Thanks - I''ll have a look at this. > > > Also I assume dnsruby can be made to ''be its own dns server'' (by-pass > > intermediate dns''es and go to the root servers)--is such the case? > > Do you mean acting as a recursive resolver rather than as a stub resolver? > I''m presuming you don''t mean acting as a nameserver for other DNS clients.Just wondering :)> Dnsruby doesn''t currently have a recursive implementation - it expects to > speak to a recursive resolver to get its replies. There''s no reason why it > can''t be a recursive resolver - I''m thinking of writing an implementation > once I get DNSSEC fully implemented. > > [I have considered writing a nameserver implementation using Dnsruby - but > I''m not convinced that it would be useful as anything other than > demonstration code]Presumably because of Ruby''s execution speed, even with EM? Take care. -Roger
alexd at nominet.org.uk
2007-Nov-20 08:29 UTC
[Eventmachine-talk] EventMachine and Dnsruby
eventmachine-talk-bounces at rubyforge.org wrote on 20/11/2007 15:49:30:> Looks nice. > One ''code snippet'' that would be useful for me would be > > "here''s how to use it with normal EM connections so that they don''t > block during DNS resolution"> EM.connect(''dns_that_takes_a_long_time_to_resolve.com'', 8289, MyClass){} Not sure if this is quite what you''re after, but one thing that springs to mind is : name = "dns_that_takes_ages.com" df = res.send_async(Message.new(name)) df.callback {|msg| EM.connect(msg.answer[0], 8289, MyClass) {} } df.errback {|msg, err| print "Sorry - can''t resolve #{name}. Error = #{err}\n" } I''m sure this could be wrapped up to be a little less unpalatable... Alex.
Thank you!> Not sure if this is quite what you''re after, but one thing that springs to > mind is : > > name = "dns_that_takes_ages.com" > df = res.send_async(Message.new(name)) > df.callback {|msg| > EM.connect(msg.answer[0], 8289, MyClass) {} > } > df.errback {|msg, err| > print "Sorry - can''t resolve #{name}. Error = #{err}\n" > } > > I''m sure this could be wrapped up to be a little less unpalatable... > > > Alex.