Forwarding this onto the list for Peter who''s having mail problems ... From: Peter Bacon Darwin [mailto:peterbacon at bacondarwin.com] Sent: Thursday, November 08, 2007 2:35 AM To: John Lam (DLR) Subject: FW: Range#=== operator I have noticed an interesting feature in the current snapshot. If you test a numeric range against a non-numeric value, such as a string you get a System.ArgumentException whereas other Ruby implentations return false. E.g. IronRuby: (1..5) === ''x'' gives System.ArgumentException: wrong number or type of arguments for `<=>'' CRuby: (1..5) === ''x'' gives false There are two interesting points here: The first is that debugging code with lightweight code generation is really difficult; tracking down what actually causes the exception to be thrown is hidden by the DynamicSite layer. If you break on the exception being thrown you have already lost the trace back to the original problem. This is because the exception is not thrown directly but via binding to an invocation site. Only by trawling through the code and putting break points all over the place was I able to ascertain that the problem is that when you apply the === operator to the Range the begin and end attributes of Range have the <=> operator applied to the argument (in this case you have something like FixNum <=> MutableString) and the current system is unable to find a suitable method to invoke. See RubyMethodGroupInfo.SetInvocationRuleInternal() and RubyMethodGroupInfo.GetMethodCandidate(). I.E. There is no implicit conversion between FixNum and MutableString. How do you guys debug situations like this? The second point is to do with this result itself. Are you aiming to produce identical results to other Ruby implementations or are there other criteria for correctness. Arguably the expression given above is invalid as there is no correct type conversion. Equally false would seem a reasonable result; certainly it is false that ''x'' is in the range of (1..5). If false is what you want, what is the accepted way of dealing with the situation? Do you want to fiddle with the protocols for type conversions? I imagine not. Is Range a specific case, in which case you would code a specific version of <=> operator for it? Interestingly the following code returns false as expected: 1 == ''x'' gives false The difference here is that the == operator is being used directly to test equality whereas early the <=> operator was being used to compare the values previously. Of interest, both IronRuby and CRuby throw argument exceptions/errors for the following comparisons. 1 > ''x'' and 1 < ''x'' What are the correct semantics here? What do you think should be done to deal with this discrepancy? Regards, Pet -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ironruby-core/attachments/20071108/8de30a5c/attachment.html
Here''s a couple of debugging tips: -X:SaveAssemblies -X:StaticMethods command line options, use these when you need to see into a DynamicMethod. Instead of DynamicMethod''s we''ll create methods via Reflection.Emit and we''ll save them to disk at process exit as snippets???.dll (where ??? is a number, usually 1). You can then open the assembly w/ reflector or ildasm to see what''s going on in the generated code. This is especially useful if you have a small repro snippet like the one here as snippets will be pretty easy to dig through and find the particular piece of generated code you want. When debugging rules in general the best place to put a break point is either the language''s RubyActionBinder.MakeRule - where you''ll see all language specific binding kick in (and boy does Ruby have a lot of language specific bindings currently :)!) or you can look at ActionBinder.GetRule to see the whole process take place. Unfortunately I don''t know the ruby code base well enough to give you better hints than that (but if you want know how to debug IronPython comparisons...) A final useful one is -X:ExceptionDetail - this will cause the CLR exception to get printed instead of a more filtered language exception. Also if the exception comes from a DynamicSite I''ll usually try and find the exceptions message somewhere in the source code which usually is in some rule generation code. Hopefully that''s helpful on the debugging side of things - and maybe some of this you''ve figured out but it might be useful for other people as well. From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of John Lam (DLR) Sent: Thursday, November 08, 2007 7:27 AM To: ironruby-core at rubyforge.org Subject: [Ironruby-core] Range#=== operator Forwarding this onto the list for Peter who''s having mail problems ... From: Peter Bacon Darwin [mailto:peterbacon at bacondarwin.com] Sent: Thursday, November 08, 2007 2:35 AM To: John Lam (DLR) Subject: FW: Range#=== operator I have noticed an interesting feature in the current snapshot. If you test a numeric range against a non-numeric value, such as a string you get a System.ArgumentException whereas other Ruby implentations return false. E.g. IronRuby: (1..5) === ''x'' gives System.ArgumentException: wrong number or type of arguments for `<=>'' CRuby: (1..5) === ''x'' gives false There are two interesting points here: The first is that debugging code with lightweight code generation is really difficult; tracking down what actually causes the exception to be thrown is hidden by the DynamicSite layer. If you break on the exception being thrown you have already lost the trace back to the original problem. This is because the exception is not thrown directly but via binding to an invocation site. Only by trawling through the code and putting break points all over the place was I able to ascertain that the problem is that when you apply the === operator to the Range the begin and end attributes of Range have the <=> operator applied to the argument (in this case you have something like FixNum <=> MutableString) and the current system is unable to find a suitable method to invoke. See RubyMethodGroupInfo.SetInvocationRuleInternal() and RubyMethodGroupInfo.GetMethodCandidate(). I.E. There is no implicit conversion between FixNum and MutableString. How do you guys debug situations like this? The second point is to do with this result itself. Are you aiming to produce identical results to other Ruby implementations or are there other criteria for correctness. Arguably the expression given above is invalid as there is no correct type conversion. Equally false would seem a reasonable result; certainly it is false that ''x'' is in the range of (1..5). If false is what you want, what is the accepted way of dealing with the situation? Do you want to fiddle with the protocols for type conversions? I imagine not. Is Range a specific case, in which case you would code a specific version of <=> operator for it? Interestingly the following code returns false as expected: 1 == ''x'' gives false The difference here is that the == operator is being used directly to test equality whereas early the <=> operator was being used to compare the values previously. Of interest, both IronRuby and CRuby throw argument exceptions/errors for the following comparisons. 1 > ''x'' and 1 < ''x'' What are the correct semantics here? What do you think should be done to deal with this discrepancy? Regards, Pet -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ironruby-core/attachments/20071108/725bf380/attachment-0001.html
Thanks Dino, I did use the SaveAssemblies and StaticMethods options ? cribbed from one of John?s blog postings, I think. From the look of it (the exception is pretty comprehensive) there is no filtered language exception going on in IronRuby at this stage. Or is this just when in debug? I haven?t tried release. The trouble with the generated code was that it appeared that the exception had already been flagged for execution at generation time; One of the snippets was an action to create and throw the ArgumentException [See RubyOps.MakeInvalidArgumentTypesError]. The way I debugged it, as you mention toward the bottom of your mail, was to search the code that creates the specific ArgumentException dynamic action (searching for the error string) and then find which bits of code use it. I also experimented with placing break points at various places around the DLR in the AST generation and action binding code. I guess it just requires a bit of getting used to and knowing where to look. Trawling through the RubyOnRails code generation (e.g. ActiveRecord) is also really hard work too. One other question though: How do you decide whether to use dynamic sites to trigger code rather than calling code directly when in coding in C#. For instance, why does Protocols.Compare invoke _CompareSharedSite rather than call a C# function directly? Sorry if I am being dumb. By the way, a simple way to get the desired results for the case equality issue mentioned before is to put a try/catch around the Compare and return false if an ArgumentException is thrown. See the attached patch. Is this too much of a hack or acceptable? Pete -----Original Message----- From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of Dino Viehland Sent: 08 November 2007 16:51 To: ironruby-core at rubyforge.org Subject: Re: [Ironruby-core] Range#=== operator Here?s a couple of debugging tips: -X:SaveAssemblies ?X:StaticMethods command line options, use these when you need to see into a DynamicMethod. Instead of DynamicMethod?s we?ll create methods via Reflection.Emit and we?ll save them to disk at process exit as snippets???.dll (where ??? is a number, usually 1). You can then open the assembly w/ reflector or ildasm to see what?s going on in the generated code. This is especially useful if you have a small repro snippet like the one here as snippets will be pretty easy to dig through and find the particular piece of generated code you want. When debugging rules in general the best place to put a break point is either the language?s RubyActionBinder.MakeRule ? where you?ll see all language specific binding kick in (and boy does Ruby have a lot of language specific bindings currently :-)!) or you can look at ActionBinder.GetRule to see the whole process take place. Unfortunately I don?t know the ruby code base well enough to give you better hints than that (but if you want know how to debug IronPython comparisons?) A final useful one is ?X:ExceptionDetail ? this will cause the CLR exception to get printed instead of a more filtered language exception. Also if the exception comes from a DynamicSite I?ll usually try and find the exceptions message somewhere in the source code which usually is in some rule generation code. Hopefully that?s helpful on the debugging side of things ? and maybe some of this you?ve figured out but it might be useful for other people as well. No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.5.503 / Virus Database: 269.15.24/1117 - Release Date: 07/11/2007 22:52 No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.503 / Virus Database: 269.15.24/1117 - Release Date: 07/11/2007 22:52 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ironruby-core/attachments/20071108/8077bc4f/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: RangeOpsCaseEquality.patch Type: application/octet-stream Size: 1251 bytes Desc: not available Url : http://rubyforge.org/pipermail/ironruby-core/attachments/20071108/8077bc4f/attachment.obj
On 11/8/07, Peter Bacon Darwin <bacondarwin at googlemail.com> wrote:> > One other question though: How do you decide whether to use dynamic > sites to trigger code rather than calling code directly when in coding in > C#. For instance, why does Protocols.Compare invoke _CompareSharedSite > rather than call a C# function directly? Sorry if I am being dumb. >Even builtins like the String or Range classes can have their comparison functions overridden by user code. So you really need to perform the comparison "in Ruby". -- Curt Hagenlocher curt at hagenlocher.org -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ironruby-core/attachments/20071108/e75b8fed/attachment.html
Of course, you are quite right. Thanks for that. -----Original Message----- From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of Curt Hagenlocher Sent: 08 November 2007 21:40 To: ironruby-core at rubyforge.org Subject: Re: [Ironruby-core] Range#=== operator On 11/8/07, Peter Bacon Darwin <HYPERLINK "mailto:bacondarwin at googlemail.com"bacondarwin at googlemail.com> wrote: One other question though: How do you decide whether to use dynamic sites to trigger code rather than calling code directly when in coding in C#. For instance, why does Protocols.Compare invoke _CompareSharedSite rather than call a C# function directly? Sorry if I am being dumb. Even builtins like the String or Range classes can have their comparison functions overridden by user code. So you really need to perform the comparison "in Ruby". -- Curt Hagenlocher HYPERLINK "mailto:curt at hagenlocher.org"curt at hagenlocher.org No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.5.503 / Virus Database: 269.15.24/1117 - Release Date: 07/11/2007 22:52 No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.503 / Virus Database: 269.15.24/1117 - Release Date: 07/11/2007 22:52 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ironruby-core/attachments/20071108/5f63aed4/attachment-0001.html
Curt Hagenlocher wrote:> On 11/8/07, Peter Bacon Darwin <bacondarwin at googlemail.com> wrote: > >> One other question though: How do you decide whether to use dynamic >> sites to trigger code rather than calling code directly when in coding >> in C#. For instance, why does Protocols.Compare invoke >> _CompareSharedSite rather than call a C# function directly? Sorry if >> I am being dumb. > > Even builtins like the String or Range classes can have their > comparison functions overridden by user code. So you really need to > perform the comparison "in Ruby".Exactly. The trick with library methods is to figure out what pattern MRI uses. Typically, they do a method lookup on the type, which in our world translates into a DynamicSite. However there are some places where they just call some library method directly and bypass any overridden/monkeypatched method. So it requires some experimentation :). Protocols.Compare is one of the stranger things I''ve discovered. MRI will call <=>, and then call < and > on the result object... which of course can be overridden to do whatever you want. Because Compare is kind of crazy I''m not surprised we get it slightly wrong. Peter Bacon Darwin wrote:> By the way, a simple way to get the desired results for the case > equality issue mentioned before is to put a try/catch around the > Compare and return false if an ArgumentException is thrown. See the > attached patch. Is this too much of a hack or acceptable?Peter, I think the right fix here is for Fixnum <=> to return nil if it gets passed a non-comparable object. That seems to be the standard MRI behavior for <=>: irb(main):013:0> (1..5) === ''x'' => false And then I think our Protocols.Compare needs to check if <=> returned null, and not call > or < if that was the case. Indeed, if you define ''<'' and ''>'' on NilClass it doesn''t change the compare behavior: irb(main):014:0> class NilClass irb(main):015:1> def < x; puts ''<''; 0; end irb(main):016:1> def > x; puts ''>''; 0; end irb(main):017:1> end => nil irb(main):018:0> (1..5) === ''x'' => false Of course, it might require more experiments to get it exactly right for all cases. - John
John Messerly wrote:> Peter, I think the right fix here is for Fixnum <=> to return nil if it > gets passed a non-comparable object. That seems to be the standard MRI > behavior for <=>: > > irb(main):013:0> (1..5) === ''x'' > => false >Oops, my example is wrong, it should be: irb(main):001:0> 1 <=> ''x'' => nil IronRuby:>>> 1 <=> ''x''System.ArgumentException: wrong number or type of arguments for `<=>'' So we definitely need another overload for Fixnum#<=>, and possibly other places as well that implement <=>. - John
Would it be fair to follow this as a general rule: when coding Ruby methods in C#, if you are calling code that is exposed as a method or operator in a Ruby class or module then invoke it via a DynamicSite, if it is purely private code to the C# method then call it directly? Pete -----Original Message----- From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of John Messerly Sent: 08 November 2007 21:55 To: ironruby-core at rubyforge.org Subject: Re: [Ironruby-core] Range#=== operator Curt Hagenlocher wrote:> > Even builtins like the String or Range classes can have their > comparison functions overridden by user code. So you really need to > perform the comparison "in Ruby".Exactly. The trick with library methods is to figure out what pattern MRI uses. Typically, they do a method lookup on the type, which in our world translates into a DynamicSite. However there are some places where they just call some library method directly and bypass any overridden/monkeypatched method. So it requires some experimentation :). No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.503 / Virus Database: 269.15.29/1124 - Release Date: 11/11/2007 10:12
Peter Bacon Darwin wrote:> Would it be fair to follow this as a general rule: when coding Ruby methods > in C#, if you are calling code that is exposed as a method or operator in a > Ruby class or module then invoke it via a DynamicSite, if it is purely > private code to the C# method then call it directly? > PeteThere is no hard rule for Ruby; sometimes you are supposed to dispatch for functionality, sometimes you call directly. There''s no one true way unfortunately. Of course, as you figure out how different methods work, document that :) - Charlie
Peter Bacon Darwin:> Would it be fair to follow this as a general rule: when coding Ruby > methods > in C#, if you are calling code that is exposed as a method or operator > in a > Ruby class or module then invoke it via a DynamicSite, if it is purely > private code to the C# method then call it directly?Yes. But since we alias types it''s sometimes not blindingly obvious that you''ll need a site. The good news is that the specs do a very good job at catching cases where there''s some observable behavior. -John
Peter Bacon Darwin wrote:> Would it be fair to follow this as a general rule: when coding Ruby methods > in C#, if you are calling code that is exposed as a method or operator in a > Ruby class or module then invoke it via a DynamicSite, if it is purely > private code to the C# method then call it directly? > PeteThis rule won''t hold for all cases, since frequently Ruby''s of two minds on whether you can override some method X used by method Y such that Y will see it. Sometimes Y will do a full dynamic dispatch and sometimes it will do a direct call, even to the same piece of code that''s bound to external (i.e. in-script) calls to X. - Charlie
Charles Oliver Nutter wrote:> Peter Bacon Darwin wrote: >> Would it be fair to follow this as a general rule: when coding Ruby methods >> in C#, if you are calling code that is exposed as a method or operator in a >> Ruby class or module then invoke it via a DynamicSite, if it is purely >> private code to the C# method then call it directly? >> Pete > > This rule won''t hold for all cases, since frequently Ruby''s of two minds > on whether you can override some method X used by method Y such that Y > will see it. Sometimes Y will do a full dynamic dispatch and sometimes > it will do a direct call, even to the same piece of code that''s bound to > external (i.e. in-script) calls to X.Pardon the duplication; the other mail must have gotten delayed... - Charlie
Hello, Where is help needed? Reading through the list archive, it looks like people are working on the zlib and yaml implementation. Is there help needed there still? Where would my time spent working be most beneficial? Thanks, Mike
As I''ve seen there''s plenty of things to do! By the way, I started to work on StringIO but never asked if someone else is already on that... is there anyone? Eduardo Flores On Nov 15, 2007 8:30 PM, Mike Jarosch <mjarosch at sbcglobal.net> wrote:> Hello, > > Where is help needed? Reading through the list archive, it looks like > people are working on the zlib and yaml implementation. Is there help > needed there still? Where would my time spent working be most beneficial? > > Thanks, > Mike > _______________________________________________ > Ironruby-core mailing list > Ironruby-core at rubyforge.org > http://rubyforge.org/mailman/listinfo/ironruby-core >
I was thinking about this issue last night. I am interested in working on the number stuff (Numeric, BigNum, FixedNum, Float, etc) but I am concerned that a) Someone internally is already doing this and just as I am about to send in a patch a new splat of code gets popped across from the internal source control to SVN and all has been wasted; b) The quality/style/efficiency is not accepted by the maintainers and my code goes to waste. It is a shame that the code is not being developed directly against the RubyForge repository as this definitely gives a feeling of them and us and makes it difficult to see what is being done, what progress is being made and it creates extra work for the maintainers ensuring that the internal and external repositories are synched up - there have been a number of issues recently about dodgy file paths and project references. Some kind of work tracking tool would be beneficial, even if it was just a priority list of things that needed to be done so that people out here could pick up something. I suppose the problem is also that, for instance, no one knows who I am and what quality of code I can produce and so it would be risky to allocate work to me. How could this be solved. My personal aim is to send in a few patches and hopefully, get enough feedback so that I am capable of producing code that is acceptable and fits in with the way the internal developers are working. Then may be the risk of me putting time in would be reduced for me and the risk of the maintainers accepting that I am going to take on a task is reduced. My 2 pence Pete -----Original Message----- From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of Eduardo A. Flores Verduzco Sent: Friday,16 November 16, 2007 02:41 To: ironruby-core at rubyforge.org Subject: Re: [Ironruby-core] Where is help needed? As I''ve seen there''s plenty of things to do! By the way, I started to work on StringIO but never asked if someone else is already on that... is there anyone? Eduardo Flores
Peter Bacon Darwin:> It is a shame that the code is not being developed directly against the > RubyForge repository as this definitely gives a feeling of them and us > and > makes it difficult to see what is being done, what progress is being > made > and it creates extra work for the maintainers ensuring that the > internal and > external repositories are synched up - there have been a number of > issues > recently about dodgy file paths and project references.Unfortunately, as much as we would love to make the "two repository" problem go away, the reality is that too much of our internal testing infrastructure (our SNAP queue, which is a lab of 20+ computers which run our check-in, sign-off, performance tests among others) is tightly coupled to TFS. There''s also another problem that isn''t visible to external folks: the DLR is undergoing a massive refactoring. Internally, all of the projects that depend on DLR (including some unannounced projects) live inside of the same TFS repository. What this means is that if a DLR change breaks something, we have to fix it otherwise SNAP won''t allow the check-in. This forcing function lets us keep all of our code sync''d with DLR changes which is goodness. It also helps us understand in real time the impact that DLR changes have on consumers. Think about this as "real time continuous integration". The latter case explains why we haven''t sync''d the SVN repository in two weeks. We had a large change (the stuff in the demos we had for RubyConf) that included some DLR changes that broke all sorts of stuff in, um, interesting ways. It''s taken us this long to get the check-in through the queue (it went in yesterday), so we''re preparing to push a new set of sources out today.> Some kind of work tracking tool would be beneficial, even if it was > just a > priority list of things that needed to be done so that people out here > could > pick up something. I suppose the problem is also that, for instance, > no one > knows who I am and what quality of code I can produce and so it would > be > risky to allocate work to me. How could this be solved.I think what we can do is get together at the beginning of each week with a quick ''scrum'' via email (probably better since we work across all sorts of different time zones). This way we can announce the things that we''re working on, so that we''re sure that we''re not stepping on other folks'' toes. Sound reasonable? I''ll kick off a scrum mail on Monday.> b) The > quality/style/efficiency is not accepted by the maintainers and my code > goes > to waste.If this were to happen, I don''t think that your code goes to waste. You''ve learned something valuable and the best way of improving is to get continuous, critical feedback on the progress of your work. Like so many other things, it''s about confidence - and there''s no better way of building confidence (in both directions) by submitting code that you''ve thought about, receiving / incorporating feedback and repeating. Thanks, -John
Peter Bacon Darwin:> I was thinking about this issue last night. I am interested in working > on the number stuff (Numeric, BigNum, FixedNum, Float, etc) but I am > concerned that a) Someone internally is already doing this and just as > I am about to send in a patch a new splat of code gets popped across > from the internal source control to SVN and all has been wasted; b) The > quality/style/efficiency is not accepted by the maintainers and my code > goes to waste.If you want to look at numeric stuff, go for it. Our numeric classes could really use some love. I know Tomas & I aren''t working on those. Regarding quality/style/efficiency--hey, my code gets rejected too sometimes :)> It is a shame that the code is not being developed directly against the > RubyForge repository as this definitely gives a feeling of them and us > and makes it difficult to see what is being done, what progress is > being made and it creates extra work for the maintainers ensuring that > the internal and external repositories are synched up - there have been > a number of issues recently about dodgy file paths and project > references.For what it''s worth, I do a lot of my work against the SVN layout, so I feel that pain too. The rakefile is still not perfect yet, but John is working on it. Well, at least I complain to him when it''s not working :)> Some kind of work tracking tool would be beneficial, even if it was > just a priority list of things that needed to be done so that people > out here could pick up something. I suppose the problem is also that, > for instance, no one knows who I am and what quality of code I can > produce and so it would be risky to allocate work to me. How could > this be solved.I really liked what Evan Phoenix said at RubyConf about "free flowing commit bit". We can''t implement it quite like Rubinius but I think the idea is anyone that does a good patch is trusted as someone who can do good quality work.> My personal aim is to send in a few patches and hopefully, get enough > feedback so that I am capable of producing code that is acceptable and > fits in with the way the internal developers are working. Then may be > the risk of me putting time in would be reduced for me and the risk of > the maintainers accepting that I am going to take on a task is reduced.I look forward to reviewing your patches! Also, keep in mind some of the feedback is just because there''s a lot of details to learn about, that no one could be expected to know without some experience with the system. I had a decent ramp-up period myself to get familiar with all of the stuff that goes in IronRuby. Cheers, John
Eduardo A. Flores Verduzco:> As I''ve seen there''s plenty of things to do! > > By the way, I started to work on StringIO but never asked if someone > else is already on that... is there anyone?Nobody is working on StringIO to my knowledge. If you want to stake a claim to it, please add your name and the class in the wiki page: http://ironruby.rubyforge.org/wiki/wiki.pl?Core Thanks! -John
Mike Jarosch:> Where is help needed? Reading through the list archive, it looks like > people are working on the zlib and yaml implementation. Is there help > needed there still? Where would my time spent working be most > beneficial?I believe that Michael Moore started to look at yaml, but then got busy with other things. Feel free to take a look, Mike - I believe that folks who have looked at yaml have been converging on porting JvYaml as a good starting point. Thanks, -John