Ironruby core - Apr 2009 - Some feedback on the compile/test process

Some notes as I go though the contributing process, in hope that will
be useful to make it smoother:
- after a clean checkout and launching dev.bat, brbd would not compile
(today). I believe it reported a missing microsoft.scripting.dll
issue.
- opening the solution in VS2008 shows 14 warnings about referenced
components that cannot be found (System, mscorlib, System.Net), not
sure if that''s an issue or not
- compiling through VS2008 makes the compile using brbd works afterwards.
- first pass at launching irtests opens a bunch of windows - most of
them fail with failure to load default.mspec (see [1]) - after
searching, the message was caused by an old .mspecrc that was sitting
in my home folder. Removing it solved the issue.
- I noticed a runfirst.cmd, should I actually run it or is it obsolete
? It seems to have an error in it (references default.mspec.rb which
has been renamed to default.mspec since).
- on a french machine, the MiscTests/NumericLiterals1 fails see [2] -
sounds like a missing InvariantCulture in a float ToString call. I''ll
look into it unless someone fixes it in between.

On irtests execution: the end of irtests launches irtest.bat, run.bat
and 4 mspec in parallels ("start" calls). Is there a need for that (ie
one process that monitors the others) or could it be run sequentially
if it''s more comfortable for me ? The load is putting my machine down
on its knees.

cheers,

-- Thibaut

===== [1] mspec error ====
IronRuby 0.3 0.3.0.0 on .NET 2.0.0.0
:0:in `load'': no such file to load --
C:\git\ironruby\Merlin\Main\..\External\Languages\IronRuby\mspec\default.mspec
(LoadError)

===== [2] NumericLiterals1 failure ====
1) NumericLiterals1
c:\git\ironruby\Merlin\Main\Languages\Ruby\IronRuby.Tests\Runtime\MiscTests.cs
: 411
Unexpected output:

''1\r\n-1\r\n1\r\n1,1\r\n-1,1\r\n1,1\r\n2.0\r\n2.0\r\n2,156\r\n''.

First difference (8):
actual = '',''
expected = ''.''

Repro: C:\git\ironruby\Merlin\Main\bin\debug\IronRuby.Tests.exe NumericLiterals1

A couple of steps later: I meet the following failures while running
irtests on an unmodified fork (by window title, see matching commands
at the end of the mail):
- smoke tests => 1 failure
- library rubyspec tests => 4 failures, 15 errors in various places
- core rubyspec tests => 1 failure, 1 error

I suspect some of these are linked to the fact that I use a french
culture on my machine.

Just do get a point of comparison and know what needs fixing, are all
these supposed to pass, or are some known to fail and that would be
normal ?

cheers,

-- Thibaut

C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Smoke Tests"
C:\git\ironruby\Merlin\Main\Languages\Ruby\Tests\Scripts\irtest.bat
C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Legacy Tests"
C:\git\ironruby\Merlin\Main\Languages\Ruby\Tests\run.bat
C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Core RubySpec
tests"
mspec ci -fd -V :core
C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Language RubySpec
tests" mspec ci -fd -V :lang
C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Library RubySpec
tests" mspec ci -fd -V :lib
C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Command Line
RubySpec tests" mspec ci -fd -V :cli :netinterop

> -----Original Message-----
> From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-
> bounces at rubyforge.org] On Behalf Of Thibaut Barr?re
> Sent: Wednesday, April 15, 2009 6:16 AM
> To: ironruby-core
> Subject: [Ironruby-core] Some feedback on the compile/test process
>
> Some notes as I go though the contributing process, in hope that will
> be useful to make it smoother:
> - after a clean checkout and launching dev.bat, brbd would not compile
> (today). I believe it reported a missing microsoft.scripting.dll
> issue.
Please remove the ClrAssembly.csproj file from the solution ... then all will
build fine. That will be removed shortly.
> - opening the solution in VS2008 shows 14 warnings about referenced
> components that cannot be found (System, mscorlib, System.Net), not
> sure if that''s an issue or not
Hmm, weird. The build configuration was definitely Debug, not "Silverlight
Debug" or something like that?
> - compiling through VS2008 makes the compile using brbd works
> afterwards.
> - first pass at launching irtests opens a bunch of windows - most of
> them fail with failure to load default.mspec (see [1]) - after
> searching, the message was caused by an old .mspecrc that was sitting
> in my home folder. Removing it solved the issue.
If you run dev.bat first, this problem should go away. Irtest opens a bunch of
windows, one for each test runner, simply for parallelism.
> - I noticed a runfirst.cmd, should I actually run it or is it obsolete
> ? It seems to have an error in it (references default.mspec.rb which
> has been renamed to default.mspec since).
Runfirst.cmd should be removed, dev.bat is what you want
> - on a french machine, the MiscTests/NumericLiterals1 fails see [2] -
> sounds like a missing InvariantCulture in a float ToString call.
I''ll
> look into it unless someone fixes it in between.
Please track this with a bug on codeplex, someone else might have context.
> On irtests execution: the end of irtests launches irtest.bat, run.bat
> and 4 mspec in parallels ("start" calls). Is there a need for
that (ie
> one process that monitors the others) or could it be run sequentially
> if it''s more comfortable for me ? The load is putting my machine
down
> on its knees.
You could make a version of irtest that does them sequentially
> cheers,
>
> -- Thibaut
>
> ===== [1] mspec error ====>
> IronRuby 0.3 0.3.0.0 on .NET 2.0.0.0
> :0:in `load'': no such file to load --
> C:\git\ironruby\Merlin\Main\..\External\Languages\IronRuby\mspec\defaul
> t.mspec
> (LoadError)
>
> ===== [2] NumericLiterals1 failure ====>
> 1) NumericLiterals1
> c:\git\ironruby\Merlin\Main\Languages\Ruby\IronRuby.Tests\Runtime\MiscT
> ests.cs
> : 411
> Unexpected output:
>
>
''1\r\n-1\r\n1\r\n1,1\r\n-1,1\r\n1,1\r\n2.0\r\n2.0\r\n2,156\r\n''.
>
> First difference (8):
> actual = '',''
> expected = ''.''
>
> Repro: C:\git\ironruby\Merlin\Main\bin\debug\IronRuby.Tests.exe
> NumericLiterals1
> _______________________________________________
> Ironruby-core mailing list
> Ironruby-core at rubyforge.org
> http://rubyforge.org/mailman/listinfo/ironruby-core

Yes, these are all suppose to pass ... they run "mspec ci" which is
the continuous integration mode ... which omits "critical" or
"fails" tagged tests.

It''d be good to know where IronRuby falls over in different cultures,
so feel free to report bugs on codeplex about these things.
> -----Original Message-----
> From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-
> bounces at rubyforge.org] On Behalf Of Thibaut Barr?re
> Sent: Wednesday, April 15, 2009 6:57 AM
> To: ironruby-core
> Subject: Re: [Ironruby-core] Some feedback on the compile/test process
>
> A couple of steps later: I meet the following failures while running
> irtests on an unmodified fork (by window title, see matching commands
> at the end of the mail):
> - smoke tests => 1 failure
> - library rubyspec tests => 4 failures, 15 errors in various places
> - core rubyspec tests => 1 failure, 1 error
>
> I suspect some of these are linked to the fact that I use a french
> culture on my machine.
>
> Just do get a point of comparison and know what needs fixing, are all
> these supposed to pass, or are some known to fail and that would be
> normal ?
>
> cheers,
>
> -- Thibaut
>
> C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Smoke Tests"
> C:\git\ironruby\Merlin\Main\Languages\Ruby\Tests\Scripts\irtest.bat
> C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Legacy
Tests"
> C:\git\ironruby\Merlin\Main\Languages\Ruby\Tests\run.bat
> C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Core RubySpec
tests"
> mspec ci -fd -V :core
> C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Language RubySpec
> tests" mspec ci -fd -V :lang
> C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Library RubySpec
> tests" mspec ci -fd -V :lib
> C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Command Line
> RubySpec tests" mspec ci -fd -V :cli :netinterop
> _______________________________________________
> Ironruby-core mailing list
> Ironruby-core at rubyforge.org
> http://rubyforge.org/mailman/listinfo/ironruby-core

Hi Jimmy,

thanks for the feedback - comments inlined below:
> Please remove the ClrAssembly.csproj file from the solution ... then all
will build fine. That will be removed shortly.
thanks.
>> - opening the solution in VS2008 shows 14 warnings about referenced
>> components that cannot be found (System, mscorlib, System.Net), not
>> sure if that''s an issue or not
>
> Hmm, weird. The build configuration was definitely Debug, not
"Silverlight Debug" or something like that?
It is Debug, not Silverlight Debug. I looked at the .csproj and saw
conditional references to these dll when SilverlightDebug is set.
It seems that the condition is not evaluated properly. Not a big deal
though, it seems to work, but could lead to weird things I guess.
>> - I noticed a runfirst.cmd, should I actually run it or is it obsolete
>> ? It seems to have an error in it (references default.mspec.rb which
>> has been renamed to default.mspec since).
>
> Runfirst.cmd should be removed, dev.bat is what you want
ok, thanks.
>
>> - on a french machine, the MiscTests/NumericLiterals1 fails see [2] -
>> sounds like a missing InvariantCulture in a float ToString call.
I''ll
>> look into it unless someone fixes it in between.
>
> Please track this with a bug on codeplex, someone else might have context.
Done: http://ironruby.codeplex.com/WorkItem/View.aspx?WorkItemId=597
>> On irtests execution: the end of irtests launches irtest.bat, run.bat
>> and 4 mspec in parallels ("start" calls). Is there a need for
that (ie
>> one process that monitors the others) or could it be run sequentially
>> if it''s more comfortable for me ? The load is putting my
machine down
>> on its knees.
>
> You could make a version of irtest that does them sequentially
Thanks - much faster.

-- Thibaut

> Yes, these are all suppose to pass ... they run "mspec ci" which
is the continuous integration mode ... which omits "critical" or
"fails" tagged tests.
>
> It''d be good to know where IronRuby falls over in different
cultures, so feel free to report bugs on codeplex about these things.
thanks - I added the first one here:
http://ironruby.codeplex.com/WorkItem/View.aspx?WorkItemId=597

and will add or fix more as I go.

-- Thibaut

I confirm that I can reproduce this bug on my system too (it-IT). See
below for a simpler test case:
>>> puts 1.1
1,1
=> nil
>>> p 1.1
1.1
=> nil

The problem is in IoOps.ToPrintedString, where there is indeed an
obj.ToString() for doubles. Converting it with Convert.ToString(obj,
System.Globalization.CultureInfo.InvariantCulture) seems to fix the
issue, but I have just tried it in the REPL without running the test
suite yet. If you want, I can commit this fix in my repository if it
passes the tests.

On Thu, Apr 16, 2009 at 09:58, Thibaut Barr?re
<thibaut.barrere at gmail.com> wrote:
>> Yes, these are all suppose to pass ... they run "mspec ci"
which is the continuous integration mode ... which omits "critical" or
"fails" tagged tests.
>>
>> It''d be good to know where IronRuby falls over in different
cultures, so feel free to report bugs on codeplex about these things.
>
> thanks - I added the first one here:
> http://ironruby.codeplex.com/WorkItem/View.aspx?WorkItemId=597
>
> and will add or fix more as I go.
>
> -- Thibaut
> _______________________________________________
> Ironruby-core mailing list
> Ironruby-core at rubyforge.org
> http://rubyforge.org/mailman/listinfo/ironruby-core
>

-- 
Daniele Alessandri
http://www.clorophilla.net/blog/
http://twitter.com/JoL1hAHN

Haya,
> The problem is in IoOps.ToPrintedString, where there is indeed an
> obj.ToString() for doubles. Converting it with Convert.ToString(obj,
> System.Globalization.CultureInfo.InvariantCulture) seems to fix the
> issue, but I have just tried it in the REPL without running the test
> suite yet. If you want, I can commit this fix in my repository if it
> passes the tests.
puts 1.1 gives 1,1 here too.

Please commit the fix, this way I''ll be able to see if this solves
other failing tests in french too (I suppose it will).

I''ll have to figure out how to cherry pick one commit from your
branch, the git way :)

-- Thibaut

I got 14 warning as well, My workaround was to change a Target  
Framework to .NET 3.5

Thanks,
-Jirapong

On Apr 15, 2009, at 8:15 PM, Thibaut Barr?re wrote:
> Some notes as I go though the contributing process, in hope that will
> be useful to make it smoother:
> - after a clean checkout and launching dev.bat, brbd would not compile
> (today). I believe it reported a missing microsoft.scripting.dll
> issue.
> - opening the solution in VS2008 shows 14 warnings about referenced
> components that cannot be found (System, mscorlib, System.Net), not
> sure if that''s an issue or not
> - compiling through VS2008 makes the compile using brbd works  
> afterwards.
> - first pass at launching irtests opens a bunch of windows - most of
> them fail with failure to load default.mspec (see [1]) - after
> searching, the message was caused by an old .mspecrc that was sitting
> in my home folder. Removing it solved the issue.
> - I noticed a runfirst.cmd, should I actually run it or is it obsolete
> ? It seems to have an error in it (references default.mspec.rb which
> has been renamed to default.mspec since).
> - on a french machine, the MiscTests/NumericLiterals1 fails see [2] -
> sounds like a missing InvariantCulture in a float ToString call.
I''ll
> look into it unless someone fixes it in between.
>
> On irtests execution: the end of irtests launches irtest.bat, run.bat
> and 4 mspec in parallels ("start" calls). Is there a need for
that (ie
> one process that monitors the others) or could it be run sequentially
> if it''s more comfortable for me ? The load is putting my machine
down
> on its knees.
>
> cheers,
>
> -- Thibaut
>
> ===== [1] mspec error ====>
> IronRuby 0.3 0.3.0.0 on .NET 2.0.0.0
> :0:in `load'': no such file to load --
> C:\git\ironruby\Merlin\Main\..\External\Languages\IronRuby\mspec 
> \default.mspec
> (LoadError)
>
> ===== [2] NumericLiterals1 failure ====>
> 1) NumericLiterals1
> c:\git\ironruby\Merlin\Main\Languages\Ruby\IronRuby.Tests\Runtime 
> \MiscTests.cs
> : 411
> Unexpected output:
>
>
''1\r\n-1\r\n1\r\n1,1\r\n-1,1\r\n1,1\r\n2.0\r\n2.0\r\n2,156\r\n''.
>
> First difference (8):
> actual = '',''
> expected = ''.''
>
> Repro: C:\git\ironruby\Merlin\Main\bin\debug\IronRuby.Tests.exe  
> NumericLiterals1
> _______________________________________________
> Ironruby-core mailing list
> Ironruby-core at rubyforge.org
> http://rubyforge.org/mailman/listinfo/ironruby-core

On Thu, Apr 16, 2009 at 12:35, Thibaut Barr?re
<thibaut.barrere at gmail.com> wrote:
> I ran the whole irtests suite after applying the attached patch, based
> on what Daniel described.
You can find a slightly modified patch attached to this mail, this is
what I''m going to push on my repository. I think this one is better
than the first one as it removes the overhead of Convert.ToString and
it reduces the number of unboxing operations from object to double.

-- 
Daniele Alessandri
http://www.clorophilla.net/blog/
http://twitter.com/JoL1hAHN
-------------- next part --------------
diff --git
a/Merlin/Main/Languages/Ruby/Libraries.LCA_RESTRICTED/Builtins/IoOps.cs
b/Merlin/Main/Languages/Ruby/Libraries.LCA_RESTRICTED/Builtins/IoOps.cs
index 6da6a3d..0d18961 100644
--- a/Merlin/Main/Languages/Ruby/Libraries.LCA_RESTRICTED/Builtins/IoOps.cs
+++ b/Merlin/Main/Languages/Ruby/Libraries.LCA_RESTRICTED/Builtins/IoOps.cs
@@ -621,22 +621,23 @@ namespace IronRuby.Builtins {
 
             if ((list = obj as List<object>) != null) {
                 return IListOps.Join(tosConversion, list, NewLine);
             } else if ((hash = obj as IDictionary<object, object>) !=
null) {
                 return IDictionaryOps.ToMutableString(tosConversion, hash);
             } else if (obj == null) {
                 return MutableString.Create("nil");
             } else if (obj is bool) {
                 return MutableString.Create((bool)obj ? "true" :
"false");
             } else if (obj is double) {
-                var result = MutableString.Create(obj.ToString());
-                if ((double)(int)(double)obj == (double)obj) {
+                double value = (double)obj;
+                var result =
MutableString.Create(value.ToString(System.Globalization.CultureInfo.InvariantCulture));
+                if ((double)(int)value == value) {
                     result.Append(".0");
                 }
                 return result;
             } else if ((str = obj as MutableString) != null) {
                 return str;
             } else {
                 return Protocols.ConvertToString(tosConversion, obj);
             }
         }

Thibaut, I have a change to irtests.bat which will give an option to run the
tests in series or in parallel. I will push this through in my next commit so
folks can chose how they want to run the tests...

The one issue with running in series is that if a task fails, you want to run
the remaining tasks as well so that you can have the full list of failures.
However, if you just keep running all tasks in series, the output of the failing
task can scroll out, and then you lose the information of which test failed. The
right solution is to redirect the output of all the tasks so that you can
inspect the log files to see the details of the failure.

Setting up the output redirection, etc, is more work. A rake task would be nice
so that all the logic can be written in Ruby rather than in irtests.bat. I will
leave this for another day. Let''s see how far irtests.bat can scale...

Thanks,
Shri

-----Original Message-----
From: ironruby-core-bounces at rubyforge.org [mailto:ironruby-core-bounces at
rubyforge.org] On Behalf Of Thibaut Barr?re
Sent: Thursday, April 16, 2009 12:58 AM
To: ironruby-core at rubyforge.org
Subject: Re: [Ironruby-core] Some feedback on the compile/test process

Hi Jimmy,

thanks for the feedback - comments inlined below:
> Please remove the ClrAssembly.csproj file from the solution ... then all
will build fine. That will be removed shortly.
thanks.
>> - opening the solution in VS2008 shows 14 warnings about referenced
>> components that cannot be found (System, mscorlib, System.Net), not
>> sure if that''s an issue or not
>
> Hmm, weird. The build configuration was definitely Debug, not
"Silverlight Debug" or something like that?
It is Debug, not Silverlight Debug. I looked at the .csproj and saw
conditional references to these dll when SilverlightDebug is set.
It seems that the condition is not evaluated properly. Not a big deal
though, it seems to work, but could lead to weird things I guess.
>> - I noticed a runfirst.cmd, should I actually run it or is it obsolete
>> ? It seems to have an error in it (references default.mspec.rb which
>> has been renamed to default.mspec since).
>
> Runfirst.cmd should be removed, dev.bat is what you want
ok, thanks.
>
>> - on a french machine, the MiscTests/NumericLiterals1 fails see [2] -
>> sounds like a missing InvariantCulture in a float ToString call.
I''ll
>> look into it unless someone fixes it in between.
>
> Please track this with a bug on codeplex, someone else might have context.
Done: http://ironruby.codeplex.com/WorkItem/View.aspx?WorkItemId=597
>> On irtests execution: the end of irtests launches irtest.bat, run.bat
>> and 4 mspec in parallels ("start" calls). Is there a need for
that (ie
>> one process that monitors the others) or could it be run sequentially
>> if it''s more comfortable for me ? The load is putting my
machine down
>> on its knees.
>
> You could make a version of irtest that does them sequentially
Thanks - much faster.

-- Thibaut
_______________________________________________
Ironruby-core mailing list
Ironruby-core at rubyforge.org
http://rubyforge.org/mailman/listinfo/ironruby-core

Hi Shri,
> Thibaut, I have a change to irtests.bat which will give an option to run
the tests in series or in parallel. I will push this through in my next commit
so folks can chose how they want to run the tests...
>
> The one issue with running in series is that if a task fails, you want to
run the remaining tasks as well so that you can have the full list of failures.
However, if you just keep running all tasks in series, the output of the failing
task can scroll out, and then you lose the information of which test failed. The
right solution is to redirect the output of all the tasks so that you can
inspect the log files to see the details of the failure.
>
> Setting up the output redirection, etc, is more work. A rake task would be
nice so that all the logic can be written in Ruby rather than in irtests.bat. I
will leave this for another day. Let''s see how far irtests.bat can
scale...
thanks for the change! And yeah, let''s see how the batch can scale. If
it gets boring, I''ll try to take a stab at redirecting output.

cheers

-- Thibaut