Ironruby core - Apr 2008 - How we will do better at community outreach

Thanks for the discussions in the rather long thread today. I wanted to start a
new thread under a better subject heading to continue the discussion.

I''ll be the first to admit that we suffer from the problem where we
work on a different source code repository, with an internal-only testing
infrastructure, and different internal tools. This causes friction and pain for
a lot of folks, and to be blunt there are no simple fixes to this. Everything
here involves a trade-off and it boils down to who''s going to feel the
pain at this point.

So let''s summarize what most folks would like to see:

1. Move the primary repository to SVN with TFS as our mirror.
2. Move more communication into the external mailing list
3. Release of binaries
4. Scrum-like status updates on the list

So let''s talk about a few of these issues in turn.

1. SVN as primary repository.

This won''t happen for several reasons. First, we have effectively four
different teams (IronRuby, IronPython, Managed JScript and DLR) working on the
same codebase. Only the IronRuby team is on RubyForge (yes, we also drop the DLR
sources there, but that''s a convenience thing for y''all and
not a permanent place for those sources). As the DLR is changed in our TFS
repository, it is the responsibility of the dev making those changes to ensure
that they don''t break any of the languages (and to fix them if the
change breaks them).

Today we gain the advantage of tight integration between these four projects.
The DLR can evolve rapidly which is goodness for folks on the outside. Remember
- we''re building the platform and the languages in parallel here.

Second, we do run what folks would call a continuous integration server. I call
it the troll that guards ''the truth'' in our repository. When a
dev wants to commit code to our repository they need to run those changes past
the troll. The troll runs a comprehensive suite of check-in tests against the
changes before it commits them to the repository after a successful test run.
This provides pretty strong guarantees that the tree is in a consistent state
which means that things continue to work when you pull the latest sources.
That''s much better than the alternative which means that you spend time
tracking down build breaks after you sync.

The troll has strong dependencies on TFS, and would be very difficult to
replicate outside of the company (the troll is really 20 servers inside one of
our labs).

2. Move more communication onto the external mailing list

This is totally doable, and something we should have done for some time now.
What I''m now proposing is this: all IronRuby code reviews will go out
through the public mailing list. We will generate a TFS ''unified
diff'' (http://msdn2.microsoft.com/en-us/library/6fd7dc73.aspx) and
attach that diff to the code review email that will be CC''d to
ironruby-core.

This way you''ll have the context for the changes (the source code)
along with a detailed email that describes what those changes are, along with
some commentary. Note that complex changes are often reviewed face-to-face on
our team (think pair-reviewing). But many changes are reviewed entirely by
email, so those follow-on emails will be captured for all to see.

3. Release of binaries

There is nothing blocking this today, and since we''re in a better
position to let folks ''kick the tires'' these days,
I''ll add binaries to our repository.

4. Scrum-like status updates to the list

This makes sense and I''ll talk tomorrow to Tomas and Jim about what
we''ll do from this point forward.

Note that these are just *my* ideas at this point. I''d like to use this
as a kick-off for some broader discussions on other issues that are on your
mind.

Thanks,
-John

The issue that Charles raised about people not knowing what others are already
working on is an important one. At a higher level, I''d like to see a
clearer statement of short term and medium term goals and plans for getting
there. Those plans require people taking ownership of certain tasks, so it would
be nice to know who is working on what at any given stage - both for people
within and external to Microsoft.

I also note that this email list serves two audiences, those interested in using
IronRuby and those interested in building it. It''s probably too early
to split the email list, but we should be conscious of this fact when we are
trying to manage the activities of this latter core-development group - there
probably aren''t that many of us! In terms of community building for
this core-development group, I''d certainly be interested to know a bit
more about who my peers are in this virtual community. Perhaps we could set up a
WIKI for core-developers to at least list their names, a little about themselves
(no more than one paragraph) and what they''re currently working on.

With respect to the code repository, the experience as an external contributor
is certainly more painful than my experience with Ruby.NET where I could
directly commit changes. I''m not suggesting that we should have such
privileges with IronRuby, just confirming that there is an issue here. I
don''t have any concrete suggestions for improvement. Certainly frequent
revisions helps (ideally 2 or 3 per week), also timely feedback on
what''s happening with submitted patches.

A number of people are working on entirely new and separate bits, for example
implementations of native standard libraries. Currently there is no common place
for external contributors to upload prototypes of bits that they are working on.
Some form of fileshare or ftp system where people could create directories and
unload files may be useful, prior to them being formally merged into the
official distribution. This would help us see what people are working on, how
far progressed they are, and to make comments.

Cheers, Wayne.

2008/4/29 John Lam (IRONRUBY) <jflam at
microsoft.com>:
>  Second, we do run what folks would call a continuous integration server. I
call it the troll that guards ''the truth'' in our repository.
When a dev wants to commit code to our repository they need to run those changes
past the troll. The troll runs a comprehensive suite of check-in tests against
the changes before it commits them to the repository after a successful test
run. This provides pretty strong guarantees that the tree is in a consistent
state which means that things continue to work when you pull the latest sources.
That''s much better than the alternative which means that you spend time
tracking down build breaks after you sync.
This is very true, but it is ironic since I get to track down build
breaks after sync anyway. Sorry, couldn''t resist.

-- 
Seo Sanghyeon

John, speaking from our perspective, as the developers of the only 
currently available IronRuby IDE ( 
http://www.sapphiresteel.com/Ruby-In-Steel-For-IronRuby ) I can tell you 
that what we really need is a clearly stated project plan - a ''road
map''
with well-defined points along the route with features and dates 
attached. This would be similar to the road map which, for example, the 
MS SDK team published. Until we have such a plan to work to, our own 
development on the IronRuby IDE remains stalled as we can''t make our
own
future plans. We have always published road maps for future versions of 
our standard Ruby IDE but until we have a firm development plan from the 
IronRuby team, we can''t make any future commitments on the IronRuby
IDE.

best wishes
Huw

SapphireSteel Software
Ruby and Rails In Visual Studio
http://www.sapphiresteel.com
-- 
Posted via http://www.ruby-forum.com/.

Sanghyeon Seo:
> This is very true, but it is ironic since I get to track down build
> breaks after sync anyway. Sorry, couldn''t resist.
Point taken :)

That said, what you''re seeing here are fairly minor build breaks since
most if not all of those build breakages are due to bugs in Mono/gmcs.

Imagine if Mono were under heavy development and changing daily. Those kinds of
breaks are what you would see if we decoupled IronRuby from DLR.

Thanks,
-John

What about the "fast-tracking the external Ruby library loading",
idea?  In
other words, allowing external "IronRuby" library code to be developed
and
compiled into a separate assembly so that it can be loaded into the Ruby
runtime via require "...".  (Not to be confused with
"requiring" a straight
.NET assembly, which of course we can do already).

This will allow separate OSS projects to be set up that really can have
direct community interaction for those libraries that are not core to the
IronRuby runtime.

The guts of IronRuby are starting to stabilize and there will be less and
less to do there but more and more to do on external libraries.  It could
encourage people to start building library code against IronRuby, therefore
providing even more feedback on both IronRuby runtime and so DLR.  Also, it
could allow the development process to scale out and potentially accelerate
toward goals such as running Rails.

In the long run, surely, this is the model that you want to have anyway.  So
why not push toward it now?

Pete

Wayne Kelly:
> The issue that Charles raised about people not knowing what others are
> already working on is an important one. At a higher level, I''d
like to
> see a clearer statement of short term and medium term goals and plans
> for getting there. Those plans require people taking ownership of
> certain tasks, so it would be nice to know who is working on what at
> any given stage - both for people within and external to Microsoft.
Wayne - do you think this would be addressed by a scheduled weekly Monday
morning scrum round of emails? Folks can post status updates out of band, of
course, but observers can watch those scrum mails for more visibility into what
various folks are working on.
> I also note that this email list serves two audiences, those interested
> in using IronRuby and those interested in building it. It''s
probably
> too early to split the email list, but we should be conscious of this
> fact when we are trying to manage the activities of this latter core-
> development group - there probably aren''t that many of us! In
terms of
> community building for this core-development group, I''d certainly
be
> interested to know a bit more about who my peers are in this virtual
> community. Perhaps we could set up a WIKI for core-developers to at
> least list their names, a little about themselves (no more than one
> paragraph) and what they''re currently working on.
I started a page here: http://ironruby.rubyforge.org/wiki/wiki.pl?Who
> Certainly frequent revisions helps (ideally 2 or 3 per
> week), also timely feedback on what''s happening with submitted
patches.
That''s my fault and I apologize for this. I''ll start by doing
daily pushes of our tree.
> A number of people are working on entirely new and separate bits, for
> example implementations of native standard libraries. Currently there
> is no common place for external contributors to upload prototypes of
> bits that they are working on. Some form of fileshare or ftp system
> where people could create directories and unload files may be useful,
> prior to them being formally merged into the official distribution.
> This would help us see what people are working on, how far progressed
> they are, and to make comments.
Let me think about this point a bit more. This is related to a point that Pete
brought up in a different thread - getting us to fix require and then making it
possible for folks to create separate projects. Part of this will involve
granting external commit access to the SVN repository - which I''m more
than happy to provide. I''ll get back to the list about this point later
today.

Thanks,
-John

/signed

This would help immensely.  As of right now all the external libraries
are loaded when you run rbx, this is in contrast to mri in that the
external libraries have to be required.

On Tue, Apr 29, 2008 at 9:31 AM, Peter Bacon Darwin
<bacondarwin at googlemail.com> wrote:
> What about the "fast-tracking the external Ruby library loading",
idea?  In
>  other words, allowing external "IronRuby" library code to be
developed and
>  compiled into a separate assembly so that it can be loaded into the Ruby
>  runtime via require "...".  (Not to be confused with
"requiring" a straight
>  .NET assembly, which of course we can do already).
>
>  This will allow separate OSS projects to be set up that really can have
>  direct community interaction for those libraries that are not core to the
>  IronRuby runtime.
>
>  The guts of IronRuby are starting to stabilize and there will be less and
>  less to do there but more and more to do on external libraries.  It could
>  encourage people to start building library code against IronRuby,
therefore
>  providing even more feedback on both IronRuby runtime and so DLR.  Also,
it
>  could allow the development process to scale out and potentially
accelerate
>  toward goals such as running Rails.
>
>  In the long run, surely, this is the model that you want to have anyway. 
So
>  why not push toward it now?
>
>  Pete
>
>
>
>
>  _______________________________________________
>  Ironruby-core mailing list
>  Ironruby-core at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/ironruby-core
>


-- 
Michael Letterle
[Polymath Prokrammer]
http://blog.prokrams.com

Peter Bacon Darwin:
> What about the "fast-tracking the external Ruby library loading",
idea?
> In other words, allowing external "IronRuby" library code to be
> developed and compiled into a separate assembly so that it can be
> loaded into the Ruby runtime via require "...".  (Not to be
confused
> with "requiring" a straight .NET assembly, which of course we can
do
> already).
Excellent idea. Will pop this to the top of the queue. This will also force us
to initialize $: correctly at startup.
> This will allow separate OSS projects to be set up that really can have
> direct community interaction for those libraries that are not core to
> the IronRuby runtime.
Let me noodle on this idea this morning. I have some ideas but I wanted to run
them past some folks around here first.
> In the long run, surely, this is the model that you want to have
> anyway.  So why not push toward it now?
Agreed.

Thanks!
-John

Hey,

On 4/29/08, John Lam (IRONRUBY) <jflam at microsoft.com>
wrote:
>  This is totally doable, and something we should have done for some time
now. What I''m now proposing is this: all IronRuby code reviews will go
out through the public mailing list. We will generate a TFS ''unified
diff'' (http://msdn2.microsoft.com/en-us/library/6fd7dc73.aspx) and
attach that diff to the code review email that will be CC''d to
ironruby-core.
>
>  This way you''ll have the context for the changes (the source
code) along with a detailed email that describes what those changes are, along
with some commentary. Note that complex changes are often reviewed face-to-face
on our team (think pair-reviewing). But many changes are reviewed entirely by
email, so those follow-on emails will be captured for all to see.
What would be neat, on top of that, is simply having a
ironruby-patches list, where automated mails are sent for each commit
on the IR repo. Pretty much something like what mono-patches provide:

http://lists.ximian.com/pipermail/mono-patches/

That would simply be great to get familiar with the code.

-- 
Jb Evain  <jb at nurv.fr>