llvm dev - Feb 2017 - LLVM GSOC Projects Criteria Consultation (before 2/28)

Agreed. I think it's worth thinking about what GSoC is actually about,
which according to the website: "Google Summer of Code is a global
program focused on introducing students to open source software
development."

Strongly favouring students with prior experience in community
involvement with LLVM is somewhat missing the point, and I think we
should be mindful of the differences in opportunities that some
students have had.

Amara

On 18 February 2017 at 02:10, Anna Zaks via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> On Feb 16, 2017, at 10:16 PM, Mehdi Amini via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>
> Hello all,
>
> GSOC is around the corner, and the LLVM projects plans to participate again
> this year. For those who don’t know about GSOC, students are proposing a
> project that they will work on for 3 months. Amongst other, one goal for
> LLVM is to mentor students to become good developers and also contributors
> to the LLVM project (or user/advocate of LLVM for building other cool
> projects).
>
> A key part of the process is about how do we select the projects. The way
> we’ve done it last year, is that the volunteer mentors shared an email
> thread and ultimately each one individually ranked the projects. We kept
the
> average grading for each project to ranked them overall.
>
> In order to make the process more transparent to student applicants, we
want
> to formalize and announce the criteria for ranking and selection project.
>
> The purpose of this email is to gather community feedback about the
> criterion that mentors should apply when ranking projects, for example:
>
> - Should we favor a student which has past contributions to LLVM compared
to
> a newcomer? Is it more important or as equally important as the quality of
> the proposal?
>
>
> While students with previous LLVM dev experience will be more productive,
> GSoC is a great way of attracting newcomers to the project! So if a
proposal
> is scoped to be completed in time even by someone not familiar with LLVM,
we
> should still accept it.
>
> - How should we rank (if any) “research or unbounded projects” vs “project
> with an immediate application”? Should we strive to keep a balance?
>
>
> My preference would be to give priority to projects that are on a
trajectory
> to have practical benefit.
>
> - What about “projects that touch LLVM/Clang/…” vs “projects that are
> building something on top of LLVM”? (For example this project was first
> proposed to be selected by LLVM before being rejected and finally picked by
> the Julia project directly:
> https://summerofcode.withgoogle.com/archive/2016/projects/6197080536121344/
> )?
>
> - Should we ask that the work is done upstream all along? In the past there
> have been project developed on GitHub outside the community that have never
> been merged. The LLVM developer policy has a full section insisting on
> incremental development (
> http://llvm.org/docs/DeveloperPolicy.html#incremental-development ).
>
>
> An invaluable part of participating in GSoC is for students to learn about
> how open source projects work. Following the developer policy is one of the
> main aspects of development in these codebases. In addition, if a project
is
> being developed on a branch it is likely not to get merged and the impact
of
> the student, the mentor, as well as the GSoC budget will be minimal.
>
> Hopefully we should be able to provide a set of guidelines to student that
> would help them fill the best application, and avoid unfortunate surprise
at
> the end of the process.
>
> We should have this discussion before receiving the projects proposals,
> which opens on 2/28.
>
> Best,
>
> --
> Mehdi
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

On Mon, Feb 20, 2017, at 03:26 PM, Amara Emerson via llvm-dev
wrote:
> Agreed. I think it's worth thinking about what GSoC is actually about,
> which according to the website: "Google Summer of Code is a global
> program focused on introducing students to open source software
> development."
> 
> Strongly favouring students with prior experience in community
> involvement with LLVM is somewhat missing the point, and I think we
> should be mindful of the differences in opportunities that some
> students have had.
Dear Mehdi,

thanks for sparking this discussion.

I believe we should evaluate students only by the quality of the student
himself without requiring previous LLVM experience. If a student can
convince us that he could be a great contributor in the future,
I am very much in favor of accepting him. In fact, many students who
never contributed to LLVM before
had great impact. Justin who pushed the PTX backend to a usable state
during GSoC, got later hired by NIVIDA and also helped upstreaming the
current PTX backend. He never had LLVM experience before.

The one most important thing which in my experience has been a very good
filter of good students is the requirement for them to have at least one
patch upstreamed and committed to LLVM. This does not mean that they
have to be long-term LLVM contributors, but this means they got in touch
with us early enough, asked for a trivial problem for them to address,
and went once through a full review process.
We can give them as much guidance as possible and help them finding very
easy bugs. However, if a student -- with all the help we can provide --
does not manage to submit a patch for a trivial problem over 6-8 weeks
it is a clear indication that he won't make a lot of progress in GSoC as
well. On the other side, if they once went through a full review
process, we have already a good feeling how this interaction will pan
out in the future.

The "one-patch-rule" is very common to a lot of GSoC projects and I
suggest we require it as well and make it very clear and explicit on our
GSoC application page how students can get support with such
a patch.

Regarding which projects to choose. I believe a good mix is great and --
assuming we have a good mix -- the quality of the proposal should help
us to decide. I agree here with arguments brought forward by Hal and
John.

Best,
Tobias



> 
> Amara
> 
> On 18 February 2017 at 02:10, Anna Zaks via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > On Feb 16, 2017, at 10:16 PM, Mehdi Amini via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> >
> > Hello all,
> >
> > GSOC is around the corner, and the LLVM projects plans to participate
again
> > this year. For those who don’t know about GSOC, students are proposing
a
> > project that they will work on for 3 months. Amongst other, one goal
for
> > LLVM is to mentor students to become good developers and also
contributors
> > to the LLVM project (or user/advocate of LLVM for building other cool
> > projects).
> >
> > A key part of the process is about how do we select the projects. The
way
> > we’ve done it last year, is that the volunteer mentors shared an email
> > thread and ultimately each one individually ranked the projects. We
kept the
> > average grading for each project to ranked them overall.
> >
> > In order to make the process more transparent to student applicants,
we want
> > to formalize and announce the criteria for ranking and selection
project.
> >
> > The purpose of this email is to gather community feedback about the
> > criterion that mentors should apply when ranking projects, for
example:
> >
> > - Should we favor a student which has past contributions to LLVM
compared to
> > a newcomer? Is it more important or as equally important as the
quality of
> > the proposal?
> >
> >
> > While students with previous LLVM dev experience will be more
productive,
> > GSoC is a great way of attracting newcomers to the project! So if a
proposal
> > is scoped to be completed in time even by someone not familiar with
LLVM, we
> > should still accept it.
> >
> > - How should we rank (if any) “research or unbounded projects” vs
“project
> > with an immediate application”? Should we strive to keep a balance?
> >
> >
> > My preference would be to give priority to projects that are on a
trajectory
> > to have practical benefit.
> >
> > - What about “projects that touch LLVM/Clang/…” vs “projects that are
> > building something on top of LLVM”? (For example this project was
first
> > proposed to be selected by LLVM before being rejected and finally
picked by
> > the Julia project directly:
> >
https://summerofcode.withgoogle.com/archive/2016/projects/6197080536121344/
> > )?
> >
> > - Should we ask that the work is done upstream all along? In the past
there
> > have been project developed on GitHub outside the community that have
never
> > been merged. The LLVM developer policy has a full section insisting on
> > incremental development (
> > http://llvm.org/docs/DeveloperPolicy.html#incremental-development ).
> >
> >
> > An invaluable part of participating in GSoC is for students to learn
about
> > how open source projects work. Following the developer policy is one
of the
> > main aspects of development in these codebases. In addition, if a
project is
> > being developed on a branch it is likely not to get merged and the
impact of
> > the student, the mentor, as well as the GSoC budget will be minimal.
> >
> > Hopefully we should be able to provide a set of guidelines to student
that
> > would help them fill the best application, and avoid unfortunate
surprise at
> > the end of the process.
> >
> > We should have this discussion before receiving the projects
proposals,
> > which opens on 2/28.
> >
> > Best,
> >
> > --
> > Mehdi
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
> >
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Going to add a "me too" to everything Tobias wrote here. I've
spent a lot
of years mentoring GSoC students and honestly have found a strong
correlation to previous work on a project, but that's the same with anyone.
Some previously unknown people have come onto the project and done quite
well. Ultimately it comes down to a project with an easy ramp up and small
bite sized chunks of work.

If I can be of any help here let me know.

-eric

On Tue, Feb 21, 2017 at 11:54 PM Tobias Grosser via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Mon, Feb 20, 2017, at 03:26 PM, Amara Emerson via llvm-dev wrote:
> > Agreed. I think it's worth thinking about what GSoC is actually
about,
> > which according to the website: "Google Summer of Code is a
global
> > program focused on introducing students to open source software
> > development."
> >
> > Strongly favouring students with prior experience in community
> > involvement with LLVM is somewhat missing the point, and I think we
> > should be mindful of the differences in opportunities that some
> > students have had.
>
> Dear Mehdi,
>
> thanks for sparking this discussion.
>
> I believe we should evaluate students only by the quality of the student
> himself without requiring previous LLVM experience. If a student can
> convince us that he could be a great contributor in the future,
> I am very much in favor of accepting him. In fact, many students who
> never contributed to LLVM before
> had great impact. Justin who pushed the PTX backend to a usable state
> during GSoC, got later hired by NIVIDA and also helped upstreaming the
> current PTX backend. He never had LLVM experience before.
>
> The one most important thing which in my experience has been a very good
> filter of good students is the requirement for them to have at least one
> patch upstreamed and committed to LLVM. This does not mean that they
> have to be long-term LLVM contributors, but this means they got in touch
> with us early enough, asked for a trivial problem for them to address,
> and went once through a full review process.
> We can give them as much guidance as possible and help them finding very
> easy bugs. However, if a student -- with all the help we can provide --
> does not manage to submit a patch for a trivial problem over 6-8 weeks
> it is a clear indication that he won't make a lot of progress in GSoC
as
> well. On the other side, if they once went through a full review
> process, we have already a good feeling how this interaction will pan
> out in the future.
>
> The "one-patch-rule" is very common to a lot of GSoC projects and
I
> suggest we require it as well and make it very clear and explicit on our
> GSoC application page how students can get support with such
> a patch.
>
> Regarding which projects to choose. I believe a good mix is great and --
> assuming we have a good mix -- the quality of the proposal should help
> us to decide. I agree here with arguments brought forward by Hal and
> John.
>
> Best,
> Tobias
>
>
>
>
> >
> > Amara
> >
> > On 18 February 2017 at 02:10, Anna Zaks via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> > >
> > > On Feb 16, 2017, at 10:16 PM, Mehdi Amini via llvm-dev
> > > <llvm-dev at lists.llvm.org> wrote:
> > >
> > > Hello all,
> > >
> > > GSOC is around the corner, and the LLVM projects plans to
participate
> again
> > > this year. For those who don’t know about GSOC, students are
proposing
> a
> > > project that they will work on for 3 months. Amongst other, one
goal
> for
> > > LLVM is to mentor students to become good developers and also
> contributors
> > > to the LLVM project (or user/advocate of LLVM for building other
cool
> > > projects).
> > >
> > > A key part of the process is about how do we select the projects.
The
> way
> > > we’ve done it last year, is that the volunteer mentors shared an
email
> > > thread and ultimately each one individually ranked the projects.
We
> kept the
> > > average grading for each project to ranked them overall.
> > >
> > > In order to make the process more transparent to student
applicants,
> we want
> > > to formalize and announce the criteria for ranking and selection
> project.
> > >
> > > The purpose of this email is to gather community feedback about
the
> > > criterion that mentors should apply when ranking projects, for
example:
> > >
> > > - Should we favor a student which has past contributions to LLVM
> compared to
> > > a newcomer? Is it more important or as equally important as the
> quality of
> > > the proposal?
> > >
> > >
> > > While students with previous LLVM dev experience will be more
> productive,
> > > GSoC is a great way of attracting newcomers to the project! So if
a
> proposal
> > > is scoped to be completed in time even by someone not familiar
with
> LLVM, we
> > > should still accept it.
> > >
> > > - How should we rank (if any) “research or unbounded projects” vs
> “project
> > > with an immediate application”? Should we strive to keep a
balance?
> > >
> > >
> > > My preference would be to give priority to projects that are on a
> trajectory
> > > to have practical benefit.
> > >
> > > - What about “projects that touch LLVM/Clang/…” vs “projects that
are
> > > building something on top of LLVM”? (For example this project was
first
> > > proposed to be selected by LLVM before being rejected and finally
> picked by
> > > the Julia project directly:
> > >
> https://summerofcode.withgoogle.com/archive/2016/projects/6197080536121344/
> > > )?
> > >
> > > - Should we ask that the work is done upstream all along? In the
past
> there
> > > have been project developed on GitHub outside the community that
have
> never
> > > been merged. The LLVM developer policy has a full section
insisting on
> > > incremental development (
> > > http://llvm.org/docs/DeveloperPolicy.html#incremental-development
).
> > >
> > >
> > > An invaluable part of participating in GSoC is for students to
learn
> about
> > > how open source projects work. Following the developer policy is
one
> of the
> > > main aspects of development in these codebases. In addition, if a
> project is
> > > being developed on a branch it is likely not to get merged and
the
> impact of
> > > the student, the mentor, as well as the GSoC budget will be
minimal.
> > >
> > > Hopefully we should be able to provide a set of guidelines to
student
> that
> > > would help them fill the best application, and avoid unfortunate
> surprise at
> > > the end of the process.
> > >
> > > We should have this discussion before receiving the projects
proposals,
> > > which opens on 2/28.
> > >
> > > Best,
> > >
> > > --
> > > Mehdi
> > >
> > > _______________________________________________
> > > LLVM Developers mailing list
> > > llvm-dev at lists.llvm.org
> > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > >
> > >
> > >
> > > _______________________________________________
> > > LLVM Developers mailing list
> > > llvm-dev at lists.llvm.org
> > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170222/cea7688d/attachment.html>