llvm dev - Apr 2020 - RFC: Switching from Bugzilla to Github Issues [UPDATED]

On 04/24/2020 03:24 AM, Sam McCall wrote:
> clangd's experience using github issues to track bugs (in a separate
repo) has been very positive, and I'm glad you're pushing on this!
> 
> Part of this has been that our issue tracker has been scoped to our
subproject only, which is a scope that the tool works well for (on the user and
developer side).
> As such I don't think we should migrate clangd to a using the monorepo
bugtracker. Email subscription to a label is better than nothing, but worse than
a separate repo.
> Removing the clangd label from the monorepo bugtracker seems like the
simplest thing, though I'm happy to work on auto-moving bugs if that's
better.
> 
> (I'd suggest considering the same for other subprojects, though I know
that's not a popular opinion here)
I think it's important for everything in the monorepo to use the same bug
tracker.

There are advantages to having code in the monorepo (e.g. free
updates for API changes, a more consistent build experience, etc.).
But there are also costs, as you have pointed out, like having to use
a less than ideal bug tracker.  It's really up to sub-projects
to make the decision about whether these benefits are worth the costs.
The flang developers have just gone through this process and have
had to make some sacrifices to get the code in, but ultimately felt the
sacrifices were worth it.

I think it hurts the ability of developers and users to collaborate effectively,
if the infrastructure for the project is spread across too many different
places.
And good collaboration is key for a project of this size with some many tightly
connected components.

Getting back to the proposal we are discussing.  Do you have any specific
feedback
for improvements that might help make it align better with the kind of
experience
the clangd users and developers are looking for?

- Tom

On Fri, Apr 24, 2020 at 9:03 PM Tom Stellard <tstellar at redhat.com>
wrote:
> On 04/24/2020 03:24 AM, Sam McCall wrote:
> > clangd's experience using github issues to track bugs (in a
separate
> repo) has been very positive, and I'm glad you're pushing on this!
> >
> > Part of this has been that our issue tracker has been scoped to our
> subproject only, which is a scope that the tool works well for (on the user
> and developer side).
> > As such I don't think we should migrate clangd to a using the
monorepo
> bugtracker. Email subscription to a label is better than nothing, but worse
> than a separate repo.
> > Removing the clangd label from the monorepo bugtracker seems like the
> simplest thing, though I'm happy to work on auto-moving bugs if
that's
> better.
> >
> > (I'd suggest considering the same for other subprojects, though I
know
> that's not a popular opinion here)
>
> I think it's important for everything in the monorepo to use the same
bug
> tracker.
>
> There are advantages to having code in the monorepo (e.g. free
> updates for API changes, a more consistent build experience, etc.).
> But there are also costs, as you have pointed out, like having to use
> a less than ideal bug tracker.  It's really up to sub-projects
> to make the decision about whether these benefits are worth the costs.
> The flang developers have just gone through this process and have
> had to make some sacrifices to get the code in, but ultimately felt the
> sacrifices were worth it.
>
> I think it hurts the ability of developers and users to collaborate
> effectively,
> if the infrastructure for the project is spread across too many different
> places.
> And good collaboration is key for a project of this size with some many
> tightly
> connected components.
>
(sorry, I should probably not tilt at this windmill. More on-topic stuff
below, I promise!)
Right, and I think having a single-project view of the LLVM organization is
a mistake: it's a graph of projects, some are highly connected and some are
not.
The monorepo has a strong technical reason: the graph is connected and
accepting a CI boundary anywhere is expensive in the absence of stable APIs.
But this is much less true for bug tracking systems: the cost to crossing
boundaries is smaller.
For clangd, the benefit of sharing a tracker with clang AST+Sema is less
than the cost of sharing a tracker with clang codegen, LLVM proper, LLD,
flang, MLIR, ... (and the opposite is true for source control/CI).
Anyway, this is going to depend on what part(s) of the project graph you
touch: people connected to many parts will want to make coordinating with
hundreds of people incrementally, while people connected to few parts are
far better served by communicating only with the people they need to
(communication famously scales badly).

Getting back to the proposal we are discussing.  Do you have any
specific
> feedback
> for improvements that might help make it align better with the kind of
> experience
> the clangd users and developers are looking for?
>
Sorry if it seemed I was trying to derail: I think sharding into multiple
repos *is* a specific improvement that should be considered, though there
are arguments against it.
If "the proposal we are discussing" doesn't admit changes, well,
I'm +1 on
its current form too :-)

Other suggestions:

Issue templates: I think you need at least one for each component.
Users will be less familiar with the bug tracker conventions than
developers are, especially given that this one is unusual in covering
multiple products. Forcing a choice between the "component" tags as
well as
guiding them to include relevant info leaves less of an unstructured mess
to triage. This helps mitigate the fact that the UI won't separate
component tags like "lld" from others like
"crash-on-invalid".
(Fortunately these don't need to be maintained centrally: editing templates
just needs commit access)

Tag namespace: I can see this becoming a mess quickly, it's hard to choose
the right tags if there are too many, so there can be a bit of a
tragedy-of-the-commons. (Seeding it with lots of "component" tags
doesn't
help). Maybe these should be centrally designed.
I wonder if the "projects" feature can be used for components?
It's not
really the intended purpose (looks like they're supposed to be time-limited
like sprints) but maybe they can work as a second tag namespace, and the
name fits...

Releases: my most active times on bugzilla are when dealing with release
blockers in the run-up to a release. As GH issues lacks blocking bugs, I'm
a bit curious what the workflow is for this (and probably worth considering
before the release is upon us). I guess "projects" might be a good
fit.

Preserving bug numbers: seems worth it to me, but I have no special insight.

Again, thanks a lot for working on this, I think it'll be a substantial
improvement to land it in any form (however much I end up using it myself).

>
> - Tom
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200424/4c7bd0f9/attachment.html>

On Fri, 24 Apr 2020 at 14:13, Sam McCall via cfe-dev <cfe-dev at
lists.llvm.org>
wrote:
> On Fri, Apr 24, 2020 at 9:03 PM Tom Stellard <tstellar at redhat.com>
wrote:
>
>> On 04/24/2020 03:24 AM, Sam McCall wrote:
>> > clangd's experience using github issues to track bugs (in a
separate
>> repo) has been very positive, and I'm glad you're pushing on
this!
>> >
>> > Part of this has been that our issue tracker has been scoped to
our
>> subproject only, which is a scope that the tool works well for (on the
user
>> and developer side).
>> > As such I don't think we should migrate clangd to a using the
monorepo
>> bugtracker. Email subscription to a label is better than nothing, but
worse
>> than a separate repo.
>> > Removing the clangd label from the monorepo bugtracker seems like
the
>> simplest thing, though I'm happy to work on auto-moving bugs if
that's
>> better.
>> >
>> > (I'd suggest considering the same for other subprojects,
though I know
>> that's not a popular opinion here)
>>
>> I think it's important for everything in the monorepo to use the
same bug
>> tracker.
>>
>> There are advantages to having code in the monorepo (e.g. free
>> updates for API changes, a more consistent build experience, etc.).
>> But there are also costs, as you have pointed out, like having to use
>> a less than ideal bug tracker.  It's really up to sub-projects
>> to make the decision about whether these benefits are worth the costs.
>> The flang developers have just gone through this process and have
>> had to make some sacrifices to get the code in, but ultimately felt the
>> sacrifices were worth it.
>>
>> I think it hurts the ability of developers and users to collaborate
>> effectively,
>> if the infrastructure for the project is spread across too many
different
>> places.
>> And good collaboration is key for a project of this size with some many
>> tightly
>> connected components.
>>
> (sorry, I should probably not tilt at this windmill. More on-topic stuff
> below, I promise!)
> Right, and I think having a single-project view of the LLVM organization
> is a mistake: it's a graph of projects, some are highly connected and
some
> are not.
> The monorepo has a strong technical reason: the graph is connected and
> accepting a CI boundary anywhere is expensive in the absence of stable
APIs.
> But this is much less true for bug tracking systems: the cost to crossing
> boundaries is smaller.
> For clangd, the benefit of sharing a tracker with clang AST+Sema is less
> than the cost of sharing a tracker with clang codegen, LLVM proper, LLD,
> flang, MLIR, ... (and the opposite is true for source control/CI).
> Anyway, this is going to depend on what part(s) of the project graph you
> touch: people connected to many parts will want to make coordinating with
> hundreds of people incrementally, while people connected to few parts are
> far better served by communicating only with the people they need to
> (communication famously scales badly).
>
I would assume we will eventually want to open up our github repository to
pull requests. When that happens, if you use a separate bugtracker then
clangd issues will be split across the two: pull requests will be sent to
the monorepo because (as far as I'm aware) a pull request for a repo can
only be sent to that repo's issue tracker, but your "regular"
issues will
be elsewhere. And even that is a rosier picture than I expect you'll
actually experience: people will file issues for clangd on the issue
tracker attached to the clangd repository, simply because that's how the
github workflow works in general. So if you keep a separate issue tracker,
I think you will de facto end up with two issue trackers for clangd, and
will spend some amount of ongoing effort managing that. So, I think you
might be underestimating the costs here if you've not taken that into
account already. That said, moving issues between repos is apparently
feasible, so it seems like we could use clangd as an experiment to find out
how major or minor these concerns actually are.

Getting back to the proposal we are discussing.  Do you have any
specific
>> feedback
>> for improvements that might help make it align better with the kind of
>> experience
>> the clangd users and developers are looking for?
>>
> Sorry if it seemed I was trying to derail: I think sharding into multiple
> repos *is* a specific improvement that should be considered, though there
> are arguments against it.
> If "the proposal we are discussing" doesn't admit changes,
well, I'm +1 on
> its current form too :-)
>
> Other suggestions:
>
> Issue templates: I think you need at least one for each component.
> Users will be less familiar with the bug tracker conventions than
> developers are, especially given that this one is unusual in covering
> multiple products. Forcing a choice between the "component" tags
as well as
> guiding them to include relevant info leaves less of an unstructured mess
> to triage. This helps mitigate the fact that the UI won't separate
> component tags like "lld" from others like
"crash-on-invalid".
> (Fortunately these don't need to be maintained centrally: editing
> templates just needs commit access)
>
> Tag namespace: I can see this becoming a mess quickly, it's hard to
choose
> the right tags if there are too many, so there can be a bit of a
> tragedy-of-the-commons. (Seeding it with lots of "component" tags
doesn't
> help). Maybe these should be centrally designed.
> I wonder if the "projects" feature can be used for components?
It's not
> really the intended purpose (looks like they're supposed to be
time-limited
> like sprints) but maybe they can work as a second tag namespace, and the
> name fits...
>
> Releases: my most active times on bugzilla are when dealing with release
> blockers in the run-up to a release. As GH issues lacks blocking bugs,
I'm
> a bit curious what the workflow is for this (and probably worth considering
> before the release is upon us). I guess "projects" might be a
good fit.
>
> Preserving bug numbers: seems worth it to me, but I have no special
> insight.
>
> Again, thanks a lot for working on this, I think it'll be a substantial
> improvement to land it in any form (however much I end up using it myself).
>
>
>>
>> - Tom
>>
>>
>>
>>
>>
>> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200424/b5ab1e54/attachment.html>

Am Fr., 24. Apr. 2020 um 16:13 Uhr schrieb Sam McCall via cfe-dev
<cfe-dev at lists.llvm.org>:
> On Fri, Apr 24, 2020 at 9:03 PM Tom Stellard <tstellar at redhat.com>
wrote:
>> On 04/24/2020 03:24 AM, Sam McCall wrote:
>> > clangd's experience using github issues to track bugs (in a
separate repo) has been very positive, and I'm glad you're pushing on
this!
>> >
>> > Part of this has been that our issue tracker has been scoped to
our subproject only, which is a scope that the tool works well for (on the user
and developer side).
>> > As such I don't think we should migrate clangd to a using the
monorepo bugtracker. Email subscription to a label is better than nothing, but
worse than a separate repo.
>> > Removing the clangd label from the monorepo bugtracker seems like
the simplest thing, though I'm happy to work on auto-moving bugs if
that's better.
>> >
>> > (I'd suggest considering the same for other subprojects,
though I know that's not a popular opinion here)
>>
>> I think it's important for everything in the monorepo to use the
same bug tracker.
>>
>> There are advantages to having code in the monorepo (e.g. free
>> updates for API changes, a more consistent build experience, etc.).
>> But there are also costs, as you have pointed out, like having to use
>> a less than ideal bug tracker.  It's really up to sub-projects
>> to make the decision about whether these benefits are worth the costs.
>> The flang developers have just gone through this process and have
>> had to make some sacrifices to get the code in, but ultimately felt the
>> sacrifices were worth it.
>>
>> I think it hurts the ability of developers and users to collaborate
effectively,
>> if the infrastructure for the project is spread across too many
different places.
>> And good collaboration is key for a project of this size with some many
tightly
>> connected components.
>
> (sorry, I should probably not tilt at this windmill. More on-topic stuff
below, I promise!)
> Right, and I think having a single-project view of the LLVM organization is
a mistake: it's a graph of projects, some are highly connected and some are
not.
> The monorepo has a strong technical reason: the graph is connected and
accepting a CI boundary anywhere is expensive in the absence of stable APIs.
> But this is much less true for bug tracking systems: the cost to crossing
boundaries is smaller.
> For clangd, the benefit of sharing a tracker with clang AST+Sema is less
than the cost of sharing a tracker with clang codegen, LLVM proper, LLD, flang,
MLIR, ... (and the opposite is true for source control/CI).
> Anyway, this is going to depend on what part(s) of the project graph you
touch: people connected to many parts will want to make coordinating with
hundreds of people incrementally, while people connected to few parts are far
better served by communicating only with the people they need to (communication
famously scales badly).
I think there is another aspect important to mention. Users not
familiar with the LLVM developers tooling will see an official GitHub
project that allows submitting bugs. It is non-obvious that there is
another GitHub project just for bug tracking, especially if there are
multiple of those for different subprojects and even worse if the main
repository itself has an active issue tracker. As with the current
GitHub projects, we already had ~200 issues filed even though we did
not even intent to use GitHub for this.

Michael

On Fri, Apr 24, 2020 at 12:04 PM Tom Stellard via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 04/24/2020 03:24 AM, Sam McCall wrote:
> > clangd's experience using github issues to track bugs (in a
separate
> repo) has been very positive, and I'm glad you're pushing on this!
> >
> > Part of this has been that our issue tracker has been scoped to our
> subproject only, which is a scope that the tool works well for (on the user
> and developer side).
> > As such I don't think we should migrate clangd to a using the
monorepo
> bugtracker. Email subscription to a label is better than nothing, but worse
> than a separate repo.
> > Removing the clangd label from the monorepo bugtracker seems like the
> simplest thing, though I'm happy to work on auto-moving bugs if
that's
> better.
> >
> > (I'd suggest considering the same for other subprojects, though I
know
> that's not a popular opinion here)
>
> I think it's important for everything in the monorepo to use the same
bug
> tracker.
>
> There are advantages to having code in the monorepo (e.g. free
> updates for API changes, a more consistent build experience, etc.).
> But there are also costs, as you have pointed out, like having to use
> a less than ideal bug tracker.  It's really up to sub-projects
> to make the decision about whether these benefits are worth the costs.
> The flang developers have just gone through this process and have
> had to make some sacrifices to get the code in, but ultimately felt the
> sacrifices were worth it.
> I think it hurts the ability of developers and users to collaborate
> effectively,
> if the infrastructure for the project is spread across too many different
> places.
> And good collaboration is key for a project of this size with some many
> tightly
> connected components.
>
+1: seems like clangd here is trying a "in-between" approach in being
halfway into a LLVM project. It was something that was strongly pushed back
against multiple times during the discussions on Flang integration, it
isn't clear to me why we'd get into a different approach with clangd. I
am
really in favor of keeping a cohesion in the project and not having a
"graph of somehow disconnected projects". There might be
sub-optimality
sometimes, but we should address them for everyone instead of one-off
improvements that may benefit one subproject on the short term but I
suspect hurt the project on the long term.

-- 
Mehdi


>
> Getting back to the proposal we are discussing.  Do you have any specific
> feedback
> for improvements that might help make it align better with the kind of
> experience
> the clangd users and developers are looking for?
>
> - Tom
>
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200425/4521b1dd/attachment.html>

On 4/25/20 10:02 PM, Mehdi AMINI via cfe-dev wrote:
>
>
> On Fri, Apr 24, 2020 at 12:04 PM Tom Stellard via llvm-dev 
> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>
>     On 04/24/2020 03:24 AM, Sam McCall wrote:
>     > clangd's experience using github issues to track bugs (in a
>     separate repo) has been very positive, and I'm glad you're
pushing
>     on this!
>     >
>     > Part of this has been that our issue tracker has been scoped to
>     our subproject only, which is a scope that the tool works well for
>     (on the user and developer side).
>     > As such I don't think we should migrate clangd to a using the
>     monorepo bugtracker. Email subscription to a label is better than
>     nothing, but worse than a separate repo.
>     > Removing the clangd label from the monorepo bugtracker seems
>     like the simplest thing, though I'm happy to work on auto-moving
>     bugs if that's better.
>     >
>     > (I'd suggest considering the same for other subprojects,
though
>     I know that's not a popular opinion here)
>
>     I think it's important for everything in the monorepo to use the
>     same bug tracker.
>
>     There are advantages to having code in the monorepo (e.g. free
>     updates for API changes, a more consistent build experience, etc.).
>     But there are also costs, as you have pointed out, like having to use
>     a less than ideal bug tracker.  It's really up to sub-projects
>     to make the decision about whether these benefits are worth the costs.
>     The flang developers have just gone through this process and have
>     had to make some sacrifices to get the code in, but ultimately
>     felt the
>     sacrifices were worth it. 
>
>
>     I think it hurts the ability of developers and users to
>     collaborate effectively,
>     if the infrastructure for the project is spread across too many
>     different places.
>     And good collaboration is key for a project of this size with some
>     many tightly
>     connected components.
>
>
> +1: seems like clangd here is trying a "in-between" approach in
being
> halfway into a LLVM project. It was something that was strongly pushed 
> back against multiple times during the discussions on Flang 
> integration, it isn't clear to me why we'd get into a different 
> approach with clangd. I am really in favor of keeping a cohesion in 
> the project and not having a "graph of somehow disconnected
projects".
> There might be sub-optimality sometimes, but we should address them 
> for everyone instead of one-off improvements that may benefit one 
> subproject on the short term but I suspect hurt the project on the 
> long term.
+1.  Agreed w/Mehdi.
>
> -- 
> Mehdi
>
>
>     Getting back to the proposal we are discussing.  Do you have any
>     specific feedback
>     for improvements that might help make it align better with the
>     kind of experience
>     the clangd users and developers are looking for?
>
>     - Tom
>
>
>
>
>
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>     https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200427/5109ddb8/attachment.html>