Xen devel - Nov 2013 - Developer Dashboards for Xen Project sub-projects (need input)

Hi all,

I have been talking to a 3rd party vendor about creating a community 
development dashboard for the Xen Project at OWF and LinuxCon EU and 
found a vendor in the community to fund a beta. This is a relatively 
cheap hosted service which supports most open source infrastructure. The 
data gets sucked into a database and is owned by the project (although 
hosted and supported by the vendor). Any code changes they make for us 
are made available as open source via https://github.com/vizgrimoire.

  * Repo activity
  * Mailing list activity (as well as forums)
  * Code reviews
  * Bug activity (not currently supporting our tracker)
  * IRC
  * Etc.

Examples are:

  * http://activity.openstack.org/dash/newbrowser/browser/
  * http://korma.wmflabs.org/browser/ (mediawiki)

The idea would be to create a dashboard for the Hypervisor, XAPI and 
Mirage projects focussing on repos, lists and code reviews first. We can 
extend out to other areas if this is attractive. This would make my life 
easier and the dashboard has some functionality to spot community problems.

I wanted to start a discussion about this and what would be useful. For 
the Hypervisor for example, we could track activity on xen.git and 
osstest.git on master as well as stable branches (not sure whether there 
is any point in tracking activity on staging branches). There is also 
stuff they can do such as map activity/authors/<almost anything you may 
want to ask> to code, etc. See the example below which maps authors to 
kernel components.

The mailing lists plugins are quite sophisticated apparently (they have 
support for the Linux kernel and its workflow) and have a lot of 
filtering and tagging capabilities. For example it should be possible to
* Model our code review process and link it to commits (assuming that in 
the majority of cases there is a mapping between patch series and 
commit, e.g. via commit message or similar). I believe in our case we do 
have a mapping which would enable this.
* It can in theory handle osstest mails and xenbugs, etc. - although 
this will probably require customization which will add extra set-up cost
* We can track specific keywords in list conversations (e.g. arm, etc. 
as useful) and create custom views if we want to

There is probably a lot more which can be done, but I believe that git 
activity, code review and list activity are most valuable for now. We 
may be able to use this for PVOPS and other upstreams too, but 
usefulness is an open question.

We can do similar things for XAPI and Mirage.

So what I am looking for is
a) a discussion on the list on what might be useful, and
b) a number of volunteers (ideally one per project) who will work with 
me and the vendor getting this off the ground

Regards
Lars
P.S.: The UI generally sucks a little (to be more correct, the 
definitions of what what means are not self explanatory in the UI)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

. snip..
> I wanted to start a discussion about this and what would be useful.
> For the Hypervisor for example, we could track activity on xen.git
> and osstest.git on master as well as stable branches (not sure
> whether there is any point in tracking activity on staging
> branches). There is also stuff they can do such as map
> activity/authors/<almost anything you may want to ask> to code, etc.
> See the example below which maps authors to kernel components.
> 
> The mailing lists plugins are quite sophisticated apparently (they
> have support for the Linux kernel and its workflow) and have a lot
> of filtering and tagging capabilities. For example it should be
> possible to
> * Model our code review process and link it to commits (assuming
> that in the majority of cases there is a mapping between patch
> series and commit, e.g. via commit message or similar). I believe in
> our case we do have a mapping which would enable this.
> * It can in theory handle osstest mails and xenbugs, etc. - although
> this will probably require customization which will add extra set-up
> cost
> * We can track specific keywords in list conversations (e.g. arm,
> etc. as useful) and create custom views if we want to
> 
> There is probably a lot more which can be done, but I believe that
> git activity, code review and list activity are most valuable for
> now. We may be able to use this for PVOPS and other upstreams too,
> but usefulness is an open question.
pvops is mostly Linus upstream. We have some patches - but every
merge window we flush them out to Linus.
> 
> We can do similar things for XAPI and Mirage.
> 
> So what I am looking for is
> a) a discussion on the list on what might be useful, and
> b) a number of volunteers (ideally one per project) who will work
> with me and the vendor getting this off the ground
I like the idea of getting an idea of ''we are getting less and less
commits in this area'' as a way to figure out what needs attention
(or perhaps no need).

But not every week, not every month either - quaterly is nice enough.
Or maybe once a year (say at Xen Summit?). But that should be possible
already right? Or is it a major pain to create those nice graphs?

What is the overall goal?

From my PoV my feeling is that:
 a). I need to hire more folks
 b). I have more things we want to do than there are
     developers.

And I think the same is true for everybody else. But I don''t know
if this dashboard will help in this regards?

Or is it a way to use that information to figure out overall
trends of changes + test coverage -> good/bad patch ratio?

On mar, 2013-11-05 at 15:14 +0000, Lars Kurth wrote:
> I have been talking to a 3rd party vendor about creating a community
> development dashboard for the Xen Project at OWF and LinuxCon EU and
> found a vendor in the community to fund a beta. This is a relatively 
> [..]
> Examples are:
>       * http://activity.openstack.org/dash/newbrowser/browser/
>       * http://korma.wmflabs.org/browser/ (mediawiki)
> 
Wow... How shiny! I like it. :-)
> The idea would be to create a dashboard for the Hypervisor, XAPI and
> Mirage projects focussing on repos, lists and code reviews first. We
> can extend out to other areas if this is attractive. This would make
> my life easier and the dashboard has some functionality to spot
> community problems.
> 
Yep, I only gave the examples above a quick glance, but it looks like
there is a lot of potentially useful information.
> I wanted to start a discussion about this and what would be useful.
>
I think it could very well be.
> We can do similar things for XAPI and Mirage.
> 
> 
> So what I am looking for is 
> a) a discussion on the list on what might be useful, and 
> b) a number of volunteers (ideally one per project) who will work with
> me and the vendor getting this off the ground
> 
Can we have some more info (if you have them yourself) on what it is
that the volunteers would need to do?

I like this thing and, in principle, I''d like to help, but it would be
nice to know what kind of effort is it, to make sure I''ll actually have
time for it, and that it matches my competences.

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On lun, 2013-11-25 at 09:35 -0500, Konrad Rzeszutek Wilk
wrote:
> > We can do similar things for XAPI and Mirage.
> > 
> > So what I am looking for is
> > a) a discussion on the list on what might be useful, and
> > b) a number of volunteers (ideally one per project) who will work
> > with me and the vendor getting this off the ground
> 
> I like the idea of getting an idea of ''we are getting less and
less
> commits in this area'' as a way to figure out what needs attention
> (or perhaps no need).
> 
Agreed.
> But not every week, not every month either - quaterly is nice enough.
> Or maybe once a year (say at Xen Summit?). But that should be possible
> already right? Or is it a major pain to create those nice graphs?
> 
Yes, I also agree that we should pick-up a proper periodicity with which
we look at this, or we risk going mad! :-)
> What is the overall goal?
> 
> From my PoV my feeling is that:
>  a). I need to hire more folks
>  b). I have more things we want to do than there are
>      developers.
> 
> And I think the same is true for everybody else. But I don''t know
> if this dashboard will help in this regards?
> 
Well, I think it goes way beyond that.

Lars is collecting numbers and plotting graphs similar to the ones this
thing is able to produce from some time now, and, honestly, I think they
provide really useful information. They tell much more than just ''there
is more effort needed in this area'', they, if done well, provide
insights about the health of a community that, in my opinion, every Open
Source project should be considering very seriously.

As an example, have a look at his ''community roundup'' from XS
in
Edinburgh:
http://www.xenproject.org/component/allvideoshare/video/latest/xpds13-welcome.html

So, if this allows for getting even more info with even less (or equal)
effort, I think it''s 100% worth it.
> Or is it a way to use that information to figure out overall
> trends of changes + test coverage -> good/bad patch ratio?
> 
That too, I guess. However, I think it''s probably something more like
"using that information to as much as possible avoid the risk of
dying",
as it happened some years ago! :-P :-P

So, as I said in the other mail, provided what would be needed is made a
bit more clear, I''m up for helping with this.

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Wed, Nov 27, 2013 at 03:19:58AM +0100, Dario Faggioli
wrote:
> On lun, 2013-11-25 at 09:35 -0500, Konrad Rzeszutek Wilk wrote:
> > > We can do similar things for XAPI and Mirage.
> > > 
> > > So what I am looking for is
> > > a) a discussion on the list on what might be useful, and
> > > b) a number of volunteers (ideally one per project) who will work
> > > with me and the vendor getting this off the ground
> > 
> > I like the idea of getting an idea of ''we are getting less
and less
> > commits in this area'' as a way to figure out what needs
attention
> > (or perhaps no need).
> > 
> Agreed.
> 
> > But not every week, not every month either - quaterly is nice enough.
> > Or maybe once a year (say at Xen Summit?). But that should be possible
> > already right? Or is it a major pain to create those nice graphs?
> > 
> Yes, I also agree that we should pick-up a proper periodicity with which
> we look at this, or we risk going mad! :-)
> 
> > What is the overall goal?
> > 
> > From my PoV my feeling is that:
> >  a). I need to hire more folks
> >  b). I have more things we want to do than there are
> >      developers.
> > 
> > And I think the same is true for everybody else. But I don''t
know
> > if this dashboard will help in this regards?
> > 
> Well, I think it goes way beyond that.
> 
> Lars is collecting numbers and plotting graphs similar to the ones this
> thing is able to produce from some time now, and, honestly, I think they
> provide really useful information. They tell much more than just
''there
> is more effort needed in this area'', they, if done well, provide
> insights about the health of a community that, in my opinion, every Open
> Source project should be considering very seriously.
> 
> As an example, have a look at his ''community roundup''
from XS in
> Edinburgh:
>
http://www.xenproject.org/component/allvideoshare/video/latest/xpds13-welcome.html
Wow. Slide #8 says that Jan Beulich is quite productive (which
I already knew). His manager should see it :-)
> 
> So, if this allows for getting even more info with even less (or equal)
> effort, I think it''s 100% worth it.
> 
> > Or is it a way to use that information to figure out overall
> > trends of changes + test coverage -> good/bad patch ratio?
> > 
> That too, I guess. However, I think it''s probably something more
like
> "using that information to as much as possible avoid the risk of
dying",
> as it happened some years ago! :-P :-P
> 
> So, as I said in the other mail, provided what would be needed is made a
> bit more clear, I''m up for helping with this.
Fantastic!
> 
> Regards,
> Dario
> 
> -- 
> <<This happens because I choose it to happen!>> (Raistlin
Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>