Xen devel - Dec 2013 - Re: Developer Dashboards for Xen Project sub-projects (need input)

On 25/11/2013 14:35, Konrad Rzeszutek Wilk wrote:
> . snip..
>> I wanted to start a discussion about this and what would be useful.
>> For the Hypervisor for example, we could track activity on xen.git
>> and osstest.git on master as well as stable branches (not sure
>> whether there is any point in tracking activity on staging
>> branches). There is also stuff they can do such as map
>> activity/authors/<almost anything you may want to ask> to code,
etc.
>> See the example below which maps authors to kernel components.
>>
>> The mailing lists plugins are quite sophisticated apparently (they
>> have support for the Linux kernel and its workflow) and have a lot
>> of filtering and tagging capabilities. For example it should be
>> possible to
>> * Model our code review process and link it to commits (assuming
>> that in the majority of cases there is a mapping between patch
>> series and commit, e.g. via commit message or similar). I believe in
>> our case we do have a mapping which would enable this.
>> * It can in theory handle osstest mails and xenbugs, etc. - although
>> this will probably require customization which will add extra set-up
>> cost
>> * We can track specific keywords in list conversations (e.g. arm,
>> etc. as useful) and create custom views if we want to
>>
>> There is probably a lot more which can be done, but I believe that
>> git activity, code review and list activity are most valuable for
>> now. We may be able to use this for PVOPS and other upstreams too,
>> but usefulness is an open question.
> pvops is mostly Linus upstream. We have some patches - but every
> merge window we flush them out to Linus.
I checked with the vendor and they could run stats on part of a git tree 
(using a git log command to just process what we care
about)
>> We can do similar things for XAPI and Mirage.
>>
>> So what I am looking for is
>> a) a discussion on the list on what might be useful, and
>> b) a number of volunteers (ideally one per project) who will work
>> with me and the vendor getting this off the ground
> I like the idea of getting an idea of ''we are getting less and
less
> commits in this area'' as a way to figure out what needs attention
> (or perhaps no need).
>
> But not every week, not every month either - quaterly is nice enough.
> Or maybe once a year (say at Xen Summit?). But that should be possible
> already right? Or is it a major pain to create those nice graphs?
It is a major pain.
> What is the overall goal?
>
>  From my PoV my feeling is that:
>   a). I need to hire more folks
>   b). I have more things we want to do than there are
>       developers.
I probably have different goals than you would have. For example:
* Creater reports that I can use to demonstrate things are going well 
for the community
* Identifying potential problems in the community
etc.
> And I think the same is true for everybody else. But I don''t know
> if this dashboard will help in this regards?
>
> Or is it a way to use that information to figure out overall
> trends of changes + test coverage -> good/bad patch ratio?
>
Good question: I don''t think this is possible with what they have now. 
If it was possible to model the review process it is conceivable to 
identify components where the reviews take longer than expected (but we 
probably already know this). On the other hand, the xen-devel mailing 
list traffic is increasing (in Nov we had more than 4500 posts, whereas 
most of this year we had 3500). In other words it may be getting 
increasingly hard to keep on top of what is actually going on.

Lars