freebsd stable - Oct 2014 - Automatically running /usr/tests on stable/10 branch under Jenkins

Hi,

I have created this Jenkins job, which you can see a graphical
representation of:

https://jenkins.freebsd.org/jenkins/view/FreeBSD_src_stable/job/FreeBSD_stable_10/848/BuildGraph/

(1)  does a buildworld/buildkernel on amd64 when someone checks new
      code into the stable/10 branch
(2)  Creates a bootable UFS image with makefs
(3)  Boots the image under bhyve
(4)  Runs these commands inside the bhyve VM:

      cd /usr/tests
      kyua test
      kyua report-junit --output=test-output.xml

(5)  Shuts down the bhyve VM
(6)  imports test-output.xml into Jenkins.

You can see a full test report here:

https://jenkins.freebsd.org/jenkins/job/FreeBSD_stable_10-tests/4/testReport/

We already do the same thing for CURRENT.

Hopefully by running the tests regularly, we can help improve the quality
of FreeBSD.


--
Craig

On Oct 12, 2014, at 23:14, Craig Rodrigues <rodrigc at FreeBSD.org> wrote:
> Hi,
> 
> I have created this Jenkins job, which you can see a graphical
> representation of:
> 
>
https://jenkins.freebsd.org/jenkins/view/FreeBSD_src_stable/job/FreeBSD_stable_10/848/BuildGraph/
?

Hi Craig!
	As much as everyone would like to take i386 out to pasture, there?s a large
degree of value in running i386 tests on 11-CURRENT and 10-STABLE (I?ve caught
some interesting build bugs and test bugs by running on my i386/CURRENT VM). Are
there any plans to have i386 executors running tests anytime soon (does bhyve
support i386?)?
Thanks!
-Garrett
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20141012/2329aa62/attachment.sig>

On Sun, Oct 12, 2014 at 11:14:45PM -0700, Craig Rodrigues
wrote:
> Hi,
> 
> I have created this Jenkins job, which you can see a graphical
> representation of:
> 
>
https://jenkins.freebsd.org/jenkins/view/FreeBSD_src_stable/job/FreeBSD_stable_10/848/BuildGraph/
> 
It is noted below stuff is being done this for current as well and in
that light:
> (1)  does a buildworld/buildkernel on amd64 when someone checks new
>       code into the stable/10 branch
Is not this excessive? I would suggest every N commits or M hours if
there were not enough commits to reach given threshold. Of course no
work in there were no commits whatsoever, but then do the work after the
first which gets in.
> (2)  Creates a bootable UFS image with makefs
any chance zfs will be used as well?
> (3)  Boots the image under bhyve
> (4)  Runs these commands inside the bhyve VM:
> 
>       cd /usr/tests
>       kyua test
>       kyua report-junit --output=test-output.xml
would be nice to run some kind of stress testing. buildworld with a high
-j is an example of a general purpose test. This could be done with
different frequency than regular tests.
> 
> (5)  Shuts down the bhyve VM
Do you have crashdumps configured in case stuff goes wrong?
> (6)  imports test-output.xml into Jenkins.
> 
> You can see a full test report here:
> 
>
https://jenkins.freebsd.org/jenkins/job/FreeBSD_stable_10-tests/4/testReport/
> 
> We already do the same thing for CURRENT.
> 
> Hopefully by running the tests regularly, we can help improve the quality
> of FreeBSD.
> 
> 
> --
> Craig
> _______________________________________________
> freebsd-virtualization at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe
at freebsd.org"
-- 
Mateusz Guzik <mjguzik gmail.com>