There was a short discussion in the comments of PR6696 about Linux
testing.
The 2.7 release is slipping primarily because there are x86-32 problems to
fix (there is also a Darwin problem and some other issues).
We have two problems with our testing:
The OSU/Google buildbots aren't running test-suite, apparently. I
didn't
realize that and as this is the public testing face for LLVM, I think this
needs to be fixed. I don't know if that's in Daniel's new testing
plan or
not but that's a pretty large hole in what those outside Apple are able to
see regularly.
The second problem is the lack of x86-32 testing on Linux. This is only
going to get worse as x86-32 desktops and servers continue to disappear.
Pretty much the only way you'll get an x86-32 these days is by purchasing an
Atom netbook or laptop. Unfortunately, these kinds of devices don't suit
themselves well to testing as they frequently aren't connected to the
'net.
I have a netbook I can add to the testing pool. If enough other people do the
same perhaps we'll have at least a few up at any one time.
Another alternative is to add a buildbot recipe that builds x86-32 targets on
x86-64 machines. Is this already available? If so, it seems we should
already be running sufficient x86-32 testing. As we apparently don't have
enough such testing, I assume we don't have this kind of recipe available
yet.
I think most Linux x86 distributions ship multilib compilers so this should
be possible without too much trouble.
Thoughts?
-Dave
On 04/06/2010 09:52 AM, David Greene wrote:> The second problem is the lack of x86-32 testing on Linux. This is only > going to get worse as x86-32 desktops and servers continue to disappear. > > Pretty much the only way you'll get an x86-32 these days is by purchasing an > Atom netbook or laptop. Unfortunately, these kinds of devices don't suit > themselves well to testing as they frequently aren't connected to the 'net.While x86-32 disappears, virtualization becomes ubiquitous along with x86-64. What about running your tests in x86-32 guests under KVM or Xen? That seems like it would solve everything on the infrastructure you already have. You can run any guest OS you want, too.> I have a netbook I can add to the testing pool. If enough other people do the > same perhaps we'll have at least a few up at any one time.I've bothered the list with stupid questions about my toy project before and plan to do so much more in the future as my real LLVM project gets moving, so I ought to see if I can donate some cycles. I know nothing about your testing infrastructure, but I do have a 32-bit laptop that is only getting occasional use. How hard would it be to put it to use at least until you ship? Dustin
On Tuesday 06 April 2010 13:08:25 Dustin Laurence wrote:> On 04/06/2010 09:52 AM, David Greene wrote: > > The second problem is the lack of x86-32 testing on Linux. This is only > > going to get worse as x86-32 desktops and servers continue to disappear. > > > > Pretty much the only way you'll get an x86-32 these days is by purchasing > > an Atom netbook or laptop. Unfortunately, these kinds of devices don't > > suit themselves well to testing as they frequently aren't connected to > > the 'net. > > While x86-32 disappears, virtualization becomes ubiquitous along with > x86-64. What about running your tests in x86-32 guests under KVM or > Xen? That seems like it would solve everything on the infrastructure > you already have. You can run any guest OS you want, too.That's an even better idea than running x86-32 on x86-64 because it better emulates a whole 32-bit environment. I'll see if I can set up such a beast on my box at home. Unfortunately, KVM+Qemu has been pretty flaky for me.> > I have a netbook I can add to the testing pool. If enough other people > > do the same perhaps we'll have at least a few up at any one time. > > I've bothered the list with stupid questions about my toy project before > and plan to do so much more in the future as my real LLVM project gets > moving, so I ought to see if I can donate some cycles. I know nothing > about your testing infrastructure, but I do have a 32-bit laptop that is > only getting occasional use. How hard would it be to put it to use at > least until you ship?It's not hard. Basically, you set up a buildbot and then contact Daniel to exchange unique identifiers and passwords that he enters into the buildbot server and you set up on the client. -Dave
On Apr 6, 2010, at 9:52 AM, David Greene wrote:> There was a short discussion in the comments of PR6696 about Linux > testing. > > The 2.7 release is slipping primarily because there are x86-32 problems to > fix (there is also a Darwin problem and some other issues). >While lack of linux testing of the testsuite is a problem, this is not why the release is slipping. We need more people to fix bugs. It can't be the same people fixing the bugs for every release. We need more people in the community to help and I can't see any way of sugar coating this message. -Tanya P.S. I'm not discounting the need for testing the testsuite on Linux. Its not 32 bit specific though.
Hi David,> The second problem is the lack of x86-32 testing on Linux. This is only > going to get worse as x86-32 desktops and servers continue to disappear.this is not really a problem if you have root access on an x86-64 box. It is quite easy to install a 32 bit chroot environment. That's what I'm using on my laptop for testing x86-32. You really can't tell it's not the real thing. The problem I have is that I don't have root access to the machines I'm using for running testers, and the administrator doesn't want to set up such a 32 bit environment.> Another alternative is to add a buildbot recipe that builds x86-32 targets on > x86-64 machines. Is this already available? If so, it seems we should > already be running sufficient x86-32 testing. As we apparently don't have > enough such testing, I assume we don't have this kind of recipe available yet. > I think most Linux x86 distributions ship multilib compilers so this should > be possible without too much trouble.I am working on such a recipe for testing 32 bit dragonegg, but so far it fails mysteriously and I didn't figure it out yet. Ciao, Duncan.
On Tuesday 06 April 2010 14:12:04 Duncan Sands wrote:> Hi David, > > > The second problem is the lack of x86-32 testing on Linux. This is only > > going to get worse as x86-32 desktops and servers continue to disappear. > > this is not really a problem if you have root access on an x86-64 box. It > is quite easy to install a 32 bit chroot environment. That's what I'm > using on my laptop for testing x86-32. You really can't tell it's not the > real thing. The problem I have is that I don't have root access to the > machines I'm using for running testers, and the administrator doesn't want > to set up such a 32 bit environment.Yes, this is doable, but it's complicated. To get adequate testing we need to make it as simple as possible for new users to participate. Either we need a step-by-step recipe or we need a simpler approach. Either this or virtualization would work but both are complicated to set up, raising the bar to participate. A chroot doesn't run a 32-bit kernel so virtualization would emulate the 32-bit environment slightly better. But I don't know if that's all that significant in the end.> > Another alternative is to add a buildbot recipe that builds x86-32 > > targets on x86-64 machines. Is this already available? If so, it seems > > we should already be running sufficient x86-32 testing. As we apparently > > don't have enough such testing, I assume we don't have this kind of > > recipe available yet. I think most Linux x86 distributions ship multilib > > compilers so this should be possible without too much trouble. > > I am working on such a recipe for testing 32 bit dragonegg, but so far it > fails mysteriously and I didn't figure it out yet.Great! It strikes me that we also have an issue with gcc version. As demonstrated by PR6616, compiler versions matter. It seems we will need to settle on a standard compiler for each release to avoid some of the confusion that can delay releases. What do the buildbots use? Unfortunately, this again makes things more complicated as it requires testers to install specific versions of compilers. Linux distributions don't always make that easy. But one step at a time. Getting more 32-bit Linux testing is the first step. -Dave
On 04/06/2010 11:45 AM, Tanya Lattner wrote:> While lack of linux testing of the testsuite is a problem, this is > not why the release is slipping. We need more people to fix bugs. It > can't be the same people fixing the bugs for every release. > > We need more people in the community to help and I can't see any way > of sugar coating this message.I don't know enough yet to help there, sorry. I think anything I could fix without knowing LLVM internals would already be fixed. Maybe down the road.... Dustin