Dan Liew
2015-Jul-07 03:35 UTC
[LLVMdev] [cfe-dev] 3.6.2-rc1 has been tagged. Testers needed.
Hi Ben, Thanks for taking the time to look at this.>> It's not necessary for CMake to be installed when building with the >> Autoconf/Makefile build system. If you look in the build directory >> after a build has been performed you should see in cmake/modules >> >> ``` >> LLVMBuildExports.cmake LLVMConfig.cmake LLVMConfigVersion.cmake >> LLVMExports.cmake Makefile >> ```` > > > I have these for the test-release build, but only in the *.obj directory, > not the install directory.Okay. So that means the files are being generated during the build but not installed for some reason>> The ``LLVMConfig.cmake`` and ``LLVMConfigExports.cmake`` file are >> generated by the build in this directory and are later installed >> (along with a bunch of other files). When I do a build of LLVM on Arch >> Linux and install it in the install directory there is a >> ``share/llvm/cmake/`` directory and it contains the following files >> (this is LLVM trunk rather than 3.6.2 but it should be very similar). >> >> ``` >> AddLLVM.cmake AddOCaml.cmake ChooseMSVCCRT.cmake >> FindOCaml.cmake GetSVN.cmake HandleLLVMStdlib.cmake >> LLVMConfig.cmake LLVMExports.cmake TableGen.cmake >> AddLLVMDefinitions.cmake AddSphinxTarget.cmake CrossCompile.cmake >> FindSphinx.cmake HandleLLVMOptions.cmake LLVM-Config.cmake >> LLVMConfigVersion.cmake LLVMProcessSources.cmake >> ``` > > I have these in the install directory of my cmake based trunk build.Yes those will definitely existing if you build LLVM using the CMake build system but since LLVM 3.5 the Autoconf/Makefile build system also generates and installs CMake files into the LLVM install tree. Perhaps you could do a clean build of LLVM using the Autoconf/Makefile build system outside of your chroot to see if there's something about your chroot environment causing the problem?>> You're going to have to do some debugging on your end because I cannot >> reproduce what ends up your tarball. I guess a first step would be to >> try building LLVM in your chroot but outside of the test-release.sh >> script. You could then try... >> >> 1. After doing a complete build have the >> ``cmake/modules/LLVMConfig.cmake`` and >> ``cmake/modules/LLVMExports.cmake`` files been generated in the build >> directory? If the CMake files are missing if you run ``make`` in the >> ``cmake/modules/`` directory the files should be generated. Running >> this manually should not be necessary though, it should happen >> automatically. >> SIDENOTE: After doing an initial configure the ``cmake/modules`` >> directory (and it's corresponding Makefile) will not exist if the >> build is out of source. The directory and the makefile will exist >> after doing a successful build. > > Yeah, I have those.Okay and those existed after a build without you needing to run ``make`` inside the ``cmake/modules/`` directory in the build tree?>> 2. Assuming the ``cmake/modules/LLVMConfig.cmake`` and >> ``cmake/modules/LLVMExports.cmake`` files were generated in the >> previous step, what happens when you run ``make install`` in the >> ``cmake/modules`` directory? Do the files actually get installed? > > Now I have the share/llvm/cmake directory with 18 files.Okay so I am right in understanding that after completing a build... - When you ran ``make install`` from the root of the build tree the CMake files are not installed into ``share/llvm/cmake``. - After trying the above you ran ``make install`` in the ``cmake/modules`` directory in the build tree and the CMake files were installed to ``share/llvm/cmake`` ?> In fact, I ran install from the root of the build directory and not a load > of .a and a few .so files appeared in *.install/libI don't quite understand what you mean. Could you explain that again? Are you saying that you ran ``make install`` again in the root of the build tree and additional libraries were installed? If so that doesn't good. Sounds like a bug. @Eric : Have you ever seen anything like this before? Thanks, Dan.
Ben Pope
2015-Jul-07 03:49 UTC
[LLVMdev] [cfe-dev] 3.6.2-rc1 has been tagged. Testers needed.
On Tuesday, July 07, 2015 11:35 AM, Dan Liew wrote:> Hi Ben, > > Thanks for taking the time to look at this. > >>> It's not necessary for CMake to be installed when building with the >>> Autoconf/Makefile build system. If you look in the build directory >>> after a build has been performed you should see in cmake/modules >>> >>> ``` >>> LLVMBuildExports.cmake LLVMConfig.cmake LLVMConfigVersion.cmake >>> LLVMExports.cmake Makefile >>> ```` >> >> I have these for the test-release build, but only in the *.obj directory, >> not the install directory. > Okay. So that means the files are being generated during the build but > not installed for some reason > >>> The ``LLVMConfig.cmake`` and ``LLVMConfigExports.cmake`` file are >>> generated by the build in this directory and are later installed >>> (along with a bunch of other files). When I do a build of LLVM on Arch >>> Linux and install it in the install directory there is a >>> ``share/llvm/cmake/`` directory and it contains the following files >>> (this is LLVM trunk rather than 3.6.2 but it should be very similar). >>> >>> ``` >>> AddLLVM.cmake AddOCaml.cmake ChooseMSVCCRT.cmake >>> FindOCaml.cmake GetSVN.cmake HandleLLVMStdlib.cmake >>> LLVMConfig.cmake LLVMExports.cmake TableGen.cmake >>> AddLLVMDefinitions.cmake AddSphinxTarget.cmake CrossCompile.cmake >>> FindSphinx.cmake HandleLLVMOptions.cmake LLVM-Config.cmake >>> LLVMConfigVersion.cmake LLVMProcessSources.cmake >>> ``` >> I have these in the install directory of my cmake based trunk build. > Yes those will definitely existing if you build LLVM using the CMake > build system but since LLVM 3.5 the Autoconf/Makefile build system > also generates and installs CMake files into the LLVM install tree. > Perhaps you could do a clean build of LLVM using the Autoconf/Makefile > build system outside of your chroot to see if there's something about > your chroot environment causing the problem?OK, I'll have a look, but it will have to be later.>>> You're going to have to do some debugging on your end because I cannot >>> reproduce what ends up your tarball. I guess a first step would be to >>> try building LLVM in your chroot but outside of the test-release.sh >>> script. You could then try... >>> >>> 1. After doing a complete build have the >>> ``cmake/modules/LLVMConfig.cmake`` and >>> ``cmake/modules/LLVMExports.cmake`` files been generated in the build >>> directory? If the CMake files are missing if you run ``make`` in the >>> ``cmake/modules/`` directory the files should be generated. Running >>> this manually should not be necessary though, it should happen >>> automatically. >>> SIDENOTE: After doing an initial configure the ``cmake/modules`` >>> directory (and it's corresponding Makefile) will not exist if the >>> build is out of source. The directory and the makefile will exist >>> after doing a successful build. >> Yeah, I have those. > Okay and those existed after a build without you needing to run > ``make`` inside the ``cmake/modules/`` directory in the build tree? > >>> 2. Assuming the ``cmake/modules/LLVMConfig.cmake`` and >>> ``cmake/modules/LLVMExports.cmake`` files were generated in the >>> previous step, what happens when you run ``make install`` in the >>> ``cmake/modules`` directory? Do the files actually get installed? >> Now I have the share/llvm/cmake directory with 18 files. > Okay so I am right in understanding that after completing a build... > > - When you ran ``make install`` from the root of the build tree the > CMake files are not installed into ``share/llvm/cmake``. > - After trying the above you ran ``make install`` in the > ``cmake/modules`` directory in the build tree and the CMake files were > installed to ``share/llvm/cmake`` > > ? > >> In fact, I ran install from the root of the build directory and not a load >> of .a and a few .so files appeared in *.install/lib > I don't quite understand what you mean. Could you explain that again? > Are you saying that you ran ``make install`` again in the root of the > build tree and additional > libraries were installed? > > If so that doesn't good. Sounds like a bug.Sorry. After I ran test-release, I didn't have the share/llvm/cmake directory. When I ran make install from the root of the build directory, I got that directory and files. It looks like it touched the .a and .so files, but I don't think new ones were created. Ben
Dan Liew
2015-Jul-07 05:26 UTC
[LLVMdev] [cfe-dev] 3.6.2-rc1 has been tagged. Testers needed.
Hi, @CC'ing Hans because this will likely be of interest to you. Right I've started trying to build LLVM inside an Ubuntu chroot (Ubuntu 14.04LTS Docker image) and I've already come across a pretty bad bug in the ``test-release.sh`` script which potentially means that builds and/or tests could potentially fail without anyone noticing (unless someone carefully looks through the logs) because in certain places if the build process fails the script will carry on executing as if nothing bad happened! The script contains ``set -e`` which is **supposed** to exit if any command fails. However because of the way the script is implemented this doesn't work. When commands are piped through the ``tee`` command (which the script does for many parts of the build process) the exit code is not exit code of the process of interest (e.g. the ``make`` or ``configure`` invocations). Here's a simple example that illustrates the problem ``` $ false ; echo $? 1 false | tee /dev/null ; echo $? 0 ``` In addition to doing ``set -e`` what we also need to do ``set -o pipefail`` in the ``test-release.sh`` script. ``` $ set -o pipefail $ false ; echo $? 1 $ false | tee /dev/null ; echo $? 1 ``` Is it okay for me to commit a fix for this to trunk? @Ben: Could you add a line just after ``set -e`` in your copy of ``test-release.sh`` that runs ``set -o pipefail`` and try another build using this patched script? This might not be the cause of the packaging problems you're having but I suspect that it might be the case that your build is failing in someway inside your chroot and that the ``test-release.sh`` script is just carrying on regardless of this failure resulting in a broken tarball being generated. Thanks, Dan.