Thanks Keith. We checked, and indeed libopenblas is not linked against libomp nor libgomp. We suspect this is because we used conda to install R and OpenBLAS. So I guess we should be barking up the conda tree instead? By the way, I also noticed on my home machine (Ubuntu), /usr/lib/libopenblas.so.0 is also not linked against those, for what that's worth. Regards, Ben On 01/10/2018 12:04 AM, Keith O'Hara wrote:> Check if libopenblas is linked against libomp or libgomp. > > I?d be curious to see any errors that arise when an OpenMP version of OpenBLAS is linked with R. > > Keith > > >> On Jan 9, 2018, at 11:01 PM, Benjamin Tyner <btyner at gmail.com> wrote: >> >> I didn't do the compile; is there a way to check whether that was used? If not, I'll inquire with our sysadmin and report back. >> >> In any case, my suggestion was motivated by the fact that some parts of R use OpenMP while others do not, in the hope that the former could have their OpenBLAS omelet without breaking the OpenMP eggs, so to speak. >> >> >> On 01/09/2018 06:41 PM, Keith O'Hara wrote: >>> Do those issues still arise when OpenBLAS is compiled with USE_OPENMP=1 ? >>> >>> Keith >>> >>>> On Jan 9, 2018, at 6:03 PM, Benjamin Tyner <btyner at gmail.com> wrote: >>>> >>>> Please pardon my ignorance, but doesn't OpenBLAS still not always play nicely with multi-threaded OpenMP? (for example, don't race conditions sometimes crop up)? If so, it might be nice to have the ability to temporarily disable multi-threaded OpenMP (effectively: omp_set_num_threads(1)) for the duration of operations using OpenBLAS. >>>> >>>> Regards >>>> Ben >>>> >>>>> Julia using OpenBLAS is *very* reassuring. >>>>> >>>>> I agree that having it included as an options(...) feature should be OK. >>>>> >>>>> On Sun, Dec 17, 2017, 3:22 PM Juan Telleria <jtelleriar at gmail.com <https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote: >>>>> >>>>>> /Julia Programming Language uses also OpenBlas, and it is actively />/maintained with bugs being fixed as I have checked it out: />//>/http://www.openblas.net/Changelog.txt />//>/So I still see it ok to be included as an options(...) feature (by default />/off, just for safety), over other Blas libraries. />//>/R could not use Intel MKL for legal reasons (I think), because as long />/that R ships with GPL libraries, shipping R by default with Non-GPL is />/illegal. />//>/Cheers, />/Juan />//>/El 17/12/2017 2:50 a. m., "Avraham Adler" <avraham.adler at gmail.com <https://stat.ethz.ch/mailman/listinfo/r-devel>> />/escribi?: />//>>/On Sat, Dec 16, 2017 at 7:41 PM, Kenny Bell <kmbell56 at gmail.com <https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote: />>/> It seems like many of the multi-threaded BLASes have some sort of />>/> fundamental problem preventing use in the way Juan suggests: />>/> />>/> - Dirk's vignette states that ATLAS "fixes the number of cores used at />>/> compile-time and cannot vary this setting at run-time", so any />>/> user-friendly implementation for R would have to compile ATLAS for 1-16 />>/> threads to allow the user to switch at run-time. This might dramatically />>/> affect install times. />>/> />>/> - MKL seems like it's been outright rejected in the past based on not />>/> being "free-enough". />>/> />>/> - OpenBLAS causes crashes. />>/> />>/> Has anyone tried ExBLAS for use with R? />>/> />>/> On Sun, Dec 17, 2017 at 1:03 PM, Peter Langfelder < />>/> peter.langfelder at gmail.com <https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote: />>/> />>/>> I would be very cautious about OpenBLAS in particular... from time to />>/>> time I get complains from users that compiled code calculations in my />>/>> WGCNA package crash or produce wrong answers with large data, and they />>/>> all come from OpenBLAS users. I am yet to reproduce any of their />>/>> crashes when using MKL and ATLAS BLAS implementations. />>/>> />>/>> Just my 2 cents... />>/>> />>/>> Peter />>//>>/I've been building R on Windows 64 bit with OpenBLAS for years and my />>/builds pass check-devel. For a while in the past it failed one check />>/as the tolerance was 5e-5 and with my build of OpenBLAS the error was />>/5.4e-5 or 5.7e-5, but that was changed around R 3.3, if I recall />>/correctly. I provide descriptions here [1], but I haven't gone so far />>/as to post compiled Rblas.dlls just yet. My personal build sets 4 />>/threads when compiling OpenBLAS itself as I'm currently on a quad-core />>/SandyBridge. In tests I ran a few years ago, both single and multi />>/threaded BLAS compile and then can be compiled into R with no issues />>/(on my platforms, at least). Most matrix operations performed better />>/with multi-threaded except for R's eigenvalue decomposition, to the />>/nest of my recollection. />>//>>/Avi />>//>>/[1] https://www.avrahamadler.com/r-tips/build-openblas-for-windows-r64/ />>//>>/______________________________________________ />>/R-devel at r-project.org <https://stat.ethz.ch/mailman/listinfo/r-devel> mailing list />>/https://stat.ethz.ch/mailman/listinfo/r-devel />>//>// >>>>> [[alternative HTML version deleted]] >>>> ______________________________________________ >>>> R-devel at r-project.org mailing list >>>> https://stat.ethz.ch/mailman/listinfo/r-devel >>
I?m not really familiar with conda, but if they?re being packaged together then an omp build might be more appropriate. Perhaps another point for Juan?s list: whether OpenBLAS is the right choice to pair with. The library itself hasn?t produced optimized kernels for any of the Intel *Lake chips yet; might be worth considering its near- and long-term future (vs something else). Keith> On Jan 10, 2018, at 8:24 PM, Benjamin Tyner <btyner at gmail.com> wrote: > > Thanks Keith. We checked, and indeed libopenblas is not linked against libomp nor libgomp. We suspect this is because we used conda to install R and OpenBLAS. So I guess we should be barking up the conda tree instead? > > By the way, I also noticed on my home machine (Ubuntu), /usr/lib/libopenblas.so.0 is also not linked against those, for what that's worth. > > Regards, > Ben > > On 01/10/2018 12:04 AM, Keith O'Hara wrote: >> Check if libopenblas is linked against libomp or libgomp. >> >> I?d be curious to see any errors that arise when an OpenMP version of OpenBLAS is linked with R. >> >> Keith >> >> >>> On Jan 9, 2018, at 11:01 PM, Benjamin Tyner <btyner at gmail.com> wrote: >>> >>> I didn't do the compile; is there a way to check whether that was used? If not, I'll inquire with our sysadmin and report back. >>> >>> In any case, my suggestion was motivated by the fact that some parts of R use OpenMP while others do not, in the hope that the former could have their OpenBLAS omelet without breaking the OpenMP eggs, so to speak. >>> >>> >>> On 01/09/2018 06:41 PM, Keith O'Hara wrote: >>>> Do those issues still arise when OpenBLAS is compiled with USE_OPENMP=1 ? >>>> >>>> Keith >>>> >>>>> On Jan 9, 2018, at 6:03 PM, Benjamin Tyner <btyner at gmail.com> wrote: >>>>> >>>>> Please pardon my ignorance, but doesn't OpenBLAS still not always play nicely with multi-threaded OpenMP? (for example, don't race conditions sometimes crop up)? If so, it might be nice to have the ability to temporarily disable multi-threaded OpenMP (effectively: omp_set_num_threads(1)) for the duration of operations using OpenBLAS. >>>>> >>>>> Regards >>>>> Ben >>>>> >>>>>> Julia using OpenBLAS is *very* reassuring. >>>>>> >>>>>> I agree that having it included as an options(...) feature should be OK. >>>>>> >>>>>> On Sun, Dec 17, 2017, 3:22 PM Juan Telleria <jtelleriar at gmail.com <https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote: >>>>>> >>>>>>> /Julia Programming Language uses also OpenBlas, and it is actively />/maintained with bugs being fixed as I have checked it out: />//>/http://www.openblas.net/Changelog.txt />//>/So I still see it ok to be included as an options(...) feature (by default />/off, just for safety), over other Blas libraries. />//>/R could not use Intel MKL for legal reasons (I think), because as long />/that R ships with GPL libraries, shipping R by default with Non-GPL is />/illegal. />//>/Cheers, />/Juan />//>/El 17/12/2017 2:50 a. m., "Avraham Adler" <avraham.adler at gmail.com <https://stat.ethz.ch/mailman/listinfo/r-devel>> />/escribi?: />//>>/On Sat, Dec 16, 2017 at 7:41 PM, Kenny Bell <kmbell56 at gmail.com <https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote: />>/> It seems like many of the multi-threaded BLASes have some sort of />>/> fundamental problem preventing use in the way Juan suggests: />>/> />>/> - Dirk's vignette states that ATLAS "fixes the number of cores used at />>/> compile-time and cannot vary this setting at run-time", so any />>/> user-friendly implementation for R would have to compile ATLAS for 1-16 />>/> threads to allow the user to switch at run-time. This might dramatically />>/> affect install times. />>/> />>/> - MKL seems like it's been outright rejected in the past based on not />>/> being "free-enough". />>/> />>/> - OpenBLAS causes crashes. />>/> />>/> Has anyone tried ExBLAS for use with R? />>/> />>/> On Sun, Dec 17, 2017 at 1:03 PM, Peter Langfelder < />>/> peter.langfelder at gmail.com <https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote: />>/> />>/>> I would be very cautious about OpenBLAS in particular... from time to />>/>> time I get complains from users that compiled code calculations in my />>/>> WGCNA package crash or produce wrong answers with large data, and they />>/>> all come from OpenBLAS users. I am yet to reproduce any of their />>/>> crashes when using MKL and ATLAS BLAS implementations. />>/>> />>/>> Just my 2 cents... />>/>> />>/>> Peter />>//>>/I've been building R on Windows 64 bit with OpenBLAS for years and my />>/builds pass check-devel. For a while in the past it failed one check />>/as the tolerance was 5e-5 and with my build of OpenBLAS the error was />>/5.4e-5 or 5.7e-5, but that was changed around R 3.3, if I recall />>/correctly. I provide descriptions here [1], but I haven't gone so far />>/as to post compiled Rblas.dlls just yet. My personal build sets 4 />>/threads when compiling OpenBLAS itself as I'm currently on a quad-core />>/SandyBridge. In tests I ran a few years ago, both single and multi />>/threaded BLAS compile and then can be compiled into R with no issues />>/(on my platforms, at least). Most matrix operations performed better />>/with multi-threaded except for R's eigenvalue decomposition, to the />>/nest of my recollection. />>//>>/Avi />>//>>/[1] https://www.avrahamadler.com/r-tips/build-openblas-for-windows-r64/ />>//>>/______________________________________________ />>/R-devel at r-project.org <https://stat.ethz.ch/mailman/listinfo/r-devel> mailing list />>/https://stat.ethz.ch/mailman/listinfo/r-devel />>//>// >>>>>> [[alternative HTML version deleted]] >>>>> ______________________________________________ >>>>> R-devel at r-project.org mailing list >>>>> https://stat.ethz.ch/mailman/listinfo/r-devel >>> > >
On Thu, Jan 11, 2018 at 10:38 AM, Keith O'Hara <keith.ohara at nyu.edu> wrote: [snip]> > Perhaps another point for Juan?s list: whether OpenBLAS is the right choice to pair with. The library itself hasn?t produced optimized kernels for any of the Intel *Lake chips yet; might be worth considering its near- and long-term future (vs something else). >Regarding this point, please see this thread on OpenBLAS-users [1], in specific the third post by Jeff Hammond who says he is an employee of Intel, where he says: "Skylake Xeon processors with AVX-512 are definitely going to require code changes to perform optimally. However, the Core i[357] processors up to and including Kaby Lake do not support AVX-512 and thus can suffice with the existing AVX2 implementation that targets Haswell. See https://ark.intel.com/#@Processors and look for "Instruction Set Extensions" for full details on on specific SKUs." I presume this holds for CoffeeLake as well, as it's pretty much a hexacore SkyLake. Thank you, Avi [1] https://groups.google.com/d/msg/openblas-users/XU6-9h-geVE/mwz2ewKrCQAJ