On Thu, Jun 27, 2019 at 5:19 PM JF Bastien <jfbastien at apple.com> wrote:> On Jun 27, 2019, at 5:05 PM, Siva Chandra <sivachandra at google.com> wrote: > On Wed, Jun 26, 2019 at 10:27 AM JF Bastien <jfbastien at apple.com> wrote: > > 3. If there is a specification, we should follow it. The scope that we need includes most of the C Standard Library; POSIX additions; and some necessary, system-specific extensions. This does not mean we should (or can) follow the entire specification -- there will be some parts which simply aren't worth implementing, and some parts which cannot be safely used in modern coding practice. > > > I’d love to hear what you have in mind with point 3 above, and see it expanded. libc++ implements C++11 and subsequent standards, and that makes me wonder: > > Which standards would this libc implement? > > > We need parts of the C standard library, parts of the POSIX > extensions, and also the linux headers. The community is of course > free to widen the surface as needed. > > > Which standard specifically? So far the responses sound like “the standard Google uses”.I was of the opinion that you were asking me to elaborate point #3 of mine from above.> I don’t think that's a good objective for such a project. For practical purposes that’s the implementation approach that makes sense to start with, but I’m looking for what the charter of this LLVM project should be.I want to refrain from talking as if this libc project has already been accepted by the LLVM. But yes, if this libc project is indeed accepted and takes off, we will definitely want a charter written down for this as an LLVM project. And I also agree that this charter cannot limit itself to Google's use cases.> Compare with libc++: https://libcxx.llvm.orgYes. Our aspirations for this libc are to be like libc++.> I think you want to fill out a proposed set of documentation pages, like libc++’s, and answer the questions libc++ answers. Not where you’ll start or in what order (though that’s useful for this discussion!), but what your proposed libc aspires to be.Absolutely!> Same as above, IMO an LLVM project should aspire to something bigger, even if practical concerns guide the initial implementation.Again, I want to wait for some sort of confirmation that we can actually start work on this as an LLVM project.> Personally I’m really interested in a project that increases the quality of all C libraries, and of the C standard. I therefore think champions of this project signing up to collaborate with WG14 is important.I do not disagree. At the same time, I am of the opinion that such a champion should grow out of this project rather than getting volunteered or nominated. This is my personal opinion and I am ready to be corrected.> I think you need write a design for how this C library will be tested.I can assure you that all this will happen once we take off.> I suggest you have a chat with Marshall Clow (CC’ed). He does a lot of really good work with libc++ and the C++ Standards Committee. I’d like this C library to be similar to libc++ in many ways, and I’d like a leader like Marshall involved in leading this C library. Talking to Marshall will help understand the type of leadership I’d like to see in this project.Experienced guidance is most welcome. And, thanks a lot for bringing up everything you have done in this email. I also apologize for the delay in my response, so thanks for your patience as well.
I think I now understand some of the disconnect you and I are having, and I think some of the pushback you’re getting from the community is the same. You’re talking about where you want to start with an LLVM libc. Many in the community (myself included) want to understand where we’ll get with this libc. At steady-state, what does it do? To a certain degree I don’t care about how you get to the steady state: sure the implementation approach is important, and which contributor cares about what parts is important in shaping that evolution, but at the end of the day what matters is where you get. So here’s what’s missing: there’s no goal. Right now, your proposal is “let’s do an LLVM libc, starting with what I care about, who’s interested?” That’s an OK place to start! You illustrated your needs, others chimed in with theirs, and now you know there’s some interest. However, you should take time now to come up with a plan. What’s this libc actually going to be? I ask a bunch of questions below that I think you need to answer as a next step. Others asked more questions which I didn’t echo, but which you should answer as well. What does this libc aspire to become? More below:> On Jun 27, 2019, at 10:29 PM, Siva Chandra <sivachandra at google.com> wrote: > > On Thu, Jun 27, 2019 at 5:19 PM JF Bastien <jfbastien at apple.com <mailto:jfbastien at apple.com>> wrote: >> On Jun 27, 2019, at 5:05 PM, Siva Chandra <sivachandra at google.com> wrote: >> On Wed, Jun 26, 2019 at 10:27 AM JF Bastien <jfbastien at apple.com> wrote: >> >> 3. If there is a specification, we should follow it. The scope that we need includes most of the C Standard Library; POSIX additions; and some necessary, system-specific extensions. This does not mean we should (or can) follow the entire specification -- there will be some parts which simply aren't worth implementing, and some parts which cannot be safely used in modern coding practice. >> >> >> I’d love to hear what you have in mind with point 3 above, and see it expanded. libc++ implements C++11 and subsequent standards, and that makes me wonder: >> >> Which standards would this libc implement? >> >> >> We need parts of the C standard library, parts of the POSIX >> extensions, and also the linux headers. The community is of course >> free to widen the surface as needed. >> >> >> Which standard specifically? So far the responses sound like “the standard Google uses”. > > I was of the opinion that you were asking me to elaborate point #3 of > mine from above.I indeed was. I’d like a list of C / POSIX standards the library will try to conform to. e.g. libc++ only really implements C++11 and later standards, some libstdc++ compatibility extensions, some experimental stuff from TSes, and a handful of other things. What’s your list?>> I don’t think that's a good objective for such a project. For practical purposes that’s the implementation approach that makes sense to start with, but I’m looking for what the charter of this LLVM project should be. > > I want to refrain from talking as if this libc project has already > been accepted by the LLVM. But yes, if this libc project is indeed > accepted and takes off, we will definitely want a charter written down > for this as an LLVM project. And I also agree that this charter cannot > limit itself to Google's use cases.You want a charter, before the project is accepted.>> Compare with libc++: https://libcxx.llvm.org <https://libcxx.llvm.org/> > > Yes. Our aspirations for this libc are to be like libc++. > >> I think you want to fill out a proposed set of documentation pages, like libc++’s, and answer the questions libc++ answers. Not where you’ll start or in what order (though that’s useful for this discussion!), but what your proposed libc aspires to be. > > Absolutely! > >> Same as above, IMO an LLVM project should aspire to something bigger, even if practical concerns guide the initial implementation. > > Again, I want to wait for some sort of confirmation that we can > actually start work on this as an LLVM project.You have enough tentative support and interested contributors to warrant writing down a plan.>> Personally I’m really interested in a project that increases the quality of all C libraries, and of the C standard. I therefore think champions of this project signing up to collaborate with WG14 is important. > > I do not disagree. At the same time, I am of the opinion that such a > champion should grow out of this project rather than getting > volunteered or nominated. This is my personal opinion and I am ready > to be corrected.Having this kind of champion is really important for an LLVM libc. I’m not sure I’d support such a project without such a person. As you come up with a plan, consider who that should be. Maybe it’s you :-)>> I think you need write a design for how this C library will be tested. > > I can assure you that all this will happen once we take off.You want a plan before it takes off. Testing standardized stuff has enough precedent that you should be able to look at what others have done, and come up with a plan up front. I really like that you want to fuzz, use sanitizers, etc. That’s pretty novel for this kind of project. Basic standards testing isn’t novel, so it should be pretty easy to figure out.>> I suggest you have a chat with Marshall Clow (CC’ed). He does a lot of really good work with libc++ and the C++ Standards Committee. I’d like this C library to be similar to libc++ in many ways, and I’d like a leader like Marshall involved in leading this C library. Talking to Marshall will help understand the type of leadership I’d like to see in this project. > > Experienced guidance is most welcome. And, thanks a lot for bringing > up everything you have done in this email. I also apologize for the > delay in my response, so thanks for your patience as well.No worries! You’ve got a lot of responses, and that’s good. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190628/7634af7e/attachment.html>
On Fri, Jun 28, 2019 at 9:29 AM JF Bastien <jfbastien at apple.com> wrote:> > I think I now understand some of the disconnect you and I are having, and I think some of the pushback you’re getting from the community is the same. You’re talking about where you want to start with an LLVM libc. Many in the community (myself included) want to understand where we’ll get with this libc. At steady-state, what does it do? To a certain degree I don’t care about how you get to the steady state: sure the implementation approach is important, and which contributor cares about what parts is important in shaping that evolution, but at the end of the day what matters is where you get. > > So here’s what’s missing: there’s no goal. Right now, your proposal is “let’s do an LLVM libc, starting with what I care about, who’s interested?” > > That’s an OK place to start! You illustrated your needs, others chimed in with theirs, and now you know there’s some interest. However, you should take time now to come up with a plan. What’s this libc actually going to be? I ask a bunch of questions below that I think you need to answer as a next step. Others asked more questions which I didn’t echo, but which you should answer as well. What does this libc aspire to become?After my first step, my first email to this thread, I was waiting for someone to drive me towards a process. Your email now has given me sufficient guidance on how to proceed forward. So thank you for that. In the coming days, I will start sharing/discussing the information you are expecting.
On Fri, Jun 28, 2019 at 9:29 AM JF Bastien <jfbastien at apple.com> wrote:> > I think I now understand some of the disconnect you and I are having, and I think some of the pushback you’re getting from the community is the same. You’re talking about where you want to start with an LLVM libc. Many in the community (myself included) want to understand where we’ll get with this libc. At steady-state, what does it do? To a certain degree I don’t care about how you get to the steady state: sure the implementation approach is important, and which contributor cares about what parts is important in shaping that evolution, but at the end of the day what matters is where you get. > > So here’s what’s missing: there’s no goal. Right now, your proposal is “let’s do an LLVM libc, starting with what I care about, who’s interested?” > > That’s an OK place to start! You illustrated your needs, others chimed in with theirs, and now you know there’s some interest. However, you should take time now to come up with a plan. What’s this libc actually going to be? I ask a bunch of questions below that I think you need to answer as a next step. Others asked more questions which I didn’t echo, but which you should answer as well. What does this libc aspire to become?I apologize for the delay. I will try to address the above questions in this email. I will shortly follow up with answers to other questions. Below is a write up which I think would qualify as the "charter" for the new libc. It is also answering questions like, "where we’ll get with this libc?", "what's this libc actually going to be?" and similar ones. I have used libcxx.llvm.org landing page as a template to write it down. ############################################### "llvm-libc" C Standard Library ======================= llvm-libc is an implementation of the C standard library targeting C11 and above. It also provides platform specific extensions as relevant. For example, on Linux it also provides pthreads, librt and other POSIX extension libraries. Documentation =========== The llvm-libc project is still in the planning phase. Stay tuned for updates soon. Features and Goals =============== * C11 and upwards conformant. * A modular libc with individual pieces implemented in the "as a library" philosophy of the LLVM project. * Ability to layer this libc over the system libc. * Provide C symbols as specified by the standards, but take advantage and use C++ language facilities for the core implementation. * Provides POSIX extensions on POSIX compliant platforms. * Provides system-specific extensions as appropriate. For example, provides the Linux API on Linux. * Vendor extensions if and only if necessary. * Designed and developed from the start to work with LLVM tooling and testing like fuzz testing and sanitizer-supported testing. * ABI independent implementation as far as possible. * Use source based implementations as far possible rather than assembly. Will try to “fix” the compiler rather than use assembly language workarounds. Why a new C Standard Library? ======================== Implementing a libc is no small task and is not be taken lightly. A natural question to ask is, "why a new implementation of the C standard library?" There is no single answer to this question, but some of the major reasons are as follows: * Most libc implementations are monolithic. It is a non-trivial porting task to pick and choose only the pieces relevant to one's platform. The new libc will be developed with sufficient modularity to make picking and choosing a straightforward task. * Most libc implementations break when built with sanitizer specific compiler options. The new libc will be developed from the start to work with those specialized compiler options. * The new libc will be developed to support and employ fuzz testing from the start. * Most libc implementations use a good amount of assembly language, and assume specific ABIs (may be platform dependent). With the new libc implementation, we want to use normal source code as much as possible so that compiler-based changes to the ABI are easy. Moreover, as part of the LLVM project, we want to use this opportunity to fix performance related compiler bugs rather than using assembly workarounds. * A large hole in the llvm toolchain will be plugged with this new libc. With the broad platform expertise in the LLVM community, and the strong license and project structure, we think that the new libc will be more tunable and robust, without sacrificing the simplicity and accessibility typical of the LLVM project. Platform Support ============= llvm-libc development is still in the planning phase. However, we envision that it will support a variety of platforms in the coming years. Interested parties are encouraged to participate in the design and implementation, and add support for their favorite platforms. Current Status =========== llvm-libc development is still in the planning phase. Build Bots ======== Coming soon. Get involved! ========== First please review our Developer's Policy. Stay tuned for llvm-libc specific information. Design Documents ============== Coming soon.
On Fri, Jun 28, 2019 at 9:29 AM JF Bastien <jfbastien at apple.com> wrote:> Personally I’m really interested in a project that increases the quality of all C libraries, and of the C standard. I therefore think champions of this project signing up to collaborate with WG14 is important. > Having this kind of champion is really important for an LLVM libc. I’m not sure I’d support such a project without such a person. As you come up with a plan, consider who that should be. Maybe it’s you :-)When the need arises, I do not mind being a "champion" like this. To begin with though, we (as in the team at Google I am representing) do not intend to participate beyond what we already do (like the C++ committee). Let me point out that I said "to begin with". So, depending on how things evolve, we might in future increase our participation with the committees. Personally, it feels like it is early days - before one goes to the committee, they should first develop some experience implementing the standard library. If there is already one such person in the community, and they would like to take the lead and engage with the committee from the start, it would be most welcome. I would only be a hand-waving participant if I were to do it today.> I think you need write a design for how this C library will be tested. > You want a plan before it takes off. Testing standardized stuff has enough precedent that you should be able to look at what others have done, and come up with a plan up front. I really like that you want to fuzz, use sanitizers, etc. That’s pretty novel for this kind of project. Basic standards testing isn’t novel, so it should be pretty easy to figure out.Beyond fuzz and sanitizer based testing, at a general level, this would be covered: 1. Extensive unit testing. 2. Standards conformance testing. 3. If relevant and possible, differential testing: We want to be able to test llvm-libc against another battle-tested libc. This is essentially to understand how we differ from other libcs. 4. If relevant and possible, test against the testsuite of an existing libc implementation. One could go into details here, but I think it is best to take them up on a case-by-case basis: when we are implementing X, we will discuss what exact kind of testing makes sense for X.