What do you expect the support for Windows to be? Certainly, I don't expect you to provide Windows support personally if you don't need it, but given that LLVM supports Windows, it should at least be done in such a way that the design lends itself to interested parties contributing Windows support. Currently clang-cl has several dependencies on having a Visual Studio installation present on your machine, and one of these is because to provide an implementation of the CRT (i.e. libc). So having a libc implementation which supports Windows and is compatible with MSVCRT would be useful for people using clang on Windows as well. On Mon, Jun 24, 2019 at 3:38 PM Jake Ehrlich via llvm-dev <llvm-dev at lists.llvm.org> wrote:> > disclaimer: I work at Google so don't take my +1 as an independent vote forward. > > We would like to use this on Fuchsia and I am particularly interested in creating a dynamic linking library for ELF with Roland McGrath's guidance. We spoke about creating a library for writing dynamic linkers internally and I don't see why this can't be upstreamed. > > On Fuchsia we critically need support for AArch64; What do you expect to be architecture dependent? I struggled to think of where the architecture and not the operating system was the issue. > > On Mon, Jun 24, 2019 at 3:23 PM Siva Chandra via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> Hello LLVM Developers, >> >> >> Within Google, we have a growing range of needs that existing libc implementations don't quite address. This is pushing us to start working on a new libc implementation. >> >> >> Informal conversations with others within the LLVM community has told us that a libc in LLVM is actually a broader need, and we are increasingly consolidating our toolchains around LLVM. Hence, we wanted to see if the LLVM project would be interested in us developing this upstream as part of the project. >> >> >> To be very clear: we don't expect our needs to exactly match everyone else's -- part of our impetus is to simplify things wherever we can, and that may not quite match what others want in a libc. That said, we do believe that the effort will still be directly beneficial and usable for the broader LLVM community, and may serve as a starting point for others in the community to flesh out an increasingly complete set of libc functionality. >> >> >> We are still in the early stages, but we do have some high-level goals and guiding principles of the initial scope we are interested in pursuing: >> >> >> The project should mesh with the "as a library" philosophy of the LLVM project: even though "the C Standard Library" is nominally "a library," most implementations are, in practice, quite monolithic. >> >> The libc should support static non-PIE and static-PIE linking. This means, providing the CRT (the C runtime) and a PIE loader for static non-PIE and static-PIE linked executables. >> >> If there is a specification, we should follow it. The scope that we need includes most of the C Standard Library; POSIX additions; and some necessary, system-specific extensions. This does not mean we should (or can) follow the entire specification -- there will be some parts which simply aren't worth implementing, and some parts which cannot be safely used in modern coding practice. >> >> Vendor extensions must be considered very carefully, and only admitted when necessary. Similar to Clang and libc++, it does seem inevitable that we will need to provide some level of compatibility with other vendors' extensions. >> >> The project should be an exemplar of developing with LLVM tooling. Two examples are fuzz testing from the start, and sanitizer-supported testing. >> >> >> There are also few areas which we do not intend to invest in at this point: >> >> >> Implement dynamic loading and linking support. >> >> Support for more architectures (we'll start with just x86-64 for simplicity). >> >> >> For these areas, the community is of course free to contribute. Our hope is that, preserving the "as a library" design philosophy will make such extensions easy, and allow retaining the simplicity when these features aren't needed. >> >> >> We intend to build the new libc in a gradual manner. To begin with, the new libc will be a layer sitting between the application and the system libc. Eventually, when the implementation is sufficiently complete, it will be able to replace the system libc at least for some use cases and contexts. >> >> >> So, what do you think about incorporating this new libc under the LLVM project? >> >> >> Thank you, >> >> Siva Chandra and the rest of the Google LLVM contributors >> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
On Mon, Jun 24, 2019 at 3:43 PM Zachary Turner <zturner at roblox.com> wrote:> What do you expect the support for Windows to be? Certainly, I don't > expect you to provide Windows support personally if you don't need it, > but given that LLVM supports Windows, it should at least be done in > such a way that the design lends itself to interested parties > contributing Windows support. >We are not going to disallow support for an item/features we do not plan to implement ourselves. Contributions will be welcome. As I have mentioned in another email, we really want to develop everything in a "as a library" fashion so that adding support for new items/features isn't blocked by design.> Currently clang-cl has several dependencies on having a Visual Studio > installation present on your machine, and one of these is because to > provide an implementation of the CRT (i.e. libc). So having a libc > implementation which supports Windows and is compatible with MSVCRT > would be useful for people using clang on Windows as well. > > On Mon, Jun 24, 2019 at 3:38 PM Jake Ehrlich via llvm-dev > <llvm-dev at lists.llvm.org> wrote: > > > > disclaimer: I work at Google so don't take my +1 as an independent vote > forward. > > > > We would like to use this on Fuchsia and I am particularly interested in > creating a dynamic linking library for ELF with Roland McGrath's guidance. > We spoke about creating a library for writing dynamic linkers internally > and I don't see why this can't be upstreamed. > > > > On Fuchsia we critically need support for AArch64; What do you expect to > be architecture dependent? I struggled to think of where the architecture > and not the operating system was the issue. > > > > On Mon, Jun 24, 2019 at 3:23 PM Siva Chandra via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> > >> Hello LLVM Developers, > >> > >> > >> Within Google, we have a growing range of needs that existing libc > implementations don't quite address. This is pushing us to start working on > a new libc implementation. > >> > >> > >> Informal conversations with others within the LLVM community has told > us that a libc in LLVM is actually a broader need, and we are increasingly > consolidating our toolchains around LLVM. Hence, we wanted to see if the > LLVM project would be interested in us developing this upstream as part of > the project. > >> > >> > >> To be very clear: we don't expect our needs to exactly match everyone > else's -- part of our impetus is to simplify things wherever we can, and > that may not quite match what others want in a libc. That said, we do > believe that the effort will still be directly beneficial and usable for > the broader LLVM community, and may serve as a starting point for others in > the community to flesh out an increasingly complete set of libc > functionality. > >> > >> > >> We are still in the early stages, but we do have some high-level goals > and guiding principles of the initial scope we are interested in pursuing: > >> > >> > >> The project should mesh with the "as a library" philosophy of the LLVM > project: even though "the C Standard Library" is nominally "a library," > most implementations are, in practice, quite monolithic. > >> > >> The libc should support static non-PIE and static-PIE linking. This > means, providing the CRT (the C runtime) and a PIE loader for static > non-PIE and static-PIE linked executables. > >> > >> If there is a specification, we should follow it. The scope that we > need includes most of the C Standard Library; POSIX additions; and some > necessary, system-specific extensions. This does not mean we should (or > can) follow the entire specification -- there will be some parts which > simply aren't worth implementing, and some parts which cannot be safely > used in modern coding practice. > >> > >> Vendor extensions must be considered very carefully, and only admitted > when necessary. Similar to Clang and libc++, it does seem inevitable that > we will need to provide some level of compatibility with other vendors' > extensions. > >> > >> The project should be an exemplar of developing with LLVM tooling. Two > examples are fuzz testing from the start, and sanitizer-supported testing. > >> > >> > >> There are also few areas which we do not intend to invest in at this > point: > >> > >> > >> Implement dynamic loading and linking support. > >> > >> Support for more architectures (we'll start with just x86-64 for > simplicity). > >> > >> > >> For these areas, the community is of course free to contribute. Our > hope is that, preserving the "as a library" design philosophy will make > such extensions easy, and allow retaining the simplicity when these > features aren't needed. > >> > >> > >> We intend to build the new libc in a gradual manner. To begin with, > the new libc will be a layer sitting between the application and the system > libc. Eventually, when the implementation is sufficiently complete, it will > be able to replace the system libc at least for some use cases and contexts. > >> > >> > >> So, what do you think about incorporating this new libc under the LLVM > project? > >> > >> > >> Thank you, > >> > >> Siva Chandra and the rest of the Google LLVM contributors > >> > >> > >> _______________________________________________ > >> LLVM Developers mailing list > >> llvm-dev at lists.llvm.org > >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190625/30d04174/attachment.html>
Not that right here at this exact moment is the right place to discuss this but a secondary email to discuss and gather requirements for an operating system abstraction layer seems to be required then. We don't want the implementation to be coupled too tightly with Linux if we want to support BSD, Windows, and Fuchsia as well. I also have hopes that hobbyist operating system developers could use this. Libc implementations for hobby OS projects were a pain point for me personally. On Tue, Jun 25, 2019 at 4:20 PM Siva Chandra <sivachandra at google.com> wrote:> On Mon, Jun 24, 2019 at 3:43 PM Zachary Turner <zturner at roblox.com> wrote: > >> What do you expect the support for Windows to be? Certainly, I don't >> expect you to provide Windows support personally if you don't need it, >> but given that LLVM supports Windows, it should at least be done in >> such a way that the design lends itself to interested parties >> contributing Windows support. >> > > We are not going to disallow support for an item/features we do not plan > to implement ourselves. Contributions will be welcome. > > As I have mentioned in another email, we really want to develop everything > in a "as a library" fashion so that adding support for new items/features > isn't blocked by design. > > >> Currently clang-cl has several dependencies on having a Visual Studio >> installation present on your machine, and one of these is because to >> provide an implementation of the CRT (i.e. libc). So having a libc >> implementation which supports Windows and is compatible with MSVCRT >> would be useful for people using clang on Windows as well. >> >> On Mon, Jun 24, 2019 at 3:38 PM Jake Ehrlich via llvm-dev >> <llvm-dev at lists.llvm.org> wrote: >> > >> > disclaimer: I work at Google so don't take my +1 as an independent vote >> forward. >> > >> > We would like to use this on Fuchsia and I am particularly interested >> in creating a dynamic linking library for ELF with Roland McGrath's >> guidance. We spoke about creating a library for writing dynamic linkers >> internally and I don't see why this can't be upstreamed. >> > >> > On Fuchsia we critically need support for AArch64; What do you expect >> to be architecture dependent? I struggled to think of where the >> architecture and not the operating system was the issue. >> > >> > On Mon, Jun 24, 2019 at 3:23 PM Siva Chandra via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >> >> >> Hello LLVM Developers, >> >> >> >> >> >> Within Google, we have a growing range of needs that existing libc >> implementations don't quite address. This is pushing us to start working on >> a new libc implementation. >> >> >> >> >> >> Informal conversations with others within the LLVM community has told >> us that a libc in LLVM is actually a broader need, and we are increasingly >> consolidating our toolchains around LLVM. Hence, we wanted to see if the >> LLVM project would be interested in us developing this upstream as part of >> the project. >> >> >> >> >> >> To be very clear: we don't expect our needs to exactly match everyone >> else's -- part of our impetus is to simplify things wherever we can, and >> that may not quite match what others want in a libc. That said, we do >> believe that the effort will still be directly beneficial and usable for >> the broader LLVM community, and may serve as a starting point for others in >> the community to flesh out an increasingly complete set of libc >> functionality. >> >> >> >> >> >> We are still in the early stages, but we do have some high-level goals >> and guiding principles of the initial scope we are interested in pursuing: >> >> >> >> >> >> The project should mesh with the "as a library" philosophy of the LLVM >> project: even though "the C Standard Library" is nominally "a library," >> most implementations are, in practice, quite monolithic. >> >> >> >> The libc should support static non-PIE and static-PIE linking. This >> means, providing the CRT (the C runtime) and a PIE loader for static >> non-PIE and static-PIE linked executables. >> >> >> >> If there is a specification, we should follow it. The scope that we >> need includes most of the C Standard Library; POSIX additions; and some >> necessary, system-specific extensions. This does not mean we should (or >> can) follow the entire specification -- there will be some parts which >> simply aren't worth implementing, and some parts which cannot be safely >> used in modern coding practice. >> >> >> >> Vendor extensions must be considered very carefully, and only admitted >> when necessary. Similar to Clang and libc++, it does seem inevitable that >> we will need to provide some level of compatibility with other vendors' >> extensions. >> >> >> >> The project should be an exemplar of developing with LLVM tooling. Two >> examples are fuzz testing from the start, and sanitizer-supported testing. >> >> >> >> >> >> There are also few areas which we do not intend to invest in at this >> point: >> >> >> >> >> >> Implement dynamic loading and linking support. >> >> >> >> Support for more architectures (we'll start with just x86-64 for >> simplicity). >> >> >> >> >> >> For these areas, the community is of course free to contribute. Our >> hope is that, preserving the "as a library" design philosophy will make >> such extensions easy, and allow retaining the simplicity when these >> features aren't needed. >> >> >> >> >> >> We intend to build the new libc in a gradual manner. To begin with, >> the new libc will be a layer sitting between the application and the system >> libc. Eventually, when the implementation is sufficiently complete, it will >> be able to replace the system libc at least for some use cases and contexts. >> >> >> >> >> >> So, what do you think about incorporating this new libc under the LLVM >> project? >> >> >> >> >> >> Thank you, >> >> >> >> Siva Chandra and the rest of the Google LLVM contributors >> >> >> >> >> >> _______________________________________________ >> >> LLVM Developers mailing list >> >> llvm-dev at lists.llvm.org >> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> > >> > _______________________________________________ >> > LLVM Developers mailing list >> > llvm-dev at lists.llvm.org >> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190625/f5bbec22/attachment.html>
The main concern I have is that Windows is so different from everything else that there is a high likelihood of decisions being baked in early on that make things very difficult for people to come along later and contribute a Windows implementation. This happened with sanitizers for example (lack of support for weak functions on Windows), LLDB (posix api calls scattered throughout the codebase), and I worry with libc it will be even more difficult to correctly design the abstraction because we have to deal with executable file format, syscalls, operating system loaders, and various linkage models. The most immediate thing I think we will run into is that you mentioned wanting this to take shape as something that sits in between system libc and application. Given that Windows' libc and other versions of libc are so different, I expect this to lead to some interesting problems. Can you elaborate more on how you envision this working with llvm libc in between application and system libc? On Tue, Jun 25, 2019 at 4:20 PM Siva Chandra <sivachandra at google.com> wrote:> > On Mon, Jun 24, 2019 at 3:43 PM Zachary Turner <zturner at roblox.com> wrote: >> >> What do you expect the support for Windows to be? Certainly, I don't >> expect you to provide Windows support personally if you don't need it, >> but given that LLVM supports Windows, it should at least be done in >> such a way that the design lends itself to interested parties >> contributing Windows support. > > > We are not going to disallow support for an item/features we do not plan to implement ourselves. Contributions will be welcome. > > As I have mentioned in another email, we really want to develop everything in a "as a library" fashion so that adding support for new items/features isn't blocked by design. > >> >> Currently clang-cl has several dependencies on having a Visual Studio >> installation present on your machine, and one of these is because to >> provide an implementation of the CRT (i.e. libc). So having a libc >> implementation which supports Windows and is compatible with MSVCRT >> would be useful for people using clang on Windows as well. >> >> On Mon, Jun 24, 2019 at 3:38 PM Jake Ehrlich via llvm-dev >> <llvm-dev at lists.llvm.org> wrote: >> > >> > disclaimer: I work at Google so don't take my +1 as an independent vote forward. >> > >> > We would like to use this on Fuchsia and I am particularly interested in creating a dynamic linking library for ELF with Roland McGrath's guidance. We spoke about creating a library for writing dynamic linkers internally and I don't see why this can't be upstreamed. >> > >> > On Fuchsia we critically need support for AArch64; What do you expect to be architecture dependent? I struggled to think of where the architecture and not the operating system was the issue. >> > >> > On Mon, Jun 24, 2019 at 3:23 PM Siva Chandra via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> >> >> Hello LLVM Developers, >> >> >> >> >> >> Within Google, we have a growing range of needs that existing libc implementations don't quite address. This is pushing us to start working on a new libc implementation. >> >> >> >> >> >> Informal conversations with others within the LLVM community has told us that a libc in LLVM is actually a broader need, and we are increasingly consolidating our toolchains around LLVM. Hence, we wanted to see if the LLVM project would be interested in us developing this upstream as part of the project. >> >> >> >> >> >> To be very clear: we don't expect our needs to exactly match everyone else's -- part of our impetus is to simplify things wherever we can, and that may not quite match what others want in a libc. That said, we do believe that the effort will still be directly beneficial and usable for the broader LLVM community, and may serve as a starting point for others in the community to flesh out an increasingly complete set of libc functionality. >> >> >> >> >> >> We are still in the early stages, but we do have some high-level goals and guiding principles of the initial scope we are interested in pursuing: >> >> >> >> >> >> The project should mesh with the "as a library" philosophy of the LLVM project: even though "the C Standard Library" is nominally "a library," most implementations are, in practice, quite monolithic. >> >> >> >> The libc should support static non-PIE and static-PIE linking. This means, providing the CRT (the C runtime) and a PIE loader for static non-PIE and static-PIE linked executables. >> >> >> >> If there is a specification, we should follow it. The scope that we need includes most of the C Standard Library; POSIX additions; and some necessary, system-specific extensions. This does not mean we should (or can) follow the entire specification -- there will be some parts which simply aren't worth implementing, and some parts which cannot be safely used in modern coding practice. >> >> >> >> Vendor extensions must be considered very carefully, and only admitted when necessary. Similar to Clang and libc++, it does seem inevitable that we will need to provide some level of compatibility with other vendors' extensions. >> >> >> >> The project should be an exemplar of developing with LLVM tooling. Two examples are fuzz testing from the start, and sanitizer-supported testing. >> >> >> >> >> >> There are also few areas which we do not intend to invest in at this point: >> >> >> >> >> >> Implement dynamic loading and linking support. >> >> >> >> Support for more architectures (we'll start with just x86-64 for simplicity). >> >> >> >> >> >> For these areas, the community is of course free to contribute. Our hope is that, preserving the "as a library" design philosophy will make such extensions easy, and allow retaining the simplicity when these features aren't needed. >> >> >> >> >> >> We intend to build the new libc in a gradual manner. To begin with, the new libc will be a layer sitting between the application and the system libc. Eventually, when the implementation is sufficiently complete, it will be able to replace the system libc at least for some use cases and contexts. >> >> >> >> >> >> So, what do you think about incorporating this new libc under the LLVM project? >> >> >> >> >> >> Thank you, >> >> >> >> Siva Chandra and the rest of the Google LLVM contributors >> >> >> >> >> >> _______________________________________________ >> >> LLVM Developers mailing list >> >> llvm-dev at lists.llvm.org >> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> > >> > _______________________________________________ >> > LLVM Developers mailing list >> > llvm-dev at lists.llvm.org >> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev