llvm dev - Sep 2021 - Incorporating LLDB Libraries into Core LLVM Project?

Greetings!

LLDB contains many useful abstractions (notably inside the Utility and Host
modules) which are not specific to the debugger or its CLI/API. We would
like to move some of these out of the LLDB subproject to use them elsewhere
in LLVM.

There is some overlap with functionality provided by LLVM. For example,
both LLVM
<https://github.com/llvm/llvm-project/blob/646299d183ca72cbafd3a2d64629ce8cb3fcdd9d/llvm/lib/Support/Timer.cpp>
and LLDB
<https://github.com/llvm/llvm-project/blob/d480f968ad8b56d3ee4a6b6df5532d485b0ad01e/lldb/include/lldb/Utility/Timer.h>
implement a Timer class. We would like for LLVM projects to make use of the
support libraries of LLDB, without requiring a large undertaking to
deduplicate and generalize all the overlapping functionality. Therefore we
would only move a few desirable components that are unique to LLDB.

As motivating examples:

   -

   the TCPSocket
  
<https://github.com/llvm/llvm-project/blob/d480f968ad8b56d3ee4a6b6df5532d485b0ad01e/lldb/include/lldb/Host/common/TCPSocket.h>
   class would be helpful for new tools such as debuginfod
   -

      It was also suggested for clangd server
     
<https://lists.llvm.org/pipermail/llvm-dev/2019-December/137591.html>,
      although that ended up using gRPC
      -

   The IOObject
  
<https://github.com/llvm/llvm-project/blob/d480f968ad8b56d3ee4a6b6df5532d485b0ad01e/lldb/include/lldb/Utility/IOObject.h>
   hierarchy, which abstracts away from the file descriptor, would be helpful
   for porting LLVM to Fuchsia (which lacks fds)

With these applications in mind, we propose to migrate just the IOObject
hierarchy
<https://lldb.llvm.org/cpp_reference/classlldb__private_1_1IOObject.html>
to the LLVM core lib. We will give some more details on our proposed use of
the TCPSocket class for debuginfod in an RFC to follow shortly, as these
are two separate topics. The associated files could be moved as follows:

   -

   From lldb/source/Host/{platform}/ to llvm/lib/Host/{platform}/
   -

   From lldb/source/Utility/ to  llvm/lib/Utility/

In fact, such a refactor was previously suggested on llvm-dev
<https://lists.llvm.org/pipermail/llvm-dev/2019-December/137591.html> to
allow for the clangd server to reuse LLDB’s TCPSocket, although it looks
like clangd-server ended up going with gRPC. We should choose a different
namespace for the IOObject classes than lldb_private, perhaps llvm_io.

We have a draft phabricator diff <https://reviews.llvm.org/D110362> but
there are several outstanding issues to be resolved.
Finally, we note that an analogous refactor to move clang support code into
LLVM (for reuse by flang) was discussed last year
<https://lists.llvm.org/pipermail/llvm-dev/2020-June/142024.html>. We
expect our proposed refactor will be simpler as the IOObject hierarchy is
not specialized to LLDB. We look forward to hearing comments and
suggestions to help clarify how we can take advantage of LLDB’s useful
treasures elsewhere in LLVM =)

Thank you in advance for comments and suggestions, Noah
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210923/795e59a8/attachment.html>

On 23/09/2021 22:41, Noah Shutty via llvm-dev wrote:
> Greetings!
> 
> LLDB contains many useful abstractions (notably inside the Utility and 
> Host modules) which are not specific to the debugger or its CLI/API. We 
> would like to move some of these out of the LLDB subproject to use them 
> elsewhere in LLVM.
> 
> 
> There is some overlap with functionality provided by LLVM. For example, 
> both LLVM 
>
<https://github.com/llvm/llvm-project/blob/646299d183ca72cbafd3a2d64629ce8cb3fcdd9d/llvm/lib/Support/Timer.cpp>and
> LLDB 
>
<https://github.com/llvm/llvm-project/blob/d480f968ad8b56d3ee4a6b6df5532d485b0ad01e/lldb/include/lldb/Utility/Timer.h>implement
> a Timer class. We would like for LLVM projects to make use of the 
> support libraries of LLDB, without requiring a large undertaking to 
> deduplicate and generalize all the overlapping functionality. Therefore 
> we would only move a few desirable components that are unique to LLDB.
> 
> As motivating examples:
> 
>   *
> 
>     the TCPSocket
>    
<https://github.com/llvm/llvm-project/blob/d480f968ad8b56d3ee4a6b6df5532d485b0ad01e/lldb/include/lldb/Host/common/TCPSocket.h>class
>     would be helpful for new tools such as debuginfod
> 
>       o
> 
>         It was also suggested for clangd server
>        
<https://lists.llvm.org/pipermail/llvm-dev/2019-December/137591.html>,
>         although that ended up using gRPC
> 
>   *
> 
>     The IOObject
>    
<https://github.com/llvm/llvm-project/blob/d480f968ad8b56d3ee4a6b6df5532d485b0ad01e/lldb/include/lldb/Utility/IOObject.h>hierarchy,
>     which abstracts away from the file descriptor, would be helpful for
>     porting LLVM to Fuchsia (which lacks fds)
> 
> With these applications in mind, we propose to migrate just the IOObject 
> hierarchy 
>
<https://lldb.llvm.org/cpp_reference/classlldb__private_1_1IOObject.html>to
> the LLVM core lib. We will give some more details on our proposed use of 
> the TCPSocket class for debuginfod in an RFC to follow shortly, as these 
> are two separate topics. The associated files could be moved as follows:
> 
>   *
> 
>      From lldb/source/Host/{platform}/to llvm/lib/Host/{platform}/
> 
>   *
> 
>      From lldb/source/Utility/to llvm/lib/Utility/
> 
> In fact, such a refactor was previously suggested on llvm-dev 
>
<https://lists.llvm.org/pipermail/llvm-dev/2019-December/137591.html>to
> allow for the clangd server to reuse LLDB’s TCPSocket, although it looks 
> like clangd-server ended up going with gRPC. We should choose a 
> different namespace for the IOObjectclasses than lldb_private, perhaps 
> llvm_io.
> 
> We have a draft phabricator diff
<https://reviews.llvm.org/D110362>but
> there are several outstanding issues to be resolved.
> 
> Finally, we note that an analogous refactor to move clang support code 
> into LLVM (for reuse by flang) was discussed last year 
> <https://lists.llvm.org/pipermail/llvm-dev/2020-June/142024.html>. We
> expect our proposed refactor will be simpler as the IOObject hierarchy 
> is not specialized to LLDB. We look forward to hearing comments and 
> suggestions to help clarify how we can take advantage of LLDB’s useful 
> treasures elsewhere in LLVM =)
> 
> Thank you in advance for comments and suggestions, Noah


Hello Noah, everyone,

first off, let me say that I think that having some kind of a socket 
library in llvm is a great idea.

/However/, I have doubts whether this library should take the form of 
lldb's Socket classes, or even if it can be obtained by 
transforming/refactoring them. LLDB has a very different standards when 
it comes to testing, code quality, and general coding style than the 
llvm core. If this code was judged by the standards for new llvm 
contributions, it wouldn't stand a chance of being accepted (it might 
not even be accepted by current lldb standards, as those have evolved 
over the years).

So, I am wondering, if instead of going the clang/flang route, it might 
not be better to use the json library as an example. Back when the idea 
of a json library was first proposed, there were also suggestions to 
reuse/move the preexisting library in lldb. This idea was eventually 
rejected due to code quality concerns. A new (much better) json library 
was added to llvm instead, and in due time the lldb library was deleted 
in favor of the llvm one.

My impression is that a similar approach would result in a higher 
quality product, and make sure it fits in well with the rest of the llvm 
libraries. I'd be happy to help with the migration of lldb to this new 
implementation.

regards,
Pavel