llvm dev - Oct 2020 - Clang API: any way to use a virtual filesystem ?

Hello,
I am shipping my app (https://ossia.io) with an embedded clang to allow
people to write extensions in C++ at run-time.

For that to work I also ship a SDK with all the required headers, libc++,
etc... which is downloaded & extracted in some folder.

Problem is: while on my linux dev. machine this operation is fast,
apparently on Windows and mac extracting a .zip with a few thousand files
takes the better part of a decade - just downloading the sdk.zip makes my
Safari get stuck for ten minutes as this idiot tries to extract it itself
(see the sdk files there:
https://github.com/ossia/score/releases/tag/v3.0.0-a14-1).

My question is: is it possible to hook something in clang to allow it to go
look for files inside a virtual file system à la
https://github.com/yevgeniy-logachev/vfspp for instance ? This way I would
only download the zip, not extract it, and it'd be just opened in ram when
needed.
Or does it necessarily go through fopen() for includes right now ?

Kind regards,
Jean-Michaël
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201005/177907f5/attachment.html>

On 2020-10-05 09:44, Jean-Michaël Celerier via llvm-dev wrote:
> My question is: is it possible to hook something in clang to allow it to 
> go look for files inside a virtual file system à la 
> https://github.com/yevgeniy-logachev/vfspp for instance ? This way I 
> would only download the zip, not extract it, and it'd be just opened in
> ram when needed.
> Or does it necessarily go through fopen() for includes right now ?
I have a tool where I use libclang (the C API). For the internal 
headers, I've embedded those inside the executable, to avoid the need to 
ship them separately or have the users to install them.

What I do is for each header file I create a `CXUnsavedFile` [1].  Then 
I can pass that to `clang_parseTranslationUnit` when parsing the source 
files.

`CXUnsavedFile` expects the filename and the content of the file. If you 
can get that, you should be able to use the same approach.

If you're using the C++ API, you can have a look at the implementation 
of `clang_parseTranslationUnit` to see how to do the same using the C++ API.

[1] https://clang.llvm.org/doxygen/structCXUnsavedFile.html
[2] 
https://clang.llvm.org/doxygen/group__CINDEX__TRANSLATION__UNIT.html#ga2baf83f8c3299788234c8bce55e4472e

-- 
/Jacob Carlborg

Thanks, those are very good starting points ! I'll try to make it work and
report if it does if that can be useful to someone else.

All the best,
Jean-Michaël


-------
Jean-Michaël Celerier
http://www.jcelerier.name


On Mon, Oct 5, 2020 at 6:01 PM Jacob Carlborg via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 2020-10-05 09:44, Jean-Michaël Celerier via llvm-dev wrote:
>
> > My question is: is it possible to hook something in clang to allow it
to
> > go look for files inside a virtual file system à la
> > https://github.com/yevgeniy-logachev/vfspp for instance ? This way I
> > would only download the zip, not extract it, and it'd be just
opened in
> > ram when needed.
> > Or does it necessarily go through fopen() for includes right now ?
>
> I have a tool where I use libclang (the C API). For the internal
> headers, I've embedded those inside the executable, to avoid the need
to
> ship them separately or have the users to install them.
>
> What I do is for each header file I create a `CXUnsavedFile` [1].  Then
> I can pass that to `clang_parseTranslationUnit` when parsing the source
> files.
>
> `CXUnsavedFile` expects the filename and the content of the file. If you
> can get that, you should be able to use the same approach.
>
> If you're using the C++ API, you can have a look at the implementation
> of `clang_parseTranslationUnit` to see how to do the same using the C++
> API.
>
> [1] https://clang.llvm.org/doxygen/structCXUnsavedFile.html
> [2]
>
>
https://clang.llvm.org/doxygen/group__CINDEX__TRANSLATION__UNIT.html#ga2baf83f8c3299788234c8bce55e4472e
>
> --
> /Jacob Carlborg
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201005/520acca2/attachment.html>