via llvm-dev
2021-Aug-26 10:32 UTC
[llvm-dev] [cfe-dev] Location (in Source Code) of String Literal in MacroExpansion
Oh, now I see why you were puzzled!
`printf()` expects a null-terminated string (“%s”), but you pass an
llvm::StringRef instead, which is not null terminated!
Thus, you will end up with undefined behavior, which ends up printing till the
end of the file.
I hope this is it!
Balazs
From: benicsbalazs at gmail.com <benicsbalazs at gmail.com>
Sent: 2021. augusztus 26., csütörtök 12:25
To: 'chiasa.men' <chiasa.men at web.de>
Cc: llvm-dev at lists.llvm.org
Subject: RE: [cfe-dev] Location (in Source Code) of String Literal in
MacroExpansion
Hi Chiasa,
For the example you attached the MacroExpansionContext seems to work as I
intended.
It records what range gets replaced by what tokens at the end of the
preprocessing, thus we get
`CONCATMACRO("TEST")` -> ` L"TEST"`
About `Lexer::getSourceText()` I’m not sure. I haven’t used that. The `at()`
function was designed as a helper defining the unit tests. I might misused
something when I created this code & tests, let me know!
Balazs
From: cfe-dev <cfe-dev-bounces at lists.llvm.org <mailto:cfe-dev-bounces
at lists.llvm.org> > On Behalf Of chiasa.men via cfe-dev
Sent: 2021. augusztus 26., csütörtök 10:03
To: cfe-dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org> >
Cc: llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
Subject: [cfe-dev] Location (in Source Code) of String Literal in MacroExpansion
https://gist.github.com/T35R6braPwgDJKq/6439fda090e4ee8440d726f8dafa4dbb#file-mectest-cpp
Above I modified the MacroExpansionContext unittest.
The relevant code:
//--------------
TEST_F(MacroExpansionContextTest, Custom) {
// From the GCC website.
const auto Ctx = getMacroExpansionContextFor(R"code(
#define CONCATMACRO(parm) L##parm
int f(wchar_t *b){}
int main()
{
int a= f(CONCATMACRO("TEST"));
int b= f(L"TEST");
return 0;
}
)code");
Ctx->dumpExpansionRanges();
//> :6:14, :6:33
Ctx->dumpExpandedTexts();
//> :6:14 -> 'L"TEST"'
printf("Exp: %s\n",Ctx->getExpandedText(at(6, 14)).getValue());
//Exp: L"TEST"
printf("Org: %s\n",Ctx->getOriginalText(at(6, 14)).getValue());
//Org: CONCATMACRO("TEST"));
// int b= f(L"TEST");
// return 0;
//}
StringRef sourceText =
clang::Lexer::getSourceText(CharSourceRange(SourceRange(at(6, 14),at(6,
33)),false),SourceMgr, LangOpts);
printf("sourceText: %s\n",sourceText);
// sourceText: CONCATMACRO("TEST"));
// int b= f(L"TEST");
// return 0;
// }
}
//--------------
I am interested in getting the range for "TEST" in the SourceText.
Since the CONCATMACRO is preprocessed to L"TEST" I thought the
MacroExpansionContext would be of help.
It tells about the ExpansionRanges: 6:14, :6:33 which is
CONCATMACRO("TEST")
What I would need is: 6:26, 6:32
Obviously I could just calculate a bit like:
range.begin (14) + macronameLen (11) +1 (opening parenthesis) = 26
to
range.end (33) - 1 (closing parenthesis) = 32
But would that be reliable and is there no other way?
Furthermore: Why do getSourceText/getOriginalText print the whole rest of the
source code and not just from (6,14) to (6,33)?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210826/dfda39c5/attachment-0001.html>
chiasa.men via llvm-dev
2021-Aug-26 15:00 UTC
[llvm-dev] [cfe-dev] Location (in Source Code) of String Literal in MacroExpansion
Thanks you were right, that fixed the minor printing issue. But the actual problem of string literals in macros still persists. Maybe someone else has an idea on that. The information isn't in reach of your MacroExpansionContext/MacroExpansionRangeRecorder, is it? On Donnerstag, 26. August 2021 12:32:22 CEST benicsbalazs at gmail.com wrote:> Oh, now I see why you were puzzled! > > > > `printf()` expects a null-terminated string (“%s”), but you pass an > llvm::StringRef instead, which is not null terminated! > > Thus, you will end up with undefined behavior, which ends up printing till > the end of the file. > > > > I hope this is it! > > > > Balazs > > > > From: benicsbalazs at gmail.com <benicsbalazs at gmail.com> > Sent: 2021. augusztus 26., csütörtök 12:25 > To: 'chiasa.men' <chiasa.men at web.de> > Cc: llvm-dev at lists.llvm.org > Subject: RE: [cfe-dev] Location (in Source Code) of String Literal in > MacroExpansion > > > > Hi Chiasa, > > > > For the example you attached the MacroExpansionContext seems to work as I > intended. > > It records what range gets replaced by what tokens at the end of the > preprocessing, thus we get > > `CONCATMACRO("TEST")` -> ` L"TEST"` > > > > About `Lexer::getSourceText()` I’m not sure. I haven’t used that. The `at()` > function was designed as a helper defining the unit tests. I might misused > something when I created this code & tests, let me know! > > > > Balazs > > > > From: cfe-dev <cfe-dev-bounces at lists.llvm.org > <mailto:cfe-dev-bounces at lists.llvm.org> > On Behalf Of chiasa.men via > cfe-dev Sent: 2021. augusztus 26., csütörtök 10:03 > To: cfe-dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org> > > Cc: llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > Subject: [cfe-dev] Location (in Source Code) of String Literal in > MacroExpansion > > > > https://gist.github.com/T35R6braPwgDJKq/6439fda090e4ee8440d726f8dafa4dbb#fil > e-mectest-cpp > > > > Above I modified the MacroExpansionContext unittest. > > The relevant code: > > //-------------- > > TEST_F(MacroExpansionContextTest, Custom) { > > // From the GCC website. > > const auto Ctx = getMacroExpansionContextFor(R"code( > > #define CONCATMACRO(parm) L##parm > > int f(wchar_t *b){} > > int main() > > { > > int a= f(CONCATMACRO("TEST")); > > int b= f(L"TEST"); > > return 0; > > } > > )code"); > > Ctx->dumpExpansionRanges(); > > //> :6:14, :6:33 > > Ctx->dumpExpandedTexts(); > > //> :6:14 -> 'L"TEST"' > > printf("Exp: %s\n",Ctx->getExpandedText(at(6, 14)).getValue()); > > //Exp: L"TEST" > > printf("Org: %s\n",Ctx->getOriginalText(at(6, 14)).getValue()); > > //Org: CONCATMACRO("TEST")); > > // int b= f(L"TEST"); > > // return 0; > > //} > > StringRef sourceText > clang::Lexer::getSourceText(CharSourceRange(SourceRange(at(6, 14),at(6, > 33)),false),SourceMgr, LangOpts); > > printf("sourceText: %s\n",sourceText); > > // sourceText: CONCATMACRO("TEST")); > > // int b= f(L"TEST"); > > // return 0; > > // } > > } > > //-------------- > > I am interested in getting the range for "TEST" in the SourceText. Since the > CONCATMACRO is preprocessed to L"TEST" I thought the MacroExpansionContext > would be of help. > > It tells about the ExpansionRanges: 6:14, :6:33 which is CONCATMACRO("TEST") > > What I would need is: 6:26, 6:32 > > Obviously I could just calculate a bit like: > > range.begin (14) + macronameLen (11) +1 (opening parenthesis) = 26 > > to > > range.end (33) - 1 (closing parenthesis) = 32 > > > > But would that be reliable and is there no other way? > > > > Furthermore: Why do getSourceText/getOriginalText print the whole rest of > the source code and not just from (6,14) to (6,33)?-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210826/f7fdde2e/attachment.html>