search for: lexer

Hello With the latest LLVM from Subversion (rev48737 from http://llvm.org/svn/llvm-project/llvm/trunk) I'm getting make[2]: Entering directory `/usr/src/Lang/llvm/_Obj/lib/AsmParser' llvm[2]: Flexing Lexer.l llvm[2]: Compiling Lexer.cpp for Debug build /usr/src/Lang/llvm/lib/AsmParser/Lexer.l: In function 'int llvmAsmlex()': /usr/src/Lang/llvm/lib/AsmParser/Lexer.l:278: error: 'PURE' was not declared in this scope /usr/src/Lang/llvm/lib/AsmParser/Lexer.l:279: error: 'CONST' w...

Hi. For weeks now I have problems compiling llvm from svn, compilation ends with llvm[2]: Compiling Lexer.cpp for Release build /home/borist/builds/llvm/lib/AsmParser/Lexer.l: In function 'int llvmAsmlex()': /home/borist/builds/llvm/lib/AsmParser/Lexer.l:278: error: 'PURE' was not declared in this scope /home/borist/builds/llvm/lib/AsmParser/Lexer.l:279: error: 'CONST' was no...

On Tue, Jan 24, 2017 at 11:29 AM, Rui Ueyama <ruiu at google.com> wrote: > Well, maybe, we should just change the Linux kernel instead of tweaking > our tokenizer too hard. > This is silly. Writing a simple and maintainable lexer is not hard (look e.g. at https://reviews.llvm.org/D10817). There are some complicated context-sensitive cases in linker scripts that break our approach of tokenizing up front (so we might want to hold off on), but we aren't going to die from implementing enough to lex basic arithmetic expressi...

...0644 lib/MC/MCParser/MachOAsmParser.cpp diff --git a/include/llvm/MC/MCParser/AsmParser.h b/include/llvm/MC/MCParser/AsmParser.h index e929fd1..dc74299 100644 --- a/include/llvm/MC/MCParser/AsmParser.h +++ b/include/llvm/MC/MCParser/AsmParser.h @@ -17,6 +17,7 @@ #include "llvm/MC/MCParser/AsmLexer.h" #include "llvm/MC/MCParser/AsmCond.h" #include "llvm/MC/MCParser/MCAsmParser.h" +#include "llvm/MC/MCParser/MCParsedAsmOperand.h" #include "llvm/MC/MCSectionMachO.h" #include "llvm/MC/MCStreamer.h" #include "llvm/MC/MCAsmInfo.h&quo...

...ressions seems to > be due to lack of context sensitivity. E.g. consider `foo*bar`. Could be a > multiplication, or could be a glob pattern. > > Looking at the code more closely, adding context sensitivity wouldn't be > that hard. In fact, our ScriptParserBase class is actually a lexer (look at > the interface; it is a lexer's interface). It shouldn't be hard to change > from an up-front tokenization to a more normal lexer approach of scanning > the text for each call that wants the next token. Roughly speaking, just > take the body of the for loop inside Scri...

Is this the appropriate place for GSoC conversations? If I understand the proposal correctly, there should be a lexer (written in R) that exposes an API; that API would be used by segregated mini-parsers (Roclets) which do the dirty work of Roxygen -> {html, LaTeX, DocBook, ...} translation. The lexer should ship with a proof-of-concept Roclet. Have I missed anything?

...m subversion and Eclipse CDT with managed make (on Ubuntu 32bit). This is the full command that's executed to link the project and its output: g++-4.2 `/home/stefan/projects/llvm/Debug/bin/llvm-config --ldflags --libs` `pkg-config --libs glibmm-2.4` -o"mojo" ./src/mojo.o ./src/mojo/lexer/state/Comment.o ./src/mojo/lexer/state/Default.o ./src/mojo/lexer/state/Numeric.o ./src/mojo/lexer/state/String.o ./src/mojo/lexer/Scanner.o ./src/mojo/lexer/Token.o ./src/mojo/ast/BinaryOperator.o ./src/mojo/ast/ConditionalBlock.o ./src/mojo/ast/FloatingPoint.o ./src/mojo/ast/Function.o ./src/mojo...

...look quite complete but I can't > > find > > any examples using them. I think a translation of the tutorial would be > > most > > welcome and about 10x shorter. ;-) > > Lexing is the one issue though. How do you mean? I'm just fiddling around with it now. The lexer, parser and AST written using camlp4 might look something like this in OCaml: type ast = | Num of float | Var of string | BinOp of [ `Add | `Sub | `Mul | `Less ] * ast * ast | Call of string * ast list | Function of string * string list * ast open Camlp4.PreCast;; let expr = Gram.Entr...

On Fri, Jan 27, 2017 at 1:31 PM, Rui Ueyama <ruiu at google.com> wrote: > Sean, > > So as you noticed that linker script tokenization rule is not very trivial > -- it is context sensitive. The current lexer is extremely simple and > almost always works well. Improving "almost always" to "perfect" is not > high priority because we have many more high priority things, but I'm fine > if someone improves it. If you are interested, please take it. Or maybe > I'll tak...

Hi, Is it possible to make the lexer and parser in clang a variability-aware? For example, if a C/C++ program contains a #ifdef, #else, #endif sequence, can clang generate AST with conditional nodes? How much effort would be needed? Thanks, -Riya -------------- next part -------------- An HTML attachment was scrubbed... URL: <http...

>> Lexing is the one issue though. > > How do you mean? I think he's observing that a majority of the tutorial code is actually spent on the lexer and parser, not on the llvm-specific pieces. > I'm just fiddling around with it now. The lexer, parser and AST > written using > camlp4 might look something like this in OCaml: Sure, the existing tutorial would be shorter if the lexer/parser used lex/yacc too, but that wasn't...

I am unknown to all the stuff happening in and around LLVM, as i am interested in the project but had not time to dig in, yet. Just reading here and there a bit. But i wanted to share my 2 cents on the Lexer topic. I have written multiple lexer/parser for private purpose and recently one for our product to compile Mathematical expressions into an internal format for evaluation. So far i usually write a Lexer which does the tokenizing on very basic rules. What is a number, what is a single symbolor a w...

>> I have a question also. You added -m elf_i386 to workaround emulation conflict issue in LLD, do you know >> does output produced by BFD boot fine after that change ? >Doesn't seem to affect BFD at all. Thanks ! ?George. -------------- next part -------------- An HTML attachment was scrubbed... URL:

Hi everyone. Because there is still a little confusion about the huge document, I want to know what llvm can provide when we customize our new compiler? For example, a normal compiler includes lexer, parser, intermediate code generator , optimizer and target code generator. According to llvm documents, it seems that llvm can provide a better intermediate code presentation. And what else can it provide? Assuming my new lanuage is very unusual, should I write all the parts( lexer, parser, int...

Hi, I am trying to understand LLVM. Can an expert tell me how to compare the kaleidoscope implementation with the LLVM's compiler implementation. I mean can anyone tell me about the lexer, parser, codegen related .cpp and .h files inside the LLVM solution project. I would like to understand the LLVM's compiler in the same way that the kaleidoscope compiler is explained. Thanks a lot. -- Kalyan Ponnala phone: 8163772059 -------------- next part -------------- An HTML attachm...

Hello, I would like to gather some ideas and opinions on how to make the default AsmLexer more flexible when dealing with Identifiers. When the lexer emits something as an "Identifier" (read. String of characters) it means that it needs to be parsed all at once in a single go, even if it contains elements that might be wanted to be parsed as separate entities. In that case it...

>Our tokenizer recognize > > [A-Za-z0-9_.$/\\~=+[]*?\-:!<>]+ > >as a token. gold uses more complex rules to tokenize. I don't think we need that much complex rules, but there seems to be >room to improve our tokenizer. In particular, I believe we can parse the Linux's linker script by changing the tokenizer rules as >follows. > >

...o drl_graph.o drl_graph_3d.o drl_layout.o drl_layout_3d.o drl_parse.o dsaitr.o dsapps.o dsaup2.o dsaupd.o dsconv.o dseigt.o dsesrt.o dseupd.o dsgets.o dsortc.o dsortr.o dstatn.o dstats.o dstqrb.o dvout.o eigen.o error.o evolver_cit.o f2c_dummy.o fast_community.o feedback_arc_set.o flow.o foreign-dl-lexer.o foreign-dl-parser.o foreign-gml-lexer.o foreign-gml-parser.o foreign-graphml.o foreign-lgl-lexer.o foreign-lgl-parser.o foreign-ncol-lexer.o foreign-ncol-parser.o foreign-pajek-lexer.o foreign-pajek-parser.o foreign.o forestfire.o fortran_intrinsics.o games.o gengraph_box_list.o gengraph_degree_s...

...ty> <op1>, <op2>`. > > Also, anybody but me that has noticed that Sphinx does not always colorize > things even if you put the proper ".. code-block:: llvm" in front? If it > is a known issue, I won't file a bug with the Sphinx people. > The Pygments lexer is sort of "brittle" and will fail to syntax highlight things if they stray too far from the lexical syntax it expects (due to being out of date or misspecified in the Pygments highlighter). You should check that the upstream has the same issue < https://bitbucket.org/birkenfeld/pygmen...

...lue into the example code to simplify things, I get the same error either way). If I run this code, LLVM asserts in the CreateGEP call with the error message "Non-pointer type for constant GetElementPtr expression" since the type I passed to CreateGEP is an Array type. Callstack: > Lexer.exe!llvm::ConstantExpr::getGetElementPtr(llvm::Constant * C, > llvm::ArrayRef<llvm::Value *> Idxs, bool InBounds, llvm::Type * > OnlyIfReducedTy) Line 2005 C++ Lexer.exe!llvm::ConstantFolder::CreateGetElementPtr(llvm::Constant * C, llvm::ArrayRef<llvm::Value *> IdxList) Line 13...