search for: scriptparserbase

...not being able to lex arithmetic expressions seems to > be due to lack of context sensitivity. E.g. consider `foo*bar`. Could be a > multiplication, or could be a glob pattern. > > Looking at the code more closely, adding context sensitivity wouldn't be > that hard. In fact, our ScriptParserBase class is actually a lexer (look at > the interface; it is a lexer's interface). It shouldn't be hard to change > from an up-front tokenization to a more normal lexer approach of scanning > the text for each call that wants the next token. Roughly speaking, just > take the body o...

...that this was high priority. Since I'm complaining so much about it maybe I should take a look this weekend :) > > As far as I know, the grammar is LL(1), so it needs only one push-back > buffer. Handling INCLUDE directive can be a bit tricky though. > > Maybe we should rename ScriptParserBase ScriptLexer. > That sounds like a good idea. -- Sean Silva > > On Fri, Jan 27, 2017 at 11:17 AM, Rafael Avila de Espindola < > rafael.espindola at gmail.com> wrote: > >> > Hmm..., the crux of not being able to lex arithmetic expressions seems >> to >> &...

On Tue, Jan 24, 2017 at 11:29 AM, Rui Ueyama <ruiu at google.com> wrote: > Well, maybe, we should just change the Linux kernel instead of tweaking > our tokenizer too hard. > This is silly. Writing a simple and maintainable lexer is not hard (look e.g. at https://reviews.llvm.org/D10817). There are some complicated context-sensitive cases in linker scripts that break our approach

...s high priority. Since I'm > complaining so much about it maybe I should take a look this weekend :) > > > > As far as I know, the grammar is LL(1), so it needs only one push-back > buffer. Handling INCLUDE directive can be a bit tricky though. > > Maybe we should rename ScriptParserBase ScriptLexer. > > > That sounds like a good idea. > > -- Sean Silva > > > > On Fri, Jan 27, 2017 at 11:17 AM, Rafael Avila de Espindola < > rafael.espindola at gmail.com> wrote: > > > Hmm..., the crux of not being able to lex arithmetic expressions seems...

>Our tokenizer recognize > > [A-Za-z0-9_.$/\\~=+[]*?\-:!<>]+ > >as a token. gold uses more complex rules to tokenize. I don't think we need that much complex rules, but there seems to be >room to improve our tokenizer. In particular, I believe we can parse the Linux's linker script by changing the tokenizer rules as >follows. > >