Discussion:
ANTLR4, Left Recursion and imported grammars
Rene Treffer
2012-11-22 12:54:29 UTC
Permalink
Hi *,

I get really strange behavior if I try to split a parser with left
recursive rules into multiple files...

I've reduced the problem to a rather primitive Example:

- MainLexer.g

lexer grammar MainLexer;

NUMBER: [0-9]+ ;
ADD: '+' ;
MUL: '*' ;

- MainParser.g

parser grammar MainParser;

options {
tokenVocab = MainLexer;
}

import MainExpression;

- MainExpression.g

parser grammar MainExpression;

expr :
expr MUL expr
| expr ADD expr
| NUMBER
;

- Running antlr4

java -jar ~/bin/antlr-4.0b4-complete.jar MainLexer.g MainParser.g
Exception in thread "main" java.util.NoSuchElementException: token
index 27 out of range 0..24
at
org.antlr.runtime.BufferedTokenStream.get(BufferedTokenStream.java:143)
at
org.antlr.v4.analysis.LeftRecursiveRuleAnalyzer.stripAltLabel(LeftRecursiveRuleAnalyzer.java:363)
at
org.antlr.v4.analysis.LeftRecursiveRuleAnalyzer.otherAlt(LeftRecursiveRuleAnalyzer.java:230)
at
org.antlr.v4.parse.LeftRecursiveRuleWalker.outerAlternative(LeftRecursiveRuleWalker.java:787)
at
org.antlr.v4.parse.LeftRecursiveRuleWalker.ruleBlock(LeftRecursiveRuleWalker.java:583)
at
org.antlr.v4.parse.LeftRecursiveRuleWalker.rec_rule(LeftRecursiveRuleWalker.java:352)
at
org.antlr.v4.analysis.LeftRecursiveRuleTransformer.translateLeftRecursiveRule(LeftRecursiveRuleTransformer.java:123)
at
org.antlr.v4.analysis.LeftRecursiveRuleTransformer.translateLeftRecursiveRules(LeftRecursiveRuleTransformer.java:89)
at
org.antlr.v4.semantics.SemanticPipeline.process(SemanticPipeline.java:93)
at org.antlr.v4.Tool.processNonCombinedGrammar(Tool.java:393)
at org.antlr.v4.Tool.process(Tool.java:381)
at org.antlr.v4.Tool.processGrammarsOnCommandLine(Tool.java:345)
at org.antlr.v4.Tool.main(Tool.java:192)

Moving the expr rule to MainParser works. Is this a Bug? Can I somehow
solve this problem?

Regards,
Rene Treffer

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
Terence Parr
2012-11-22 20:26:38 UTC
Permalink
hi. fixed. pushed to antlr/antlr4 at github if you want to build.
T
Post by Rene Treffer
Hi *,
I get really strange behavior if I try to split a parser with left
recursive rules into multiple files...
- MainLexer.g
lexer grammar MainLexer;
NUMBER: [0-9]+ ;
ADD: '+' ;
MUL: '*' ;
- MainParser.g
parser grammar MainParser;
options {
tokenVocab = MainLexer;
}
import MainExpression;
- MainExpression.g
parser grammar MainExpression;
expr MUL expr
| expr ADD expr
| NUMBER
;
- Running antlr4
java -jar ~/bin/antlr-4.0b4-complete.jar MainLexer.g MainParser.g
Exception in thread "main" java.util.NoSuchElementException: token
index 27 out of range 0..24
at
org.antlr.runtime.BufferedTokenStream.get(BufferedTokenStream.java:143)
at
org.antlr.v4.analysis.LeftRecursiveRuleAnalyzer.stripAltLabel(LeftRecursiveRuleAnalyzer.java:363)
at
org.antlr.v4.analysis.LeftRecursiveRuleAnalyzer.otherAlt(LeftRecursiveRuleAnalyzer.java:230)
at
org.antlr.v4.parse.LeftRecursiveRuleWalker.outerAlternative(LeftRecursiveRuleWalker.java:787)
at
org.antlr.v4.parse.LeftRecursiveRuleWalker.ruleBlock(LeftRecursiveRuleWalker.java:583)
at
org.antlr.v4.parse.LeftRecursiveRuleWalker.rec_rule(LeftRecursiveRuleWalker.java:352)
at
org.antlr.v4.analysis.LeftRecursiveRuleTransformer.translateLeftRecursiveRule(LeftRecursiveRuleTransformer.java:123)
at
org.antlr.v4.analysis.LeftRecursiveRuleTransformer.translateLeftRecursiveRules(LeftRecursiveRuleTransformer.java:89)
at
org.antlr.v4.semantics.SemanticPipeline.process(SemanticPipeline.java:93)
at org.antlr.v4.Tool.processNonCombinedGrammar(Tool.java:393)
at org.antlr.v4.Tool.process(Tool.java:381)
at org.antlr.v4.Tool.processGrammarsOnCommandLine(Tool.java:345)
at org.antlr.v4.Tool.main(Tool.java:192)
Moving the expr rule to MainParser works. Is this a Bug? Can I somehow
solve this problem?
Regards,
Rene Treffer
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
Loading...