问题描述
我想在 Python 中使用 ANTLR 构建一个 Java 解析器.
I want to build a Java parser using ANTLR in Python.
我从 ANTLR 存储库下载了语法:
I downloaded the grammars from the ANTLR repository:
词法分析器:https://github.com/antlr/grammars-v4/blob/master/java/java/JavaLexer.g4
解析器:https://github.com/antlr/grammars-v4/blob/master/java/java/JavaParser.g4
然后我用我的 script.bat 来生成我需要的 python 代码:
Then I used my script.bat to generate the python code I need :
java -jar antlr-4.8-complete.jar -Dlanguage=Python3 Java8Lexer.g4
java -jar antlr-4.8-complete.jar -Dlanguage=Python3 Java8Parser.g4
antlr-4.8-complete.jar
在此处下载:https://www.antlr.org/download/antlr-4.8-complete.jar
这生成了这个文件列表:
This generated this list of files:
- Java8Lexer.interp
- Java8Lexer.py
- Java8Lexer.tokens
- Java8Parser.interp
- Java8Parser.py
- Java8Parser.tokens
- Java8ParserListener.py
然后我写了这段代码来解析一个java文件:
Then I wrote this code to parse a java file:
import antlr4
from antlr4 import *
from java.antlr_unit2 import Java8Parser, Java8Lexer
def main():
code = open('test.txt', 'r').read()
lexer = Java8Lexer.Java8Lexer(antlr4.InputStream(code))
stream = antlr4.CommonTokenStream(lexer)
parser = Java8Parser.Java8Parser(stream)
tree = parser.expression()
print (tree)
if __name__ == '__main__':
main()
我的测试java代码test.txt
是这样的:
My test java code test.txt
is something like this:
package org.jabref.gui.fieldeditors;
import java.util.ArrayList;
/**
* This class contains some code
*/
public class TextInputControlBehavior {
private static final boolean SHOW_HANDLES = Properties.IS_TOUCH_SUPPORTED && !OS.OS_X;
}
由于太短,这里是我想解析的代码示例:https://pastebin.com/KNxfasKQ
Since this is too short, here is an example of code I want to parse: https://pastebin.com/KNxfasKQ
当我运行这段代码时,我得到了这个:
When I run this code I get this :
line 1:0 extraneous input 'package' expecting {'boolean', 'byte', 'char', 'double', 'float', 'int', 'long', 'new', 'short', 'super', 'this', 'void', IntegerLiteral, FloatingPointLiteral, BooleanLiteral, CharacterLiteral, StringLiteral, 'null', '(', '!', '~', '++', '--', '+', '-', Identifier, '@'}
[]
我做错了吗?语法不是我写的,我只是从ANTLR repo里拿来的.
Am I doing something wrong? I didn't wrote the grammar, I just took it from ANTLR repo.
编辑:Pavel Smirnov 的回答帮助了我,现在我没有收到警告.但是现在程序看起来很慢,我得到一个空树作为输出.
EDIT: Pavel Smirnov's answer helped me and now I don't get the warning. But now the program seems really slow and i get an empty tree as output.
解决:我正在打印tree
,但我不得不print(tree.toStringTree(recog=parser))
SOLVED: I was printing tree
but I had to print(tree.toStringTree(recog=parser))
所以最后的代码是:
import antlr4
from antlr4 import *
from java.antlr_unit2 import Java8Parser, Java8Lexer
def main():
code = open('test.txt', 'r').read()
lexer = Java8Lexer.Java8Lexer(antlr4.InputStream(code))
stream = antlr4.CommonTokenStream(lexer)
parser = Java8Parser.Java8Parser(stream)
tree = parser.compilationUnit()
print(tree.toStringTree(recog=parser))
if __name__ == '__main__':
main()
推荐答案
您的文本文件包含一个 compilationUnit
,而不是您尝试使用
Your text file contains a compilationUnit
, not an expression
you try to parse with
tree = parser.expression()
仔细看解析器规则,你需要的规则是
Look carefully through the parser rules, the rule you need is
compilationUnit
: packageDeclaration? importDeclaration* typeDeclaration* EOF
;
必须被称为
tree = parser.compilationUnit()
这篇关于使用 ANTLR 用 Python 解析一些 Java 代码的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!