我对pyparsing和Python很陌生,所以这是一个警告,我可能做错了什么。
我要做的是构建一个sql解析器,并用节点构建树,然后我可以遍历这些节点。
我正试图从yacc/bison grammer文件中复制这种东西:
| scalar_exp '^' scalar_exp
{ $$ = new QgsSearchTreeNode(QgsSearchTreeNode::opPOW, $1, $3);
joinTmpNodes($$,$1,$3); }
这是我在python中的代码:
LPAR = Suppress('(')
RPAR = Suppress(')')
COMMA = Suppress(',')
AND = CaselessKeyword('AND')
ASC = CaselessKeyword('ASC')
DESC = CaselessKeyword('DESC')
ON = CaselessKeyword('ON')
USING = CaselessKeyword("USING")
INNER = CaselessKeyword("INNER")
JOIN = CaselessKeyword("JOIN")
AS = CaselessKeyword("AS")
NOT = CaselessKeyword("NOT")
SELECT = CaselessKeyword("SELECT")
FROM = CaselessKeyword("FROM")
WHERE = CaselessKeyword("WHERE")
GROUP = CaselessKeyword("GROUP")
BY = CaselessKeyword("BY")
ORDER = CaselessKeyword("ORDER")
LIMIT = CaselessKeyword("LIMIT")
BETWEEN = CaselessKeyword("BETWEEN")
UNARY = 1
BINARY = 2
TERNARY = 3
keyword = MatchFirst(( ASC, DESC, ON, USING, INNER,
JOIN, AS, NOT, SELECT, FROM, WHERE, GROUP, BY,
ORDER, BY, LIMIT,BETWEEN))
identifier = ~keyword + Word(alphas, alphanums+"_")
collation_name = identifier.copy()
column_name = Suppress('[') + ~keyword + Word(alphas, alphanums+"_") + Suppress(']')
column_alias = identifier.copy()
table_name = identifier.copy()
table_alias = identifier.copy()
index_name = identifier.copy()
function_name = identifier.copy()
parameter_name = identifier.copy()
expr = Forward().setName("expression")
select_stmt = Forward().setName("select statement")
integer = Regex(r"[+-]?\d+")
numeric_literal = Regex(r"\d+(\.\d*)?([eE][+-]?\d+)?")
string_literal = QuotedString("'")
literal_value = ( numeric_literal | string_literal)
expr_term = (
function_name + LPAR + Optional(delimitedList(expr)) + RPAR |
literal_value |
identifier |
column_name
)
expr << operatorPrecedence(expr_term,
[
(oneOf('- + ~') | NOT, UNARY, opAssoc.LEFT, setObject),
('||', BINARY, opAssoc.LEFT),
(oneOf('* / %'), BINARY, opAssoc.LEFT,setObject),
(oneOf('+ -'), BINARY, opAssoc.LEFT),
(oneOf('<< >> & |'), BINARY, opAssoc.LEFT),
(oneOf('< <= > >='), BINARY, opAssoc.LEFT),
(oneOf('= == != <>') , BINARY, opAssoc.LEFT),
('||', BINARY, opAssoc.LEFT),
((BETWEEN,AND), TERNARY, opAssoc.LEFT),
])
ordering_term = expr + Optional(ASC | DESC)
join_constraint = ON + expr('join_expression')
join_op = COMMA | (INNER + JOIN)
join_source = Forward()
single_source = ( table_name("table") +
Optional(Optional(AS) + table_alias("table_alias")))
join_source << single_source + Group(ZeroOrMore(join_op + single_source + Optional(join_constraint)))("join")
result_column = "*" | table_name + "." + "*" | (expr + Optional(Optional(AS) + column_alias))
select_core = (SELECT + Group(delimitedList(result_column))("columns") +
Optional(FROM + join_source).setParseAction(setObject) +
Optional(WHERE + expr("where_expr")) +
Optional(GROUP + BY + Group(delimitedList(ordering_term)("group_by_terms")))
)
select_stmt << (select_core + ZeroOrMore(select_core) +
Optional(ORDER + BY + Group(delimitedList(ordering_term))("order_by_terms"))
)
注:这是Paul McGuire编写的select_parser.py的一个精简版本
我想我必须使用setparseaction,但是每当我这样做的时候,我调用的方法中的标记总是得不到。我得到完整的字符串和位置,但没有标记。
调用setParseAction复制yacc/bison逻辑的最佳位置是哪里?
最佳答案
OperatorReceidence操作列表中的第一个条目应该是右关联的,而不是左关联的。在做了那个更改(并启用packrat解析)之后,这个解析器开始为我工作。
但是对于构建语法树,我会让pyparsing为您构建该树。将解析操作附加到不同的语法元素,这些语法元素返回将在setobject中创建的类的实例。比如:
class ExpressionNode(object):
def __init__(self, tokens):
self.tokens = tokens
def __repr__(self):
return "%s:\n%s" % (self.__class__.__name__, self.tokens.dump(indent=' '))
def __getattr__(self, attr):
return getattr(self.tokens, attr)
class SelectNode(ExpressionNode): pass
select_stmt.setParseAction(SelectNode)
stmtobj = select_stmt.parseString("SELECT * FROM B")[0]
print stmtobj.columns