从显式令牌列表中获取ocamlyacc解析器?

问题描述

是否可以向OCamlYacc生成的解析器提供显式令牌列表以进行分析?

Is it possible to feed an OCamlYacc-generated parser an explicit token list for analysis?

我想使用OCamlLex显式生成令牌列表，然后稍后使用Yacc生成的解析器对其进行分析.但是，标准用例会生成一个解析器，该解析器隐式调用下一个标记的词法分析器.在此，令牌是在yacc分析期间而不是之前进行计算的.从概念上讲，解析器应该只对令牌起作用，但是Yacc生成的解析器提供了依赖于词法分析器的接口，在我的情况下，我不需要.

I'd like to use OCamlLex to explicitly generate a token list which I then analyze using a Yacc-generated parser later. However, the standard use case generates a parser that calls a lexer implicitly for the next token. Here tokens are computed during the yacc analysis rather than before. Conceptually a parser should only work on tokens but a Yacc-generated parser provides an interface that relies on a lexer which in my case I don't need.

推荐答案

如果您已经有了令牌列表，则可以采用丑陋的方法，而完全忽略词法缓冲区.毕竟，解析器期望的parse-from-lexbuf函数是一个非纯函数:

If you already have a list of tokens, you can just go the ugly way and ignore the lexing buffer altogether. After all, the parse-from-lexbuf function that your parser expects is a non-pure function :

let my_tokens = ref [ (* WHATEVER *) ]
let token lexbuf =
  match !my_tokens with
    | []     -> EOF
    | h :: t -> my_tokens := t ; h

let ast = Parser.parse token (Lexbuf.from_string "")

另一方面，从您的注释中可以看出，您实际上具有类型为Lexing.lexbuf -> token list的函数，您正试图将其放入解析器的Lexing.lexbuf -> token签名中.如果是这样，您可以轻松地使用队列在两种类型之间编写转换器:

On the other hand, it looks from your comments that you actually have a function of type Lexing.lexbuf -> token list that you're trying to fit into the Lexing.lexbuf -> token signature of your parser. If that is the case, you can easily use a queue to write a converter between the two types:

let deflate token =
  let q = Queue.create () in
  fun lexbuf ->
    if not (Queue.is_empty q) then Queue.pop q else
      match token lexbuf with
        | [   ] -> EOF
        | [tok] -> tok
        | hd::t -> List.iter (fun tok -> Queue.add tok q) t ; hd

let ast = Parser.parse (deflate my_lexer) lexbuf

这篇关于从显式令牌列表中获取ocamlyacc解析器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！