为Flume-ng编写自定义HTTPSource处理程序

问题描述

是否有任何资源可以帮助我为Flume-ng的HTTPSource编写自定义处理程序.我阅读了文档，并且有一个Json的示例处理程序，但是我想知道是否有人需要编写一个处理程序来从XML消息主体创建Flume事件.HttpSource现在在Flume-ng 1.3.1中可用，但是我们需要处理程序来解释我们的数据.

Are there any resources available to help me out in writing a custom handler for a HTTPSource for Flume-ng. I read the documentation and there is a sample handler for Json but I am wondering if anybody has had the need to write a handler for creating Flume events from a XML message body.The HttpSource is now available in Flume-ng 1.3.1 but we need handlers to interpret our data.

谢谢.

推荐答案

您是否看过 JSONHandler源? XMLHandler的唯一区别是使用了一些XML反序列化器而不是Gson.

Did you look at JSONHandler source? The only difference for XMLHandler would be usage of some XML deserializer instead of Gson.

您只需要转换以下内容:

You just need to convert something like:

<dataList>
  <data>
     ...
  </data>
  <data>
     ...
  </data>
</dataList>

进入列表< SimpleEvent >(或引入您自己的HTTPEvent，就像Flume开发人员使用 JSONEvent (如果需要处理不同的编码).这些事件的正文将包含您的<data>..</data>块，为字符串的字节表示形式.

into List<SimpleEvent> (or introduce your own HTTPEvent, as Flume developers did with JSONEvent, if you need to handle different encodings). These events' bodies will contain your <data>..</data> chunks as byte representation of String.

我不建议在Flume中实现解析这些事件的任何其他业务逻辑，因为业务规则往往会经常更改，而Flume作为基础结构软件应该是稳定的.

I don't recommend to implement any additional business logic of parsing these events in Flume, because business rules tends to change often and Flume as infrastructure software should be stable.

最后，将代码打包到jar中，然后将该jar放入Flume的lib目录(如果是Cloudera的rpm，则为/usr/lib/flume-ng/lib)，在HTTPSource的flume.conf"handler"属性中指定为"com.vicky.flume.source.http.XMLHandler"(或类似的东西)，重新启动代理即可.

Finally, you pack your code into jar, place this jar into Flume's lib directory (/usr/lib/flume-ng/lib in case of Cloudera's rpm distribution), specify in flume.conf "handler" property of HTTPSource as "com.vicky.flume.source.http.XMLHandler" (or something like that), restart the agent and that's all.

希望这会有所帮助.很高兴回答您的问题.

Hope this helps. Will be glad to answer your questions.

这篇关于为Flume-ng编写自定义HTTPSource处理程序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！