不多说,直接上干货!

Flume Sources官网剖析(博主推荐)

Flume Channels官网剖析(博主推荐)

Flume Channel Selectors官网剖析(博主推荐)

一切来源于flume官网

http://flume.apache.org/FlumeUserGuide.html

Flume Sinks官网剖析(博主推荐)-LMLPHP

Flume Sinks

HDFS Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

官网给我们的参考例子是

a1.channels = c1
a1.sinks = k1
a1.sinks.k1.type = hdfs
a1.sinks.k1.channel = c1
a1.sinks.k1.hdfs.path = /flume/events/%y-%m-%d/%H%M/%S
a1.sinks.k1.hdfs.filePrefix = events-
a1.sinks.k1.hdfs.round = true
a1.sinks.k1.hdfs.roundValue =
a1.sinks.k1.hdfs.roundUnit = minute

我们一般是

agent1.sinks = hdfs-sink1

# Define and configure a hdfs sink
agent1.sinks.hdfs-sink1.channel = ch1
agent1.sinks.hdfs-sink1.type = hdfs
agent1.sinks.hdfs-sink1.hdfs.path = hdfs://master:8030/flume/%Y%m%d
agent1.sinks.hdfs-sink1.hdfs.filePrefix = flume
agent1.sinks.hdfs-sink1.hdfs.fileSuffix = .log
agent1.sinks.hdfs-sink1.hdfs.useLocalTimeStamp = true
agent1.sinks.hdfs-sink1.hdfs.rollInterval =
agent1.sinks.hdfs-sink1.hdfs.rollSize =
agent1.sinks.hdfs-sink1.hdfs.rollCount =
agent1.sinks.hdfs-sink1.hdfs.codeC = snappy(当然这行,你得自行去安装snappy才可用)

Hive Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

Logger Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

Avro Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

Thrift Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

IRC Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

File Roll Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

Null Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

HBaseSinks(这里包含HBaseSink  和  AsyncHBaseSink)

Flume Sinks官网剖析(博主推荐)-LMLPHP

Flume Sinks官网剖析(博主推荐)-LMLPHP

  自行去官网看吧!

MorphlineSolrSink

Flume Sinks官网剖析(博主推荐)-LMLPHP

ElasticSearchSink

Flume Sinks官网剖析(博主推荐)-LMLPHP

官网提供给我们的参考案例是

a1.channels = c1
a1.sinks = k1
a1.sinks.k1.type = elasticsearch
a1.sinks.k1.hostNames = 127.0.0.1:,127.0.0.2:
a1.sinks.k1.indexName = foo_index
a1.sinks.k1.indexType = bar_type
a1.sinks.k1.clusterName = foobar_cluster
a1.sinks.k1.batchSize =
a1.sinks.k1.ttl = 5d
a1.sinks.k1.serializer = org.apache.flume.sink.elasticsearch.ElasticSearchDynamicSerializer
a1.sinks.k1.channel = c1

Kite Dataset Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

Kafka Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

  

  这里,官网还提供了Security and Kafka Sink、TLS and Kafka Sink、Kerberos and Kafka Sink,自行去看官网吧!

Custom Sink

Flume Sinks官网剖析(博主推荐)-LMLPHP

大家自己去看,罗列出来,是让大家,不要局限于博客本身,眼光要放宽和多看官网,一切来源于官网。

对于大数据无论是各种开源项目,看官网是学习的最好办法,再加上他人的中文博客。不要觉得英文恐惧,专业英语也就那样!变成高手和大牛,加油吧,zhouls!

05-18 08:53