首先应确定master和slave上都安装了Flume,如果slave上没有,可以将master上的传过去

scp -r /opt/flume/ [email protected]:/opt/

配置一级Flume

在master上创建netcat-avro.conf,意思是一级flume从444端口收集信息,再从555端口以avro格式发到二级flume

cd /opt/flume/conf
vi netcat-avro.conf
=>
a1.sources = r1  
a1.channels = c1  
a1.sinks = k1

a1.sources.r1.type = netcat  
a1.sources.r1.bind = master  
a1.sources.r1.port = 444

a1.sinks.k1.type = avro  
a1.sinks.k1.channel = c1  
a1.sinks.k1.hostname=slave1  
a1.sinks.k1.port = 555

a1.channels.c1.type = memory  
a1.channels.c1.capacity = 100  
a1.channels.c1.transactionCapacity = 100

a1.sources.r1.channels = c1  
a1.sinks.k1.channel = c1
<=

配置二级Flume

在slave上创建avro-logger.conf,意思是二级flume从555端口收集信息

cd /opt/flume/conf
vi avro-logger.conf
=>
a1.sources = r1  
a1.channels = c1  
a1.sinks = k1

a1.sources.r1.type = avro  
a1.sources.r1.bind = slave1  
a1.sources.r1.port = 555

  
a1.sinks.k1.type = logger

a1.channels.c1.type = memory  
a1.channels.c1.capacity = 100  
a1.channels.c1.transactionCapacity = 100

a1.sources.r1.channels = c1  
a1.sinks.k1.channel = c1
<=

启动一级和二级Flume

首先在slave上启动二级Flume

cd /opt/flume
bin/flume-ng agent --conf conf --conf-file conf/avro-logger.conf --name a1 -Dflume.root.logger=INFO,console

二级

接着在master上启动一级Flume

cd /opt/flume
bin/flume-ng agent --conf conf --conf-file conf/netcat-avro.conf --name a1 -Dflume.root.logger=INFO,console

一级

测试

使用telnet命令连接master

telnet master 444

并在telnet中输入数据 telnet

可以看到slave的二级Flume节点接受到了数据 slaveflume