Flume安装部署

安装部署

安装包连接:链接:https://pan.baidu.com/s/1m0d5O3Q2eH14BpWsGGfbLw?pwd=6666

(1)将apache-flume-1.10.1-bin.tar.gz上传到linux的/opt/software目录下

(2)解压apache-flume-1.10.1-bin.tar.gz到/opt/moudle/目录下

复制代码
tar -zxf /opt/software/apache-flume-1.10.1-bin.tar.gz -C /opt/moudle/

(3)修改apache-flume-1.10.1-bin的名称为flume

复制代码
mv apache-flume-1.10.1-bin/ flume

(4)修改conf目录下的log4j2.xml配置文件,配置日志文件路径

<?xml version="1.0" encoding="UTF-8"?>

<!--

Licensed to the Apache Software Foundation (ASF) under one or more

contributor license agreements. See the NOTICE file distributed with

this work for additional information regarding copyright ownership.

The ASF licenses this file to You under the Apache License, Version 2.0

(the "License"); you may not use this file except in compliance with

the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

-->

<Configuration status="ERROR">

<Properties>

<Property name="LOG_DIR">/opt/moudle/flume/log</Property>

</Properties>

<Appenders>

<Console name="Console" target="SYSTEM_ERR">

<PatternLayout pattern="%d (%t) [%p - %l] %m%n" />

</Console>

<RollingFile name="LogFile" fileName="{LOG_DIR}/flume.log" filePattern="{LOG_DIR}/archive/flume.log.%d{yyyyMMdd}-%i">

<PatternLayout pattern="%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %equals{%x}{[]}{} - %m%n" />

<Policies>

<!-- Roll every night at midnight or when the file reaches 100MB -->

<SizeBasedTriggeringPolicy size="100 MB"/>

<CronTriggeringPolicy schedule="0 0 0 * * ?"/>

</Policies>

<DefaultRolloverStrategy min="1" max="20">

<Delete basePath="${LOG_DIR}/archive">

<!-- Nested conditions: the inner condition is only evaluated on files for which the outer conditions are true. -->

<IfFileName glob="flume.log.*">

<!-- Only allow 1 GB of files to accumulate -->

<IfAccumulatedFileSize exceeds="1 GB"/>

</IfFileName>

</Delete>

</DefaultRolloverStrategy>

</RollingFile>

</Appenders>

<Loggers>

<Logger name="org.apache.flume.lifecycle" level="info"/>

<Logger name="org.jboss" level="WARN"/>

<Logger name="org.apache.avro.ipc.netty.NettyTransceiver" level="WARN"/>

<Logger name="org.apache.hadoop" level="INFO"/>

<Logger name="org.apache.hadoop.hive" level="ERROR"/>

引入控制台输出,方便学习查看日志

<Root level="INFO">

<AppenderRef ref="LogFile" />

<AppenderRef ref="Console" />

</Root>

</Loggers>

</Configuration>

(5)分发flume(当前位置/opt/moudle/)

复制代码
xsync flume/
相关推荐
小北方城市网8 分钟前
第1课:架构设计核心认知|从0建立架构思维(架构系列入门课)
大数据·网络·数据结构·python·架构·数据库架构
收获不止数据库25 分钟前
黄仁勋2026CES演讲复盘:旧世界,裂开了!
大数据·数据库·人工智能·职场和发展
老胡全房源系统26 分钟前
房产中介管理系统哪一款性价比高
大数据·人工智能·房产经纪人培训
黄焖鸡能干四碗33 分钟前
信息安全网络安全评估报告(WORD)
大数据·网络·人工智能·安全·web安全·制造·需求分析
汤姆yu1 小时前
基于python大数据的协同过滤音乐推荐系统
大数据·开发语言·python
Data_agent1 小时前
Cssbuy 模式淘宝 / 1688 代购系统南美市场搭建指南
大数据·python
川西胖墩墩1 小时前
团队协作泳道图制作工具 PC中文免费
大数据·论文阅读·人工智能·架构·流程图
云启数智YQ2 小时前
企业进行大数据迁移的注意事项有些什么?
大数据·大文件传输·跨国文件传输·内外网文件传输·大文件传输软件
房产中介行业研习社2 小时前
嘉兴国商区2026年1月品质楼盘推荐
大数据·人工智能·房产直播技巧·房产直播培训
巧克力味的桃子3 小时前
Spark 课程核心知识点复习汇总
大数据·分布式·spark