Flume安装部署

安装部署

安装包连接:链接:https://pan.baidu.com/s/1m0d5O3Q2eH14BpWsGGfbLw?pwd=6666

(1)将apache-flume-1.10.1-bin.tar.gz上传到linux的/opt/software目录下

(2)解压apache-flume-1.10.1-bin.tar.gz到/opt/moudle/目录下

复制代码
tar -zxf /opt/software/apache-flume-1.10.1-bin.tar.gz -C /opt/moudle/

(3)修改apache-flume-1.10.1-bin的名称为flume

复制代码
mv apache-flume-1.10.1-bin/ flume

(4)修改conf目录下的log4j2.xml配置文件,配置日志文件路径

<?xml version="1.0" encoding="UTF-8"?>

<!--

Licensed to the Apache Software Foundation (ASF) under one or more

contributor license agreements. See the NOTICE file distributed with

this work for additional information regarding copyright ownership.

The ASF licenses this file to You under the Apache License, Version 2.0

(the "License"); you may not use this file except in compliance with

the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

-->

<Configuration status="ERROR">

<Properties>

<Property name="LOG_DIR">/opt/moudle/flume/log</Property>

</Properties>

<Appenders>

<Console name="Console" target="SYSTEM_ERR">

<PatternLayout pattern="%d (%t) [%p - %l] %m%n" />

</Console>

<RollingFile name="LogFile" fileName="{LOG_DIR}/flume.log" filePattern="{LOG_DIR}/archive/flume.log.%d{yyyyMMdd}-%i">

<PatternLayout pattern="%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %equals{%x}{[]}{} - %m%n" />

<Policies>

<!-- Roll every night at midnight or when the file reaches 100MB -->

<SizeBasedTriggeringPolicy size="100 MB"/>

<CronTriggeringPolicy schedule="0 0 0 * * ?"/>

</Policies>

<DefaultRolloverStrategy min="1" max="20">

<Delete basePath="${LOG_DIR}/archive">

<!-- Nested conditions: the inner condition is only evaluated on files for which the outer conditions are true. -->

<IfFileName glob="flume.log.*">

<!-- Only allow 1 GB of files to accumulate -->

<IfAccumulatedFileSize exceeds="1 GB"/>

</IfFileName>

</Delete>

</DefaultRolloverStrategy>

</RollingFile>

</Appenders>

<Loggers>

<Logger name="org.apache.flume.lifecycle" level="info"/>

<Logger name="org.jboss" level="WARN"/>

<Logger name="org.apache.avro.ipc.netty.NettyTransceiver" level="WARN"/>

<Logger name="org.apache.hadoop" level="INFO"/>

<Logger name="org.apache.hadoop.hive" level="ERROR"/>

引入控制台输出,方便学习查看日志

<Root level="INFO">

<AppenderRef ref="LogFile" />

<AppenderRef ref="Console" />

</Root>

</Loggers>

</Configuration>

(5)分发flume(当前位置/opt/moudle/)

复制代码
xsync flume/
相关推荐
STLearner1 天前
WSDM 2026 | 时间序列(Time Series)论文总结【预测,表示学习,因果】
大数据·论文阅读·人工智能·深度学习·学习·机器学习·数据挖掘
亚马逊云开发者1 天前
【Bedrock AgentCore】AI Agent 回答不一致怎么办?双 Memory 架构实现服务标准化(附完整代码)
大数据·人工智能·架构
大嘴皮猴儿1 天前
从零开始学商品图翻译:小白也能快速掌握的多语言文字处理与上架技巧
大数据·ide·人工智能·macos·新媒体运营·xcode·自动翻译
雷工笔记1 天前
《为什么 10 倍增长比 2 倍更容易》读书笔记:反内卷的指数级增长破局法
大数据
captain_AIouo1 天前
OZON航海引领者Captain AI指引运营新航向
大数据·人工智能·经验分享·aigc
K3v1 天前
【git】删除本地以及远端已经合并到master的分支
大数据·git·elasticsearch
53AI1 天前
智能调度赋能交通行业:从经验驱动到数据智能的跨越
大数据·人工智能·知识库·智能调度·53ai
黎阳之光1 天前
黎阳之光核工厂202应急管控平台|全域实景孪生,筑牢核安全最后一道防线
大数据·人工智能·算法·安全·数字孪生
roman_日积跬步-终至千里1 天前
【系统架构设计师-综合题-知识点(1)】系统工程与信息技术基础
大数据
Elastic 中国社区官方博客2 天前
Elasticsearch:快速近似 ES|QL - 第二部分
大数据·数据库·sql·elasticsearch·搜索引擎·全文检索