Flume安装部署

安装部署

安装包连接:链接:https://pan.baidu.com/s/1m0d5O3Q2eH14BpWsGGfbLw?pwd=6666

(1)将apache-flume-1.10.1-bin.tar.gz上传到linux的/opt/software目录下

(2)解压apache-flume-1.10.1-bin.tar.gz到/opt/moudle/目录下

复制代码
tar -zxf /opt/software/apache-flume-1.10.1-bin.tar.gz -C /opt/moudle/

(3)修改apache-flume-1.10.1-bin的名称为flume

复制代码
mv apache-flume-1.10.1-bin/ flume

(4)修改conf目录下的log4j2.xml配置文件,配置日志文件路径

<?xml version="1.0" encoding="UTF-8"?>

<!--

Licensed to the Apache Software Foundation (ASF) under one or more

contributor license agreements. See the NOTICE file distributed with

this work for additional information regarding copyright ownership.

The ASF licenses this file to You under the Apache License, Version 2.0

(the "License"); you may not use this file except in compliance with

the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

-->

<Configuration status="ERROR">

<Properties>

<Property name="LOG_DIR">/opt/moudle/flume/log</Property>

</Properties>

<Appenders>

<Console name="Console" target="SYSTEM_ERR">

<PatternLayout pattern="%d (%t) [%p - %l] %m%n" />

</Console>

<RollingFile name="LogFile" fileName="{LOG_DIR}/flume.log" filePattern="{LOG_DIR}/archive/flume.log.%d{yyyyMMdd}-%i">

<PatternLayout pattern="%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %equals{%x}{[]}{} - %m%n" />

<Policies>

<!-- Roll every night at midnight or when the file reaches 100MB -->

<SizeBasedTriggeringPolicy size="100 MB"/>

<CronTriggeringPolicy schedule="0 0 0 * * ?"/>

</Policies>

<DefaultRolloverStrategy min="1" max="20">

<Delete basePath="${LOG_DIR}/archive">

<!-- Nested conditions: the inner condition is only evaluated on files for which the outer conditions are true. -->

<IfFileName glob="flume.log.*">

<!-- Only allow 1 GB of files to accumulate -->

<IfAccumulatedFileSize exceeds="1 GB"/>

</IfFileName>

</Delete>

</DefaultRolloverStrategy>

</RollingFile>

</Appenders>

<Loggers>

<Logger name="org.apache.flume.lifecycle" level="info"/>

<Logger name="org.jboss" level="WARN"/>

<Logger name="org.apache.avro.ipc.netty.NettyTransceiver" level="WARN"/>

<Logger name="org.apache.hadoop" level="INFO"/>

<Logger name="org.apache.hadoop.hive" level="ERROR"/>

引入控制台输出,方便学习查看日志

<Root level="INFO">

<AppenderRef ref="LogFile" />

<AppenderRef ref="Console" />

</Root>

</Loggers>

</Configuration>

(5)分发flume(当前位置/opt/moudle/)

复制代码
xsync flume/
相关推荐
青岛前景互联信息技术有限公司4 小时前
OpenClaw 重构智慧消防:AI时代的平台融合实践
大数据·人工智能
梦梦代码精5 小时前
BuildingAI 上部署自定义工作流智能体:5 个实用技巧
大数据·人工智能·算法·开源软件
极客老王说Agent5 小时前
2026智造前瞻:实在Agent生产排期智能助理核心功能与使用方法详解
大数据·人工智能·ai·chatgpt
数智化精益手记局6 小时前
什么是设备维护管理?设备维护管理包含哪些内容?
大数据·网络·人工智能·安全·信息可视化
AllData公司负责人7 小时前
通过Postgresql同步到Doris,全视角演示AllData数据中台核心功能效果,涵盖:数据入湖仓,数据同步,数据处理,数据服务,BI可视化驾驶舱
java·大数据·数据库·数据仓库·人工智能·python·postgresql
桃花键神7 小时前
Bright Data Web Scraping指南 2026: 使用 MCP + Dify 自动采集海外社交媒体数据
大数据·前端·人工智能
程序鉴定师10 小时前
西安App开发推荐与业界认可的优秀实践
大数据·小程序
workflower10 小时前
从拿订单到看方向
大数据·人工智能·设计模式·机器人·动态规划
CableTech_SQH11 小时前
F5G 全光网,赋能智慧校园数字化建设
大数据·网络·5g·运维开发·信息与通信
goyeer11 小时前
【ITIL4】- 服务价值体系
大数据·运维·信息化·自动运维·itil