Flume安装部署

安装部署

安装包连接:链接:https://pan.baidu.com/s/1m0d5O3Q2eH14BpWsGGfbLw?pwd=6666

(1)将apache-flume-1.10.1-bin.tar.gz上传到linux的/opt/software目录下

(2)解压apache-flume-1.10.1-bin.tar.gz到/opt/moudle/目录下

复制代码
tar -zxf /opt/software/apache-flume-1.10.1-bin.tar.gz -C /opt/moudle/

(3)修改apache-flume-1.10.1-bin的名称为flume

复制代码
mv apache-flume-1.10.1-bin/ flume

(4)修改conf目录下的log4j2.xml配置文件,配置日志文件路径

<?xml version="1.0" encoding="UTF-8"?>

<!--

Licensed to the Apache Software Foundation (ASF) under one or more

contributor license agreements. See the NOTICE file distributed with

this work for additional information regarding copyright ownership.

The ASF licenses this file to You under the Apache License, Version 2.0

(the "License"); you may not use this file except in compliance with

the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

-->

<Configuration status="ERROR">

<Properties>

<Property name="LOG_DIR">/opt/moudle/flume/log</Property>

</Properties>

<Appenders>

<Console name="Console" target="SYSTEM_ERR">

<PatternLayout pattern="%d (%t) %p - %l %m%n" />

</Console>

<RollingFile name="LogFile" fileName="{LOG_DIR}/flume.log" filePattern="{LOG_DIR}/archive/flume.log.%d{yyyyMMdd}-%i">

<PatternLayout pattern="%d{dd MMM yyyy HH:mm:ss,SSS} %-5p %t (%C.%M:%L) %equals{%x}{\[\]}{} - %m%n" />

<Policies>

<!-- Roll every night at midnight or when the file reaches 100MB -->

<SizeBasedTriggeringPolicy size="100 MB"/>

<CronTriggeringPolicy schedule="0 0 0 * * ?"/>

</Policies>

<DefaultRolloverStrategy min="1" max="20">

<Delete basePath="${LOG_DIR}/archive">

<!-- Nested conditions: the inner condition is only evaluated on files for which the outer conditions are true. -->

<IfFileName glob="flume.log.*">

<!-- Only allow 1 GB of files to accumulate -->

<IfAccumulatedFileSize exceeds="1 GB"/>

</IfFileName>

</Delete>

</DefaultRolloverStrategy>

</RollingFile>

</Appenders>

<Loggers>

<Logger name="org.apache.flume.lifecycle" level="info"/>

<Logger name="org.jboss" level="WARN"/>

<Logger name="org.apache.avro.ipc.netty.NettyTransceiver" level="WARN"/>

<Logger name="org.apache.hadoop" level="INFO"/>

<Logger name="org.apache.hadoop.hive" level="ERROR"/>

引入控制台输出,方便学习查看日志

<Root level="INFO">

<AppenderRef ref="LogFile" />

<AppenderRef ref="Console" />

</Root>

</Loggers>

</Configuration>

(5)分发flume(当前位置/opt/moudle/)

复制代码
xsync flume/
相关推荐
互联网科技看点9 分钟前
2026年短信服务性价比分析:飞鸽云与头部云厂商
大数据·网络
2601_9567436824 分钟前
上海AI Agent智能体开发选型:从工程架构看落地路径解析
大数据·人工智能·ai·架构·开发经验·上海
subin9930 分钟前
圣阳蓄电池企业口碑优势与核心运营策略FAQ解析
大数据·运维·人工智能·电源·ups
ApacheSeaTunnel30 分钟前
一套替代四套!同程旅行用 Apache SeaTunnel 搭建多模态统一数据通道
大数据·数据库·ai·开源·数据集成·seatunnel·技术分享·数据同步
段一凡-华北理工大学33 分钟前
向量数据库实战:选型、调优与落地~系列文章02:Embedding 嵌入模型选型指南:OpenAI、BGE、Jina、Cohere 横评
大数据·数据库·人工智能·embedding·高炉炼铁·jina·工业智能体
Java面试题总结1 小时前
robots.txt 完全指南:从入门到精通
大数据
SNKXD_11 小时前
避开选型认知误区:2026年实测8款智能AI数字人平台总结筛选标准
大数据·人工智能·机器学习
Spider Cat 蜘蛛猫1 小时前
Keycloak私有化部署落地Shopify类跨境SaaS独立站完整方案
大数据
AC赳赳老秦1 小时前
GTD 工作法落地:用 OpenClaw 搭建收集-整理-执行-复盘全流程自动化工作流
大数据·运维·人工智能·信息可视化·自动化·deepseek·openclaw
DataX_ruby821 小时前
DCMM 2.0 贯标评估全解读:数据中台如何支撑九大能力域(2026版)
大数据·运维·人工智能·数据治理·数据中台