Flume安装部署

安装部署

安装包连接:链接:https://pan.baidu.com/s/1m0d5O3Q2eH14BpWsGGfbLw?pwd=6666

(1)将apache-flume-1.10.1-bin.tar.gz上传到linux的/opt/software目录下

(2)解压apache-flume-1.10.1-bin.tar.gz到/opt/moudle/目录下

复制代码
tar -zxf /opt/software/apache-flume-1.10.1-bin.tar.gz -C /opt/moudle/

(3)修改apache-flume-1.10.1-bin的名称为flume

复制代码
mv apache-flume-1.10.1-bin/ flume

(4)修改conf目录下的log4j2.xml配置文件,配置日志文件路径

<?xml version="1.0" encoding="UTF-8"?>

<!--

Licensed to the Apache Software Foundation (ASF) under one or more

contributor license agreements. See the NOTICE file distributed with

this work for additional information regarding copyright ownership.

The ASF licenses this file to You under the Apache License, Version 2.0

(the "License"); you may not use this file except in compliance with

the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

-->

<Configuration status="ERROR">

<Properties>

<Property name="LOG_DIR">/opt/moudle/flume/log</Property>

</Properties>

<Appenders>

<Console name="Console" target="SYSTEM_ERR">

<PatternLayout pattern="%d (%t) %p - %l %m%n" />

</Console>

<RollingFile name="LogFile" fileName="{LOG_DIR}/flume.log" filePattern="{LOG_DIR}/archive/flume.log.%d{yyyyMMdd}-%i">

<PatternLayout pattern="%d{dd MMM yyyy HH:mm:ss,SSS} %-5p %t (%C.%M:%L) %equals{%x}{\[\]}{} - %m%n" />

<Policies>

<!-- Roll every night at midnight or when the file reaches 100MB -->

<SizeBasedTriggeringPolicy size="100 MB"/>

<CronTriggeringPolicy schedule="0 0 0 * * ?"/>

</Policies>

<DefaultRolloverStrategy min="1" max="20">

<Delete basePath="${LOG_DIR}/archive">

<!-- Nested conditions: the inner condition is only evaluated on files for which the outer conditions are true. -->

<IfFileName glob="flume.log.*">

<!-- Only allow 1 GB of files to accumulate -->

<IfAccumulatedFileSize exceeds="1 GB"/>

</IfFileName>

</Delete>

</DefaultRolloverStrategy>

</RollingFile>

</Appenders>

<Loggers>

<Logger name="org.apache.flume.lifecycle" level="info"/>

<Logger name="org.jboss" level="WARN"/>

<Logger name="org.apache.avro.ipc.netty.NettyTransceiver" level="WARN"/>

<Logger name="org.apache.hadoop" level="INFO"/>

<Logger name="org.apache.hadoop.hive" level="ERROR"/>

引入控制台输出,方便学习查看日志

<Root level="INFO">

<AppenderRef ref="LogFile" />

<AppenderRef ref="Console" />

</Root>

</Loggers>

</Configuration>

(5)分发flume(当前位置/opt/moudle/)

复制代码
xsync flume/
相关推荐
百年੭ ᐕ)੭*⁾⁾16 分钟前
Redash部署指南(Windows docker部署)
大数据·windows·docker·数据可视化·redash
乐迪信息21 分钟前
防爆AI摄像机实时监测船舶异常姿态
大数据·人工智能·算法·安全·目标跟踪
. . . . .22 分钟前
现代大数据技术栈核心知识总结
大数据
hangzhouhuanyu1 小时前
从培训到创业:AI工具在数字经济中的落地路径与避坑指南
大数据·人工智能
Lalolander1 小时前
WorkBuddy App 怎么发起任务?输入栏与发送全流程
大数据·人工智能·金蝶erp·workbuddy
2501_912784082 小时前
跨境建站避坑:为什么通用电商架构不适配反向代购业务
大数据·人工智能·架构·taoify
数智启示录2 小时前
实时数据湖 flink CDC + Kafka +Doris 【企业级实战】Checkpoint、Offset、事务与幂等如何闭环 06
大数据·flink·kafka
智慧大脑搬运工2 小时前
环保装备制造业高质量发展政策框架解析:揭榜挂帅机制与专精特新培育路径
大数据
老林说收银2 小时前
溯引 GEO 优化系统落地实战指南
大数据·人工智能
Safeploy安策数据2 小时前
服务器防勒索实战指南:从攻击链路到防御体系构建
大数据