Flume安装部署

安装部署

安装包连接:链接:https://pan.baidu.com/s/1m0d5O3Q2eH14BpWsGGfbLw?pwd=6666

(1)将apache-flume-1.10.1-bin.tar.gz上传到linux的/opt/software目录下

(2)解压apache-flume-1.10.1-bin.tar.gz到/opt/moudle/目录下

复制代码
tar -zxf /opt/software/apache-flume-1.10.1-bin.tar.gz -C /opt/moudle/

(3)修改apache-flume-1.10.1-bin的名称为flume

复制代码
mv apache-flume-1.10.1-bin/ flume

(4)修改conf目录下的log4j2.xml配置文件,配置日志文件路径

<?xml version="1.0" encoding="UTF-8"?>

<!--

Licensed to the Apache Software Foundation (ASF) under one or more

contributor license agreements. See the NOTICE file distributed with

this work for additional information regarding copyright ownership.

The ASF licenses this file to You under the Apache License, Version 2.0

(the "License"); you may not use this file except in compliance with

the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

-->

<Configuration status="ERROR">

<Properties>

<Property name="LOG_DIR">/opt/moudle/flume/log</Property>

</Properties>

<Appenders>

<Console name="Console" target="SYSTEM_ERR">

<PatternLayout pattern="%d (%t) [%p - %l] %m%n" />

</Console>

<RollingFile name="LogFile" fileName="{LOG_DIR}/flume.log" filePattern="{LOG_DIR}/archive/flume.log.%d{yyyyMMdd}-%i">

<PatternLayout pattern="%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %equals{%x}{[]}{} - %m%n" />

<Policies>

<!-- Roll every night at midnight or when the file reaches 100MB -->

<SizeBasedTriggeringPolicy size="100 MB"/>

<CronTriggeringPolicy schedule="0 0 0 * * ?"/>

</Policies>

<DefaultRolloverStrategy min="1" max="20">

<Delete basePath="${LOG_DIR}/archive">

<!-- Nested conditions: the inner condition is only evaluated on files for which the outer conditions are true. -->

<IfFileName glob="flume.log.*">

<!-- Only allow 1 GB of files to accumulate -->

<IfAccumulatedFileSize exceeds="1 GB"/>

</IfFileName>

</Delete>

</DefaultRolloverStrategy>

</RollingFile>

</Appenders>

<Loggers>

<Logger name="org.apache.flume.lifecycle" level="info"/>

<Logger name="org.jboss" level="WARN"/>

<Logger name="org.apache.avro.ipc.netty.NettyTransceiver" level="WARN"/>

<Logger name="org.apache.hadoop" level="INFO"/>

<Logger name="org.apache.hadoop.hive" level="ERROR"/>

引入控制台输出,方便学习查看日志

<Root level="INFO">

<AppenderRef ref="LogFile" />

<AppenderRef ref="Console" />

</Root>

</Loggers>

</Configuration>

(5)分发flume(当前位置/opt/moudle/)

复制代码
xsync flume/
相关推荐
thesky12345614 分钟前
Agno Agent
大数据·人工智能·深度学习
IT果果日记2 小时前
没有Kafka怎么办?Flink SQL 创建 mysql-cdc 作业
大数据·后端·flink
siliconstorm.ai3 小时前
穿越周期:AIoT产业的真实突破口与实践路径
大数据·人工智能
数据智研3 小时前
【数据分享】安徽省安庆市地理基础数据(道路、水系、铁路、行政边界(含乡镇)、DEM等)
大数据
云飞云共享云桌面4 小时前
共享云服务器替代传统电脑做三维设计会卡顿吗
大数据·运维·服务器·数据库·自动化
smilejingwei4 小时前
数据分析编程第五步:数据准备与整理
大数据·开发语言·数据分析·esprocspl
Lx3524 小时前
Hadoop集群资源管理:合理分配CPU和内存资源
大数据·hadoop
fanstuck5 小时前
2014-2024高教社杯全国大学生数学建模竞赛赛题汇总预览分析
大数据·人工智能·数学建模·数据挖掘·数据分析
thesky1234567 小时前
camel agent
大数据·人工智能·深度学习·智能体
新手村-小钻风7 小时前
AI-Agent 深度科普:从概念到架构、应用与未来趋势
大数据·人工智能