记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

复制代码
export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
闪亮的路灯4 小时前
Linux 安装mqtt服务mosquitto
linux·运维·服务器
szxinmai主板定制专家7 小时前
X86 工控机 vs RK3588 异构主控:AMR 控制器 BOM、散热、长期运维对比
运维·stm32·单片机·嵌入式硬件·fpga开发·am62x
叼烟扛炮7 小时前
Linux 学习笔记(四十一):TCP 并发演进——从单进程到多线程服务器
linux·运维·服务器·网络·笔记·学习·tcp/ip
Decay__8 小时前
09_MySQL 闪回与时间点恢复 —— binlog 回放、mysqlbinlog、PITR 与误删恢复
运维·数据库·sql·mysql·架构
久曲健的测试窝8 小时前
2026软件测试工具选型指南:双模式如何化解UI脚本维护难题
运维·人工智能·科技·aigc
DarkAthena9 小时前
wslc ps 卡死的排查:一次「停容器不停 VM」引发的连环故障
docker
NailiConveyor10 小时前
柔性物流方案如何适配工厂升级?自动化输送流水线定制要点
大数据·运维·自动化·制造·业界资讯
小沈同学呀10 小时前
【Agent开发第十五期】部署上线:把 Alex 搬进浏览器,15 期全栈串联收官
docker·agent·从零搭建ai助手
Lsetea10 小时前
Certbot webroot申请SSL证书总是404:Nginx挑战目录与路径映射检查
运维·nginx·https·ssl证书·certbot
如若12310 小时前
Windows SSH + Headscale/Tailscale 组网及免密登录完整配置教程
运维·windows·ubuntu·ssh·内网穿透