记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

复制代码
export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
云栖梦泽3 分钟前
网络设备驱动(3)
linux·运维·服务器·网络·嵌入式硬件
云生信1 小时前
服务器上新 OmicOS,体验生信 AI 计算
运维·服务器·人工智能
DFT计算杂谈1 小时前
无图形界面服务器用 Codex 终端连接本地部署的 DeepSeek
运维·服务器·网络
CAE虚拟与现实2 小时前
docker desktop中的build功能是要build什么
java·docker·容器
fb_123452 小时前
Docker入门-第1-2章-容器生态系统与架构详解
docker·容器·架构
allnlei4 小时前
s6-overlay - 装在 Docker 容器里的轻量级管家
java·docker·容器
一技安身10 小时前
【Docker】 镜像导出 & 导入,实验环境aarch64(arm64)Linux操作系统
linux·docker·容器
两点王爷13 小时前
解决 Linux 中 version `CXXABI_1.3.8‘ not found 报错
linux·运维·服务器
0+11113 小时前
Linux --线程概念与控制
linux·运维·服务器·jvm
宽海智能仓储物流14 小时前
航发自动化立体库异构系统改造:宽海智能30天联调技术全解析
运维·数据仓库·自动化