记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

复制代码
export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
木易双人青5 小时前
01-Docker-简介、安装与使用
运维·docker·容器
专注API从业者7 小时前
Python + 淘宝 API 开发:自动化采集商品数据的完整流程
大数据·运维·前端·数据挖掘·自动化
Lovyk8 小时前
Linux 正则表达式
linux·运维
ac.char9 小时前
在CentOS系统中查询已删除但仍占用磁盘空间的文件
linux·运维·centos
罗不俷11 小时前
【RH134知识点问答题】第13章:运行容器
容器·rhel
中科米堆11 小时前
中科米堆CASAIM自动化三维测量设备测量汽车壳体直径尺寸
运维·自动化·汽车·视觉检测
缘华工业智维11 小时前
CNN 在故障诊断中的应用:原理、案例与优势
大数据·运维·cnn
开航母的李大13 小时前
软件系统运维常见问题
运维·服务器·系统架构·运维开发
AI大模型13 小时前
基于 Docker 的 LLaMA-Factory 全流程部署指南
docker·llm·llama
华强笔记14 小时前
Linux内存管理系统性总结
linux·运维·网络