记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

复制代码
export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
吴声子夜歌7 小时前
Nginx应用与运维——Nginx负载均衡应用实战(二)
运维·nginx·负载均衡
lisanmengmeng7 小时前
NRPE 添加命令(一)
linux·运维·服务器
pt10437 小时前
CML网络仿真入门-2:使用CML模拟网络实验环境
运维·网络协议
沫璃染墨8 小时前
《从零入门Linux系统篇(五十七):线程篇·十——生产者消费者模型进阶:从环形缓冲区到POSIX信号量》
linux·运维·服务器·开发语言·c++·系统架构·信号处理
HAHAXX89 小时前
电商RPA批量上架通用方案:一套流程如何同时跑通拼多多、抖店、淘宝和跨境平台
java·运维·rpa
零基础1239 小时前
LLM Agent 驱动的物模型构建:从设备手册到边缘接入的自动化实践
运维·人工智能·经验分享·python·自动化
更深兼春远10 小时前
第2篇:Dify 部署保姆级教程:用 Docker跑起来
docker·容器·dify
程序猿老A10 小时前
从入门型到企业型:云服务器开放共享型到独享型规格升级
运维·服务器
Ruiery11 小时前
Linux 6.6内核 CPU 深度解析(九):时钟与 TSC — 内核怎么从 PIT/HPET 切到 TSC
linux·运维·服务器
吴声子夜歌12 小时前
Nginx应用与运维——Nginx监控配置及管理(二)
运维·nginx