记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

复制代码
export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
聆风吟º6 小时前
CANN开源项目深度实践:基于amct-toolkit实现自动化模型量化与精度保障策略
运维·开源·自动化·cann
较劲男子汉9 小时前
CANN Runtime零拷贝传输技术源码实战 彻底打通Host与Device的数据传输壁垒
运维·服务器·数据库·cann
风流倜傥唐伯虎9 小时前
Spring Boot Jar包生产级启停脚本
java·运维·spring boot
Doro再努力9 小时前
【Linux操作系统10】Makefile深度解析:从依赖推导到有效编译
android·linux·运维·服务器·编辑器·vim
senijusene9 小时前
Linux软件编程:IO编程,标准IO(1)
linux·运维·服务器
忧郁的橙子.10 小时前
02-本地部署Ollama、Python
linux·运维·服务器
醇氧10 小时前
【linux】查看发行版信息
linux·运维·服务器
lpruoyu10 小时前
【Docker进阶-03】存储原理
docker·容器
No8g攻城狮10 小时前
【Linux】Windows11 安装 WSL2 并运行 Ubuntu 22.04 详细操作步骤
linux·运维·ubuntu
酷酷的崽79811 小时前
CANN 生态可维护性与可观测性:构建生产级边缘 AI 系统的运维体系
运维·人工智能