记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
华纳云IDC服务商33 分钟前
如何自动解决服务器弹性伸缩问题?
运维·服务器
是芽芽哩!1 小时前
【Kubernetes 指南】基础入门——Kubernetes 基本概念(二)
云原生·容器·kubernetes
soragui1 小时前
【ChatGPT】OpenAI 如何使用流模式进行回答
linux·运维·游戏
Logintern092 小时前
Linux如何设置redis可以外网访问—执行使用指定配置文件启动redis
linux·运维·redis
娶不到胡一菲的汪大东2 小时前
Linux之ARM(MX6U)裸机篇----1.开发环境搭建
linux·运维·服务器
vvw&2 小时前
如何在 Ubuntu 22.04 上安装和使用 Composer
linux·运维·服务器·前端·ubuntu·php·composer
Hi202402172 小时前
ubuntu22.04上安装win10虚拟机,并采用noVNC+frp,让远程通过web访问桌面
运维·kvm·云桌面
几维安全3 小时前
如何通过运行时威胁洞察提升反欺诈策略
运维·网络·安全
X__cheng3 小时前
【C++】list模拟实现
c++·容器
怡雪~3 小时前
k8s使用ceph
ceph·容器·kubernetes