记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

复制代码
export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
人工智能训练4 小时前
OpenEnler等Linux系统中安装git工具的方法
linux·运维·服务器·git·vscode·python·ubuntu
QT 小鲜肉5 小时前
【Linux命令大全】001.文件管理之which命令(实操篇)
linux·运维·服务器·前端·chrome·笔记
Drqf6 小时前
NAS一键安装脚本
docker
fantasy5_57 小时前
Linux 动态进度条实战:从零掌握开发工具与核心原理
linux·运维·服务器
weixin_462446237 小时前
exo + tinygrad:Linux 节点设备能力自动探测(NVIDIA / AMD / CPU 安全兜底)
linux·运维·python·安全
莫逸风7 小时前
【局域网服务方案】:无需找运营商,低成本拥有高性能服务器
运维·服务器
oMcLin7 小时前
CentOS 7 频繁出现 “Connection Refused” 错误的原因分析与解决
linux·运维·centos
饭九钦vlog8 小时前
银河麒麟修复openssh升级脚本
linux·运维
❀͜͡傀儡师9 小时前
docker部署PostgreSQL数据库的监控和管理工具
数据库·docker·postgresql
Amy_au9 小时前
Linux week 01
linux·运维·服务器