记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

复制代码
export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
DuelCode10 分钟前
Windows VMWare Centos Docker部署Springboot 应用实现文件上传返回文件http链接
java·spring boot·mysql·nginx·docker·centos·mybatis
你想考研啊2 小时前
四、jenkins自动构建和设置邮箱
运维·jenkins
Code blocks2 小时前
使用Jenkins完成springboot项目快速更新
java·运维·spring boot·后端·jenkins
饥饿的半导体3 小时前
Linux快速入门
linux·运维
杨浦老苏4 小时前
开源服务运行监控工具Lunalytics
docker·群晖·网站监控
还是奇怪5 小时前
Linux - 安全排查 2
linux·运维·安全
牛奶咖啡136 小时前
Linux系统的常用操作命令——文件远程传输、文件编辑、软件安装的四种方式
运维·服务器·软件安装·linux云计算·scp文件远程传输·vi文件编辑·设置yum的阿里云源
难受啊马飞2.06 小时前
如何判断 AI 将优先自动化哪些任务?
运维·人工智能·ai·语言模型·程序员·大模型·大模型学习
会又不会6 小时前
Jenkins-Email Extension 插件插件
运维·jenkins
电脑能手7 小时前
[保姆级教程] 解决不同局域网电脑无法SSH的问题
运维·ssh·电脑