记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

复制代码
export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
xhbh6667 分钟前
服务器转发表完全教程:路由表vs转发表的区别与联系
运维·服务器
花生壳儿8 分钟前
Docker容器安装MySQL数据库
数据库·mysql·docker
互联科技报10 分钟前
2026 年度甄选:国内高可靠性 AI 算力服务器供应商
运维·服务器·人工智能
l1t13 分钟前
DeepSeek总结的Quack:DuckDB 客户端-服务器协议
运维·服务器·数据库·duckdb
楼田莉子14 分钟前
仿Muduo的高并发服务器:基于HTTP的HTTP服务器及其测试
运维·服务器·http
kyle~15 分钟前
Linux时间系统3---时间同步控制机制(step、slew、offset、frequency)
linux·运维·服务器
Agent手记26 分钟前
物流对账全流程自动化,落地实操与财务打通方案:基于LLM+智能体驱动的业财一体化实践
运维·人工智能·ai·自动化
Cat_Rocky27 分钟前
K8S中的优先级
云原生·容器·kubernetes
铅笔小新z27 分钟前
【Linux】进程间通信(IPC)
java·linux·运维
极客先躯29 分钟前
高级java每日一道面试题-2025年12月11日-实战篇[Docker]-如何配置 Docker 的资源限制(CPU、内存、磁盘)?
java·docker·如何配置docker的资源限制·资源限制的底层支柱·linux cgroups·cpu 限制·从逻辑到策略