记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

复制代码
export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
小黑蛋学java11 分钟前
Nginx 接口耗时 Prometheus + Grafana 监控实施方案
运维·nginx·负载均衡·grafana·prometheus
鹤鸣的日常1 小时前
前端运行时动态环境变量方案
前端·react.js·docker·前端框架·vue·gitlab
asyxchenchong8882 小时前
最新Hermes Agent 技能封装与科研自动化:以 Meta-Analysis 为例-实现从文献检索到绘图的一站式工作流
运维·人工智能·自动化
tianyuanwo2 小时前
项目内自我管理:一名OS领域DevOps的破局之路
运维·devops
三十..2 小时前
Redis 核心原理与高可用架构实践
运维·数据库·redis
我是谁??4 小时前
ubuntu22.04 通过docker部署vLLM(Qwen3-0.6B)大模型+New API+OpenWebUI
docker·容器·vllm
jinglong.zha4 小时前
LScript-从零基础到商业变现的AI自动化学习平台
运维·学习·自动化
运维瓦工5 小时前
DevOps 生态介绍(十):Docker Compose 核心 YAML 配置详解与常用命令大全
spring cloud·docker·容器
Adorable老犀牛5 小时前
Telegraf:InfluxData 出品的指标采集代理
运维·telegraf
云烟成雨TD5 小时前
Spring AI 1.x 系列【59】容器化开发支持:Docker Compose 与 Testcontainers
人工智能·spring·docker