记录《现有docker中安装spark3.4.1》

基础docker环境中存储hadoop3--方便后续查看

参考:

实践:

复制代码
export JAVA_HOME=/opt/apache/jdk1.8.0_333
export SPARK_MASTER_IP=192.168.0.220
export SPARK_WORKER_MEMORY=4g
export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=4g
export HADOOP_HOME=/opt/apache/hadoop-3.2.2/
export HADOOP_CONF_DIR=/opt/apache/hadoop-3.2.2/etc/hadoop
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/apache/jdk1.8.0_333/jre/lib/amd64

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop220
hadoop221
hadoop222
相关推荐
Yana.nice3 小时前
Linux 只保留 30 天内日志(find命令删除日志文件)
linux·运维·chrome
2601_960567965 小时前
电商套图自动化效率的工程量化分析——从逐张生成到批量套图的架构演进
运维·架构·自动化
吳所畏惧6 小时前
宝塔面板Redis密码修改指南:SSH命令修改 vs 面板UI界面修改,哪个更靠谱?
运维·服务器·数据库·redis·缓存·ssh
张忠琳6 小时前
【NVIDIA】 NVIDIA Container Toolkit v1.19.1 — OCI 模块超深度分析之三
云原生·容器·架构·kubernetes·nvidia
HiDev_7 小时前
【非标自动化】2、认识元器件(确定目标)
运维·自动化
Zhang~Ling7 小时前
从 fopen 到 struct file:从零开始拆解 Linux 文件 I/O
linux·运维·服务器
爱写代码的森8 小时前
蒙三方库 | harmony-utils之FileUtil文件重命名与属性查询详解
linux·运维·服务器·华为·harmonyos·鸿蒙·huawei
Elastic 中国社区官方博客8 小时前
将你的 Grafana Kubernetes 仪表板迁移到 Elastic Observability:相同的 PromQL,30 倍更快的查询
大数据·人工智能·elasticsearch·搜索引擎·容器·kubernetes·grafana
中微极客8 小时前
2026主流AI Agent框架技术选型与性能对比
运维·网络·人工智能
久曲健的测试窝8 小时前
智能座舱自动化测试落地方案:数字仿真搭配实车实测,解决用例运维与场景失真难题
运维