bigtop-3.4.0-SNAPSHOT ubuntu22.04打包

环境:

window11

Docker Desktop 4.29.0

操作步骤:

docker pull bigtop/slaves:3.3.0-ubuntu-22.04 #拉取ubuntu镜像

docker run -it bigtop/slaves:3.3.0-ubuntu-22.04 /bin/bash #启动镜像并进入

adduser bigtop #创建bigtop用户

su - bigtop #登录bigtop用户

git clone https://github.com/apache/bigtop.git #获取bigtop代码

cd bigtop #进入bigtop目录

./gradlew deb #开始编译打包

结果:

经过漫长的编译打包,终于成功,再执行./gradlew deb,显示如下:

bigtop@c9d11691e2da:~/bigtop$ ./gradlew deb

> Task :alluxio-download

File /home/doris/bigtop/dl/alluxio-2.9.3.tar.gz appears to be already downloaded. Exiting...

> Task :alluxio-tar

Nothing to do. Exiting...

> Task :alluxio-sdeb

Nothing to do. Exiting...

> Task :alluxio-deb

Nothing to do. Exiting...

> Task :bigtop-groovy-download

File /home/doris/bigtop/dl/bigtop-groovy-2.5.4.tar.gz appears to be already downloaded. Exiting...

> Task :bigtop-groovy-tar

Nothing to do. Exiting...

> Task :bigtop-groovy-sdeb

Nothing to do. Exiting...

> Task :bigtop-groovy-deb

Nothing to do. Exiting...

> Task :bigtop-jsvc-download

File /home/doris/bigtop/dl/commons-daemon-1.2.4.tar.gz appears to be already downloaded. Exiting...

> Task :bigtop-jsvc-tar

Nothing to do. Exiting...

> Task :bigtop-jsvc-sdeb

Nothing to do. Exiting...

> Task :bigtop-jsvc-deb

Nothing to do. Exiting...

> Task :bigtop-utils-tar

Nothing to do. Exiting...

> Task :bigtop-utils-sdeb

Nothing to do. Exiting...

> Task :bigtop-utils-deb

Nothing to do. Exiting...

> Task :flink-download

File /home/doris/bigtop/dl/flink-1.16.2.tar.gz appears to be already downloaded. Exiting...

> Task :flink-tar

Nothing to do. Exiting...

> Task :flink-sdeb

Nothing to do. Exiting...

> Task :flink-deb

Nothing to do. Exiting...

> Task :hadoop-download

File /home/doris/bigtop/dl/hadoop-3.3.6.tar.gz appears to be already downloaded. Exiting...

> Task :hadoop-tar

Nothing to do. Exiting...

> Task :hadoop-sdeb

Nothing to do. Exiting...

> Task :hadoop-deb

Nothing to do. Exiting...

> Task :hbase-download

File /home/doris/bigtop/dl/hbase-2.4.17.tar.gz appears to be already downloaded. Exiting...

> Task :hbase-tar

Nothing to do. Exiting...

> Task :hbase-sdeb

Nothing to do. Exiting...

> Task :hbase-deb

Nothing to do. Exiting...

> Task :hive-download

File /home/doris/bigtop/dl/apache-hive-3.1.3-src.tar.gz appears to be already downloaded. Exiting...

> Task :hive-tar

Nothing to do. Exiting...

> Task :hive-sdeb

Nothing to do. Exiting...

> Task :hive-deb

Nothing to do. Exiting...

> Task :kafka-download

File /home/doris/bigtop/dl/kafka-2.8.2.tar.gz appears to be already downloaded. Exiting...

> Task :kafka-tar

Nothing to do. Exiting...

> Task :kafka-sdeb

Nothing to do. Exiting...

> Task :kafka-deb

Nothing to do. Exiting...

> Task :livy-download

File /home/doris/bigtop/dl/livy-0.8.0.zip appears to be already downloaded. Exiting...

> Task :livy-tar

Nothing to do. Exiting...

> Task :livy-sdeb

Nothing to do. Exiting...

> Task :livy-deb

Nothing to do. Exiting...

> Task :phoenix-download

File /home/doris/bigtop/dl/phoenix-5.1.3-src.tar.gz appears to be already downloaded. Exiting...

> Task :phoenix-tar

Nothing to do. Exiting...

> Task :phoenix-sdeb

Nothing to do. Exiting...

> Task :phoenix-deb

Nothing to do. Exiting...

> Task :ranger-download

File /home/doris/bigtop/dl/release-ranger-2.4.0.tar.gz appears to be already downloaded. Exiting...

> Task :ranger-tar

Nothing to do. Exiting...

> Task :ranger-sdeb

Nothing to do. Exiting...

> Task :ranger-deb

Nothing to do. Exiting...

> Task :solr-download

File /home/doris/bigtop/dl/solr-8.11.2-src.tgz appears to be already downloaded. Exiting...

> Task :solr-tar

Nothing to do. Exiting...

> Task :solr-sdeb

Nothing to do. Exiting...

> Task :solr-deb

Nothing to do. Exiting...

> Task :spark-download

File /home/doris/bigtop/dl/spark-3.3.4.tar.gz appears to be already downloaded. Exiting...

> Task :spark-tar

Nothing to do. Exiting...

> Task :spark-sdeb

Nothing to do. Exiting...

> Task :spark-deb

Nothing to do. Exiting...

> Task :tez-download

File /home/doris/bigtop/dl/apache-tez-0.10.2-src.tar.gz appears to be already downloaded. Exiting...

> Task :tez-tar

Nothing to do. Exiting...

> Task :tez-sdeb

Nothing to do. Exiting...

> Task :tez-deb

Nothing to do. Exiting...

> Task :zeppelin-download

File /home/doris/bigtop/dl/zeppelin-0.11.0.tar.gz appears to be already downloaded. Exiting...

> Task :zeppelin-tar

Nothing to do. Exiting...

> Task :zeppelin-sdeb

Nothing to do. Exiting...

> Task :zeppelin-deb

Nothing to do. Exiting...

> Task :zookeeper-download

File /home/doris/bigtop/dl/apache-zookeeper-3.7.2.tar.gz appears to be already downloaded. Exiting...

> Task :zookeeper-tar

Nothing to do. Exiting...

> Task :zookeeper-sdeb

Nothing to do. Exiting...

> Task :zookeeper-deb

Nothing to do. Exiting...

BUILD SUCCESSFUL in 16s

86 actionable tasks: 86 executed

共生成89个deb包

bigtop@c9d11691e2da:~/bigtop/output$ find . -name *.deb | wc -l

89

生成的deb包如下:

bigtop@c9d11691e2da:~/bigtop/output$ find . -name *.deb

./zeppelin/zeppelin_0.11.0-1_all.deb

./alluxio/alluxio_2.9.3-1_all.deb

./bigtop-jsvc/bigtop-jsvc_1.2.4-1_amd64.deb

./hadoop/hadoop-kms_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-router_3.3.6-1_amd64.deb

./hadoop/hadoop-doc_3.3.6-1_all.deb

./hadoop/hadoop-httpfs_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-timelineserver_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-nodemanager_3.3.6-1_amd64.deb

./hadoop/hadoop-client_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-fuse_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-proxyserver_3.3.6-1_amd64.deb

./hadoop/libhdfs0_3.3.6-1_amd64.deb

./hadoop/libhdfs0-dev_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-journalnode_3.3.6-1_amd64.deb

./hadoop/hadoop_3.3.6-1_amd64.deb

./hadoop/libhdfspp-dev_3.3.6-1_amd64.deb

./hadoop/hadoop-conf-pseudo_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-secondarynamenode_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs_3.3.6-1_amd64.deb

./hadoop/hadoop-mapreduce_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-datanode_3.3.6-1_amd64.deb

./hadoop/libhdfspp_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-zkfc_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-resourcemanager_3.3.6-1_amd64.deb

./hadoop/hadoop-mapreduce-historyserver_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-namenode_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-dfsrouter_3.3.6-1_amd64.deb

./flink/flink-taskmanager_1.16.2-1_all.deb

./flink/flink-jobmanager_1.16.2-1_all.deb

./flink/flink_1.16.2-1_all.deb

./kafka/kafka_2.8.2-1_all.deb

./kafka/kafka-server_2.8.2-1_all.deb

./tez/tez_0.10.2-1_all.deb

./spark/spark-thriftserver_3.3.4-1_all.deb

./spark/spark-master_3.3.4-1_all.deb

./spark/spark-sparkr_3.3.4-1_all.deb

./spark/spark-core_3.3.4-1_all.deb

./spark/spark-external_3.3.4-1_all.deb

./spark/spark-datanucleus_3.3.4-1_all.deb

./spark/spark-history-server_3.3.4-1_all.deb

./spark/spark-worker_3.3.4-1_all.deb

./spark/spark-yarn-shuffle_3.3.4-1_all.deb

./spark/spark-python_3.3.4-1_all.deb

./livy/livy_0.8.0-1_all.deb

./hive/hive-hcatalog-server_3.1.3-1_all.deb

./hive/hive-hcatalog_3.1.3-1_all.deb

./hive/hive-hbase_3.1.3-1_all.deb

./hive/hive-metastore_3.1.3-1_all.deb

./hive/hive-jdbc_3.1.3-1_all.deb

./hive/hive_3.1.3-1_all.deb

./hive/hive-webhcat_3.1.3-1_all.deb

./hive/hive-server2_3.1.3-1_all.deb

./hive/hive-webhcat-server_3.1.3-1_all.deb

./bigtop-utils/bigtop-utils_3.4.0-1_all.deb

./zookeeper/zookeeper-server_3.7.2-1_all.deb

./zookeeper/zookeeper_3.7.2-1_all.deb

./zookeeper/zookeeper-native_3.7.2-1_amd64.deb

./zookeeper/zookeeper-rest_3.7.2-1_all.deb

./solr/solr_8.11.2-2_all.deb

./solr/solr-server_8.11.2-2_all.deb

./solr/solr-doc_8.11.2-2_all.deb

./phoenix/phoenix_5.1.3-1_all.deb

./hbase/hbase-regionserver_2.4.17-1_all.deb

./hbase/hbase-thrift_2.4.17-1_all.deb

./hbase/hbase_2.4.17-1_amd64.deb

./hbase/hbase-rest_2.4.17-1_all.deb

./hbase/hbase-master_2.4.17-1_all.deb

./hbase/hbase-doc_2.4.17-1_all.deb

./bigtop-groovy/bigtop-groovy_2.5.4-1_all.deb

./ranger/ranger-yarn-plugin_2.4.0-1_all.deb

./ranger/ranger-presto-plugin_2.4.0-1_all.deb

./ranger/ranger-hbase-plugin_2.4.0-1_all.deb

./ranger/ranger-kylin-plugin_2.4.0-1_all.deb

./ranger/ranger-atlas-plugin_2.4.0-1_all.deb

./ranger/ranger-tagsync_2.4.0-1_all.deb

./ranger/ranger-knox-plugin_2.4.0-1_all.deb

./ranger/ranger-hive-plugin_2.4.0-1_all.deb

./ranger/ranger-elasticsearch-plugin_2.4.0-1_all.deb

./ranger/ranger-storm-plugin_2.4.0-1_all.deb

./ranger/ranger-hdfs-plugin_2.4.0-1_all.deb

./ranger/ranger_2.4.0-1_all.deb

./ranger/ranger-usersync_2.4.0-1_all.deb

./ranger/ranger-kms_2.4.0-1_all.deb

./ranger/ranger-solr-plugin_2.4.0-1_all.deb

./ranger/ranger-admin_2.4.0-1_all.deb

./ranger/ranger-sqoop-plugin_2.4.0-1_all.deb

./ranger/ranger-kafka-plugin_2.4.0-1_all.deb

总结:

编译打包没有报错,共生成89个deb包,涉及的组件有18个,由于greenplum github上的源码最近改为只读,开源前景不明朗,已经在bigtop最新的版本中去掉了。列表如下:

|----|---------------|
| 1 | solr |
| 2 | hive |
| 3 | hbase |
| 4 | phoenix |
| 5 | ranger |
| 6 | zookeeper |
| 7 | flink |
| 8 | bigtop-groovy |
| 9 | bigtop-utils |
| 10 | bigtop-select |
| 11 | bigtop-jsvc |
| 12 | kafka |
| 13 | livy |
| 14 | zeppelin |
| 15 | spark |
| 16 | hadoop |
| 17 | tez |
| 18 | alluxio |

注意:虽然编译打包没有报错,但是没有bigtop-select的deb包,不清楚为什么,后续会追踪这个问题

相关推荐
王小王-1231 天前
基于 Hive 的网易云音乐数据分析及可视化系统
hive·hadoop·数据分析·音乐数据分析·网易云音乐分析·hive音乐分析·hadoop网易云
极光代码工作室1 天前
基于数据仓库的电商数据分析平台
大数据·hadoop·python·spark·数据可视化
JLWcai202510091 天前
铸造领域树脂砂轮|金利威多场景解决方案,20 + 配方覆盖全需求
mongodb·zookeeper·eureka·spark·rabbitmq·memcached·storm
Chris _data1 天前
WPF 学习第三天 — Modbus RTU 串口通信
hadoop·学习·wpf
ACP广源盛139246256731 天前
GSV9001S@ACP#1080P 级视频处理芯片,物理 AI 普及终端的高性价比选择
大数据·人工智能·分布式·嵌入式硬件·spark
知识分享小能手1 天前
Hadoop学习教程,从入门到精通,Flume日志采集系统 — 完整知识点与案例代码(9)
hadoop·学习·flume
木心术11 天前
AMD Ryzen AI Halo与NVIDIA RTX Spark/DGX Spark两款AI个人主机的差异和优劣势
大数据·人工智能·spark
ACP广源盛139246256732 天前
GSV5600@ACP#多接口协议转换芯片,物理 AI 便携终端的互联核心
大数据·人工智能·分布式·嵌入式硬件·spark
KaMeidebaby2 天前
卡梅德生物技术快报 | 噬菌体展示 12 肽文库在蛋白表位定位中的应用与实验数据
大数据·人工智能·架构·spark·新浪微博
Francek Chen2 天前
【大数据处理与分析】MapReduce:06 MapReduce编程实践
大数据·hadoop·分布式·mapreduce