bigtop-3.4.0-SNAPSHOT ubuntu22.04打包

环境:

window11

Docker Desktop 4.29.0

操作步骤:

docker pull bigtop/slaves:3.3.0-ubuntu-22.04 #拉取ubuntu镜像

docker run -it bigtop/slaves:3.3.0-ubuntu-22.04 /bin/bash #启动镜像并进入

adduser bigtop #创建bigtop用户

su - bigtop #登录bigtop用户

git clone https://github.com/apache/bigtop.git #获取bigtop代码

cd bigtop #进入bigtop目录

./gradlew deb #开始编译打包

结果:

经过漫长的编译打包,终于成功,再执行./gradlew deb,显示如下:

bigtop@c9d11691e2da:~/bigtop$ ./gradlew deb

> Task :alluxio-download

File /home/doris/bigtop/dl/alluxio-2.9.3.tar.gz appears to be already downloaded. Exiting...

> Task :alluxio-tar

Nothing to do. Exiting...

> Task :alluxio-sdeb

Nothing to do. Exiting...

> Task :alluxio-deb

Nothing to do. Exiting...

> Task :bigtop-groovy-download

File /home/doris/bigtop/dl/bigtop-groovy-2.5.4.tar.gz appears to be already downloaded. Exiting...

> Task :bigtop-groovy-tar

Nothing to do. Exiting...

> Task :bigtop-groovy-sdeb

Nothing to do. Exiting...

> Task :bigtop-groovy-deb

Nothing to do. Exiting...

> Task :bigtop-jsvc-download

File /home/doris/bigtop/dl/commons-daemon-1.2.4.tar.gz appears to be already downloaded. Exiting...

> Task :bigtop-jsvc-tar

Nothing to do. Exiting...

> Task :bigtop-jsvc-sdeb

Nothing to do. Exiting...

> Task :bigtop-jsvc-deb

Nothing to do. Exiting...

> Task :bigtop-utils-tar

Nothing to do. Exiting...

> Task :bigtop-utils-sdeb

Nothing to do. Exiting...

> Task :bigtop-utils-deb

Nothing to do. Exiting...

> Task :flink-download

File /home/doris/bigtop/dl/flink-1.16.2.tar.gz appears to be already downloaded. Exiting...

> Task :flink-tar

Nothing to do. Exiting...

> Task :flink-sdeb

Nothing to do. Exiting...

> Task :flink-deb

Nothing to do. Exiting...

> Task :hadoop-download

File /home/doris/bigtop/dl/hadoop-3.3.6.tar.gz appears to be already downloaded. Exiting...

> Task :hadoop-tar

Nothing to do. Exiting...

> Task :hadoop-sdeb

Nothing to do. Exiting...

> Task :hadoop-deb

Nothing to do. Exiting...

> Task :hbase-download

File /home/doris/bigtop/dl/hbase-2.4.17.tar.gz appears to be already downloaded. Exiting...

> Task :hbase-tar

Nothing to do. Exiting...

> Task :hbase-sdeb

Nothing to do. Exiting...

> Task :hbase-deb

Nothing to do. Exiting...

> Task :hive-download

File /home/doris/bigtop/dl/apache-hive-3.1.3-src.tar.gz appears to be already downloaded. Exiting...

> Task :hive-tar

Nothing to do. Exiting...

> Task :hive-sdeb

Nothing to do. Exiting...

> Task :hive-deb

Nothing to do. Exiting...

> Task :kafka-download

File /home/doris/bigtop/dl/kafka-2.8.2.tar.gz appears to be already downloaded. Exiting...

> Task :kafka-tar

Nothing to do. Exiting...

> Task :kafka-sdeb

Nothing to do. Exiting...

> Task :kafka-deb

Nothing to do. Exiting...

> Task :livy-download

File /home/doris/bigtop/dl/livy-0.8.0.zip appears to be already downloaded. Exiting...

> Task :livy-tar

Nothing to do. Exiting...

> Task :livy-sdeb

Nothing to do. Exiting...

> Task :livy-deb

Nothing to do. Exiting...

> Task :phoenix-download

File /home/doris/bigtop/dl/phoenix-5.1.3-src.tar.gz appears to be already downloaded. Exiting...

> Task :phoenix-tar

Nothing to do. Exiting...

> Task :phoenix-sdeb

Nothing to do. Exiting...

> Task :phoenix-deb

Nothing to do. Exiting...

> Task :ranger-download

File /home/doris/bigtop/dl/release-ranger-2.4.0.tar.gz appears to be already downloaded. Exiting...

> Task :ranger-tar

Nothing to do. Exiting...

> Task :ranger-sdeb

Nothing to do. Exiting...

> Task :ranger-deb

Nothing to do. Exiting...

> Task :solr-download

File /home/doris/bigtop/dl/solr-8.11.2-src.tgz appears to be already downloaded. Exiting...

> Task :solr-tar

Nothing to do. Exiting...

> Task :solr-sdeb

Nothing to do. Exiting...

> Task :solr-deb

Nothing to do. Exiting...

> Task :spark-download

File /home/doris/bigtop/dl/spark-3.3.4.tar.gz appears to be already downloaded. Exiting...

> Task :spark-tar

Nothing to do. Exiting...

> Task :spark-sdeb

Nothing to do. Exiting...

> Task :spark-deb

Nothing to do. Exiting...

> Task :tez-download

File /home/doris/bigtop/dl/apache-tez-0.10.2-src.tar.gz appears to be already downloaded. Exiting...

> Task :tez-tar

Nothing to do. Exiting...

> Task :tez-sdeb

Nothing to do. Exiting...

> Task :tez-deb

Nothing to do. Exiting...

> Task :zeppelin-download

File /home/doris/bigtop/dl/zeppelin-0.11.0.tar.gz appears to be already downloaded. Exiting...

> Task :zeppelin-tar

Nothing to do. Exiting...

> Task :zeppelin-sdeb

Nothing to do. Exiting...

> Task :zeppelin-deb

Nothing to do. Exiting...

> Task :zookeeper-download

File /home/doris/bigtop/dl/apache-zookeeper-3.7.2.tar.gz appears to be already downloaded. Exiting...

> Task :zookeeper-tar

Nothing to do. Exiting...

> Task :zookeeper-sdeb

Nothing to do. Exiting...

> Task :zookeeper-deb

Nothing to do. Exiting...

BUILD SUCCESSFUL in 16s

86 actionable tasks: 86 executed

共生成89个deb包

bigtop@c9d11691e2da:~/bigtop/output$ find . -name *.deb | wc -l

89

生成的deb包如下:

bigtop@c9d11691e2da:~/bigtop/output$ find . -name *.deb

./zeppelin/zeppelin_0.11.0-1_all.deb

./alluxio/alluxio_2.9.3-1_all.deb

./bigtop-jsvc/bigtop-jsvc_1.2.4-1_amd64.deb

./hadoop/hadoop-kms_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-router_3.3.6-1_amd64.deb

./hadoop/hadoop-doc_3.3.6-1_all.deb

./hadoop/hadoop-httpfs_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-timelineserver_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-nodemanager_3.3.6-1_amd64.deb

./hadoop/hadoop-client_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-fuse_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-proxyserver_3.3.6-1_amd64.deb

./hadoop/libhdfs0_3.3.6-1_amd64.deb

./hadoop/libhdfs0-dev_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-journalnode_3.3.6-1_amd64.deb

./hadoop/hadoop_3.3.6-1_amd64.deb

./hadoop/libhdfspp-dev_3.3.6-1_amd64.deb

./hadoop/hadoop-conf-pseudo_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-secondarynamenode_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs_3.3.6-1_amd64.deb

./hadoop/hadoop-mapreduce_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-datanode_3.3.6-1_amd64.deb

./hadoop/libhdfspp_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-zkfc_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-resourcemanager_3.3.6-1_amd64.deb

./hadoop/hadoop-mapreduce-historyserver_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-namenode_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-dfsrouter_3.3.6-1_amd64.deb

./flink/flink-taskmanager_1.16.2-1_all.deb

./flink/flink-jobmanager_1.16.2-1_all.deb

./flink/flink_1.16.2-1_all.deb

./kafka/kafka_2.8.2-1_all.deb

./kafka/kafka-server_2.8.2-1_all.deb

./tez/tez_0.10.2-1_all.deb

./spark/spark-thriftserver_3.3.4-1_all.deb

./spark/spark-master_3.3.4-1_all.deb

./spark/spark-sparkr_3.3.4-1_all.deb

./spark/spark-core_3.3.4-1_all.deb

./spark/spark-external_3.3.4-1_all.deb

./spark/spark-datanucleus_3.3.4-1_all.deb

./spark/spark-history-server_3.3.4-1_all.deb

./spark/spark-worker_3.3.4-1_all.deb

./spark/spark-yarn-shuffle_3.3.4-1_all.deb

./spark/spark-python_3.3.4-1_all.deb

./livy/livy_0.8.0-1_all.deb

./hive/hive-hcatalog-server_3.1.3-1_all.deb

./hive/hive-hcatalog_3.1.3-1_all.deb

./hive/hive-hbase_3.1.3-1_all.deb

./hive/hive-metastore_3.1.3-1_all.deb

./hive/hive-jdbc_3.1.3-1_all.deb

./hive/hive_3.1.3-1_all.deb

./hive/hive-webhcat_3.1.3-1_all.deb

./hive/hive-server2_3.1.3-1_all.deb

./hive/hive-webhcat-server_3.1.3-1_all.deb

./bigtop-utils/bigtop-utils_3.4.0-1_all.deb

./zookeeper/zookeeper-server_3.7.2-1_all.deb

./zookeeper/zookeeper_3.7.2-1_all.deb

./zookeeper/zookeeper-native_3.7.2-1_amd64.deb

./zookeeper/zookeeper-rest_3.7.2-1_all.deb

./solr/solr_8.11.2-2_all.deb

./solr/solr-server_8.11.2-2_all.deb

./solr/solr-doc_8.11.2-2_all.deb

./phoenix/phoenix_5.1.3-1_all.deb

./hbase/hbase-regionserver_2.4.17-1_all.deb

./hbase/hbase-thrift_2.4.17-1_all.deb

./hbase/hbase_2.4.17-1_amd64.deb

./hbase/hbase-rest_2.4.17-1_all.deb

./hbase/hbase-master_2.4.17-1_all.deb

./hbase/hbase-doc_2.4.17-1_all.deb

./bigtop-groovy/bigtop-groovy_2.5.4-1_all.deb

./ranger/ranger-yarn-plugin_2.4.0-1_all.deb

./ranger/ranger-presto-plugin_2.4.0-1_all.deb

./ranger/ranger-hbase-plugin_2.4.0-1_all.deb

./ranger/ranger-kylin-plugin_2.4.0-1_all.deb

./ranger/ranger-atlas-plugin_2.4.0-1_all.deb

./ranger/ranger-tagsync_2.4.0-1_all.deb

./ranger/ranger-knox-plugin_2.4.0-1_all.deb

./ranger/ranger-hive-plugin_2.4.0-1_all.deb

./ranger/ranger-elasticsearch-plugin_2.4.0-1_all.deb

./ranger/ranger-storm-plugin_2.4.0-1_all.deb

./ranger/ranger-hdfs-plugin_2.4.0-1_all.deb

./ranger/ranger_2.4.0-1_all.deb

./ranger/ranger-usersync_2.4.0-1_all.deb

./ranger/ranger-kms_2.4.0-1_all.deb

./ranger/ranger-solr-plugin_2.4.0-1_all.deb

./ranger/ranger-admin_2.4.0-1_all.deb

./ranger/ranger-sqoop-plugin_2.4.0-1_all.deb

./ranger/ranger-kafka-plugin_2.4.0-1_all.deb

总结:

编译打包没有报错,共生成89个deb包,涉及的组件有18个,由于greenplum github上的源码最近改为只读,开源前景不明朗,已经在bigtop最新的版本中去掉了。列表如下:

|----|---------------|
| 1 | solr |
| 2 | hive |
| 3 | hbase |
| 4 | phoenix |
| 5 | ranger |
| 6 | zookeeper |
| 7 | flink |
| 8 | bigtop-groovy |
| 9 | bigtop-utils |
| 10 | bigtop-select |
| 11 | bigtop-jsvc |
| 12 | kafka |
| 13 | livy |
| 14 | zeppelin |
| 15 | spark |
| 16 | hadoop |
| 17 | tez |
| 18 | alluxio |

注意:虽然编译打包没有报错,但是没有bigtop-select的deb包,不清楚为什么,后续会追踪这个问题

相关推荐
coding随想2 小时前
深入浅出Hadoop:大数据时代的“瑞士军刀”
大数据·hadoop·分布式
鸭鸭鸭进京赶烤5 小时前
第六届电子通讯与人工智能国际学术会议(ICECAI 2025)
大数据·图像处理·人工智能·hadoop·机器学习·数据挖掘
忘了ʷºᵇₐ6 小时前
MapReduce-Top N程序编写与运行
大数据·hadoop·maven·mapreduce·idea
LiRuiJie1 天前
深度剖析ZooKeeper
java·hadoop·分布式·zookeeper
线条11 天前
Zookeeper 集群安装与脚本化管理详解
分布式·zookeeper·云原生
好吃的肘子1 天前
Zookeeper入门(三)
分布式·zookeeper·云原生
岁月的眸1 天前
Hadoop-HA高可用集群启动nameNode莫名挂掉,排错解决
大数据·hadoop·分布式
wuli玉shell1 天前
SparkContext介绍
hadoop·spark·sparkcontext
北随琛烬入1 天前
Spark(32)SparkSQL操作Mysql
大数据·mysql·spark
想你依然心痛1 天前
Spark大数据分与实践笔记(第五章 HBase分布式数据库-02)
大数据·数据库·分布式·hbase