bigtop-3.4.0-SNAPSHOT ubuntu22.04打包

环境:

window11

Docker Desktop 4.29.0

操作步骤:

docker pull bigtop/slaves:3.3.0-ubuntu-22.04 #拉取ubuntu镜像

docker run -it bigtop/slaves:3.3.0-ubuntu-22.04 /bin/bash #启动镜像并进入

adduser bigtop #创建bigtop用户

su - bigtop #登录bigtop用户

git clone https://github.com/apache/bigtop.git #获取bigtop代码

cd bigtop #进入bigtop目录

./gradlew deb #开始编译打包

结果:

经过漫长的编译打包,终于成功,再执行./gradlew deb,显示如下:

bigtop@c9d11691e2da:~/bigtop$ ./gradlew deb

> Task :alluxio-download

File /home/doris/bigtop/dl/alluxio-2.9.3.tar.gz appears to be already downloaded. Exiting...

> Task :alluxio-tar

Nothing to do. Exiting...

> Task :alluxio-sdeb

Nothing to do. Exiting...

> Task :alluxio-deb

Nothing to do. Exiting...

> Task :bigtop-groovy-download

File /home/doris/bigtop/dl/bigtop-groovy-2.5.4.tar.gz appears to be already downloaded. Exiting...

> Task :bigtop-groovy-tar

Nothing to do. Exiting...

> Task :bigtop-groovy-sdeb

Nothing to do. Exiting...

> Task :bigtop-groovy-deb

Nothing to do. Exiting...

> Task :bigtop-jsvc-download

File /home/doris/bigtop/dl/commons-daemon-1.2.4.tar.gz appears to be already downloaded. Exiting...

> Task :bigtop-jsvc-tar

Nothing to do. Exiting...

> Task :bigtop-jsvc-sdeb

Nothing to do. Exiting...

> Task :bigtop-jsvc-deb

Nothing to do. Exiting...

> Task :bigtop-utils-tar

Nothing to do. Exiting...

> Task :bigtop-utils-sdeb

Nothing to do. Exiting...

> Task :bigtop-utils-deb

Nothing to do. Exiting...

> Task :flink-download

File /home/doris/bigtop/dl/flink-1.16.2.tar.gz appears to be already downloaded. Exiting...

> Task :flink-tar

Nothing to do. Exiting...

> Task :flink-sdeb

Nothing to do. Exiting...

> Task :flink-deb

Nothing to do. Exiting...

> Task :hadoop-download

File /home/doris/bigtop/dl/hadoop-3.3.6.tar.gz appears to be already downloaded. Exiting...

> Task :hadoop-tar

Nothing to do. Exiting...

> Task :hadoop-sdeb

Nothing to do. Exiting...

> Task :hadoop-deb

Nothing to do. Exiting...

> Task :hbase-download

File /home/doris/bigtop/dl/hbase-2.4.17.tar.gz appears to be already downloaded. Exiting...

> Task :hbase-tar

Nothing to do. Exiting...

> Task :hbase-sdeb

Nothing to do. Exiting...

> Task :hbase-deb

Nothing to do. Exiting...

> Task :hive-download

File /home/doris/bigtop/dl/apache-hive-3.1.3-src.tar.gz appears to be already downloaded. Exiting...

> Task :hive-tar

Nothing to do. Exiting...

> Task :hive-sdeb

Nothing to do. Exiting...

> Task :hive-deb

Nothing to do. Exiting...

> Task :kafka-download

File /home/doris/bigtop/dl/kafka-2.8.2.tar.gz appears to be already downloaded. Exiting...

> Task :kafka-tar

Nothing to do. Exiting...

> Task :kafka-sdeb

Nothing to do. Exiting...

> Task :kafka-deb

Nothing to do. Exiting...

> Task :livy-download

File /home/doris/bigtop/dl/livy-0.8.0.zip appears to be already downloaded. Exiting...

> Task :livy-tar

Nothing to do. Exiting...

> Task :livy-sdeb

Nothing to do. Exiting...

> Task :livy-deb

Nothing to do. Exiting...

> Task :phoenix-download

File /home/doris/bigtop/dl/phoenix-5.1.3-src.tar.gz appears to be already downloaded. Exiting...

> Task :phoenix-tar

Nothing to do. Exiting...

> Task :phoenix-sdeb

Nothing to do. Exiting...

> Task :phoenix-deb

Nothing to do. Exiting...

> Task :ranger-download

File /home/doris/bigtop/dl/release-ranger-2.4.0.tar.gz appears to be already downloaded. Exiting...

> Task :ranger-tar

Nothing to do. Exiting...

> Task :ranger-sdeb

Nothing to do. Exiting...

> Task :ranger-deb

Nothing to do. Exiting...

> Task :solr-download

File /home/doris/bigtop/dl/solr-8.11.2-src.tgz appears to be already downloaded. Exiting...

> Task :solr-tar

Nothing to do. Exiting...

> Task :solr-sdeb

Nothing to do. Exiting...

> Task :solr-deb

Nothing to do. Exiting...

> Task :spark-download

File /home/doris/bigtop/dl/spark-3.3.4.tar.gz appears to be already downloaded. Exiting...

> Task :spark-tar

Nothing to do. Exiting...

> Task :spark-sdeb

Nothing to do. Exiting...

> Task :spark-deb

Nothing to do. Exiting...

> Task :tez-download

File /home/doris/bigtop/dl/apache-tez-0.10.2-src.tar.gz appears to be already downloaded. Exiting...

> Task :tez-tar

Nothing to do. Exiting...

> Task :tez-sdeb

Nothing to do. Exiting...

> Task :tez-deb

Nothing to do. Exiting...

> Task :zeppelin-download

File /home/doris/bigtop/dl/zeppelin-0.11.0.tar.gz appears to be already downloaded. Exiting...

> Task :zeppelin-tar

Nothing to do. Exiting...

> Task :zeppelin-sdeb

Nothing to do. Exiting...

> Task :zeppelin-deb

Nothing to do. Exiting...

> Task :zookeeper-download

File /home/doris/bigtop/dl/apache-zookeeper-3.7.2.tar.gz appears to be already downloaded. Exiting...

> Task :zookeeper-tar

Nothing to do. Exiting...

> Task :zookeeper-sdeb

Nothing to do. Exiting...

> Task :zookeeper-deb

Nothing to do. Exiting...

BUILD SUCCESSFUL in 16s

86 actionable tasks: 86 executed

共生成89个deb包

bigtop@c9d11691e2da:~/bigtop/output$ find . -name *.deb | wc -l

89

生成的deb包如下:

bigtop@c9d11691e2da:~/bigtop/output$ find . -name *.deb

./zeppelin/zeppelin_0.11.0-1_all.deb

./alluxio/alluxio_2.9.3-1_all.deb

./bigtop-jsvc/bigtop-jsvc_1.2.4-1_amd64.deb

./hadoop/hadoop-kms_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-router_3.3.6-1_amd64.deb

./hadoop/hadoop-doc_3.3.6-1_all.deb

./hadoop/hadoop-httpfs_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-timelineserver_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-nodemanager_3.3.6-1_amd64.deb

./hadoop/hadoop-client_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-fuse_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-proxyserver_3.3.6-1_amd64.deb

./hadoop/libhdfs0_3.3.6-1_amd64.deb

./hadoop/libhdfs0-dev_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-journalnode_3.3.6-1_amd64.deb

./hadoop/hadoop_3.3.6-1_amd64.deb

./hadoop/libhdfspp-dev_3.3.6-1_amd64.deb

./hadoop/hadoop-conf-pseudo_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-secondarynamenode_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs_3.3.6-1_amd64.deb

./hadoop/hadoop-mapreduce_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-datanode_3.3.6-1_amd64.deb

./hadoop/libhdfspp_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-zkfc_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-resourcemanager_3.3.6-1_amd64.deb

./hadoop/hadoop-mapreduce-historyserver_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-namenode_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-dfsrouter_3.3.6-1_amd64.deb

./flink/flink-taskmanager_1.16.2-1_all.deb

./flink/flink-jobmanager_1.16.2-1_all.deb

./flink/flink_1.16.2-1_all.deb

./kafka/kafka_2.8.2-1_all.deb

./kafka/kafka-server_2.8.2-1_all.deb

./tez/tez_0.10.2-1_all.deb

./spark/spark-thriftserver_3.3.4-1_all.deb

./spark/spark-master_3.3.4-1_all.deb

./spark/spark-sparkr_3.3.4-1_all.deb

./spark/spark-core_3.3.4-1_all.deb

./spark/spark-external_3.3.4-1_all.deb

./spark/spark-datanucleus_3.3.4-1_all.deb

./spark/spark-history-server_3.3.4-1_all.deb

./spark/spark-worker_3.3.4-1_all.deb

./spark/spark-yarn-shuffle_3.3.4-1_all.deb

./spark/spark-python_3.3.4-1_all.deb

./livy/livy_0.8.0-1_all.deb

./hive/hive-hcatalog-server_3.1.3-1_all.deb

./hive/hive-hcatalog_3.1.3-1_all.deb

./hive/hive-hbase_3.1.3-1_all.deb

./hive/hive-metastore_3.1.3-1_all.deb

./hive/hive-jdbc_3.1.3-1_all.deb

./hive/hive_3.1.3-1_all.deb

./hive/hive-webhcat_3.1.3-1_all.deb

./hive/hive-server2_3.1.3-1_all.deb

./hive/hive-webhcat-server_3.1.3-1_all.deb

./bigtop-utils/bigtop-utils_3.4.0-1_all.deb

./zookeeper/zookeeper-server_3.7.2-1_all.deb

./zookeeper/zookeeper_3.7.2-1_all.deb

./zookeeper/zookeeper-native_3.7.2-1_amd64.deb

./zookeeper/zookeeper-rest_3.7.2-1_all.deb

./solr/solr_8.11.2-2_all.deb

./solr/solr-server_8.11.2-2_all.deb

./solr/solr-doc_8.11.2-2_all.deb

./phoenix/phoenix_5.1.3-1_all.deb

./hbase/hbase-regionserver_2.4.17-1_all.deb

./hbase/hbase-thrift_2.4.17-1_all.deb

./hbase/hbase_2.4.17-1_amd64.deb

./hbase/hbase-rest_2.4.17-1_all.deb

./hbase/hbase-master_2.4.17-1_all.deb

./hbase/hbase-doc_2.4.17-1_all.deb

./bigtop-groovy/bigtop-groovy_2.5.4-1_all.deb

./ranger/ranger-yarn-plugin_2.4.0-1_all.deb

./ranger/ranger-presto-plugin_2.4.0-1_all.deb

./ranger/ranger-hbase-plugin_2.4.0-1_all.deb

./ranger/ranger-kylin-plugin_2.4.0-1_all.deb

./ranger/ranger-atlas-plugin_2.4.0-1_all.deb

./ranger/ranger-tagsync_2.4.0-1_all.deb

./ranger/ranger-knox-plugin_2.4.0-1_all.deb

./ranger/ranger-hive-plugin_2.4.0-1_all.deb

./ranger/ranger-elasticsearch-plugin_2.4.0-1_all.deb

./ranger/ranger-storm-plugin_2.4.0-1_all.deb

./ranger/ranger-hdfs-plugin_2.4.0-1_all.deb

./ranger/ranger_2.4.0-1_all.deb

./ranger/ranger-usersync_2.4.0-1_all.deb

./ranger/ranger-kms_2.4.0-1_all.deb

./ranger/ranger-solr-plugin_2.4.0-1_all.deb

./ranger/ranger-admin_2.4.0-1_all.deb

./ranger/ranger-sqoop-plugin_2.4.0-1_all.deb

./ranger/ranger-kafka-plugin_2.4.0-1_all.deb

总结:

编译打包没有报错,共生成89个deb包,涉及的组件有18个,由于greenplum github上的源码最近改为只读,开源前景不明朗,已经在bigtop最新的版本中去掉了。列表如下:

|----|---------------|
| 1 | solr |
| 2 | hive |
| 3 | hbase |
| 4 | phoenix |
| 5 | ranger |
| 6 | zookeeper |
| 7 | flink |
| 8 | bigtop-groovy |
| 9 | bigtop-utils |
| 10 | bigtop-select |
| 11 | bigtop-jsvc |
| 12 | kafka |
| 13 | livy |
| 14 | zeppelin |
| 15 | spark |
| 16 | hadoop |
| 17 | tez |
| 18 | alluxio |

注意:虽然编译打包没有报错,但是没有bigtop-select的deb包,不清楚为什么,后续会追踪这个问题

相关推荐
闯闯桑9 小时前
Spark 中spark.implicits._ 中的 toDF和DataFrame 类本身的 toDF 方法
大数据·ajax·spark·scala
Lx35213 小时前
Hadoop数据处理模式:批处理与流处理结合技巧
大数据·hadoop
皆过客,揽星河14 小时前
Linux上安装MySQL8详细教程
android·linux·hadoop·mysql·linux安装mysql·数据库安装·详细教程
core51216 小时前
Hive实战(二)
数据仓库·hive·hadoop
武子康17 小时前
大数据-94 Spark核心三剑客:RDD、DataFrame、Dataset与SparkSession全面解析
大数据·后端·spark
奋斗的蛋黄18 小时前
大数据与云计算知识点
大数据·hadoop·云计算
计算机编程-吉哥1 天前
大数据毕业设计-基于Python的中文起点网小说数据分析平台(高分计算机毕业设计选题·定制开发·真正大数据)
大数据·hadoop·计算机毕业设计选题·机器学习毕业设计·大数据毕业设计·大数据毕业设计选题推荐·大数据毕设项目
Agatha方艺璇1 天前
Hive基础简介
数据仓库·hive·hadoop
IT研究室2 天前
大数据毕业设计选题推荐-基于大数据的国内旅游景点游客数据分析系统-Spark-Hadoop-Bigdata
大数据·hadoop·spark·毕业设计·源码·数据可视化·bigdata
XueminXu2 天前
Spark提交任务的资源配置和优化
spark·并行度·spark-submit·driver-memory·num-executors·executor-memory·executor-cores