bigtop-3.4.0-SNAPSHOT ubuntu22.04打包

环境:

window11

Docker Desktop 4.29.0

操作步骤:

docker pull bigtop/slaves:3.3.0-ubuntu-22.04 #拉取ubuntu镜像

docker run -it bigtop/slaves:3.3.0-ubuntu-22.04 /bin/bash #启动镜像并进入

adduser bigtop #创建bigtop用户

su - bigtop #登录bigtop用户

git clone https://github.com/apache/bigtop.git #获取bigtop代码

cd bigtop #进入bigtop目录

./gradlew deb #开始编译打包

结果:

经过漫长的编译打包,终于成功,再执行./gradlew deb,显示如下:

bigtop@c9d11691e2da:~/bigtop$ ./gradlew deb

> Task :alluxio-download

File /home/doris/bigtop/dl/alluxio-2.9.3.tar.gz appears to be already downloaded. Exiting...

> Task :alluxio-tar

Nothing to do. Exiting...

> Task :alluxio-sdeb

Nothing to do. Exiting...

> Task :alluxio-deb

Nothing to do. Exiting...

> Task :bigtop-groovy-download

File /home/doris/bigtop/dl/bigtop-groovy-2.5.4.tar.gz appears to be already downloaded. Exiting...

> Task :bigtop-groovy-tar

Nothing to do. Exiting...

> Task :bigtop-groovy-sdeb

Nothing to do. Exiting...

> Task :bigtop-groovy-deb

Nothing to do. Exiting...

> Task :bigtop-jsvc-download

File /home/doris/bigtop/dl/commons-daemon-1.2.4.tar.gz appears to be already downloaded. Exiting...

> Task :bigtop-jsvc-tar

Nothing to do. Exiting...

> Task :bigtop-jsvc-sdeb

Nothing to do. Exiting...

> Task :bigtop-jsvc-deb

Nothing to do. Exiting...

> Task :bigtop-utils-tar

Nothing to do. Exiting...

> Task :bigtop-utils-sdeb

Nothing to do. Exiting...

> Task :bigtop-utils-deb

Nothing to do. Exiting...

> Task :flink-download

File /home/doris/bigtop/dl/flink-1.16.2.tar.gz appears to be already downloaded. Exiting...

> Task :flink-tar

Nothing to do. Exiting...

> Task :flink-sdeb

Nothing to do. Exiting...

> Task :flink-deb

Nothing to do. Exiting...

> Task :hadoop-download

File /home/doris/bigtop/dl/hadoop-3.3.6.tar.gz appears to be already downloaded. Exiting...

> Task :hadoop-tar

Nothing to do. Exiting...

> Task :hadoop-sdeb

Nothing to do. Exiting...

> Task :hadoop-deb

Nothing to do. Exiting...

> Task :hbase-download

File /home/doris/bigtop/dl/hbase-2.4.17.tar.gz appears to be already downloaded. Exiting...

> Task :hbase-tar

Nothing to do. Exiting...

> Task :hbase-sdeb

Nothing to do. Exiting...

> Task :hbase-deb

Nothing to do. Exiting...

> Task :hive-download

File /home/doris/bigtop/dl/apache-hive-3.1.3-src.tar.gz appears to be already downloaded. Exiting...

> Task :hive-tar

Nothing to do. Exiting...

> Task :hive-sdeb

Nothing to do. Exiting...

> Task :hive-deb

Nothing to do. Exiting...

> Task :kafka-download

File /home/doris/bigtop/dl/kafka-2.8.2.tar.gz appears to be already downloaded. Exiting...

> Task :kafka-tar

Nothing to do. Exiting...

> Task :kafka-sdeb

Nothing to do. Exiting...

> Task :kafka-deb

Nothing to do. Exiting...

> Task :livy-download

File /home/doris/bigtop/dl/livy-0.8.0.zip appears to be already downloaded. Exiting...

> Task :livy-tar

Nothing to do. Exiting...

> Task :livy-sdeb

Nothing to do. Exiting...

> Task :livy-deb

Nothing to do. Exiting...

> Task :phoenix-download

File /home/doris/bigtop/dl/phoenix-5.1.3-src.tar.gz appears to be already downloaded. Exiting...

> Task :phoenix-tar

Nothing to do. Exiting...

> Task :phoenix-sdeb

Nothing to do. Exiting...

> Task :phoenix-deb

Nothing to do. Exiting...

> Task :ranger-download

File /home/doris/bigtop/dl/release-ranger-2.4.0.tar.gz appears to be already downloaded. Exiting...

> Task :ranger-tar

Nothing to do. Exiting...

> Task :ranger-sdeb

Nothing to do. Exiting...

> Task :ranger-deb

Nothing to do. Exiting...

> Task :solr-download

File /home/doris/bigtop/dl/solr-8.11.2-src.tgz appears to be already downloaded. Exiting...

> Task :solr-tar

Nothing to do. Exiting...

> Task :solr-sdeb

Nothing to do. Exiting...

> Task :solr-deb

Nothing to do. Exiting...

> Task :spark-download

File /home/doris/bigtop/dl/spark-3.3.4.tar.gz appears to be already downloaded. Exiting...

> Task :spark-tar

Nothing to do. Exiting...

> Task :spark-sdeb

Nothing to do. Exiting...

> Task :spark-deb

Nothing to do. Exiting...

> Task :tez-download

File /home/doris/bigtop/dl/apache-tez-0.10.2-src.tar.gz appears to be already downloaded. Exiting...

> Task :tez-tar

Nothing to do. Exiting...

> Task :tez-sdeb

Nothing to do. Exiting...

> Task :tez-deb

Nothing to do. Exiting...

> Task :zeppelin-download

File /home/doris/bigtop/dl/zeppelin-0.11.0.tar.gz appears to be already downloaded. Exiting...

> Task :zeppelin-tar

Nothing to do. Exiting...

> Task :zeppelin-sdeb

Nothing to do. Exiting...

> Task :zeppelin-deb

Nothing to do. Exiting...

> Task :zookeeper-download

File /home/doris/bigtop/dl/apache-zookeeper-3.7.2.tar.gz appears to be already downloaded. Exiting...

> Task :zookeeper-tar

Nothing to do. Exiting...

> Task :zookeeper-sdeb

Nothing to do. Exiting...

> Task :zookeeper-deb

Nothing to do. Exiting...

BUILD SUCCESSFUL in 16s

86 actionable tasks: 86 executed

共生成89个deb包

bigtop@c9d11691e2da:~/bigtop/output$ find . -name *.deb | wc -l

89

生成的deb包如下:

bigtop@c9d11691e2da:~/bigtop/output$ find . -name *.deb

./zeppelin/zeppelin_0.11.0-1_all.deb

./alluxio/alluxio_2.9.3-1_all.deb

./bigtop-jsvc/bigtop-jsvc_1.2.4-1_amd64.deb

./hadoop/hadoop-kms_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-router_3.3.6-1_amd64.deb

./hadoop/hadoop-doc_3.3.6-1_all.deb

./hadoop/hadoop-httpfs_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-timelineserver_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-nodemanager_3.3.6-1_amd64.deb

./hadoop/hadoop-client_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-fuse_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-proxyserver_3.3.6-1_amd64.deb

./hadoop/libhdfs0_3.3.6-1_amd64.deb

./hadoop/libhdfs0-dev_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-journalnode_3.3.6-1_amd64.deb

./hadoop/hadoop_3.3.6-1_amd64.deb

./hadoop/libhdfspp-dev_3.3.6-1_amd64.deb

./hadoop/hadoop-conf-pseudo_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-secondarynamenode_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs_3.3.6-1_amd64.deb

./hadoop/hadoop-mapreduce_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-datanode_3.3.6-1_amd64.deb

./hadoop/libhdfspp_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-zkfc_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn-resourcemanager_3.3.6-1_amd64.deb

./hadoop/hadoop-mapreduce-historyserver_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-namenode_3.3.6-1_amd64.deb

./hadoop/hadoop-yarn_3.3.6-1_amd64.deb

./hadoop/hadoop-hdfs-dfsrouter_3.3.6-1_amd64.deb

./flink/flink-taskmanager_1.16.2-1_all.deb

./flink/flink-jobmanager_1.16.2-1_all.deb

./flink/flink_1.16.2-1_all.deb

./kafka/kafka_2.8.2-1_all.deb

./kafka/kafka-server_2.8.2-1_all.deb

./tez/tez_0.10.2-1_all.deb

./spark/spark-thriftserver_3.3.4-1_all.deb

./spark/spark-master_3.3.4-1_all.deb

./spark/spark-sparkr_3.3.4-1_all.deb

./spark/spark-core_3.3.4-1_all.deb

./spark/spark-external_3.3.4-1_all.deb

./spark/spark-datanucleus_3.3.4-1_all.deb

./spark/spark-history-server_3.3.4-1_all.deb

./spark/spark-worker_3.3.4-1_all.deb

./spark/spark-yarn-shuffle_3.3.4-1_all.deb

./spark/spark-python_3.3.4-1_all.deb

./livy/livy_0.8.0-1_all.deb

./hive/hive-hcatalog-server_3.1.3-1_all.deb

./hive/hive-hcatalog_3.1.3-1_all.deb

./hive/hive-hbase_3.1.3-1_all.deb

./hive/hive-metastore_3.1.3-1_all.deb

./hive/hive-jdbc_3.1.3-1_all.deb

./hive/hive_3.1.3-1_all.deb

./hive/hive-webhcat_3.1.3-1_all.deb

./hive/hive-server2_3.1.3-1_all.deb

./hive/hive-webhcat-server_3.1.3-1_all.deb

./bigtop-utils/bigtop-utils_3.4.0-1_all.deb

./zookeeper/zookeeper-server_3.7.2-1_all.deb

./zookeeper/zookeeper_3.7.2-1_all.deb

./zookeeper/zookeeper-native_3.7.2-1_amd64.deb

./zookeeper/zookeeper-rest_3.7.2-1_all.deb

./solr/solr_8.11.2-2_all.deb

./solr/solr-server_8.11.2-2_all.deb

./solr/solr-doc_8.11.2-2_all.deb

./phoenix/phoenix_5.1.3-1_all.deb

./hbase/hbase-regionserver_2.4.17-1_all.deb

./hbase/hbase-thrift_2.4.17-1_all.deb

./hbase/hbase_2.4.17-1_amd64.deb

./hbase/hbase-rest_2.4.17-1_all.deb

./hbase/hbase-master_2.4.17-1_all.deb

./hbase/hbase-doc_2.4.17-1_all.deb

./bigtop-groovy/bigtop-groovy_2.5.4-1_all.deb

./ranger/ranger-yarn-plugin_2.4.0-1_all.deb

./ranger/ranger-presto-plugin_2.4.0-1_all.deb

./ranger/ranger-hbase-plugin_2.4.0-1_all.deb

./ranger/ranger-kylin-plugin_2.4.0-1_all.deb

./ranger/ranger-atlas-plugin_2.4.0-1_all.deb

./ranger/ranger-tagsync_2.4.0-1_all.deb

./ranger/ranger-knox-plugin_2.4.0-1_all.deb

./ranger/ranger-hive-plugin_2.4.0-1_all.deb

./ranger/ranger-elasticsearch-plugin_2.4.0-1_all.deb

./ranger/ranger-storm-plugin_2.4.0-1_all.deb

./ranger/ranger-hdfs-plugin_2.4.0-1_all.deb

./ranger/ranger_2.4.0-1_all.deb

./ranger/ranger-usersync_2.4.0-1_all.deb

./ranger/ranger-kms_2.4.0-1_all.deb

./ranger/ranger-solr-plugin_2.4.0-1_all.deb

./ranger/ranger-admin_2.4.0-1_all.deb

./ranger/ranger-sqoop-plugin_2.4.0-1_all.deb

./ranger/ranger-kafka-plugin_2.4.0-1_all.deb

总结:

编译打包没有报错,共生成89个deb包,涉及的组件有18个,由于greenplum github上的源码最近改为只读,开源前景不明朗,已经在bigtop最新的版本中去掉了。列表如下:

|----|---------------|
| 1 | solr |
| 2 | hive |
| 3 | hbase |
| 4 | phoenix |
| 5 | ranger |
| 6 | zookeeper |
| 7 | flink |
| 8 | bigtop-groovy |
| 9 | bigtop-utils |
| 10 | bigtop-select |
| 11 | bigtop-jsvc |
| 12 | kafka |
| 13 | livy |
| 14 | zeppelin |
| 15 | spark |
| 16 | hadoop |
| 17 | tez |
| 18 | alluxio |

注意:虽然编译打包没有报错,但是没有bigtop-select的deb包,不清楚为什么,后续会追踪这个问题

相关推荐
阿里云大数据AI技术10 小时前
活动报名 | Apache Spark Meetup · 上海站,助力企业构建高效数据平台
spark
阿杰同学13 小时前
Hadoop 面试题及答案整理,最新面试题
大数据·hadoop·分布式
T1ssy16 小时前
Spark性能优化:核心技巧与实战指南
大数据·spark
皓空揽月16 小时前
Ubuntu 单机安装 Hadoop 3.2.4教程
linux·hadoop·ubuntu
bigdata-rookie17 小时前
Spark 运行架构及相关概念
大数据·架构·spark
beijingliushao17 小时前
101-Spark之Standalone环境安装步骤
大数据·分布式·spark
云 祁17 小时前
Spark SQL 深度优化实战指南:从原理到生产的完整方法论
大数据·数据库·分布式·sql·spark
雨中飘荡的记忆1 天前
HBase实战指南
大数据·数据库·hbase
写代码的【黑咖啡】1 天前
HDFS简介及其存储机制详解
大数据·hadoop·hdfs
俊哥大数据2 天前
【项目实战1】大数据项目开发案例---新闻资讯离线分析|实时分析|大数据仓库|推荐系统|数据可视化项目
数据仓库·hadoop·flink·spark·推荐系统·实时分析·离线分析