HDFS和HBase跨集群数据迁移 源码

HDFS集群间数据迁移（hadoop distcp）

hadoop distcp \
-pb \
hdfs://XX.14.36.205:8020/user/hive/warehouse/dp_fk_tmp.db/ph_cash_order \
hdfs://XX.18.32.21:8020/user/hive/warehouse/dp_fksx_mart.db/

HBase集群间数据（hbase ExportSnapshot）

/usr/local/fqlhadoop/hbase/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot \
-Dmapreduce.job.name=ExportSnapshot-portrait_info_fql_20241106 \
-Dmapreduce.job.queuename=lx_realtime \
-Dsnapshot.export.skip.tmp=true \
-Dmapreduce.map.memory.mb=4096 \
-Dmapreduce.map.java.opts=-Xmx3686m \
-overwrite \
-snapshot snapshot_portrait_info_fql_20241106 \
-copy-from hdfs://11.portraitszl.hbase.fql.com:8020/hbase \
-copy-to hdfs://1.compareportrait.hbase.fql.com:8020/hbase \
-mappers 150 \
-bandwidth 100