原sqoop脚本:
bash
sqoop import \
--connect "jdbc:oracle:thin:@//10.82.88.223:61521/LZY2" \
--username LZSHARE \
--password 'LZ^share$5105' \
--driver oracle.jdbc.driver.OracleDriver \
--query "SELECT
TO_NUMBER(TO_CHAR(GCRQ, 'YYYY')) AS gcrq_year,
TO_NUMBER(TO_CHAR(GCRQ, 'MM')) AS gcrq_month,
TO_NUMBER(TO_CHAR(GCRQ, 'DD')) AS gcrq_day,
YEAR,
GCRQ,
GCZBS,
HOUR,
MINUTE,
CDH,
XSFX,
SBSBM,
DCSJLX,
SJXH,
CLZQ,
XKC,
DKC,
XHC,
ZHC,
DHC,
TDH,
JZX,
MTC,
TLJ,
XKCS,
DKCS,
XHCS,
ZHCS,
DHCS,
TDHS,
JZXS,
MTCS,
TLJS,
GCBFB,
PJCTJJ,
SJZYL,
FLAGS,
OFF_MINS,
ERR_CODE,
ERR_DESC,
DELETE_BY,
DELETE_TIME,
CREATE_BY,
CREATE_TIME,
UPDATE_BY,
UPDATE_TIME,
INSERT_TIME
FROM LZJHGX.DAT_DCSJ_TIME
WHERE TO_CHAR(GCRQ , 'YYYY-MM-DD') = TO_CHAR(SYSDATE, 'YYYY-MM-DD') AND \$CONDITIONS" \
--split-by "YEAR" \
--hive-import \
--hive-table ods_pre_dat_dcsj_time \
--hive-partition-key "gcrq_year,gcrq_month,gcrq_day" \
--target-dir /user/hive/warehouse/ods_pre_dat_dcsj_time \
--hive-partition-value "gcrq_year=${gcrq_year},gcrq_month=${gcrq_month},gcrq_day=${gcrq_day}" \
-- --hive-drop-import-delims
出现
关键在于sqoop在计算split-by切片字段时:OriginalSql = SELECT MIN(YEAR), MAX(YEAR) FROM (SELECT
TO_NUMBER(TO_CHAR(GCRQ, 'YYYY')) AS gcrq_year,
TO_NUMBER(TO_CHAR(GCRQ, 'MM')) AS gcrq_month,
TO_NUMBER(TO_CHAR(GCRQ, 'DD')) AS gcrq_day,
.......
FROM LZJHGX.DAT_DCSJ_TIME
WHERE TO_CHAR(GCRQ , 'YYYY-MM-DD') = TO_CHAR(SYSDATE, 'YYYY-MM-DD') AND (1 = 1) ) AS t1,
使用 AS t1的语法,Oracle是不支持select * from (select * from A) as t1,这种子查询AS 命名表名的写法。
那么sqoop脚本中如何避免呢?
解决办法:删除掉driver参数--driver oracle.jdbc.driver.OracleDriver