用sqoop1写出个能用的mysql to hive脚本

xiaoxiao2021-02-27  261

hive的test库下新建表:

Create table log(col1 bigint,col2 tinyint,col3 bigint) partitioned by (createDate string) row format delimited fields terminated by ‘\t’ #!/bin/sh #取前一天日期,格式YYYYMMDD lastDay=`date "+%Y%m%d" -d "-1days"` #删除分区 hive -S -e "alter table test.logdrop partition(createDate=$lastDay)" #给frequency_monitor_log增加分区 hive -S -e "alter table test.log add if not exists partition(createDate=$lastDay)" sqoop import \ --connect jdbc:mysql://ip:3306/test \ --username root \ --password 123456 \ --query "select col1,col2,col3 from log where YEAR(create_time)='2017' AND MONTH(create_time)='04' AND DAYOFMONTH(create_time)='25' and \$CONDITIONS" \ --hive-import -m 1 \ --target-dir /hive/warehouse/test.db/tmp \ --hive-table test.log\ --hive-partition-key createDate \ --hive-partition-value $lastDay \ --fields-terminated-by '\t'
转载请注明原文地址: https://www.6miu.com/read-5001.html

最新回复(0)