Skip to content

M
mobvista-dmp
Commit e1702143 by WangJinfeng
Options

switch spark 3.1

parent 33095b19
Showing with 2 additions and 2 deletions
azkaban/joypac/joypac_result_etl.sh
......@@ -18,7 +18,7 @@ database="dwh"
table="joypac_daily"
part='etl'
INPUT_PATH="s3://mob-emr-test/dataplatform/DataWareHouse/data/dwh/stage_joypac_upload_daily/$date_path/*/*.gz"
INPUT_PATH="s3://mob-emr-test/dataplatform/DataWareHouse/data/dwh/stage_joypac_upload_daily/$date_path/*/*"
OUTPUT_PATH="s3://mob-emr-test/dataplatform/DataWareHouse/data/dwh/joypac_result/${date_path}/${part}"
spark-submit --class mobvista.dmp.datasource.joypac.JoypacResultEtl \
......
azkaban/joypac/joypc_sdk_daily.sh
......@@ -16,7 +16,7 @@ month=$(date -d "$ScheduleTime 1 days ago" +"%m")
day=$(date -d "$ScheduleTime 1 days ago" +"%d")
INPUT_PATH="s3://mob-emr-test/dataplatform/DataWareHouse/data/dwh/stage_joypac_upload_daily/$date_path/*/*.gz"
INPUT_PATH="s3://mob-emr-test/dataplatform/DataWareHouse/data/dwh/stage_joypac_upload_daily/$date_path/*/*"
## OUTPUT_PATH="s3://mob-emr-test/dataplatform/DataWareHouse/data/dwh/etl_joypc_sdk_daily/$date_path"
OUTPUT_PATH="${JOYPAC_DAILY_LOG_PATH}/$date_path"
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment