Skip to content

M
mobvista-dmp
Commit c9ec291d by fan.jiang
Options

fix reyun_daily.sh

parent 911d96f8
Showing with 1 addition and 0 deletions
azkaban/ali/reyun/reyun_daily.sh
...@@ -24,6 +24,7 @@ hadoop fs -rm -r "${OUTPUT_PATH}" ...@@ -24,6 +24,7 @@ hadoop fs -rm -r "${OUTPUT_PATH}"
spark-submit --class mobvista.dmp.datasource.reyun.ReyunDaily \ spark-submit --class mobvista.dmp.datasource.reyun.ReyunDaily \
--conf spark.network.timeout=720s \ --conf spark.network.timeout=720s \
--conf spark.sql.mapKeyDedupPolicy=LAST_WIN \
--conf spark.driver.maxResultSize=4g \ --conf spark.driver.maxResultSize=4g \
--conf spark.default.parallelism=2000 \ --conf spark.default.parallelism=2000 \
--conf spark.sql.shuffle.partitions=2000 \ --conf spark.sql.shuffle.partitions=2000 \
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment