Skip to content

M
mobvista-dmp
Commit bc7b36ac by fan.jiang
Options

fix btop_tiktokrv.sh memory bug

parent f6303229
Showing with 1 addition and 1 deletion
azkaban/dm/pseudo_package_to_other_business/btop_tiktokrv.sh
...@@ -30,7 +30,7 @@ spark-submit --class mobvista.dmp.datasource.dm.BtopTiktokrv \ ...@@ -30,7 +30,7 @@ spark-submit --class mobvista.dmp.datasource.dm.BtopTiktokrv \
--conf spark.default.parallelism=3000 \ --conf spark.default.parallelism=3000 \
--conf spark.sql.shuffle.partitions=3000 \ --conf spark.sql.shuffle.partitions=3000 \
--conf spark.network.timeout=720s \ --conf spark.network.timeout=720s \
--master yarn --deploy-mode cluster --executor-memory 8g --driver-memory 10g --executor-cores 5 --num-executors 70 \ --master yarn --deploy-mode cluster --executor-memory 8g --driver-memory 12g --executor-cores 5 --num-executors 70 \
../../${JAR} -begin_day ${begin_day} -begin_day02 ${begin_day02} -end_day ${end_day} -output01 ${OUTPUT_PATH01} -output02 ${OUTPUT_PATH02} -output03 ${OUTPUT_PATH03} -output04 ${OUTPUT_PATH04} -coalesce 200 ../../${JAR} -begin_day ${begin_day} -begin_day02 ${begin_day02} -end_day ${end_day} -output01 ${OUTPUT_PATH01} -output02 ${OUTPUT_PATH02} -output03 ${OUTPUT_PATH03} -output04 ${OUTPUT_PATH04} -coalesce 200
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment