Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
M
mobvista-dmp
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
王金锋
mobvista-dmp
Commits
f6303229
Commit
f6303229
authored
Oct 22, 2021
by
WangJinfeng
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
drop partition
parent
7d875c3a
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
8 additions
and
3 deletions
+8
-3
ods_dmp_user_info_daily.sh
azkaban/userinfo/ods_dmp_user_info_daily.sh
+7
-2
ods_dmp_user_info_daily_mapping.sh
azkaban/userinfo/ods_dmp_user_info_daily_mapping.sh
+1
-1
No files found.
azkaban/userinfo/ods_dmp_user_info_daily.sh
View file @
f6303229
...
...
@@ -130,10 +130,14 @@ spark-submit --class mobvista.dmp.datasource.device.OdsDmpUserInfoDailyV2 \
../
${
JAR
}
\
-cur_day
${
date
}
-gender_date
${
GET_GENDER_DATE
}
-output
${
OUTPUT_PATH
}
-coalesce
2000
if
[[
$?
-ne
0
]]
;
then
exit
255
fi
hadoop fs
-touchz
${
OUTPUT_PATH
}
/_SUCCESS
yes_bef10_day
=
`
date
-d
"
$ScheduleTime
10 days ago"
+%Y%m%d
`
UNMOUNT_OUTPUT_PATH
=
${
ODS_DMP_USER_INFO_DAILY
}
/
${
yes_bef10_day
}
unmount_partition
"ods_dmp_user_info_daily"
"dt='
${
yes_bef10_day
}
'"
"
${
UNMOUNT_OUTPUT_PATH
}
"
\ No newline at end of file
azkaban/userinfo/ods_dmp_user_info_daily_mapping.sh
View file @
f6303229
...
...
@@ -6,7 +6,7 @@ LOG_TIME=$(date +%Y%m%d -d "-1 day $ScheduleTime")
yes_bef1_slack
=
$(
date
+%Y/%m/%d
-d
"-1 day
$ScheduleTime
"
)
INPUT_PATH
=
""
INPUT_PATH
=
"
s3://mob-emr-test/dataplatform/DataWareHouse/data/device_mapping/
${
yes_bef1_slack
}
"
OUTPUT_PATH
=
"
${
ODS_DMP_USER_INFO_DAILY
}
_mapping/
${
yes_bef1_slack
}
"
spark-submit
--class
mobvista.dmp.datasource.device.OdsDmpUserInfoDailyMapping
\
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment