Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
AppTag
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
dataplatform
AppTag
Commits
6ff13b0b
Commit
6ff13b0b
authored
5 years ago
by
jinfeng.wang
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
fix
parent
1aa5967a
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
113 additions
and
1 deletion
+113
-1
adx_package_tag.job
azkaban/adx_package_tag.job
+4
-0
adx_package_tag.sh
azkaban/adx_package_tag.sh
+42
-0
GetPackageFromAdxFile.java
...java/com/mobvista/apptag/utils/GetPackageFromAdxFile.java
+66
-0
MySqlUtil.java
src/main/java/com/mobvista/apptag/utils/MySqlUtil.java
+1
-1
No files found.
azkaban/adx_package_tag.job
0 → 100644
View file @
6ff13b0b
type=command
retries=3
command=sh -x adx_package_tag.sh
\ No newline at end of file
This diff is collapsed.
Click to expand it.
azkaban/adx_package_tag.sh
0 → 100644
View file @
6ff13b0b
#!/usr/sh
check_await
()
{
while
[[
true
]]
;
do
if
hadoop fs
-ls
"
$1
"
>
/dev/null 2>&1
then
break
fi
sleep
300
done
}
today
=
${
ScheduleTime
:-
$1
}
date
=
$(
date
+
"%Y%m%d"
-d
"-1 day
$today
"
)
date_path
=
$(
date
+%Y/%m/%d
-d
"-1 day
$today
"
)
INPUT
=
"s3://mob-emr-test/dataplatform/DataWareHouse/data/dwh/dim_adn_adx_package/
${
date_path
}
"
check_await
${
INPUT
}
/_SUCCESS
root_dir
=
`
pwd
`
if
[[
!
-d
"
${
root_dir
}
/adx_package/
${
date
}
"
]]
;
then
mkdir
-p
${
root_dir
}
/adx_package/
${
date
}
else
rm
-rf
${
root_dir
}
/adx_package/
${
date
}
mkdir
-p
${
root_dir
}
/adx_package/
${
date
}
fi
result
=
"
${
root_dir
}
/adx_package/
${
date
}
/package.txt"
hdfs dfs
-text
${
INPUT
}
/
*
>
${
result
}
if
[[
$?
-ne
0
]]
;
then
exit
255
fi
java
-jar
-Dloader
.main
=
com.mobvista.apptag.utils.GetPackageFromAdxFile apptag.jar
${
result
}
if
[[
$?
-ne
0
]]
;
then
exit
255
fi
\ No newline at end of file
This diff is collapsed.
Click to expand it.
src/main/java/com/mobvista/apptag/utils/GetPackageFromAdxFile.java
0 → 100644
View file @
6ff13b0b
package
com
.
mobvista
.
apptag
.
utils
;
import
java.io.BufferedReader
;
import
java.io.File
;
import
java.io.FileInputStream
;
import
java.io.InputStreamReader
;
import
java.util.HashSet
;
import
java.util.Set
;
/**
* @package: com.mobvista.apptag.utils
* @author: wangjf
* @date: 2019-07-17
* @time: 11:38
* @email: jinfeng.wang@mobvista.com
* @phone: 152-1062-7698
*/
public
class
GetPackageFromAdxFile
{
public
static
void
main
(
String
[]
args
)
{
try
{
String
filePath
=
""
;
if
(
args
.
length
==
1
)
{
filePath
=
args
[
0
];
}
else
{
System
.
out
.
println
(
"Please check input!"
);
System
.
exit
(
1
);
}
Set
<
String
>
set
=
readTxtFileIntoStringSet
(
filePath
);
System
.
out
.
println
(
"packageNames.size ===>>> "
+
set
.
size
());
// MySqlUtil mySqlUtil = new MySqlUtil();
// mySqlUtil.insert(set);
}
catch
(
Exception
e
)
{
System
.
err
.
println
(
e
.
getClass
().
getName
()
+
": "
+
e
.
getMessage
());
}
}
public
static
Set
<
String
>
readTxtFileIntoStringSet
(
String
filePath
)
{
Set
<
String
>
set
=
new
HashSet
<>();
try
{
String
encoding
=
"GBK"
;
File
file
=
new
File
(
filePath
);
// 判断文件是否存在
if
(
file
.
isFile
()
&&
file
.
exists
())
{
// 考虑到编码格式
InputStreamReader
read
=
new
InputStreamReader
(
new
FileInputStream
(
file
),
encoding
);
BufferedReader
bufferedReader
=
new
BufferedReader
(
read
);
String
lineTxt
;
while
((
lineTxt
=
bufferedReader
.
readLine
())
!=
null
)
{
String
packageName
=
lineTxt
;
set
.
add
(
packageName
);
}
bufferedReader
.
close
();
read
.
close
();
}
else
{
System
.
out
.
println
(
"找不到指定的文件"
);
}
}
catch
(
Exception
e
)
{
System
.
out
.
println
(
"读取文件内容出错"
);
e
.
printStackTrace
();
}
return
set
;
}
}
This diff is collapsed.
Click to expand it.
src/main/java/com/mobvista/apptag/utils/MySqlUtil.java
View file @
6ff13b0b
...
...
@@ -30,7 +30,7 @@ public class MySqlUtil {
Pattern
iosPkgPtn
=
Pattern
.
compile
(
"^\\d+$"
);
Pattern
adrPkgPtn
=
Pattern
.
compile
(
"^[0-9a-zA-Z\\.]+$"
);
if
(
iosPkgPtn
.
matcher
(
packageName
).
matches
()
||
adrPkgPtn
.
matcher
(
packageName
).
matches
())
{
if
(
packageName
.
contains
(
"."
))
{
if
(
adrPkgPtn
.
matcher
(
packageName
).
matches
(
))
{
platform
=
"Android"
;
}
else
{
platform
=
"IOS"
;
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment