Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
M
mobvista-dmp
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
王金锋
mobvista-dmp
Commits
88fee28d
Commit
88fee28d
authored
Jun 16, 2021
by
wang-jinfeng
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
fix rtdmp bug
parent
f604014c
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
9 additions
and
5 deletions
+9
-5
RTDmpASV2.scala
src/main/scala/mobvista/dmp/datasource/rtdmp/RTDmpASV2.scala
+9
-5
No files found.
src/main/scala/mobvista/dmp/datasource/rtdmp/RTDmpASV2.scala
View file @
88fee28d
package
mobvista.dmp.datasource.rtdmp
import
java.net.URI
import
com.alibaba.fastjson.
{
JSONArray
,
JSONObject
}
import
mobvista.dmp.common.
{
CommonSparkJob
,
MobvistaConstant
}
import
mobvista.dmp.format.TextMultipleOutputFormat
...
...
@@ -13,6 +12,7 @@ import org.apache.hadoop.io.compress.{CompressionCodec, GzipCodec}
import
org.apache.spark.sql.SparkSession
import
org.apache.spark.storage.StorageLevel
import
java.net.URI
import
scala.collection.JavaConversions._
import
scala.collection.mutable
import
scala.collection.mutable.ArrayBuffer
...
...
@@ -55,10 +55,14 @@ class RTDmpASV2 extends CommonSparkJob with Serializable {
// 筛选所有的人群包,给一个最大值和一个最小值
var
audience_date_utime_start
=
1577811600L
var
audience_date_utime_end
=
4100731200L
var
update_time_start
=
"2000-01-01 00:00:00"
var
update_time_end
=
"2099-12-31 23:59:59"
var
update_time_start
=
DateUtil
.
format
(
DateUtil
.
getDay
(
time
+
":00:00"
,
"yyyy-MM-dd HH:mm:ss"
,
-
30
),
"yyyy-MM-dd HH:mm:ss"
)
var
update_time_end
=
DateUtil
.
format
(
time
+
":59:59"
,
"yyyy-MM-dd HH:mm:ss"
)
var
audience_date_utime_start
=
DateUtil
.
parse
(
update_time_start
,
"yyyy-MM-dd HH:mm:ss"
).
getTime
/
1000
-
28800
var
audience_date_utime_end
=
DateUtil
.
parse
(
update_time_end
,
"yyyy-MM-dd HH:mm:ss"
).
getTime
/
1000
-
28800
// var audience_date_utime_start = 1577811600L
// var audience_date_utime_end = 4100731200L
// var update_time_start = "2000-01-01 00:00:00"
// var update_time_end = "2099-12-31 23:59:59"
import
scala.collection.JavaConverters._
val
ids
=
ServerUtil
.
request
(
update_time_start
,
update_time_end
,
audience_date_utime_start
,
audience_date_utime_end
,
0
,
0
,
4
).
asScala
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment