Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
S
saasio
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
reyun
saasio
Commits
7d708f59
Commit
7d708f59
authored
6 years ago
by
carrieyzzhang
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
export
parent
cc658f18
Hide whitespace changes
Inline
Side-by-side
Showing
15 changed files
with
605 additions
and
14 deletions
+605
-14
pom.xml
pom.xml
+15
-0
saasio.iml
saasio.iml
+5
-0
EventStatsController.java
src/main/java/com/reyun/controller/EventStatsController.java
+2
-2
FunnelController.java
src/main/java/com/reyun/controller/FunnelController.java
+1
-1
IntelligentPathController.java
.../java/com/reyun/controller/IntelligentPathController.java
+1
-1
RetentionController.java
src/main/java/com/reyun/controller/RetentionController.java
+2
-2
AccountFlowRestrictServiceImpl.java
...om/reyun/service/impl/AccountFlowRestrictServiceImpl.java
+1
-1
ExportReportTask.java
src/main/java/com/reyun/task/ExportReportTask.java
+75
-0
ExportReportThread.java
src/main/java/com/reyun/taskexecute/ExportReportThread.java
+486
-0
AwsS3Util.java
src/main/java/com/reyun/util/AwsS3Util.java
+2
-2
Constant.java
src/main/java/com/reyun/util/Constant.java
+7
-2
SendCommonPostMail.java
src/main/java/com/reyun/util/SendCommonPostMail.java
+2
-3
applicationContext-schedule.xml
src/main/resources/applicationContext-schedule.xml
+2
-0
common.properties
src/main/resources/common.properties
+1
-0
key.properties
src/main/resources/key.properties
+3
-0
No files found.
pom.xml
View file @
7d708f59
...
...
@@ -91,6 +91,11 @@
<account.check.url>
http://localhost:8091
</account.check.url>
<ddb.accesskey>
AKIAOS2UEXIFNVER2O6A
</ddb.accesskey>
<ddb.secretkey>
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
</ddb.secretkey>
<report.export.task>
ExportReportTask
</report.export.task>
</properties>
</profile>
<profile>
...
...
@@ -201,6 +206,11 @@
<account.check.url>
http://52.80.113.103:8006
</account.check.url>
<ddb.accesskey>
AKIAOS2UEXIFNVER2O6A
</ddb.accesskey>
<ddb.secretkey>
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
</ddb.secretkey>
<report.export.task>
ExportReportTask
</report.export.task>
</properties>
</profile>
...
...
@@ -271,6 +281,11 @@
<account.check.url>
http://io.reyun.com
</account.check.url>
<ddb.accesskey>
AKIAPMJ4QRXP6BECFUVQ
</ddb.accesskey>
<ddb.secretkey>
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
</ddb.secretkey>
<report.export.task>
ExportReportTask
</report.export.task>
</properties>
</profile>
</profiles>
...
...
This diff is collapsed.
Click to expand it.
saasio.iml
View file @
7d708f59
...
...
@@ -12,6 +12,11 @@
</webroots>
</configuration>
</facet>
<facet
type=
"Spring"
name=
"Spring"
>
<configuration>
<fileset
id=
"fileset"
name=
"Spring Application Context"
removed=
"false"
/>
</configuration>
</facet>
</component>
<component
name=
"NewModuleRootManager"
LANGUAGE_LEVEL=
"JDK_1_8"
inherit-compiler-output=
"false"
>
<output
url=
"file://$MODULE_DIR$/target/classes"
/>
...
...
This diff is collapsed.
Click to expand it.
src/main/java/com/reyun/controller/EventStatsController.java
View file @
7d708f59
...
...
@@ -43,7 +43,7 @@ public class EventStatsController {
if
(!
StringUtil
.
isEmpty
(
eventCondition
)
&&
!
StringUtil
.
isEmpty
(
startDate
)
&&
!
StringUtil
.
isEmpty
(
endDate
))
{
return
ResultModel
.
OK
(
complicateEventsService
.
queryComplicatedEvent
(
appId
,
account
.
getId
(),
startDate
,
endDate
,
eventCondition
,
viewFlag
,
1
));
eventCondition
,
viewFlag
,
Constant
.
iscache
));
}
else
{
return
ResultModel
.
ERROR
(
ResultStatus
.
PARAM_INVALID
);
...
...
@@ -64,7 +64,7 @@ public class EventStatsController {
if
(!
StringUtil
.
isEmpty
(
eventCondition
)
&&
!
StringUtil
.
isEmpty
(
startDate
)
&&
!
StringUtil
.
isEmpty
(
endDate
))
{
return
ResultModel
.
OK
(
complicateEventsService
.
queryDistinctTotalData
(
appId
,
account
.
getId
(),
startDate
,
endDate
,
eventCondition
,
viewFlag
,
Constant
.
i
a
cache
));
eventCondition
,
viewFlag
,
Constant
.
i
s
cache
));
}
else
{
return
ResultModel
.
ERROR
(
ResultStatus
.
PARAM_INVALID
);
...
...
This diff is collapsed.
Click to expand it.
src/main/java/com/reyun/controller/FunnelController.java
View file @
7d708f59
...
...
@@ -58,7 +58,7 @@ public class FunnelController {
isProfile
=
true
;
}
boolean
isDevice
=
StringUtil
.
isEmpty
(
isDeviceStr
)?
false
:
"true"
.
equals
(
isDeviceStr
)?
true
:
false
;
Map
<
String
,
List
>
result
=
funnelService
.
funnelReport
(
funnel
,
startDate
,
endDate
,
usergroupStr
,
isList
,
dimentionStr
,
loginAccount
,
isDevice
,
isProfile
,
Constant
.
i
a
cache
);
Map
<
String
,
List
>
result
=
funnelService
.
funnelReport
(
funnel
,
startDate
,
endDate
,
usergroupStr
,
isList
,
dimentionStr
,
loginAccount
,
isDevice
,
isProfile
,
Constant
.
i
s
cache
);
if
(
result
.
isEmpty
())
{
return
ResultModel
.
ERROR
(
ResultStatus
.
NETWORK_ERROR
);
...
...
This diff is collapsed.
Click to expand it.
src/main/java/com/reyun/controller/IntelligentPathController.java
View file @
7d708f59
...
...
@@ -75,6 +75,6 @@ public class IntelligentPathController {
intelligentPath
.
setUserGroup
(
Long
.
parseLong
(
userGroup
));
}
return
ResultModel
.
OK
(
intelligentPathService
.
queryIntelligentPath
(
loginAccount
,
intelligentPath
,
Constant
.
i
a
cache
));
return
ResultModel
.
OK
(
intelligentPathService
.
queryIntelligentPath
(
loginAccount
,
intelligentPath
,
Constant
.
i
s
cache
));
}
}
This diff is collapsed.
Click to expand it.
src/main/java/com/reyun/controller/RetentionController.java
View file @
7d708f59
...
...
@@ -86,7 +86,7 @@ public class RetentionController {
String
endDate
=
this
.
getEndDateByReportView
(
end
,
reportView
);
Map
<
String
,
List
>
result
=
retentionService
.
retentionDetailReport
(
loginAccount
,
startDate
,
endDate
,
isList
,
eventInfo
,
app
,
userGroupStr
,
dimensionStr
,
reportView
,
interval
,
isDevice
,
retentiontype
,
isProfile
,
Constant
.
i
a
cache
);
userGroupStr
,
dimensionStr
,
reportView
,
interval
,
isDevice
,
retentiontype
,
isProfile
,
Constant
.
i
s
cache
);
if
(
retentiontype
.
equals
(
"define"
))
{
userLogService
.
insertLog
(
loginAccount
,
LogEnumType
.
FIND
.
getCode
(),
LogEnumType
.
FIND
.
getName
()
+
"留存明细报表"
,
...
...
@@ -146,7 +146,7 @@ public class RetentionController {
Map
<
String
,
List
>
result
=
retentionService
.
retentionReport
(
startDate
,
endDate
,
isList
,
eventInfo
,
app
,
userGroupStr
,
dimensionStr
,
reportView
,
interval
,
isDevice
,
loginAccount
.
getId
(),
retentiontype
,
isProfile
,
Constant
.
ia
cache
);
dimensionStr
,
reportView
,
interval
,
isDevice
,
loginAccount
.
getId
(),
retentiontype
,
isProfile
,
Constant
.
is
cache
);
if
(
retentiontype
.
equals
(
"define"
))
{
userLogService
.
insertLog
(
loginAccount
,
LogEnumType
.
FIND
.
getCode
(),
LogEnumType
.
FIND
.
getName
()
+
"留存报表"
,
...
...
This diff is collapsed.
Click to expand it.
src/main/java/com/reyun/service/impl/AccountFlowRestrictServiceImpl.java
View file @
7d708f59
...
...
@@ -391,7 +391,7 @@ public class AccountFlowRestrictServiceImpl implements AccountFlowRestrictServic
conditions
.
put
(
"startdate"
,
createTime
);
conditions
.
put
(
"enddate"
,
pastDate
);
conditions
.
put
(
"datatype"
,
"list"
);
conditions
.
put
(
"iscache"
,
Constant
.
iscache
);
conditions
.
put
(
"iscache"
,
Constant
.
iscache
.
toString
()
);
String
url
=
Constant
.
reportUrl
+
"/api/trackingio/"
+
reportName
+
"/"
+
appId
;
String
responseJson
=
HttpClientUtil
.
doHttpPostRequest
(
url
,
"trackingio"
,
conditions
);
HashMap
<
String
,
String
>
resultValMap
=
new
HashMap
<>();
...
...
This diff is collapsed.
Click to expand it.
src/main/java/com/reyun/task/ExportReportTask.java
0 → 100644
View file @
7d708f59
package
com
.
reyun
.
task
;
import
com.reyun.dic.DownloadStatusEnumType
;
import
com.reyun.model.ExportReportInfo
;
import
com.reyun.service.ExportReportInfoService
;
import
com.reyun.taskexecute.ExportReportThread
;
import
com.reyun.util.AwsS3Util
;
import
com.reyun.util.Constant
;
import
com.reyun.util.RedisUtilNew
;
import
org.apache.commons.lang.time.DateUtils
;
import
org.joda.time.DateTime
;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
org.springframework.beans.factory.annotation.Autowired
;
import
org.springframework.stereotype.Service
;
import
java.util.List
;
/**
* Created by zxy on 17/4/27.
* description:报表导出
*/
@Service
public
class
ExportReportTask
{
private
static
Logger
logger
=
LoggerFactory
.
getLogger
(
ExportReportTask
.
class
);
private
static
int
EXPIRE_PERIOD
=
(
int
)
DateUtils
.
MILLIS_PER_MINUTE
*
1
/
1000
;
private
static
final
String
S3_DOWNLOAD_BUCKET
=
"reyundownload-test"
;
private
static
final
String
DOWNLOAD_PATH
=
"trackingio/reportdownload/"
;
@Autowired
private
ExportReportInfoService
exportService
;
public
void
run
(){
logger
.
info
(
"导出报表结果数据................//Begin, 执行时间: {}"
,
DateTime
.
now
());
if
(
RedisUtilNew
.
getInstance
().
tryLock
(
Constant
.
reportExportTask
,
EXPIRE_PERIOD
)
!=
null
)
{
//生成数据
List
<
ExportReportInfo
>
exportList
=
exportService
.
findExport2Deal
();
for
(
ExportReportInfo
export
:
exportList
)
{
logger
.
info
(
export
.
getFileName
());
Thread
exportReportThread
=
new
ExportReportThread
(
export
);
exportReportThread
.
start
();
}
//删除已过期文件
List
<
ExportReportInfo
>
invalidExportInfoList
=
exportService
.
findInvalidExportList
();
for
(
ExportReportInfo
invalidExport
:
invalidExportInfoList
)
{
final
String
s3key
=
DOWNLOAD_PATH
+
invalidExport
.
getFileName
();
//删除文件
if
(
AwsS3Util
.
getInstance
().
getAmazonS3
().
doesObjectExist
(
S3_DOWNLOAD_BUCKET
,
s3key
))
{
AwsS3Util
.
getInstance
().
deleteS3Object
(
S3_DOWNLOAD_BUCKET
,
s3key
);
}
//跟新文件为失效状态
exportService
.
updateStatus
(
invalidExport
.
getId
(),
DownloadStatusEnumType
.
INVALID
.
getCode
());
}
}
else
{
logger
.
info
(
"ExportReportTask正在执行中......"
);
}
logger
.
info
(
"导出报表结果数据................//End"
);
}
}
This diff is collapsed.
Click to expand it.
src/main/java/com/reyun/taskexecute/ExportReportThread.java
0 → 100644
View file @
7d708f59
package
com
.
reyun
.
taskexecute
;
import
com.reyun.context.AppUtils
;
import
com.reyun.dic.DownloadStatusEnumType
;
import
com.reyun.dic.FunctionEnumType
;
import
com.reyun.model.*
;
import
com.reyun.service.*
;
import
com.reyun.util.*
;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
org.springframework.util.CollectionUtils
;
import
org.springframework.util.StringUtils
;
import
java.io.IOException
;
import
java.net.URLDecoder
;
import
java.nio.charset.StandardCharsets
;
import
java.text.ParseException
;
import
java.text.SimpleDateFormat
;
import
java.util.*
;
/**
* Created by zxy on 17/4/27.
* description:生成报表导出结果文件
*/
public
class
ExportReportThread
extends
Thread
{
private
static
final
Logger
logger
=
LoggerFactory
.
getLogger
(
ExportReportThread
.
class
);
private
ExportReportInfo
export
;
private
ExportReportInfoService
exportService
=
AppUtils
.
getApplicationContext
().
getBean
(
ExportReportInfoService
.
class
);
private
FunnelService
funnelService
=
AppUtils
.
getApplicationContext
().
getBean
(
FunnelService
.
class
);
private
AuthService
authService
=
AppUtils
.
getApplicationContext
().
getBean
(
AuthService
.
class
);
private
EventStatsService
eventStatsService
=
AppUtils
.
getApplicationContext
().
getBean
(
EventStatsService
.
class
);
private
ComplicateEventsService
complicateEventsService
=
AppUtils
.
getApplicationContext
().
getBean
(
ComplicateEventsService
.
class
);
private
RetentionService
retentionService
=
AppUtils
.
getApplicationContext
().
getBean
(
RetentionService
.
class
);
private
static
final
String
S3_DOWNLOAD_BUCKET
=
"reyundownload"
;
private
static
final
String
DOWNLOAD_PATH
=
"trackingio/reportdownload/"
;
private
static
final
String
VIEW_TYPE_DAY
=
"day"
;
private
static
final
String
VIEW_TYPE_WEEK
=
"week"
;
private
static
final
String
VIEW_TYPE_MONTH
=
"month"
;
public
ExportReportThread
(
ExportReportInfo
export
)
{
this
.
export
=
export
;
this
.
setName
(
"ExportReportThread"
);
}
@Override
public
void
run
()
{
boolean
success
=
true
;
try
{
Map
<
String
,
String
>
conditions
=
ExportReportUtil
.
changeCondition2Map
(
export
.
getConditions
());
StringBuffer
sb
=
new
StringBuffer
();
Account
account
=
authService
.
getSubAccountAuthById
(
export
.
getAccount
());
if
(
export
.
getFunctionType
().
equals
(
FunctionEnumType
.
FUNNEL
.
getKey
()))
{
//必传参数:funnel、startdate、enddate
Long
funnel
=
export
.
getFunctionId
();
String
startDate
=
conditions
.
get
(
"startdate"
);
String
endDate
=
conditions
.
get
(
"enddate"
);
if
(
funnel
==
null
||
StringUtil
.
isEmpty
(
startDate
)
||
StringUtil
.
isEmpty
(
endDate
))
{
logger
.
info
(
"漏斗下载条件不全:startDate:"
+
startDate
+
", endDate:"
+
endDate
+
", funnel:"
+
funnel
);
success
=
false
;
}
else
{
String
userGroupStr
=
conditions
.
get
(
"usergroupid"
);
String
dimensionStr
=
conditions
.
get
(
"dimention"
);
String
isDeviceStr
=
conditions
.
get
(
"isdevice"
);
boolean
isList
=
conditions
.
get
(
"datatype"
)
!=
null
&&
"list"
.
equals
(
conditions
.
get
(
"datatype"
));
boolean
isDevice
=
!
StringUtil
.
isEmpty
(
isDeviceStr
)
&&
"true"
.
equals
(
isDeviceStr
);
String
eventType
=
conditions
.
get
(
"eventType"
);
boolean
isProfile
=
false
;
if
(!
StringUtils
.
isEmpty
(
eventType
)
&&
"profile"
.
equals
(
eventType
)){
isProfile
=
true
;
}
Map
<
String
,
List
>
result
=
funnelService
.
funnelReport
(
funnel
,
startDate
,
endDate
,
userGroupStr
,
isList
,
dimensionStr
,
account
,
isDevice
,
isProfile
,
Constant
.
iscache
);
this
.
dealReportRateName
(
result
);
sb
=
getExportContent
(
result
);
}
}
else
if
(
export
.
getFunctionType
().
equals
(
FunctionEnumType
.
EVENTSTATS
.
getKey
()))
{
//必传参数:eventCondition、startdate、enddate
String
startDate
=
conditions
.
get
(
"startdate"
);
String
endDate
=
conditions
.
get
(
"enddate"
);
String
eventCondition
=
conditions
.
get
(
"eventCondition"
);
if
(
StringUtil
.
isEmpty
(
eventCondition
)
||
StringUtil
.
isEmpty
(
startDate
)
||
StringUtil
.
isEmpty
(
endDate
))
{
logger
.
info
(
"事件分析下载条件不全:startDate:"
+
startDate
+
", endDate:"
+
endDate
+
", event:"
+
eventCondition
);
success
=
false
;
}
else
{
String
groupCondition
=
conditions
.
get
(
"dimension"
);
String
viewFlag
=
conditions
.
get
(
"viewflag"
);
boolean
isDevice
=
Boolean
.
valueOf
(
conditions
.
get
(
"isdevice"
));
Map
<
String
,
List
>
result
=
eventStatsService
.
queryEventStatsData
(
export
.
getApp
(),
account
.
getId
(),
startDate
,
endDate
,
groupCondition
,
isDevice
,
eventCondition
,
viewFlag
);
sb
=
getExportContent
(
result
);
}
}
else
if
(
FunctionEnumType
.
COMPLICATED_EVENTS
.
getKey
().
equals
(
export
.
getFunctionType
()))
{
//必传参数:eventCondition、startdate、enddate
String
startDate
=
conditions
.
get
(
"startdate"
);
String
endDate
=
conditions
.
get
(
"enddate"
);
String
eventCondition
=
conditions
.
get
(
"eventCondition"
);
eventCondition
=
URLDecoder
.
decode
(
eventCondition
,
StandardCharsets
.
UTF_8
.
toString
());
if
(
StringUtil
.
isEmpty
(
eventCondition
)
||
StringUtil
.
isEmpty
(
startDate
)
||
StringUtil
.
isEmpty
(
endDate
))
{
logger
.
info
(
"复杂事件下载条件不全:startDate:"
+
startDate
+
", endDate:"
+
endDate
+
", event:"
+
eventCondition
);
success
=
false
;
}
else
{
String
viewFlag
=
conditions
.
get
(
"viewflag"
);
Map
<
String
,
List
>
result
=
complicateEventsService
.
queryComplicatedEvent
(
export
.
getApp
(),
account
.
getId
(),
startDate
,
endDate
,
eventCondition
,
viewFlag
,
Constant
.
iscache
);
sb
=
getTotalExportContent
(
result
,
export
.
getStartDate
(),
export
.
getEndDate
());
//sb = getExportContent(result);
}
}
else
if
(
export
.
getFunctionType
().
equals
(
FunctionEnumType
.
CUSTOMRETENTION
.
getKey
()))
{
//必传参数:eventinfo、startdate、enddate
String
start
=
conditions
.
get
(
"startdate"
);
String
end
=
conditions
.
get
(
"enddate"
);
String
eventInfo
=
conditions
.
get
(
"eventinfo"
);
String
retentiontype
=
conditions
.
get
(
"retentiontype"
);
if
(
StringUtil
.
isEmpty
(
retentiontype
)
||
(
"define"
.
equals
(
retentiontype
)
&&
StringUtil
.
isEmpty
(
eventInfo
))
||
StringUtil
.
isEmpty
(
start
)
||
StringUtil
.
isEmpty
(
end
))
{
logger
.
info
(
"自定义留存概览下载条件不全:startDate:"
+
start
+
", endDate:"
+
end
+
", event:"
+
eventInfo
);
success
=
false
;
}
else
{
String
isDeviceStr
=
conditions
.
get
(
"isdevice"
);
String
reportView
=
conditions
.
get
(
"reportview"
);
String
userGroupStr
=
conditions
.
get
(
"usergroupid"
);
String
dimensionStr
=
conditions
.
get
(
"dimention"
);
boolean
isDevice
=
!
StringUtil
.
isEmpty
(
isDeviceStr
)
&&
"true"
.
equals
(
isDeviceStr
);
boolean
isList
=
conditions
.
get
(
"datatype"
)
!=
null
&&
"list"
.
equals
(
conditions
.
get
(
"datatype"
));
String
type
=
conditions
.
get
(
"retentiontype"
);
if
(
StringUtil
.
isEmpty
(
reportView
))
{
reportView
=
"day"
;
}
int
interval
=
this
.
getIntervalByViewType
(
start
,
end
,
reportView
);
String
startDate
=
this
.
getStartDateByViewType
(
start
,
reportView
);
String
endDate
=
this
.
getEndDateByViewType
(
end
,
reportView
);
String
eventType
=
conditions
.
get
(
"eventType"
);
boolean
isProfile
=
false
;
if
(!
StringUtils
.
isEmpty
(
eventType
)
&&
"profile"
.
equals
(
eventType
)){
isProfile
=
true
;
}
Map
<
String
,
List
>
result
=
retentionService
.
retentionReport
(
startDate
,
endDate
,
isList
,
eventInfo
,
export
.
getApp
(),
userGroupStr
,
dimensionStr
,
reportView
,
interval
,
isDevice
,
account
.
getId
(),
retentiontype
,
isProfile
,
Constant
.
iscache
);
this
.
dealRetentionName
(
result
);
sb
=
getExportContent
(
result
);
}
}
else
if
(
export
.
getFunctionType
().
equals
(
FunctionEnumType
.
DETAILCUSTOMRETENTION
.
getKey
()))
{
//必传参数:eventinfo、startdate、enddate
String
start
=
conditions
.
get
(
"startdate"
);
String
end
=
conditions
.
get
(
"enddate"
);
String
eventInfo
=
conditions
.
get
(
"eventinfo"
);
if
(
StringUtil
.
isEmpty
(
eventInfo
)
||
StringUtil
.
isEmpty
(
start
)
||
StringUtil
.
isEmpty
(
end
))
{
logger
.
info
(
"自定义留存明细下载条件不全:startDate:"
+
start
+
", endDate:"
+
end
+
", event:"
+
eventInfo
);
success
=
false
;
}
else
{
String
isDeviceStr
=
conditions
.
get
(
"isdevice"
);
String
reportView
=
conditions
.
get
(
"reportview"
);
String
userGroupStr
=
conditions
.
get
(
"usergroupid"
);
String
dimensionStr
=
conditions
.
get
(
"dimention"
);
boolean
isDevice
=
!
StringUtil
.
isEmpty
(
isDeviceStr
)
&&
"true"
.
equals
(
isDeviceStr
);
boolean
isList
=
conditions
.
get
(
"datatype"
)
!=
null
&&
"list"
.
equals
(
conditions
.
get
(
"datatype"
));
String
type
=
conditions
.
get
(
"retentiontype"
);
if
(
StringUtil
.
isEmpty
(
reportView
))
{
reportView
=
"day"
;
}
int
interval
=
this
.
getIntervalByViewType
(
start
,
end
,
reportView
);
String
startDate
=
this
.
getStartDateByViewType
(
start
,
reportView
);
String
endDate
=
this
.
getEndDateByViewType
(
end
,
reportView
);
String
eventType
=
conditions
.
get
(
"eventType"
);
boolean
isProfile
=
false
;
if
(!
StringUtils
.
isEmpty
(
eventType
)
&&
"profile"
.
equals
(
eventType
)){
isProfile
=
true
;
}
Map
<
String
,
List
>
result
=
retentionService
.
retentionDetailReport
(
account
,
startDate
,
endDate
,
isList
,
eventInfo
,
export
.
getApp
(),
userGroupStr
,
dimensionStr
,
reportView
,
interval
,
isDevice
,
type
,
isProfile
,
Constant
.
iscache
);
this
.
dealRetentionName
(
result
);
sb
=
getExportContent
(
result
);
}
}
logger
.
info
(
export
.
getId
()
+
":"
+
export
.
getFileName
()
+
":\n"
+
sb
.
toString
());
//上传到S3服务器
uploadFileToS3
(
sb
,
export
.
getFileName
());
}
catch
(
Exception
e
)
{
success
=
false
;
e
.
printStackTrace
();
logger
.
error
(
e
.
getMessage
());
}
//更新导出记录
if
(
success
)
{
updateStatus
(
export
.
getId
(),
DownloadStatusEnumType
.
COMPLETE
.
getCode
());
}
else
{
updateStatus
(
export
.
getId
(),
DownloadStatusEnumType
.
FAILED
.
getCode
());
}
}
/**
* 根据日周月获取开始时间
* created by sunhao 20170510
*/
private
String
getStartDateByViewType
(
String
startDate
,
String
viewType
)
{
String
resultStartDate
=
startDate
;
switch
(
viewType
)
{
case
VIEW_TYPE_DAY:
resultStartDate
=
startDate
;
break
;
case
VIEW_TYPE_WEEK:
resultStartDate
=
DateUtil
.
getFirstDayStrOfWeek
(
DateUtil
.
parseDate
(
startDate
));
break
;
case
VIEW_TYPE_MONTH:
resultStartDate
=
DateUtil
.
getFirstDayOfMonth
(
DateUtil
.
parseDate
(
startDate
));
break
;
}
return
resultStartDate
;
}
/**
* 根据日周月获取结束时间
* created by sunhao 20170510
*/
private
String
getEndDateByViewType
(
String
endDate
,
String
viewType
)
{
String
resultEndDate
=
endDate
;
switch
(
viewType
)
{
case
VIEW_TYPE_DAY:
resultEndDate
=
endDate
;
break
;
case
VIEW_TYPE_WEEK:
resultEndDate
=
DateUtil
.
getLastDayStrOfWeek
(
DateUtil
.
parseDate
(
endDate
));
break
;
case
VIEW_TYPE_MONTH:
resultEndDate
=
DateUtil
.
getLastDayOfMonth
(
DateUtil
.
parseDate
(
endDate
));
break
;
}
return
resultEndDate
;
}
/**
* 根据日周月获取时间间隔
* created by sunhao 20170510
*/
private
int
getIntervalByViewType
(
String
startDate
,
String
endDate
,
String
viewType
)
{
int
interval
=
0
;
switch
(
viewType
)
{
case
VIEW_TYPE_DAY:
interval
=
DateUtil
.
getDateInterval
(
startDate
,
endDate
).
size
();
break
;
case
VIEW_TYPE_WEEK:
interval
=
DateUtil
.
getEveryWeek
(
startDate
,
endDate
).
size
();
break
;
case
VIEW_TYPE_MONTH:
interval
=
DateUtil
.
getEveryMonth
(
startDate
,
endDate
).
size
();
break
;
}
return
interval
;
}
private
void
dealRetentionName
(
Map
<
String
,
List
>
result
)
{
List
<
String
>
names
=
result
.
get
(
"name"
);
List
<
String
>
columnKey
=
result
.
get
(
"columnkey"
);
List
<
String
>
newColumnKey
=
new
ArrayList
<>();
List
<
String
>
newName
=
new
ArrayList
<>();
for
(
int
i
=
0
;
i
<
columnKey
.
size
();
i
++)
{
if
(
i
>
1
)
{
newColumnKey
.
add
(
2
*
i
-
2
,
columnKey
.
get
(
i
));
newName
.
add
(
2
*
i
-
2
,
names
.
get
(
i
));
newColumnKey
.
add
(
2
*
i
-
1
,
"rate_"
+
columnKey
.
get
(
i
));
newName
.
add
(
2
*
i
-
1
,
names
.
get
(
i
)
+
"留存率"
);
}
else
{
newColumnKey
.
add
(
i
,
columnKey
.
get
(
i
));
newName
.
add
(
i
,
names
.
get
(
i
));
}
}
result
.
put
(
"name"
,
newName
);
result
.
put
(
"columnkey"
,
newColumnKey
);
}
/**
* 处理报表中带rate的名字
* created by sunhao 20170510
*/
private
void
dealReportRateName
(
Map
<
String
,
List
>
result
)
{
List
<
String
>
name
=
result
.
get
(
"name"
);
List
<
String
>
columnKey
=
result
.
get
(
"columnkey"
);
List
<
String
>
newColumnKey
=
new
ArrayList
<>();
List
<
String
>
newName
=
new
ArrayList
<>();
for
(
int
i
=
0
;
i
<
columnKey
.
size
();
i
++)
{
if
(
i
!=
0
)
{
newColumnKey
.
add
(
2
*
i
-
1
,
columnKey
.
get
(
i
));
newName
.
add
(
2
*
i
-
1
,
name
.
get
(
i
));
newColumnKey
.
add
(
2
*
i
,
"rate_"
+
columnKey
.
get
(
i
));
newName
.
add
(
2
*
i
,
name
.
get
(
i
)
+
"率"
);
}
else
{
newColumnKey
.
add
(
i
,
columnKey
.
get
(
i
));
newName
.
add
(
i
,
name
.
get
(
i
));
}
}
result
.
put
(
"name"
,
newName
);
result
.
put
(
"columnkey"
,
newColumnKey
);
}
private
StringBuffer
getExportContent
(
Map
<
String
,
List
>
result
)
{
StringBuffer
sb
=
new
StringBuffer
();
List
<
String
>
name
=
result
.
get
(
"name"
);
List
<
String
>
columnKey
=
result
.
get
(
"columnkey"
);
List
<
Map
<
String
,
Object
>>
value
=
result
.
get
(
"val"
);
if
(!
CollectionUtils
.
isEmpty
(
name
))
{
sb
.
append
(
String
.
join
(
","
,
name
)).
append
(
"\r\n"
);
for
(
Map
<
String
,
Object
>
v
:
value
)
{
for
(
int
i
=
0
;
i
<
columnKey
.
size
();
i
++)
{
sb
.
append
(
v
.
get
(
columnKey
.
get
(
i
))).
append
(
columnKey
.
get
(
i
).
startsWith
(
"rate_"
)
?
"%"
:
""
)
.
append
(
i
==
columnKey
.
size
()
-
1
?
"\r\n"
:
","
);
}
}
}
return
sb
;
}
private
StringBuffer
getTotalExportContent
(
Map
<
String
,
List
>
result
,
String
startDate
,
String
endDate
)
{
StringBuffer
sb
=
new
StringBuffer
();
List
<
String
>
name
=
result
.
get
(
"name"
);
List
<
String
>
columnKey
=
result
.
get
(
"columnkey"
);
List
<
Map
<
String
,
Object
>>
value
=
result
.
get
(
"val"
);
List
<
Map
<
String
,
Object
>>
totalValue
=
result
.
get
(
"totalval"
);
SimpleDateFormat
sdf1
=
new
SimpleDateFormat
(
"yyyy年MM月dd日"
);
SimpleDateFormat
sdf2
=
new
SimpleDateFormat
(
"yyyy-MM-dd"
);
Date
parseStartDate
=
new
Date
();
Date
parseEndDate
=
new
Date
();
try
{
parseStartDate
=
sdf2
.
parse
(
startDate
);
parseEndDate
=
sdf2
.
parse
(
endDate
);
}
catch
(
ParseException
e
)
{
e
.
printStackTrace
();
}
String
startDate1
=
sdf1
.
format
(
parseStartDate
);
String
endDate1
=
sdf1
.
format
(
parseEndDate
);
String
dateStr
=
startDate1
+
"-"
+
endDate1
;
if
(!
CollectionUtils
.
isEmpty
(
name
))
{
sb
.
append
(
String
.
join
(
","
,
name
)).
append
(
"\r\n"
);
sb
.
append
(
"汇总"
+
"\r\n"
);
for
(
Map
<
String
,
Object
>
v
:
totalValue
)
{
for
(
int
i
=
0
;
i
<
columnKey
.
size
();
i
++)
{
if
(
i
==
0
)
{
sb
.
append
(
dateStr
+
","
);
}
else
{
sb
.
append
(
v
.
get
(
columnKey
.
get
(
i
))).
append
(
columnKey
.
get
(
i
).
startsWith
(
"rate_"
)
?
"%"
:
""
)
.
append
(
i
==
columnKey
.
size
()
-
1
?
"\r\n"
:
","
);
}
}
}
sb
.
append
(
"明细"
+
"\r\n"
);
for
(
Map
<
String
,
Object
>
v
:
value
)
{
for
(
int
i
=
0
;
i
<
columnKey
.
size
();
i
++)
{
sb
.
append
(
v
.
get
(
columnKey
.
get
(
i
))).
append
(
columnKey
.
get
(
i
).
startsWith
(
"rate_"
)
?
"%"
:
""
)
.
append
(
i
==
columnKey
.
size
()
-
1
?
"\r\n"
:
","
);
}
}
}
return
sb
;
}
private
StringBuffer
getExportContent
(
ReportChartResult
reportChartResult
)
{
StringBuffer
sb
=
new
StringBuffer
();
List
<
String
>
column
=
new
ArrayList
<>();
column
.
addAll
(
reportChartResult
.
getColumn
());
column
.
addAll
(
reportChartResult
.
getName
());
List
<
String
>
key
=
reportChartResult
.
getKey
();
List
<
Object
[]>
value
=
reportChartResult
.
getVal
();
if
(!
CollectionUtils
.
isEmpty
(
column
))
{
sb
.
append
(
String
.
join
(
","
,
column
)).
append
(
"\r\n"
);
for
(
int
i
=
0
;
i
<
key
.
size
();
i
++)
{
sb
.
append
(
key
.
get
(
i
)).
append
(
","
);
for
(
int
j
=
0
;
j
<
value
.
size
();
j
++)
{
sb
.
append
(
value
.
get
(
j
)[
i
]).
append
(
j
==
value
.
size
()
-
1
?
"\r\n"
:
","
);
}
}
}
return
sb
;
}
private
void
updateStatus
(
Long
id
,
String
status
)
{
exportService
.
updateStatus
(
id
,
status
);
}
/**
* 上传文件到S3
* created by sunhao 20170509
*/
private
void
uploadFileToS3
(
StringBuffer
fileContent
,
String
fileName
)
throws
IOException
{
final
String
s3key
=
DOWNLOAD_PATH
+
fileName
;
//删除重复文件
if
(
AwsS3Util
.
getInstance
().
getAmazonS3
().
doesObjectExist
(
S3_DOWNLOAD_BUCKET
,
s3key
))
{
AwsS3Util
.
getInstance
().
deleteS3Object
(
S3_DOWNLOAD_BUCKET
,
s3key
);
}
//上传文件
if
(
fileContent
!=
null
)
{
//fileContent无数据
if
(
fileContent
.
capacity
()
==
0
)
{
fileContent
.
append
(
"-无记录-"
);
}
AwsS3Util
.
getInstance
().
uploadStringToS3
(
S3_DOWNLOAD_BUCKET
,
s3key
,
fileContent
.
toString
());
}
}
}
This diff is collapsed.
Click to expand it.
src/main/java/com/reyun/util/AwsS3Util.java
View file @
7d708f59
...
...
@@ -24,8 +24,8 @@ public class AwsS3Util {
private
AmazonS3
s3
;
private
AwsS3Util
()
{
String
accessKey
=
""
;
String
secretKey
=
""
;
String
accessKey
=
Constant
.
accessKey
;
String
secretKey
=
Constant
.
secretKey
;
if
((
accessKey
!=
null
)
&&
(
secretKey
!=
null
))
{
AWSCredentials
credentials
=
new
BasicAWSCredentials
(
accessKey
,
secretKey
);
...
...
This diff is collapsed.
Click to expand it.
src/main/java/com/reyun/util/Constant.java
View file @
7d708f59
...
...
@@ -35,11 +35,11 @@ public class Constant {
public
static
String
reportUrl
=
commonBundle
.
getString
(
"report.url"
);
public
static
String
adminUrl
=
commonBundle
.
getString
(
"admin.url"
);
public
static
String
trackingUrl
=
commonBundle
.
getString
(
"trackingio.url"
);
public
static
String
iscache
=
commonBundle
.
getString
(
"iscache"
);
public
static
String
receiverHost
=
commonBundle
.
getString
(
"receiver.host"
);
public
static
String
receiverPort
=
commonBundle
.
getString
(
"receiver.port"
);
public
static
String
demoUrl
=
commonBundle
.
getString
(
"demo.url"
);
public
static
String
flowneed
=
commonBundle
.
getString
(
"flowneed"
);
public
static
String
reportExportTask
=
commonBundle
.
getString
(
"report.export.task"
);
...
...
@@ -56,5 +56,10 @@ public class Constant {
public
static
ResourceBundle
redis
=
ResourceBundle
.
getBundle
(
"redis"
);
public
static
String
REDIS_IP_REQUEST_TIMES
=
redis
.
getString
(
"redis.ip.requesttimes"
);
public
static
long
cacheLiveTime
=
Long
.
parseLong
(
redis
.
getString
(
"cache.livetime"
));
public
static
Integer
iacache
=
Integer
.
parseInt
(
redis
.
getString
(
"iscache"
));
public
static
Integer
iscache
=
Integer
.
parseInt
(
redis
.
getString
(
"iscache"
));
public
static
ResourceBundle
keyBuddle
=
ResourceBundle
.
getBundle
(
"key"
);
public
static
String
accessKey
=
keyBuddle
.
getString
(
"ddb.accesskey"
);
public
static
String
secretKey
=
keyBuddle
.
getString
(
"ddb.secretkey"
);
}
This diff is collapsed.
Click to expand it.
src/main/java/com/reyun/util/SendCommonPostMail.java
View file @
7d708f59
...
...
@@ -17,7 +17,6 @@ import java.util.List;
public
class
SendCommonPostMail
{
protected
static
Logger
logger
=
LoggerFactory
.
getLogger
(
SendCommonPostMail
.
class
);
static
String
url
=
"http://api.sendcloud.net/apiv2/mail/send"
;
static
String
apiUser
=
"service24h"
;
...
...
@@ -56,11 +55,11 @@ public class SendCommonPostMail {
String
result
=
EntityUtils
.
toString
(
response
.
getEntity
());
success
=
true
;
}
else
{
logger
.
error
(
"sendcloud response status is error......."
);
System
.
out
.
println
(
"sendcloud response status is error......."
);
}
httPost
.
releaseConnection
();
}
catch
(
Exception
e
)
{
logger
.
error
(
"fail to send mail to "
+
toEmail
);
System
.
out
.
println
(
"fail to send mail to "
+
toEmail
);
}
return
success
;
}
...
...
This diff is collapsed.
Click to expand it.
src/main/resources/applicationContext-schedule.xml
View file @
7d708f59
...
...
@@ -10,8 +10,10 @@
<description>
调度配置
</description>
<bean
id=
"autoCalculateFlow"
class=
"com.reyun.task.AutoCalculateFlow"
></bean>
<!--<bean id="exportReportTask" class="com.reyun.task.ExportReportTask"></bean>-->
<task:scheduled-tasks>
<task:scheduled
ref=
"autoCalculateFlow"
method=
"run"
cron=
"0 0 7 * * ?"
/>
<!--<task:scheduled ref="exportReportTask" method="run" cron="0 */2 * * * ?"/>-->
</task:scheduled-tasks>
</beans>
This diff is collapsed.
Click to expand it.
src/main/resources/common.properties
View file @
7d708f59
...
...
@@ -2,6 +2,7 @@ profile.table=${profile.table}
event.table
=
${event.table}
usergroup.table
=
${usergroup.table}
account.check.url
=
${account.check.url}
report.export.task
=
${report.export.task}
receiver.host
=
${receiver.host}
receiver.port
=
${receiver.port}
...
...
This diff is collapsed.
Click to expand it.
src/main/resources/key.properties
0 → 100644
View file @
7d708f59
# Redis common settings
ddb.accesskey
=
${ddb.accesskey}
ddb.secretkey
=
${ddb.secretkey}
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment