Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
atlas
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
dataplatform
atlas
Commits
116fb62c
Commit
116fb62c
authored
6 years ago
by
Ashutosh Mestry
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
ATLAS-2845: Rename AtlasCluster to AtlasServer.
parent
c0a91c7e
Show whitespace changes
Inline
Side-by-side
Showing
18 changed files
with
270 additions
and
220 deletions
+270
-220
0010-base_model.json
addons/models/0000-Area0/0010-base_model.json
+12
-4
001-base_model_replication_attributes.json
...-Area0/patches/001-base_model_replication_attributes.json
+4
-4
AtlasBaseClient.java
...ommon/src/main/java/org/apache/atlas/AtlasBaseClient.java
+5
-5
Constants.java
.../src/main/java/org/apache/atlas/repository/Constants.java
+2
-2
AtlasServer.java
.../main/java/org/apache/atlas/model/impexp/AtlasServer.java
+63
-49
ExportImportAuditEntry.java
...org/apache/atlas/model/impexp/ExportImportAuditEntry.java
+4
-4
AtlasServerService.java
...rg/apache/atlas/repository/impexp/AtlasServerService.java
+36
-14
AuditsWriter.java
...java/org/apache/atlas/repository/impexp/AuditsWriter.java
+43
-40
ExportImportAuditService.java
...che/atlas/repository/impexp/ExportImportAuditService.java
+7
-7
AtlasServerDTO.java
.../java/org/apache/atlas/repository/ogm/AtlasServerDTO.java
+20
-17
ExportImportAuditEntryDTO.java
...pache/atlas/repository/ogm/ExportImportAuditEntryDTO.java
+7
-7
TestModules.java
repository/src/test/java/org/apache/atlas/TestModules.java
+2
-2
AtlasServerServiceTest.java
...pache/atlas/repository/impexp/AtlasServerServiceTest.java
+20
-20
ExportImportAuditServiceTest.java
...atlas/repository/impexp/ExportImportAuditServiceTest.java
+7
-7
ReplicationEntityAttributeTest.java
...las/repository/impexp/ReplicationEntityAttributeTest.java
+8
-11
replicationAttrs.json
...st/resources/json/stocksDB-Entities/replicationAttrs.json
+2
-2
AdminResource.java
...in/java/org/apache/atlas/web/resources/AdminResource.java
+19
-16
AdminExportImportTestIT.java
...g/apache/atlas/web/resources/AdminExportImportTestIT.java
+9
-9
No files found.
addons/models/0000-Area0/0010-base_model.json
View file @
116fb62c
...
...
@@ -100,12 +100,20 @@
]
},
{
"name"
:
"Atlas
Clust
er"
,
"name"
:
"Atlas
Serv
er"
,
"typeVersion"
:
"1.0"
,
"superTypes"
:
[
],
"attributeDefs"
:
[
{
"name"
:
"name"
,
"typeName"
:
"string"
,
"cardinality"
:
"SINGLE"
,
"isIndexable"
:
true
,
"isOptional"
:
false
,
"isUnique"
:
false
},
{
"name"
:
"displayName"
,
"typeName"
:
"string"
,
"cardinality"
:
"SINGLE"
,
...
...
@@ -114,7 +122,7 @@
"isUnique"
:
false
},
{
"name"
:
"
qualified
Name"
,
"name"
:
"
full
Name"
,
"typeName"
:
"string"
,
"cardinality"
:
"SINGLE"
,
"isIndexable"
:
true
,
...
...
@@ -258,7 +266,7 @@
"isUnique"
:
false
},
{
"name"
:
"source
Clust
erName"
,
"name"
:
"source
Serv
erName"
,
"typeName"
:
"string"
,
"cardinality"
:
"SINGLE"
,
"isIndexable"
:
true
,
...
...
@@ -266,7 +274,7 @@
"isUnique"
:
false
},
{
"name"
:
"target
Clust
erName"
,
"name"
:
"target
Serv
erName"
,
"typeName"
:
"string"
,
"cardinality"
:
"SINGLE"
,
"isIndexable"
:
true
,
...
...
This diff is collapsed.
Click to expand it.
addons/models/0000-Area0/patches/001-base_model_replication_attributes.json
View file @
116fb62c
...
...
@@ -8,16 +8,16 @@
"params"
:
null
,
"attributeDefs"
:
[
{
"name"
:
"replicatedFrom
Cluster
"
,
"typeName"
:
"array<Atlas
Clust
er>"
,
"name"
:
"replicatedFrom"
,
"typeName"
:
"array<Atlas
Serv
er>"
,
"cardinality"
:
"SET"
,
"isIndexable"
:
false
,
"isOptional"
:
true
,
"isUnique"
:
false
},
{
"name"
:
"replicatedTo
Cluster
"
,
"typeName"
:
"array<Atlas
Clust
er>"
,
"name"
:
"replicatedTo"
,
"typeName"
:
"array<Atlas
Serv
er>"
,
"cardinality"
:
"SET"
,
"isIndexable"
:
false
,
"isOptional"
:
true
,
...
...
This diff is collapsed.
Click to expand it.
client/common/src/main/java/org/apache/atlas/AtlasBaseClient.java
View file @
116fb62c
...
...
@@ -37,7 +37,7 @@ import com.sun.jersey.multipart.MultiPart;
import
com.sun.jersey.multipart.file.FileDataBodyPart
;
import
com.sun.jersey.multipart.file.StreamDataBodyPart
;
import
com.sun.jersey.multipart.impl.MultiPartWriter
;
import
org.apache.atlas.model.impexp.Atlas
Clust
er
;
import
org.apache.atlas.model.impexp.Atlas
Serv
er
;
import
org.apache.atlas.model.impexp.AtlasExportRequest
;
import
org.apache.atlas.model.impexp.AtlasImportRequest
;
import
org.apache.atlas.model.impexp.AtlasImportResult
;
...
...
@@ -80,7 +80,7 @@ public abstract class AtlasBaseClient {
public
static
final
String
ADMIN_METRICS
=
"admin/metrics"
;
public
static
final
String
ADMIN_IMPORT
=
"admin/import"
;
public
static
final
String
ADMIN_EXPORT
=
"admin/export"
;
public
static
final
String
ADMIN_
CLUSTER_TEMPLATE
=
"%sadmin/clust
er/%s"
;
public
static
final
String
ADMIN_
SERVER_TEMPLATE
=
"%sadmin/serv
er/%s"
;
public
static
final
String
QUERY
=
"query"
;
public
static
final
String
LIMIT
=
"limit"
;
...
...
@@ -526,9 +526,9 @@ public abstract class AtlasBaseClient {
return
new
FormDataBodyPart
(
IMPORT_REQUEST_PARAMTER
,
AtlasType
.
toJson
(
request
),
MediaType
.
APPLICATION_JSON_TYPE
);
}
public
Atlas
Cluster
getCluster
(
String
clust
erName
)
throws
AtlasServiceException
{
API
api
=
new
API
(
String
.
format
(
ADMIN_
CLUSTER_TEMPLATE
,
BASE_URI
,
clust
erName
),
HttpMethod
.
GET
,
Response
.
Status
.
OK
);
return
callAPI
(
api
,
Atlas
Clust
er
.
class
,
null
);
public
Atlas
Server
getServer
(
String
serv
erName
)
throws
AtlasServiceException
{
API
api
=
new
API
(
String
.
format
(
ADMIN_
SERVER_TEMPLATE
,
BASE_URI
,
serv
erName
),
HttpMethod
.
GET
,
Response
.
Status
.
OK
);
return
callAPI
(
api
,
Atlas
Serv
er
.
class
,
null
);
}
boolean
isRetryableException
(
ClientHandlerException
che
)
{
...
...
This diff is collapsed.
Click to expand it.
common/src/main/java/org/apache/atlas/repository/Constants.java
View file @
116fb62c
...
...
@@ -164,8 +164,8 @@ public final class Constants {
*/
public
static
final
String
ATTR_NAME_REFERENCEABLE
=
"Referenceable."
;
public
static
final
String
ATTR_NAME_REPLICATED_TO
_CLUSTER
=
"replicatedToCluster
"
;
public
static
final
String
ATTR_NAME_REPLICATED_FROM
_CLUSTER
=
"replicatedFromCluster
"
;
public
static
final
String
ATTR_NAME_REPLICATED_TO
=
"replicatedTo
"
;
public
static
final
String
ATTR_NAME_REPLICATED_FROM
=
"replicatedFrom
"
;
private
Constants
()
{
}
...
...
This diff is collapsed.
Click to expand it.
intg/src/main/java/org/apache/atlas/model/impexp/Atlas
Clust
er.java
→
intg/src/main/java/org/apache/atlas/model/impexp/Atlas
Serv
er.java
View file @
116fb62c
...
...
@@ -35,25 +35,28 @@ import static com.fasterxml.jackson.annotation.JsonAutoDetect.Visibility.PUBLIC_
@JsonAutoDetect
(
getterVisibility
=
PUBLIC_ONLY
,
setterVisibility
=
PUBLIC_ONLY
,
fieldVisibility
=
NONE
)
@JsonSerialize
(
include
=
JsonSerialize
.
Inclusion
.
NON_NULL
)
@JsonIgnoreProperties
(
ignoreUnknown
=
true
)
public
class
Atlas
Clust
er
extends
AtlasBaseModelObject
implements
Serializable
{
public
class
Atlas
Serv
er
extends
AtlasBaseModelObject
implements
Serializable
{
private
static
final
long
serialVersionUID
=
1L
;
public
static
final
String
KEY_REPLICATION_DETAILS
=
"REPL_DETAILS"
;
private
String
name
;
private
String
qualifiedName
;
private
Map
<
String
,
String
>
additionalInfo
;
private
List
<
String
>
urls
;
private
String
fullName
;
private
String
displayName
;
private
Map
<
String
,
String
>
additionalInfo
=
new
HashMap
<>();
private
List
<
String
>
urls
=
new
ArrayList
<>();
public
AtlasCluster
()
{
urls
=
new
ArrayList
<>();
additionalInfo
=
new
HashMap
<>();
public
AtlasServer
()
{
}
public
AtlasServer
(
String
name
,
String
fullName
)
{
this
(
name
,
name
,
fullName
);
}
public
AtlasCluster
(
String
name
,
String
qualifiedName
)
{
this
();
public
AtlasServer
(
String
name
,
String
displayName
,
String
fullName
)
{
this
.
name
=
name
;
this
.
qualifiedName
=
qualifiedName
;
this
.
displayName
=
displayName
;
this
.
fullName
=
fullName
;
}
public
void
setName
(
String
name
)
{
...
...
@@ -64,12 +67,45 @@ public class AtlasCluster extends AtlasBaseModelObject implements Serializable {
return
this
.
name
;
}
public
String
getFullName
()
{
return
fullName
;
}
public
void
setFullName
(
String
fullName
)
{
this
.
fullName
=
fullName
;
}
public
String
getDisplayName
()
{
return
displayName
;
}
public
void
setDisplayName
(
String
displayName
)
{
this
.
displayName
=
displayName
;
}
public
void
setAdditionalInfo
(
Map
<
String
,
String
>
additionalInfo
)
{
this
.
additionalInfo
=
additionalInfo
;
}
public
Map
<
String
,
String
>
getAdditionalInfo
()
{
return
this
.
additionalInfo
;
}
public
String
getAdditionalInfo
(
String
key
)
{
return
additionalInfo
.
get
(
key
);
}
public
void
setUrls
(
List
<
String
>
urls
)
{
this
.
urls
=
urls
;
}
public
List
<
String
>
getUrls
()
{
return
this
.
urls
;
}
public
void
setAdditionalInfo
(
String
key
,
String
value
)
{
if
(
additionalInfo
==
null
)
{
if
(
additionalInfo
==
null
)
{
additionalInfo
=
new
HashMap
<>();
}
...
...
@@ -79,15 +115,15 @@ public class AtlasCluster extends AtlasBaseModelObject implements Serializable {
public
void
setAdditionalInfoRepl
(
String
guid
,
long
modifiedTimestamp
)
{
Map
<
String
,
Object
>
replicationDetailsMap
=
null
;
if
(
additionalInfo
!=
null
&&
additionalInfo
.
containsKey
(
KEY_REPLICATION_DETAILS
))
{
if
(
additionalInfo
!=
null
&&
additionalInfo
.
containsKey
(
KEY_REPLICATION_DETAILS
))
{
replicationDetailsMap
=
AtlasType
.
fromJson
(
getAdditionalInfo
().
get
(
KEY_REPLICATION_DETAILS
),
Map
.
class
);
}
if
(
replicationDetailsMap
==
null
)
{
if
(
replicationDetailsMap
==
null
)
{
replicationDetailsMap
=
new
HashMap
<>();
}
if
(
modifiedTimestamp
==
0
)
{
if
(
modifiedTimestamp
==
0
)
{
replicationDetailsMap
.
remove
(
guid
);
}
else
{
replicationDetailsMap
.
put
(
guid
,
modifiedTimestamp
);
...
...
@@ -96,14 +132,8 @@ public class AtlasCluster extends AtlasBaseModelObject implements Serializable {
updateReplicationMap
(
replicationDetailsMap
);
}
private
void
updateReplicationMap
(
Map
<
String
,
Object
>
replicationDetailsMap
)
{
String
json
=
AtlasType
.
toJson
(
replicationDetailsMap
);
setAdditionalInfo
(
KEY_REPLICATION_DETAILS
,
json
);
}
public
Object
getAdditionalInfoRepl
(
String
guid
)
{
if
(
additionalInfo
==
null
||
!
additionalInfo
.
containsKey
(
KEY_REPLICATION_DETAILS
))
{
if
(
additionalInfo
==
null
||
!
additionalInfo
.
containsKey
(
KEY_REPLICATION_DETAILS
))
{
return
null
;
}
...
...
@@ -111,44 +141,28 @@ public class AtlasCluster extends AtlasBaseModelObject implements Serializable {
String
mapJson
=
additionalInfo
.
get
(
KEY_REPLICATION_DETAILS
);
Map
<
String
,
String
>
replicationDetailsMap
=
AtlasType
.
fromJson
(
mapJson
,
Map
.
class
);
if
(!
replicationDetailsMap
.
containsKey
(
key
))
{
if
(!
replicationDetailsMap
.
containsKey
(
key
))
{
return
null
;
}
return
replicationDetailsMap
.
get
(
key
);
}
public
Map
<
String
,
String
>
getAdditionalInfo
()
{
return
this
.
additionalInfo
;
}
public
String
getAdditionalInfo
(
String
key
)
{
return
additionalInfo
.
get
(
key
);
}
public
String
getQualifiedName
()
{
return
qualifiedName
;
}
public
void
setQualifiedName
(
String
qualifiedName
)
{
this
.
qualifiedName
=
qualifiedName
;
}
public
void
setUrls
(
List
<
String
>
urls
)
{
this
.
urls
=
urls
;
}
public
List
<
String
>
getUrls
()
{
return
this
.
urls
;
}
@Override
public
StringBuilder
toString
(
StringBuilder
sb
)
{
sb
.
append
(
", name="
).
append
(
name
);
sb
.
append
(
",
qualifiedName="
).
append
(
getQualifiedName
()
);
sb
.
append
(
",
urls="
).
append
(
urls
);
sb
.
append
(
",
fullName="
).
append
(
fullName
);
sb
.
append
(
",
displayName="
).
append
(
displayName
);
sb
.
append
(
", additionalInfo="
).
append
(
additionalInfo
);
sb
.
append
(
"}"
);
sb
.
append
(
", urls="
).
append
(
urls
);
return
sb
;
}
private
void
updateReplicationMap
(
Map
<
String
,
Object
>
replicationDetailsMap
)
{
String
json
=
AtlasType
.
toJson
(
replicationDetailsMap
);
setAdditionalInfo
(
KEY_REPLICATION_DETAILS
,
json
);
}
}
This diff is collapsed.
Click to expand it.
intg/src/main/java/org/apache/atlas/model/impexp/ExportImportAuditEntry.java
View file @
116fb62c
...
...
@@ -93,19 +93,19 @@ public class ExportImportAuditEntry extends AtlasBaseModelObject implements Seri
return
this
.
endTime
;
}
public
String
getTarget
Clust
erName
()
{
public
String
getTarget
Serv
erName
()
{
return
this
.
targetClusterName
;
}
public
String
getSource
Clust
erName
()
{
public
String
getSource
Serv
erName
()
{
return
this
.
sourceClusterName
;
}
public
void
setSource
Clust
erName
(
String
sourceClusterName
)
{
public
void
setSource
Serv
erName
(
String
sourceClusterName
)
{
this
.
sourceClusterName
=
sourceClusterName
;
}
public
void
setTarget
Clust
erName
(
String
targetClusterName
)
{
public
void
setTarget
Serv
erName
(
String
targetClusterName
)
{
this
.
targetClusterName
=
targetClusterName
;
}
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/impexp/
Clust
erService.java
→
repository/src/main/java/org/apache/atlas/repository/impexp/
AtlasServ
erService.java
View file @
116fb62c
...
...
@@ -21,7 +21,7 @@ package org.apache.atlas.repository.impexp;
import
org.apache.atlas.annotation.AtlasService
;
import
org.apache.atlas.annotation.GraphTransaction
;
import
org.apache.atlas.exception.AtlasBaseException
;
import
org.apache.atlas.model.impexp.Atlas
Clust
er
;
import
org.apache.atlas.model.impexp.Atlas
Serv
er
;
import
org.apache.atlas.model.instance.AtlasEntity
;
import
org.apache.atlas.model.instance.AtlasObjectId
;
import
org.apache.atlas.repository.graphdb.AtlasVertex
;
...
...
@@ -42,8 +42,8 @@ import java.util.ArrayList;
import
java.util.List
;
@AtlasService
public
class
Clust
erService
{
private
static
final
Logger
LOG
=
LoggerFactory
.
getLogger
(
Clust
erService
.
class
);
public
class
AtlasServ
erService
{
private
static
final
Logger
LOG
=
LoggerFactory
.
getLogger
(
AtlasServ
erService
.
class
);
private
final
DataAccess
dataAccess
;
private
final
AtlasEntityStore
entityStore
;
...
...
@@ -51,46 +51,68 @@ public class ClusterService {
private
final
EntityGraphRetriever
entityGraphRetriever
;
@Inject
public
ClusterService
(
DataAccess
dataAccess
,
AtlasEntityStore
entityStore
,
AtlasTypeRegistry
typeRegistry
,
EntityGraphRetriever
entityGraphRetriever
)
{
public
AtlasServerService
(
DataAccess
dataAccess
,
AtlasEntityStore
entityStore
,
AtlasTypeRegistry
typeRegistry
,
EntityGraphRetriever
entityGraphRetriever
)
{
this
.
dataAccess
=
dataAccess
;
this
.
entityStore
=
entityStore
;
this
.
typeRegistry
=
typeRegistry
;
this
.
entityGraphRetriever
=
entityGraphRetriever
;
}
public
Atlas
Cluster
get
(
AtlasCluster
clust
er
)
throws
AtlasBaseException
{
public
Atlas
Server
get
(
AtlasServer
serv
er
)
throws
AtlasBaseException
{
try
{
return
dataAccess
.
load
(
clust
er
);
return
dataAccess
.
load
(
serv
er
);
}
catch
(
AtlasBaseException
e
)
{
LOG
.
error
(
"dataAccess"
,
e
);
throw
e
;
}
}
public
AtlasServer
getCreateAtlasServer
(
String
name
,
String
fullName
)
throws
AtlasBaseException
{
AtlasServer
defaultServer
=
new
AtlasServer
(
name
,
fullName
);
AtlasServer
server
=
getAtlasServer
(
defaultServer
);
if
(
server
==
null
)
{
return
save
(
defaultServer
);
}
return
server
;
}
private
AtlasServer
getAtlasServer
(
AtlasServer
server
)
{
try
{
return
get
(
server
);
}
catch
(
AtlasBaseException
ex
)
{
return
null
;
}
}
@GraphTransaction
public
AtlasCluster
save
(
AtlasCluster
clusterInfo
)
{
public
AtlasServer
save
(
AtlasServer
server
)
{
try
{
return
dataAccess
.
save
(
clusterInfo
);
return
dataAccess
.
save
(
server
);
}
catch
(
AtlasBaseException
e
)
{
return
clusterInfo
;
return
server
;
}
}
@GraphTransaction
public
void
updateEntitiesWith
Cluster
(
AtlasCluster
clust
er
,
List
<
String
>
entityGuids
,
String
attributeName
)
throws
AtlasBaseException
{
if
(
cluster
!=
null
&&
StringUtils
.
isEmpty
(
clust
er
.
getGuid
()))
{
public
void
updateEntitiesWith
Server
(
AtlasServer
serv
er
,
List
<
String
>
entityGuids
,
String
attributeName
)
throws
AtlasBaseException
{
if
(
server
!=
null
&&
StringUtils
.
isEmpty
(
serv
er
.
getGuid
()))
{
return
;
}
AtlasObjectId
objectId
=
getObjectId
(
clust
er
);
AtlasObjectId
objectId
=
getObjectId
(
serv
er
);
for
(
String
guid
:
entityGuids
)
{
AtlasEntity
.
AtlasEntityWithExtInfo
entityWithExtInfo
=
entityStore
.
getById
(
guid
);
updateAttribute
(
entityWithExtInfo
,
attributeName
,
objectId
);
}
}
private
AtlasObjectId
getObjectId
(
Atlas
Cluster
clust
er
)
{
return
new
AtlasObjectId
(
cluster
.
getGuid
(),
AtlasClust
er
.
class
.
getSimpleName
());
private
AtlasObjectId
getObjectId
(
Atlas
Server
serv
er
)
{
return
new
AtlasObjectId
(
server
.
getGuid
(),
AtlasServ
er
.
class
.
getSimpleName
());
}
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/impexp/AuditsWriter.java
View file @
116fb62c
...
...
@@ -22,7 +22,7 @@ import org.apache.atlas.ApplicationProperties;
import
org.apache.atlas.AtlasConstants
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.exception.AtlasBaseException
;
import
org.apache.atlas.model.impexp.Atlas
Clust
er
;
import
org.apache.atlas.model.impexp.Atlas
Serv
er
;
import
org.apache.atlas.model.impexp.AtlasExportRequest
;
import
org.apache.atlas.model.impexp.AtlasExportResult
;
import
org.apache.atlas.model.impexp.AtlasImportRequest
;
...
...
@@ -45,15 +45,15 @@ public class AuditsWriter {
private
static
final
Logger
LOG
=
LoggerFactory
.
getLogger
(
AuditsWriter
.
class
);
private
static
final
String
CLUSTER_NAME_DEFAULT
=
"default"
;
private
ClusterService
clust
erService
;
private
AtlasServerService
atlasServ
erService
;
private
ExportImportAuditService
auditService
;
private
ExportAudits
auditForExport
=
new
ExportAudits
();
private
ImportAudits
auditForImport
=
new
ImportAudits
();
@Inject
public
AuditsWriter
(
ClusterService
clust
erService
,
ExportImportAuditService
auditService
)
{
this
.
clusterService
=
clust
erService
;
public
AuditsWriter
(
AtlasServerService
atlasServ
erService
,
ExportImportAuditService
auditService
)
{
this
.
atlasServerService
=
atlasServ
erService
;
this
.
auditService
=
auditService
;
}
...
...
@@ -63,7 +63,9 @@ public class AuditsWriter {
auditForExport
.
add
(
userName
,
result
,
startTime
,
endTime
,
entityCreationOrder
);
}
public
void
write
(
String
userName
,
AtlasImportResult
result
,
long
startTime
,
long
endTime
,
List
<
String
>
entityCreationOrder
)
throws
AtlasBaseException
{
public
void
write
(
String
userName
,
AtlasImportResult
result
,
long
startTime
,
long
endTime
,
List
<
String
>
entityCreationOrder
)
throws
AtlasBaseException
{
auditForImport
.
add
(
userName
,
result
,
startTime
,
endTime
,
entityCreationOrder
);
}
...
...
@@ -72,7 +74,7 @@ public class AuditsWriter {
}
private
void
updateReplicationAttribute
(
boolean
isReplicationSet
,
String
clust
erName
,
String
serv
erName
,
List
<
String
>
exportedGuids
,
String
attrNameReplicated
,
long
lastModifiedTimestamp
)
throws
AtlasBaseException
{
...
...
@@ -80,30 +82,32 @@ public class AuditsWriter {
return
;
}
Atlas
Cluster
cluster
=
saveCluster
(
clust
erName
,
exportedGuids
.
get
(
0
),
lastModifiedTimestamp
);
clusterService
.
updateEntitiesWithCluster
(
clust
er
,
exportedGuids
,
attrNameReplicated
);
Atlas
Server
server
=
saveServer
(
serv
erName
,
exportedGuids
.
get
(
0
),
lastModifiedTimestamp
);
atlasServerService
.
updateEntitiesWithServer
(
serv
er
,
exportedGuids
,
attrNameReplicated
);
}
private
String
getClusterNameFromOptions
(
Map
options
,
String
key
)
{
return
options
.
containsKey
(
key
)
?
(
String
)
options
.
get
(
key
)
:
""
;
:
StringUtils
.
EMPTY
;
}
private
AtlasCluster
saveCluster
(
String
clusterName
)
throws
AtlasBaseException
{
AtlasCluster
cluster
=
new
AtlasCluster
(
clusterName
,
clusterName
);
return
clusterService
.
save
(
cluster
);
private
AtlasServer
saveServer
(
String
name
)
throws
AtlasBaseException
{
return
atlasServerService
.
save
(
new
AtlasServer
(
name
,
name
));
}
private
AtlasCluster
saveCluster
(
String
clusterName
,
String
entityGuid
,
long
lastModifiedTimestamp
)
throws
AtlasBaseException
{
AtlasCluster
cluster
=
new
AtlasCluster
(
clusterName
,
clusterName
);
cluster
.
setAdditionalInfoRepl
(
entityGuid
,
lastModifiedTimestamp
);
private
AtlasServer
saveServer
(
String
name
,
String
entityGuid
,
long
lastModifiedTimestamp
)
throws
AtlasBaseException
{
AtlasServer
server
=
new
AtlasServer
(
name
,
name
);
server
.
setAdditionalInfoRepl
(
entityGuid
,
lastModifiedTimestamp
);
if
(
LOG
.
isDebugEnabled
())
{
LOG
.
debug
(
"save
Cluster: {}"
,
clust
er
);
LOG
.
debug
(
"save
Server: {}"
,
serv
er
);
}
return
clusterService
.
save
(
clust
er
);
return
atlasServerService
.
save
(
serv
er
);
}
public
static
String
getCurrentClusterName
()
{
...
...
@@ -113,12 +117,12 @@ public class AuditsWriter {
LOG
.
error
(
"getCurrentClusterName"
,
e
);
}
return
""
;
return
StringUtils
.
EMPTY
;
}
private
class
ExportAudits
{
private
AtlasExportRequest
request
;
private
String
target
Clust
erName
;
private
String
target
Serv
erName
;
private
String
optionKeyReplicatedTo
;
private
boolean
replicationOptionState
;
...
...
@@ -129,9 +133,9 @@ public class AuditsWriter {
request
=
result
.
getRequest
();
replicationOptionState
=
isReplicationOptionSet
(
request
.
getOptions
(),
optionKeyReplicatedTo
);
save
Clust
ers
();
save
Serv
ers
();
auditService
.
add
(
userName
,
getCurrentClusterName
(),
target
Clust
erName
,
auditService
.
add
(
userName
,
getCurrentClusterName
(),
target
Serv
erName
,
ExportImportAuditEntry
.
OPERATION_EXPORT
,
AtlasType
.
toJson
(
result
),
startTime
,
endTime
,
!
entityGuids
.
isEmpty
());
...
...
@@ -139,16 +143,16 @@ public class AuditsWriter {
return
;
}
updateReplicationAttribute
(
replicationOptionState
,
target
Clust
erName
,
entityGuids
,
Constants
.
ATTR_NAME_REPLICATED_TO
_CLUSTER
,
result
.
getLastModifiedTimestamp
());
updateReplicationAttribute
(
replicationOptionState
,
target
Serv
erName
,
entityGuids
,
Constants
.
ATTR_NAME_REPLICATED_TO
,
result
.
getLastModifiedTimestamp
());
}
private
void
save
Clust
ers
()
throws
AtlasBaseException
{
save
Clust
er
(
getCurrentClusterName
());
private
void
save
Serv
ers
()
throws
AtlasBaseException
{
save
Serv
er
(
getCurrentClusterName
());
target
Clust
erName
=
getClusterNameFromOptions
(
request
.
getOptions
(),
optionKeyReplicatedTo
);
if
(
StringUtils
.
isNotEmpty
(
target
Clust
erName
))
{
save
Cluster
(
targetClust
erName
);
target
Serv
erName
=
getClusterNameFromOptions
(
request
.
getOptions
(),
optionKeyReplicatedTo
);
if
(
StringUtils
.
isNotEmpty
(
target
Serv
erName
))
{
save
Server
(
targetServ
erName
);
}
}
}
...
...
@@ -156,8 +160,7 @@ public class AuditsWriter {
private
class
ImportAudits
{
private
AtlasImportRequest
request
;
private
boolean
replicationOptionState
;
private
String
sourceClusterName
;
private
AtlasCluster
sourceCluster
;
private
String
sourceServerName
;
private
String
optionKeyReplicatedFrom
;
public
void
add
(
String
userName
,
AtlasImportResult
result
,
...
...
@@ -167,10 +170,10 @@ public class AuditsWriter {
request
=
result
.
getRequest
();
replicationOptionState
=
isReplicationOptionSet
(
request
.
getOptions
(),
optionKeyReplicatedFrom
);
save
Clust
ers
();
save
Serv
ers
();
auditService
.
add
(
userName
,
source
Clust
erName
,
getCurrentClusterName
(),
source
Serv
erName
,
getCurrentClusterName
(),
ExportImportAuditEntry
.
OPERATION_IMPORT
,
AtlasType
.
toJson
(
result
),
startTime
,
endTime
,
!
entityGuids
.
isEmpty
());
...
...
@@ -178,23 +181,23 @@ public class AuditsWriter {
return
;
}
updateReplicationAttribute
(
replicationOptionState
,
this
.
source
Clust
erName
,
entityGuids
,
Constants
.
ATTR_NAME_REPLICATED_FROM
_CLUSTER
,
result
.
getExportResult
().
getLastModifiedTimestamp
());
updateReplicationAttribute
(
replicationOptionState
,
this
.
source
Serv
erName
,
entityGuids
,
Constants
.
ATTR_NAME_REPLICATED_FROM
,
result
.
getExportResult
().
getLastModifiedTimestamp
());
}
private
void
save
Clust
ers
()
throws
AtlasBaseException
{
save
Clust
er
(
getCurrentClusterName
());
private
void
save
Serv
ers
()
throws
AtlasBaseException
{
save
Serv
er
(
getCurrentClusterName
());
source
Clust
erName
=
getClusterNameFromOptionsState
();
if
(
StringUtils
.
isNotEmpty
(
source
Clust
erName
))
{
this
.
sourceCluster
=
saveCluster
(
sourceClust
erName
);
source
Serv
erName
=
getClusterNameFromOptionsState
();
if
(
StringUtils
.
isNotEmpty
(
source
Serv
erName
))
{
saveServer
(
sourceServ
erName
);
}
}
private
String
getClusterNameFromOptionsState
()
{
return
replicationOptionState
?
getClusterNameFromOptions
(
request
.
getOptions
(),
optionKeyReplicatedFrom
)
:
""
;
:
StringUtils
.
EMPTY
;
}
}
}
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/impexp/ExportImportAuditService.java
View file @
116fb62c
...
...
@@ -120,10 +120,10 @@ public class ExportImportAuditService {
addParameterIfValueNotEmpty
(
criteria
,
ExportImportAuditEntryDTO
.
PROPERTY_START_TIME
,
startTime
);
addParameterIfValueNotEmpty
(
criteria
,
ExportImportAuditEntryDTO
.
PROPERTY_END_TIME
,
endTime
);
add
Clust
erFilterCriteria
(
criteria
,
cluster
);
add
Serv
erFilterCriteria
(
criteria
,
cluster
);
}
private
void
add
Clust
erFilterCriteria
(
SearchParameters
.
FilterCriteria
parentCriteria
,
String
cluster
)
{
private
void
add
Serv
erFilterCriteria
(
SearchParameters
.
FilterCriteria
parentCriteria
,
String
cluster
)
{
if
(
StringUtils
.
isEmpty
(
cluster
))
{
return
;
}
...
...
@@ -132,8 +132,8 @@ public class ExportImportAuditService {
criteria
.
setCondition
(
SearchParameters
.
FilterCriteria
.
Condition
.
OR
);
criteria
.
setCriterion
(
new
ArrayList
<>());
addParameterIfValueNotEmpty
(
criteria
,
ExportImportAuditEntryDTO
.
PROPERTY_SOURCE_
CLUST
ER_NAME
,
cluster
);
addParameterIfValueNotEmpty
(
criteria
,
ExportImportAuditEntryDTO
.
PROPERTY_TARGET_
CLUST
ER_NAME
,
cluster
);
addParameterIfValueNotEmpty
(
criteria
,
ExportImportAuditEntryDTO
.
PROPERTY_SOURCE_
SERV
ER_NAME
,
cluster
);
addParameterIfValueNotEmpty
(
criteria
,
ExportImportAuditEntryDTO
.
PROPERTY_TARGET_
SERV
ER_NAME
,
cluster
);
parentCriteria
.
getCriterion
().
add
(
criteria
);
}
...
...
@@ -158,8 +158,8 @@ public class ExportImportAuditService {
ExportImportAuditEntry
entry
=
new
ExportImportAuditEntry
();
entry
.
setUserName
(
userName
);
entry
.
setSource
Clust
erName
(
sourceCluster
);
entry
.
setTarget
Clust
erName
(
targetCluster
);
entry
.
setSource
Serv
erName
(
sourceCluster
);
entry
.
setTarget
Serv
erName
(
targetCluster
);
entry
.
setOperation
(
operation
);
entry
.
setResultSummary
(
result
);
entry
.
setStartTime
(
startTime
);
...
...
@@ -167,6 +167,6 @@ public class ExportImportAuditService {
save
(
entry
);
LOG
.
info
(
"addAuditEntry: user: {}, source: {}, target: {}, operation: {}"
,
entry
.
getUserName
(),
entry
.
getSource
ClusterName
(),
entry
.
getTargetClust
erName
(),
entry
.
getOperation
());
entry
.
getSource
ServerName
(),
entry
.
getTargetServ
erName
(),
entry
.
getOperation
());
}
}
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/ogm/Atlas
Clust
erDTO.java
→
repository/src/main/java/org/apache/atlas/repository/ogm/Atlas
Serv
erDTO.java
View file @
116fb62c
...
...
@@ -18,7 +18,7 @@
package
org
.
apache
.
atlas
.
repository
.
ogm
;
import
org.apache.atlas.model.impexp.Atlas
Clust
er
;
import
org.apache.atlas.model.impexp.Atlas
Serv
er
;
import
org.apache.atlas.model.instance.AtlasEntity
;
import
org.apache.atlas.type.AtlasTypeRegistry
;
import
org.springframework.stereotype.Component
;
...
...
@@ -29,39 +29,42 @@ import java.util.List;
import
java.util.Map
;
@Component
public
class
AtlasClusterDTO
extends
AbstractDataTransferObject
<
AtlasCluster
>
{
private
final
String
PROPERTY_CLUSTER_NAME
=
"displayName"
;
private
final
String
PROPERTY_QUALIFIED_NAME
=
"qualifiedName"
;
public
class
AtlasServerDTO
extends
AbstractDataTransferObject
<
AtlasServer
>
{
private
final
String
PROPERTY_NAME
=
"name"
;
private
final
String
PROPERTY_DISPLAY_NAME
=
"displayName"
;
private
final
String
PROPERTY_FULL_NAME
=
"fullName"
;
private
final
String
PROPERTY_ADDITIONAL_INFO
=
"additionalInfo"
;
private
final
String
PROPERTY_URLS
=
"urls"
;
@Inject
public
Atlas
Clust
erDTO
(
AtlasTypeRegistry
typeRegistry
)
{
super
(
typeRegistry
,
Atlas
Cluster
.
class
,
AtlasClust
er
.
class
.
getSimpleName
());
public
Atlas
Serv
erDTO
(
AtlasTypeRegistry
typeRegistry
)
{
super
(
typeRegistry
,
Atlas
Server
.
class
,
AtlasServ
er
.
class
.
getSimpleName
());
}
public
Atlas
Clust
er
from
(
AtlasEntity
entity
)
{
Atlas
Cluster
cluster
=
new
AtlasClust
er
();
public
Atlas
Serv
er
from
(
AtlasEntity
entity
)
{
Atlas
Server
cluster
=
new
AtlasServ
er
();
setGuid
(
cluster
,
entity
);
cluster
.
setName
((
String
)
entity
.
getAttribute
(
PROPERTY_CLUSTER_NAME
));
cluster
.
setQualifiedName
((
String
)
entity
.
getAttribute
(
PROPERTY_QUALIFIED_NAME
));
cluster
.
setName
((
String
)
entity
.
getAttribute
(
PROPERTY_NAME
));
cluster
.
setFullName
((
String
)
entity
.
getAttribute
(
PROPERTY_FULL_NAME
));
cluster
.
setDisplayName
((
String
)
entity
.
getAttribute
(
PROPERTY_DISPLAY_NAME
));
cluster
.
setAdditionalInfo
((
Map
<
String
,
String
>)
entity
.
getAttribute
(
PROPERTY_ADDITIONAL_INFO
));
cluster
.
setUrls
((
List
<
String
>)
entity
.
getAttribute
(
PROPERTY_URLS
));
return
cluster
;
}
public
Atlas
Clust
er
from
(
AtlasEntity
.
AtlasEntityWithExtInfo
entityWithExtInfo
)
{
public
Atlas
Serv
er
from
(
AtlasEntity
.
AtlasEntityWithExtInfo
entityWithExtInfo
)
{
return
from
(
entityWithExtInfo
.
getEntity
());
}
@Override
public
AtlasEntity
toEntity
(
Atlas
Clust
er
obj
)
{
public
AtlasEntity
toEntity
(
Atlas
Serv
er
obj
)
{
AtlasEntity
entity
=
getDefaultAtlasEntity
(
obj
);
entity
.
setAttribute
(
PROPERTY_CLUSTER_NAME
,
obj
.
getName
());
entity
.
setAttribute
(
PROPERTY_QUALIFIED_NAME
,
obj
.
getQualifiedName
());
entity
.
setAttribute
(
PROPERTY_NAME
,
obj
.
getName
());
entity
.
setAttribute
(
PROPERTY_DISPLAY_NAME
,
obj
.
getDisplayName
());
entity
.
setAttribute
(
PROPERTY_FULL_NAME
,
obj
.
getFullName
());
entity
.
setAttribute
(
PROPERTY_ADDITIONAL_INFO
,
obj
.
getAdditionalInfo
());
entity
.
setAttribute
(
PROPERTY_URLS
,
obj
.
getUrls
());
...
...
@@ -69,14 +72,14 @@ public class AtlasClusterDTO extends AbstractDataTransferObject<AtlasCluster> {
}
@Override
public
AtlasEntity
.
AtlasEntityWithExtInfo
toEntityWithExtInfo
(
Atlas
Clust
er
obj
)
{
public
AtlasEntity
.
AtlasEntityWithExtInfo
toEntityWithExtInfo
(
Atlas
Serv
er
obj
)
{
return
new
AtlasEntity
.
AtlasEntityWithExtInfo
(
toEntity
(
obj
));
}
@Override
public
Map
<
String
,
Object
>
getUniqueAttributes
(
final
Atlas
Clust
er
obj
)
{
public
Map
<
String
,
Object
>
getUniqueAttributes
(
final
Atlas
Serv
er
obj
)
{
return
new
HashMap
<
String
,
Object
>()
{{
put
(
PROPERTY_
QUALIFIED_NAME
,
obj
.
getQualified
Name
());
put
(
PROPERTY_
FULL_NAME
,
obj
.
getFull
Name
());
}};
}
}
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/ogm/ExportImportAuditEntryDTO.java
View file @
116fb62c
...
...
@@ -40,14 +40,14 @@ public class ExportImportAuditEntryDTO extends AbstractDataTransferObject<Export
public
static
final
String
PROPERTY_START_TIME
=
"operationStartTime"
;
public
static
final
String
PROPERTY_END_TIME
=
"operationEndTime"
;
public
static
final
String
PROPERTY_RESULT_SUMMARY
=
"resultSummary"
;
public
static
final
String
PROPERTY_SOURCE_
CLUSTER_NAME
=
"sourceClust
erName"
;
public
static
final
String
PROPERTY_TARGET_
CLUSTER_NAME
=
"targetClust
erName"
;
public
static
final
String
PROPERTY_SOURCE_
SERVER_NAME
=
"sourceServ
erName"
;
public
static
final
String
PROPERTY_TARGET_
SERVER_NAME
=
"targetServ
erName"
;
private
static
final
Set
<
String
>
ATTRIBUTE_NAMES
=
new
HashSet
<>(
Arrays
.
asList
(
PROPERTY_USER_NAME
,
PROPERTY_OPERATION
,
PROPERTY_OPERATION_PARAMS
,
PROPERTY_START_TIME
,
PROPERTY_END_TIME
,
PROPERTY_RESULT_SUMMARY
,
PROPERTY_SOURCE_
CLUSTER_NAME
,
PROPERTY_TARGET_CLUST
ER_NAME
));
PROPERTY_SOURCE_
SERVER_NAME
,
PROPERTY_TARGET_SERV
ER_NAME
));
@Inject
public
ExportImportAuditEntryDTO
(
AtlasTypeRegistry
typeRegistry
)
{
...
...
@@ -68,8 +68,8 @@ public class ExportImportAuditEntryDTO extends AbstractDataTransferObject<Export
entry
.
setOperationParams
((
String
)
attributes
.
get
(
PROPERTY_OPERATION_PARAMS
));
entry
.
setStartTime
((
long
)
attributes
.
get
(
PROPERTY_START_TIME
));
entry
.
setEndTime
((
long
)
attributes
.
get
(
PROPERTY_END_TIME
));
entry
.
setSource
ClusterName
((
String
)
attributes
.
get
(
PROPERTY_SOURCE_CLUST
ER_NAME
));
entry
.
setTarget
ClusterName
((
String
)
attributes
.
get
(
PROPERTY_TARGET_CLUST
ER_NAME
));
entry
.
setSource
ServerName
((
String
)
attributes
.
get
(
PROPERTY_SOURCE_SERV
ER_NAME
));
entry
.
setTarget
ServerName
((
String
)
attributes
.
get
(
PROPERTY_TARGET_SERV
ER_NAME
));
entry
.
setResultSummary
((
String
)
attributes
.
get
(
PROPERTY_RESULT_SUMMARY
));
return
entry
;
...
...
@@ -94,8 +94,8 @@ public class ExportImportAuditEntryDTO extends AbstractDataTransferObject<Export
entity
.
setAttribute
(
PROPERTY_OPERATION_PARAMS
,
obj
.
getOperationParams
());
entity
.
setAttribute
(
PROPERTY_START_TIME
,
obj
.
getStartTime
());
entity
.
setAttribute
(
PROPERTY_END_TIME
,
obj
.
getEndTime
());
entity
.
setAttribute
(
PROPERTY_SOURCE_
CLUSTER_NAME
,
obj
.
getSourceClust
erName
());
entity
.
setAttribute
(
PROPERTY_TARGET_
CLUSTER_NAME
,
obj
.
getTargetClust
erName
());
entity
.
setAttribute
(
PROPERTY_SOURCE_
SERVER_NAME
,
obj
.
getSourceServ
erName
());
entity
.
setAttribute
(
PROPERTY_TARGET_
SERVER_NAME
,
obj
.
getTargetServ
erName
());
entity
.
setAttribute
(
PROPERTY_RESULT_SUMMARY
,
obj
.
getResultSummary
());
return
entity
;
...
...
This diff is collapsed.
Click to expand it.
repository/src/test/java/org/apache/atlas/TestModules.java
View file @
116fb62c
...
...
@@ -41,7 +41,7 @@ import org.apache.atlas.repository.graphdb.AtlasGraph;
import
org.apache.atlas.repository.graphdb.GraphDBMigrator
;
import
org.apache.atlas.repository.graphdb.janus.migration.GraphDBGraphSONMigrator
;
import
org.apache.atlas.repository.impexp.ExportService
;
import
org.apache.atlas.repository.ogm.Atlas
Clust
erDTO
;
import
org.apache.atlas.repository.ogm.Atlas
Serv
erDTO
;
import
org.apache.atlas.repository.ogm.ExportImportAuditEntryDTO
;
import
org.apache.atlas.repository.ogm.profiles.AtlasSavedSearchDTO
;
import
org.apache.atlas.repository.ogm.profiles.AtlasUserProfileDTO
;
...
...
@@ -172,7 +172,7 @@ public class TestModules {
availableDTOs
.
addBinding
().
to
(
AtlasGlossaryDTO
.
class
);
availableDTOs
.
addBinding
().
to
(
AtlasGlossaryTermDTO
.
class
);
availableDTOs
.
addBinding
().
to
(
AtlasGlossaryCategoryDTO
.
class
);
availableDTOs
.
addBinding
().
to
(
Atlas
Clust
erDTO
.
class
);
availableDTOs
.
addBinding
().
to
(
Atlas
Serv
erDTO
.
class
);
availableDTOs
.
addBinding
().
to
(
ExportImportAuditEntryDTO
.
class
);
bind
(
DTORegistry
.
class
).
asEagerSingleton
();
...
...
This diff is collapsed.
Click to expand it.
repository/src/test/java/org/apache/atlas/repository/impexp/
Clust
erServiceTest.java
→
repository/src/test/java/org/apache/atlas/repository/impexp/
AtlasServ
erServiceTest.java
View file @
116fb62c
...
...
@@ -20,7 +20,7 @@ package org.apache.atlas.repository.impexp;
import
org.apache.atlas.TestModules
;
import
org.apache.atlas.exception.AtlasBaseException
;
import
org.apache.atlas.model.impexp.Atlas
Clust
er
;
import
org.apache.atlas.model.impexp.Atlas
Serv
er
;
import
org.apache.atlas.model.instance.AtlasObjectId
;
import
org.apache.atlas.repository.Constants
;
import
org.apache.atlas.store.AtlasTypeDefStore
;
...
...
@@ -41,13 +41,12 @@ import static org.testng.Assert.assertNotEquals;
import
static
org
.
testng
.
Assert
.
assertNotNull
;
@Guice
(
modules
=
TestModules
.
TestOnlyModule
.
class
)
public
class
Clust
erServiceTest
{
public
class
AtlasServ
erServiceTest
{
private
final
String
TOP_LEVEL_ENTITY_NAME
=
"db1@cl1"
;
private
final
String
CLUSTER_NAME
=
"testCl1"
;
private
final
String
TARGET_
CLUSTER_NAME
=
"testCl2"
;
private
final
String
SERVER_NAME
=
"testCl1"
;
private
final
String
TARGET_
SERVER_NAME
=
"testCl2"
;
private
final
String
QUALIFIED_NAME_STOCKS
=
"stocks@cl1"
;
private
final
String
TYPE_HIVE_DB
=
"hive_db"
;
private
final
String
topLevelEntityGuid
=
"AAA-BBB-CCC"
;
@Inject
private
AtlasTypeDefStore
typeDefStore
;
...
...
@@ -56,7 +55,8 @@ public class ClusterServiceTest {
private
AtlasTypeRegistry
typeRegistry
;
@Inject
private
ClusterService
clusterService
;
private
AtlasServerService
atlasServerService
;
private
String
topLevelEntityGuid
=
"AAA-BBB-CCC"
;
@BeforeClass
public
void
setup
()
throws
IOException
,
AtlasBaseException
{
...
...
@@ -64,15 +64,15 @@ public class ClusterServiceTest {
}
@Test
public
void
saveAndRetrieve
Clust
erInfo
()
throws
AtlasBaseException
{
Atlas
Cluster
expected
=
getCluster
(
CLUSTER_NAME
+
"_1"
,
TOP_LEVEL_ENTITY_NAME
,
"EXPORT"
,
0
l
,
TARGET_CLUST
ER_NAME
);
Atlas
Cluster
expected2
=
getCluster
(
TARGET_CLUSTER_NAME
+
"_1"
,
TOP_LEVEL_ENTITY_NAME
,
"IMPORT"
,
0L
,
TARGET_CLUST
ER_NAME
);
Atlas
Cluster
expected3
=
getCluster
(
TARGET_CLUSTER_NAME
+
"_3"
,
TOP_LEVEL_ENTITY_NAME
,
"IMPORT"
,
0
,
TARGET_CLUST
ER_NAME
);
public
void
saveAndRetrieve
Serv
erInfo
()
throws
AtlasBaseException
{
Atlas
Server
expected
=
getServer
(
SERVER_NAME
+
"_1"
,
TOP_LEVEL_ENTITY_NAME
,
"EXPORT"
,
0
l
,
TARGET_SERV
ER_NAME
);
Atlas
Server
expected2
=
getServer
(
TARGET_SERVER_NAME
+
"_1"
,
TOP_LEVEL_ENTITY_NAME
,
"IMPORT"
,
0L
,
TARGET_SERV
ER_NAME
);
Atlas
Server
expected3
=
getServer
(
TARGET_SERVER_NAME
+
"_3"
,
TOP_LEVEL_ENTITY_NAME
,
"IMPORT"
,
0
,
TARGET_SERV
ER_NAME
);
Atlas
Cluster
actual
=
clust
erService
.
save
(
expected
);
Atlas
Cluster
actual2
=
clust
erService
.
save
(
expected2
);
Atlas
Cluster
actual3
=
clust
erService
.
save
(
expected3
);
Atlas
Cluster
actual2x
=
clust
erService
.
get
(
expected2
);
Atlas
Server
actual
=
atlasServ
erService
.
save
(
expected
);
Atlas
Server
actual2
=
atlasServ
erService
.
save
(
expected2
);
Atlas
Server
actual3
=
atlasServ
erService
.
save
(
expected3
);
Atlas
Server
actual2x
=
atlasServ
erService
.
get
(
expected2
);
assertNotNull
(
actual
.
getGuid
());
assertNotNull
(
actual2
.
getGuid
());
...
...
@@ -83,18 +83,18 @@ public class ClusterServiceTest {
assertEquals
(
actual
.
getName
(),
expected
.
getName
());
assertEquals
(
actual
.
get
QualifiedName
(),
expected
.
getQualified
Name
());
assertEquals
(
actual
.
get
FullName
(),
expected
.
getFull
Name
());
}
private
Atlas
Cluster
getCluster
(
String
clusterName
,
String
topLevelEntity
,
String
operation
,
long
nextModifiedTimestamp
,
String
targetClust
erName
)
{
Atlas
Cluster
cluster
=
new
AtlasCluster
(
clusterName
,
clust
erName
);
private
Atlas
Server
getServer
(
String
serverName
,
String
topLevelEntity
,
String
operation
,
long
nextModifiedTimestamp
,
String
targetServ
erName
)
{
Atlas
Server
cluster
=
new
AtlasServer
(
serverName
,
serv
erName
);
Map
<
String
,
String
>
syncMap
=
new
HashMap
<>();
syncMap
.
put
(
"topLevelEntity"
,
topLevelEntity
);
syncMap
.
put
(
"operation"
,
operation
);
syncMap
.
put
(
"nextModifiedTimestamp"
,
Long
.
toString
(
nextModifiedTimestamp
));
syncMap
.
put
(
"targetCluster"
,
target
Clust
erName
);
syncMap
.
put
(
"targetCluster"
,
target
Serv
erName
);
cluster
.
setAdditionalInfo
(
syncMap
);
...
...
@@ -105,13 +105,13 @@ public class ClusterServiceTest {
public
void
verifyAdditionalInfo
()
throws
AtlasBaseException
{
final
long
expectedLastModifiedTimestamp
=
200L
;
Atlas
Cluster
expectedCluster
=
new
AtlasCluster
(
CLUSTER_NAME
,
CLUST
ER_NAME
);
Atlas
Server
expectedCluster
=
atlasServerService
.
getCreateAtlasServer
(
SERVER_NAME
,
SERV
ER_NAME
);
String
qualifiedNameAttr
=
Constants
.
QUALIFIED_NAME
.
replace
(
ATTR_NAME_REFERENCEABLE
,
""
);
AtlasObjectId
objectId
=
new
AtlasObjectId
(
TYPE_HIVE_DB
,
qualifiedNameAttr
,
QUALIFIED_NAME_STOCKS
);
expectedCluster
.
setAdditionalInfoRepl
(
topLevelEntityGuid
,
expectedLastModifiedTimestamp
);
Atlas
Cluster
actualCluster
=
clust
erService
.
save
(
expectedCluster
);
Atlas
Server
actualCluster
=
atlasServ
erService
.
save
(
expectedCluster
);
assertEquals
(
actualCluster
.
getName
(),
expectedCluster
.
getName
());
int
actualModifiedTimestamp
=
(
int
)
actualCluster
.
getAdditionalInfoRepl
(
topLevelEntityGuid
);
...
...
This diff is collapsed.
Click to expand it.
repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportAuditServiceTest.java
View file @
116fb62c
...
...
@@ -76,14 +76,14 @@ public class ExportImportAuditServiceTest extends ExportImportTestBase {
assertNotEquals
(
actualEntry
.
getGuid
(),
actualEntry2
.
getGuid
());
assertNotNull
(
actualEntry
.
getGuid
());
assertEquals
(
actualEntry
.
getSource
ClusterName
(),
entry
.
getSourceClust
erName
());
assertEquals
(
actualEntry
.
getTarget
ClusterName
(),
entry
.
getTargetClust
erName
());
assertEquals
(
actualEntry
.
getSource
ServerName
(),
entry
.
getSourceServ
erName
());
assertEquals
(
actualEntry
.
getTarget
ServerName
(),
entry
.
getTargetServ
erName
());
assertEquals
(
actualEntry
.
getOperation
(),
entry
.
getOperation
());
}
@Test
public
void
numberOfSavedEntries_Retrieved
()
throws
AtlasBaseException
,
InterruptedException
{
final
String
source1
=
"
clust
er1"
;
final
String
source1
=
"
serv
er1"
;
final
String
target1
=
"cly"
;
int
MAX_ENTRIES
=
5
;
...
...
@@ -100,7 +100,7 @@ public class ExportImportAuditServiceTest extends ExportImportTestBase {
private
ExportImportAuditEntry
retrieveEntry
(
ExportImportAuditEntry
entry
)
throws
AtlasBaseException
{
List
<
ExportImportAuditEntry
>
result
=
auditService
.
get
(
entry
.
getUserName
(),
entry
.
getOperation
(),
entry
.
getSource
Clust
erName
(),
entry
.
getSource
Serv
erName
(),
Long
.
toString
(
entry
.
getStartTime
()),
""
,
10
,
0
);
assertNotNull
(
result
);
assertEquals
(
result
.
size
(),
1
);
...
...
@@ -108,10 +108,10 @@ public class ExportImportAuditServiceTest extends ExportImportTestBase {
return
auditService
.
get
(
entry
);
}
private
ExportImportAuditEntry
saveAndGet
(
String
source
ClusterName
,
String
operation
,
String
targetClust
erName
)
throws
AtlasBaseException
{
ExportImportAuditEntry
entry
=
new
ExportImportAuditEntry
(
source
Clust
erName
,
operation
);
private
ExportImportAuditEntry
saveAndGet
(
String
source
ServerName
,
String
operation
,
String
targetServ
erName
)
throws
AtlasBaseException
{
ExportImportAuditEntry
entry
=
new
ExportImportAuditEntry
(
source
Serv
erName
,
operation
);
entry
.
setTarget
ClusterName
(
targetClust
erName
);
entry
.
setTarget
ServerName
(
targetServ
erName
);
entry
.
setUserName
(
"default"
);
entry
.
setStartTime
(
System
.
currentTimeMillis
());
entry
.
setEndTime
(
System
.
currentTimeMillis
()
+
1000L
);
...
...
This diff is collapsed.
Click to expand it.
repository/src/test/java/org/apache/atlas/repository/impexp/ReplicationEntityAttributeTest.java
View file @
116fb62c
...
...
@@ -24,7 +24,7 @@ import org.apache.atlas.RequestContext;
import
org.apache.atlas.TestModules
;
import
org.apache.atlas.TestUtilsV2
;
import
org.apache.atlas.exception.AtlasBaseException
;
import
org.apache.atlas.model.impexp.Atlas
Clust
er
;
import
org.apache.atlas.model.impexp.Atlas
Serv
er
;
import
org.apache.atlas.model.impexp.AtlasExportRequest
;
import
org.apache.atlas.model.impexp.AtlasImportRequest
;
import
org.apache.atlas.model.impexp.AtlasImportResult
;
...
...
@@ -63,9 +63,6 @@ public class ReplicationEntityAttributeTest extends ExportImportTestBase {
private
final
String
EXPORT_REQUEST_FILE
=
"export-replicatedTo"
;
private
final
String
IMPORT_REQUEST_FILE
=
"import-replicatedFrom"
;
private
final
String
DB_GUID
=
"1637a33e-6512-447b-ade7-249c8cb5344b"
;
private
final
String
TABLE_GUID
=
"df122fc3-5555-40f8-a30f-3090b8a622f8"
;
private
String
REPLICATED_TO_CLUSTER_NAME
=
""
;
private
String
REPLICATED_FROM_CLUSTER_NAME
=
""
;
...
...
@@ -85,7 +82,7 @@ public class ReplicationEntityAttributeTest extends ExportImportTestBase {
ImportService
importService
;
@Inject
ClusterService
clust
erService
;
AtlasServerService
atlasServ
erService
;
private
AtlasEntityChangeNotifier
mockChangeNotifier
=
mock
(
AtlasEntityChangeNotifier
.
class
);
private
AtlasEntityStoreV2
entityStore
;
...
...
@@ -120,7 +117,7 @@ public class ReplicationEntityAttributeTest extends ExportImportTestBase {
assertEquals
(
zipSource
.
getCreationOrder
().
size
(),
expectedEntityCount
);
assertCluster
(
REPLICATED_TO_CLUSTER_NAME
,
null
);
assertReplicationAttribute
(
Constants
.
ATTR_NAME_REPLICATED_TO
_CLUSTER
);
assertReplicationAttribute
(
Constants
.
ATTR_NAME_REPLICATED_TO
);
}
@Test
(
dependsOnMethods
=
"exportWithReplicationToOption_AddsClusterObjectIdToReplicatedFromAttribute"
,
enabled
=
false
)
...
...
@@ -129,7 +126,7 @@ public class ReplicationEntityAttributeTest extends ExportImportTestBase {
AtlasImportResult
importResult
=
runImportWithParameters
(
importService
,
request
,
zipSource
);
assertCluster
(
REPLICATED_FROM_CLUSTER_NAME
,
importResult
);
assertReplicationAttribute
(
Constants
.
ATTR_NAME_REPLICATED_FROM
_CLUSTER
);
assertReplicationAttribute
(
Constants
.
ATTR_NAME_REPLICATED_FROM
);
}
private
void
assertReplicationAttribute
(
String
attrNameReplication
)
throws
AtlasBaseException
{
...
...
@@ -139,13 +136,13 @@ public class ReplicationEntityAttributeTest extends ExportImportTestBase {
Object
ex
=
e
.
getAttribute
(
attrNameReplication
);
assertNotNull
(
ex
);
List
<
String
>
clusterNameSyncTyp
e
=
(
List
)
ex
;
assertEquals
(
clusterNameSyncTyp
e
.
size
(),
1
);
List
<
String
>
attrValu
e
=
(
List
)
ex
;
assertEquals
(
attrValu
e
.
size
(),
1
);
}
}
private
void
assertCluster
(
String
name
,
AtlasImportResult
importResult
)
throws
AtlasBaseException
{
Atlas
Cluster
actual
=
clusterService
.
get
(
new
AtlasClust
er
(
name
,
name
));
Atlas
Server
actual
=
atlasServerService
.
get
(
new
AtlasServ
er
(
name
,
name
));
assertNotNull
(
actual
);
assertEquals
(
actual
.
getName
(),
name
);
...
...
@@ -155,7 +152,7 @@ public class ReplicationEntityAttributeTest extends ExportImportTestBase {
}
}
private
void
assertClusterAdditionalInfo
(
Atlas
Clust
er
cluster
,
AtlasImportResult
importResult
)
throws
AtlasBaseException
{
private
void
assertClusterAdditionalInfo
(
Atlas
Serv
er
cluster
,
AtlasImportResult
importResult
)
throws
AtlasBaseException
{
AtlasExportRequest
request
=
importResult
.
getExportResult
().
getRequest
();
AtlasEntityType
type
=
(
AtlasEntityType
)
typeRegistry
.
getType
(
request
.
getItemsToExport
().
get
(
0
).
getTypeName
());
AtlasEntity
.
AtlasEntityWithExtInfo
entity
=
entityStore
.
getByUniqueAttributes
(
type
,
request
.
getItemsToExport
().
get
(
0
).
getUniqueAttributes
());
...
...
This diff is collapsed.
Click to expand it.
repository/src/test/resources/json/stocksDB-Entities/replicationAttrs.json
View file @
116fb62c
{
"name"
:
"replicatedFrom
Cluster
"
,
"typeName"
:
"array<Atlas
Clust
er>"
,
"name"
:
"replicatedFrom"
,
"typeName"
:
"array<Atlas
Serv
er>"
,
"cardinality"
:
"SET"
,
"isIndexable"
:
false
,
"isOptional"
:
true
,
...
...
This diff is collapsed.
Click to expand it.
webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
View file @
116fb62c
...
...
@@ -29,18 +29,21 @@ import org.apache.atlas.authorize.AtlasAuthorizationUtils;
import
org.apache.atlas.discovery.SearchContext
;
import
org.apache.atlas.exception.AtlasBaseException
;
import
org.apache.atlas.model.discovery.AtlasSearchResult
;
import
org.apache.atlas.model.impexp.AtlasServer
;
import
org.apache.atlas.model.impexp.AtlasExportRequest
;
import
org.apache.atlas.model.impexp.AtlasExportResult
;
import
org.apache.atlas.model.impexp.AtlasImportRequest
;
import
org.apache.atlas.model.impexp.AtlasImportResult
;
import
org.apache.atlas.model.impexp.*
;
import
org.apache.atlas.model.impexp.ExportImportAuditEntry
;
import
org.apache.atlas.model.impexp.MigrationStatus
;
import
org.apache.atlas.model.metrics.AtlasMetrics
;
import
org.apache.atlas.repository.impexp.AtlasServerService
;
import
org.apache.atlas.repository.impexp.ExportImportAuditService
;
import
org.apache.atlas.repository.impexp.ExportService
;
import
org.apache.atlas.repository.impexp.ImportService
;
import
org.apache.atlas.repository.impexp.MigrationProgressService
;
import
org.apache.atlas.repository.impexp.ZipSink
;
import
org.apache.atlas.repository.impexp.ZipSource
;
import
org.apache.atlas.repository.impexp.ExportImportAuditService
;
import
org.apache.atlas.repository.impexp.*
;
import
org.apache.atlas.services.MetricsService
;
import
org.apache.atlas.type.AtlasType
;
import
org.apache.atlas.type.AtlasTypeRegistry
;
...
...
@@ -129,7 +132,7 @@ public class AdminResource {
private
final
MigrationProgressService
migrationProgressService
;
private
final
ReentrantLock
importExportOperationLock
;
private
final
ExportImportAuditService
exportImportAuditService
;
private
final
ClusterService
clust
erService
;
private
final
AtlasServerService
atlasServ
erService
;
static
{
try
{
...
...
@@ -143,7 +146,7 @@ public class AdminResource {
public
AdminResource
(
ServiceState
serviceState
,
MetricsService
metricsService
,
AtlasTypeRegistry
typeRegistry
,
ExportService
exportService
,
ImportService
importService
,
SearchTracker
activeSearches
,
MigrationProgressService
migrationProgressService
,
ClusterService
clust
erService
,
AtlasServerService
serv
erService
,
ExportImportAuditService
exportImportAuditService
)
{
this
.
serviceState
=
serviceState
;
this
.
metricsService
=
metricsService
;
...
...
@@ -152,7 +155,7 @@ public class AdminResource {
this
.
activeSearches
=
activeSearches
;
this
.
typeRegistry
=
typeRegistry
;
this
.
migrationProgressService
=
migrationProgressService
;
this
.
clusterService
=
clust
erService
;
this
.
atlasServerService
=
serv
erService
;
this
.
exportImportAuditService
=
exportImportAuditService
;
this
.
importExportOperationLock
=
new
ReentrantLock
();
}
...
...
@@ -448,24 +451,24 @@ public class AdminResource {
/**
* Fetch details of a cluster.
* @param
clust
erName name of target cluster with which it is paired
* @return Atlas
Clust
er
* @param
serv
erName name of target cluster with which it is paired
* @return Atlas
Serv
er
* @throws AtlasBaseException
*/
@GET
@Path
(
"/
cluster/{clust
erName}"
)
@Path
(
"/
server/{serv
erName}"
)
@Consumes
(
Servlets
.
JSON_MEDIA_TYPE
)
@Produces
(
Servlets
.
JSON_MEDIA_TYPE
)
public
Atlas
Cluster
getCluster
(
@PathParam
(
"clusterName"
)
String
clust
erName
)
throws
AtlasBaseException
{
public
Atlas
Server
getCluster
(
@PathParam
(
"serverName"
)
String
serv
erName
)
throws
AtlasBaseException
{
AtlasPerfTracer
perf
=
null
;
try
{
if
(
AtlasPerfTracer
.
isPerfTraceEnabled
(
PERF_LOG
))
{
perf
=
AtlasPerfTracer
.
getPerfTracer
(
PERF_LOG
,
"cluster.get
Cluster("
+
clust
erName
+
")"
);
perf
=
AtlasPerfTracer
.
getPerfTracer
(
PERF_LOG
,
"cluster.get
Server("
+
serv
erName
+
")"
);
}
Atlas
Cluster
cluster
=
new
AtlasCluster
(
clusterName
,
clust
erName
);
return
clust
erService
.
get
(
cluster
);
Atlas
Server
cluster
=
new
AtlasServer
(
serverName
,
serv
erName
);
return
atlasServ
erService
.
get
(
cluster
);
}
finally
{
AtlasPerfTracer
.
log
(
perf
);
}
...
...
@@ -475,7 +478,7 @@ public class AdminResource {
@Path
(
"/expimp/audit"
)
@Consumes
(
Servlets
.
JSON_MEDIA_TYPE
)
@Produces
(
Servlets
.
JSON_MEDIA_TYPE
)
public
List
<
ExportImportAuditEntry
>
getExportImportAudit
(
@QueryParam
(
"
clusterName"
)
String
cluster
,
public
List
<
ExportImportAuditEntry
>
getExportImportAudit
(
@QueryParam
(
"
serverName"
)
String
serverName
,
@QueryParam
(
"userName"
)
String
userName
,
@QueryParam
(
"operation"
)
String
operation
,
@QueryParam
(
"startTime"
)
String
startTime
,
...
...
@@ -486,10 +489,10 @@ public class AdminResource {
try
{
if
(
AtlasPerfTracer
.
isPerfTraceEnabled
(
PERF_LOG
))
{
perf
=
AtlasPerfTracer
.
getPerfTracer
(
PERF_LOG
,
"getExportImportAudit("
+
cluster
+
")"
);
perf
=
AtlasPerfTracer
.
getPerfTracer
(
PERF_LOG
,
"getExportImportAudit("
+
serverName
+
")"
);
}
return
exportImportAuditService
.
get
(
userName
,
operation
,
cluster
,
startTime
,
endTime
,
limit
,
offset
);
return
exportImportAuditService
.
get
(
userName
,
operation
,
serverName
,
startTime
,
endTime
,
limit
,
offset
);
}
finally
{
AtlasPerfTracer
.
log
(
perf
);
}
...
...
This diff is collapsed.
Click to expand it.
webapp/src/test/java/org/apache/atlas/web/resources/AdminExportImportTestIT.java
View file @
116fb62c
...
...
@@ -21,7 +21,7 @@ package org.apache.atlas.web.resources;
import
org.apache.atlas.AtlasServiceException
;
import
org.apache.atlas.exception.AtlasBaseException
;
import
org.apache.atlas.model.impexp.Atlas
Clust
er
;
import
org.apache.atlas.model.impexp.Atlas
Serv
er
;
import
org.apache.atlas.model.impexp.AtlasExportRequest
;
import
org.apache.atlas.model.impexp.AtlasImportRequest
;
import
org.apache.atlas.model.impexp.AtlasImportResult
;
...
...
@@ -44,10 +44,10 @@ import static org.testng.Assert.assertTrue;
public
class
AdminExportImportTestIT
extends
BaseResourceIT
{
private
final
String
FILE_TO_IMPORT
=
"stocks-base.zip"
;
private
final
String
EXPORT_REQUEST_FILE
=
"export-incremental"
;
private
final
String
SOURCE_
CLUST
ER_NAME
=
"cl1"
;
private
final
String
SOURCE_
SERV
ER_NAME
=
"cl1"
;
static
final
String
IMPORT_TRANSFORM_CLEAR_ATTRS
=
"{ \"Asset\": { \"*\":[ \"clearAttrValue:replicatedTo
Cluster,replicatedFromCluster
\" ] } }"
;
"{ \"Asset\": { \"*\":[ \"clearAttrValue:replicatedTo
,replicatedFrom
\" ] } }"
;
static
final
String
IMPORT_TRANSFORM_SET_DELETED
=
"{ \"Asset\": { \"*\":[ \"setDeleted\" ] } }"
;
...
...
@@ -77,7 +77,7 @@ public class AdminExportImportTestIT extends BaseResourceIT {
private
void
performImport
(
String
fileToImport
)
throws
AtlasServiceException
{
AtlasImportRequest
request
=
new
AtlasImportRequest
();
request
.
getOptions
().
put
(
AtlasImportRequest
.
OPTION_KEY_REPLICATED_FROM
,
SOURCE_
CLUST
ER_NAME
);
request
.
getOptions
().
put
(
AtlasImportRequest
.
OPTION_KEY_REPLICATED_FROM
,
SOURCE_
SERV
ER_NAME
);
request
.
getOptions
().
put
(
AtlasImportRequest
.
TRANSFORMS_KEY
,
IMPORT_TRANSFORM_CLEAR_ATTRS
);
performImport
(
fileToImport
,
request
);
...
...
@@ -100,11 +100,11 @@ public class AdminExportImportTestIT extends BaseResourceIT {
assertEquals
(
result
.
getProcessedEntities
().
size
(),
37
);
}
private
void
assertReplicationData
(
String
clust
erName
)
throws
AtlasServiceException
{
Atlas
Cluster
cluster
=
atlasClientV2
.
getCluster
(
clust
erName
);
assertNotNull
(
clust
er
);
assertNotNull
(
clust
er
.
getAdditionalInfo
());
assertTrue
(
clust
er
.
getAdditionalInfo
().
size
()
>
0
);
private
void
assertReplicationData
(
String
serv
erName
)
throws
AtlasServiceException
{
Atlas
Server
server
=
atlasClientV2
.
getServer
(
serv
erName
);
assertNotNull
(
serv
er
);
assertNotNull
(
serv
er
.
getAdditionalInfo
());
assertTrue
(
serv
er
.
getAdditionalInfo
().
size
()
>
0
);
}
@AfterClass
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment