Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
atlas
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
dataplatform
atlas
Commits
8127189c
Commit
8127189c
authored
5 years ago
by
Le Ma
Committed by
Sarath Subramanian
5 years ago
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
ATLAS-3327-2 Create internal vertex property for classification names and…
ATLAS-3327-2 Create internal vertex property for classification names and propagated classification names
parent
f912aa89
Hide whitespace changes
Inline
Side-by-side
Showing
8 changed files
with
114 additions
and
49 deletions
+114
-49
Constants.java
.../src/main/java/org/apache/atlas/repository/Constants.java
+1
-0
GraphBackedSearchIndexer.java
...ache/atlas/repository/graph/GraphBackedSearchIndexer.java
+2
-0
GraphHelper.java
...n/java/org/apache/atlas/repository/graph/GraphHelper.java
+13
-14
ClassificationTextPatch.java
...che/atlas/repository/patches/ClassificationTextPatch.java
+1
-2
DeleteHandlerV1.java
...ache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
+38
-8
EntityGraphMapper.java
...he/atlas/repository/store/graph/v2/EntityGraphMapper.java
+42
-18
EntityStateChecker.java
...e/atlas/repository/store/graph/v2/EntityStateChecker.java
+8
-0
TestEntityREST.java
...st/java/org/apache/atlas/web/adapters/TestEntityREST.java
+9
-7
No files found.
common/src/main/java/org/apache/atlas/repository/Constants.java
View file @
8127189c
...
@@ -180,6 +180,7 @@ public final class Constants {
...
@@ -180,6 +180,7 @@ public final class Constants {
public
static
final
String
CLASSIFICATION_EDGE_IS_PROPAGATED_PROPERTY_KEY
=
encodePropertyKey
(
INTERNAL_PROPERTY_KEY_PREFIX
+
"isPropagated"
);
public
static
final
String
CLASSIFICATION_EDGE_IS_PROPAGATED_PROPERTY_KEY
=
encodePropertyKey
(
INTERNAL_PROPERTY_KEY_PREFIX
+
"isPropagated"
);
public
static
final
String
CLASSIFICATION_EDGE_STATE_PROPERTY_KEY
=
STATE_PROPERTY_KEY
;
public
static
final
String
CLASSIFICATION_EDGE_STATE_PROPERTY_KEY
=
STATE_PROPERTY_KEY
;
public
static
final
String
CLASSIFICATION_LABEL
=
"classifiedAs"
;
public
static
final
String
CLASSIFICATION_LABEL
=
"classifiedAs"
;
public
static
final
String
CLASSIFICATION_NAME_DELIMITER
=
"|"
;
public
static
final
String
TERM_ASSIGNMENT_LABEL
=
"r:AtlasGlossarySemanticAssignment"
;
public
static
final
String
TERM_ASSIGNMENT_LABEL
=
"r:AtlasGlossarySemanticAssignment"
;
public
static
final
String
ATTRIBUTE_INDEX_PROPERTY_KEY
=
encodePropertyKey
(
INTERNAL_PROPERTY_KEY_PREFIX
+
"index"
);
public
static
final
String
ATTRIBUTE_INDEX_PROPERTY_KEY
=
encodePropertyKey
(
INTERNAL_PROPERTY_KEY_PREFIX
+
"index"
);
public
static
final
String
ATTRIBUTE_KEY_PROPERTY_KEY
=
encodePropertyKey
(
INTERNAL_PROPERTY_KEY_PREFIX
+
"key"
);
public
static
final
String
ATTRIBUTE_KEY_PROPERTY_KEY
=
encodePropertyKey
(
INTERNAL_PROPERTY_KEY_PREFIX
+
"key"
);
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
View file @
8127189c
...
@@ -319,6 +319,8 @@ public class GraphBackedSearchIndexer implements SearchIndexer, ActiveStateChang
...
@@ -319,6 +319,8 @@ public class GraphBackedSearchIndexer implements SearchIndexer, ActiveStateChang
createCommonVertexIndex
(
management
,
CREATED_BY_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
SINGLE
,
false
,
false
);
createCommonVertexIndex
(
management
,
CREATED_BY_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
SINGLE
,
false
,
false
);
createCommonVertexIndex
(
management
,
CLASSIFICATION_TEXT_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
SINGLE
,
false
,
false
);
createCommonVertexIndex
(
management
,
CLASSIFICATION_TEXT_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
SINGLE
,
false
,
false
);
createCommonVertexIndex
(
management
,
MODIFIED_BY_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
SINGLE
,
false
,
false
);
createCommonVertexIndex
(
management
,
MODIFIED_BY_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
SINGLE
,
false
,
false
);
createCommonVertexIndex
(
management
,
CLASSIFICATION_NAMES_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
SINGLE
,
true
,
false
);
createCommonVertexIndex
(
management
,
PROPAGATED_CLASSIFICATION_NAMES_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
SINGLE
,
true
,
false
);
createCommonVertexIndex
(
management
,
TRAIT_NAMES_PROPERTY_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
SET
,
true
,
true
);
createCommonVertexIndex
(
management
,
TRAIT_NAMES_PROPERTY_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
SET
,
true
,
true
);
createCommonVertexIndex
(
management
,
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
LIST
,
true
,
true
);
createCommonVertexIndex
(
management
,
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
,
UniqueKind
.
NONE
,
String
.
class
,
LIST
,
true
,
true
);
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
View file @
8127189c
...
@@ -85,6 +85,8 @@ import static org.apache.atlas.repository.Constants.CLASSIFICATION_ENTITY_GUID;
...
@@ -85,6 +85,8 @@ import static org.apache.atlas.repository.Constants.CLASSIFICATION_ENTITY_GUID;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_ENTITY_STATUS
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_ENTITY_STATUS
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_LABEL
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_LABEL
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_EDGE_NAME_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_EDGE_NAME_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_NAME_DELIMITER
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_VERTEX_NAME_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_VERTEX_NAME_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_VERTEX_PROPAGATE_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_VERTEX_PROPAGATE_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_VERTEX_REMOVE_PROPAGATIONS_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_VERTEX_REMOVE_PROPAGATIONS_KEY
;
...
@@ -92,8 +94,6 @@ import static org.apache.atlas.repository.Constants.CREATED_BY_KEY;
...
@@ -92,8 +94,6 @@ import static org.apache.atlas.repository.Constants.CREATED_BY_KEY;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
ENTITY_TYPE_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
ENTITY_TYPE_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
MODIFICATION_TIMESTAMP_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
MODIFICATION_TIMESTAMP_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
MODIFIED_BY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
MODIFIED_BY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
PROPAGATED_CLASSIFICATION_NAMES_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
RELATIONSHIPTYPE_BLOCKED_PROPAGATED_CLASSIFICATIONS_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
RELATIONSHIPTYPE_BLOCKED_PROPAGATED_CLASSIFICATIONS_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
RELATIONSHIPTYPE_TAG_PROPAGATION_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
RELATIONSHIPTYPE_TAG_PROPAGATION_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
STATE_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
STATE_PROPERTY_KEY
;
...
@@ -659,18 +659,6 @@ public final class GraphHelper {
...
@@ -659,18 +659,6 @@ public final class GraphHelper {
return
element
.
toString
();
return
element
.
toString
();
}
}
public
static
void
addToPropagatedClassificationAndTraitNames
(
AtlasVertex
entityVertex
,
String
classificationName
)
{
if
(
LOG
.
isDebugEnabled
())
{
LOG
.
debug
(
"Adding property {} = \"{}\" to vertex {}"
,
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
,
classificationName
,
string
(
entityVertex
));
}
entityVertex
.
addListProperty
(
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
,
classificationName
);
String
propClsNames
=
entityVertex
.
getProperty
(
PROPAGATED_CLASSIFICATION_NAMES_KEY
,
String
.
class
);
propClsNames
=
StringUtils
.
isEmpty
(
propClsNames
)?
classificationName
:
propClsNames
+
"|"
+
classificationName
;
entityVertex
.
setProperty
(
PROPAGATED_CLASSIFICATION_NAMES_KEY
,
propClsNames
);
}
/**
/**
* Remove the specified edge from the graph.
* Remove the specified edge from the graph.
*
*
...
@@ -1849,4 +1837,14 @@ public final class GraphHelper {
...
@@ -1849,4 +1837,14 @@ public final class GraphHelper {
public
static
boolean
getDefaultRemovePropagations
()
{
public
static
boolean
getDefaultRemovePropagations
()
{
return
removePropagations
;
return
removePropagations
;
}
}
public
static
String
getDelimitedClassificationNames
(
Collection
<
String
>
classificationNames
)
{
String
ret
=
null
;
if
(
CollectionUtils
.
isNotEmpty
(
classificationNames
))
{
ret
=
CLASSIFICATION_NAME_DELIMITER
+
StringUtils
.
join
(
classificationNames
,
CLASSIFICATION_NAME_DELIMITER
)
+
CLASSIFICATION_NAME_DELIMITER
;
}
return
ret
;
}
}
}
\ No newline at end of file
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/patches/ClassificationTextPatch.java
View file @
8127189c
...
@@ -71,8 +71,7 @@ public class ClassificationTextPatch extends AtlasPatchHandler {
...
@@ -71,8 +71,7 @@ public class ClassificationTextPatch extends AtlasPatchHandler {
LOG
.
debug
(
"processItem(typeName={}, vertexId={})"
,
typeName
,
vertexId
);
LOG
.
debug
(
"processItem(typeName={}, vertexId={})"
,
typeName
,
vertexId
);
}
}
getEntityGraphMapper
().
updateClassificationText
(
vertex
);
getEntityGraphMapper
().
updateClassificationTextAndNames
(
vertex
);
getEntityGraphMapper
().
setClassificationNames
(
vertex
);
if
(
LOG
.
isDebugEnabled
())
{
if
(
LOG
.
isDebugEnabled
())
{
LOG
.
debug
(
"processItem(typeName={}, vertexId={}): Done!"
,
typeName
,
vertexId
);
LOG
.
debug
(
"processItem(typeName={}, vertexId={}): Done!"
,
typeName
,
vertexId
);
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
View file @
8127189c
...
@@ -62,6 +62,7 @@ import static org.apache.atlas.model.typedef.AtlasRelationshipDef.PropagateTags.
...
@@ -62,6 +62,7 @@ import static org.apache.atlas.model.typedef.AtlasRelationshipDef.PropagateTags.
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_EDGE_NAME_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_EDGE_NAME_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_ENTITY_STATUS
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_ENTITY_STATUS
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_LABEL
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_LABEL
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_NAME_DELIMITER
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
MODIFICATION_TIMESTAMP_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
MODIFICATION_TIMESTAMP_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
MODIFIED_BY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
MODIFIED_BY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
PROPAGATED_CLASSIFICATION_NAMES_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
PROPAGATED_CLASSIFICATION_NAMES_KEY
;
...
@@ -450,7 +451,7 @@ public abstract class DeleteHandlerV1 {
...
@@ -450,7 +451,7 @@ public abstract class DeleteHandlerV1 {
graphHelper
.
addClassificationEdge
(
propagatedEntityVertex
,
classificationVertex
,
true
);
graphHelper
.
addClassificationEdge
(
propagatedEntityVertex
,
classificationVertex
,
true
);
addToPropagatedClassification
AndTrait
Names
(
propagatedEntityVertex
,
classificationName
);
addToPropagatedClassificationNames
(
propagatedEntityVertex
,
classificationName
);
// record add propagation details to send notifications at the end
// record add propagation details to send notifications at the end
RequestContext
context
=
RequestContext
.
get
();
RequestContext
context
=
RequestContext
.
get
();
...
@@ -614,7 +615,7 @@ public abstract class DeleteHandlerV1 {
...
@@ -614,7 +615,7 @@ public abstract class DeleteHandlerV1 {
graphHelper
.
removeEdge
(
propagatedEdge
);
graphHelper
.
removeEdge
(
propagatedEdge
);
removeFromPropagatedClassification
AndTrait
Names
(
impactedEntityVertex
,
classificationName
);
removeFromPropagatedClassificationNames
(
impactedEntityVertex
,
classificationName
);
}
else
{
}
else
{
if
(
LOG
.
isDebugEnabled
())
{
if
(
LOG
.
isDebugEnabled
())
{
LOG
.
debug
(
" --> Not removing propagated classification edge from [{}] --> [{}][{}] using edge label: [{}], since edge doesn't exist"
,
LOG
.
debug
(
" --> Not removing propagated classification edge from [{}] --> [{}][{}] using edge label: [{}], since edge doesn't exist"
,
...
@@ -666,7 +667,7 @@ public abstract class DeleteHandlerV1 {
...
@@ -666,7 +667,7 @@ public abstract class DeleteHandlerV1 {
getTypeName
(
entityVertex
),
GraphHelper
.
getGuid
(
entityVertex
),
CLASSIFICATION_LABEL
);
getTypeName
(
entityVertex
),
GraphHelper
.
getGuid
(
entityVertex
),
CLASSIFICATION_LABEL
);
}
}
removeFromPropagatedClassification
AndTrait
Names
(
entityVertex
,
classificationName
);
removeFromPropagatedClassificationNames
(
entityVertex
,
classificationName
);
deleteEdge
(
edge
,
true
);
deleteEdge
(
edge
,
true
);
...
@@ -987,18 +988,47 @@ public abstract class DeleteHandlerV1 {
...
@@ -987,18 +988,47 @@ public abstract class DeleteHandlerV1 {
return
Objects
.
nonNull
(
entityType
)
&&
entityType
.
isInternalType
();
return
Objects
.
nonNull
(
entityType
)
&&
entityType
.
isInternalType
();
}
}
private
void
removeFromPropagatedClassificationAndTraitNames
(
AtlasVertex
entityVertex
,
String
classificationName
)
{
private
void
addToPropagatedClassificationNames
(
AtlasVertex
entityVertex
,
String
classificationName
)
{
if
(
LOG
.
isDebugEnabled
())
{
LOG
.
debug
(
"Adding property {} = \"{}\" to vertex {}"
,
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
,
classificationName
,
string
(
entityVertex
));
}
entityVertex
.
addListProperty
(
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
,
classificationName
);
String
propClsNames
=
entityVertex
.
getProperty
(
PROPAGATED_CLASSIFICATION_NAMES_KEY
,
String
.
class
);
propClsNames
=
StringUtils
.
isEmpty
(
propClsNames
)
?
CLASSIFICATION_NAME_DELIMITER
+
classificationName
:
(
propClsNames
+
classificationName
);
propClsNames
=
propClsNames
+
CLASSIFICATION_NAME_DELIMITER
;
entityVertex
.
setProperty
(
PROPAGATED_CLASSIFICATION_NAMES_KEY
,
propClsNames
);
}
private
void
removeFromPropagatedClassificationNames
(
AtlasVertex
entityVertex
,
String
classificationName
)
{
if
(
entityVertex
!=
null
&&
StringUtils
.
isNotEmpty
(
classificationName
))
{
if
(
entityVertex
!=
null
&&
StringUtils
.
isNotEmpty
(
classificationName
))
{
List
<
String
>
propagatedTraitNames
=
getTraitNames
(
entityVertex
,
true
);
List
<
String
>
propagatedTraitNames
=
getTraitNames
(
entityVertex
,
true
);
propagatedTraitNames
.
remove
(
classificationName
);
propagatedTraitNames
.
remove
(
classificationName
);
entityVertex
.
removeProperty
(
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
);
setPropagatedClassificationNames
(
entityVertex
,
propagatedTraitNames
);
entityVertex
.
removeProperty
(
PROPAGATED_CLASSIFICATION_NAMES_KEY
);
}
}
private
void
setPropagatedClassificationNames
(
AtlasVertex
entityVertex
,
List
<
String
>
classificationNames
)
{
if
(
LOG
.
isDebugEnabled
())
{
LOG
.
debug
(
"Adding property {} = \"{}\" to vertex {}"
,
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
,
classificationNames
,
string
(
entityVertex
));
}
for
(
String
propagatedTraitName
:
propagatedTraitNames
)
{
entityVertex
.
removeProperty
(
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
);
addToPropagatedClassificationAndTraitNames
(
entityVertex
,
propagatedTraitName
);
entityVertex
.
removeProperty
(
PROPAGATED_CLASSIFICATION_NAMES_KEY
);
if
(
CollectionUtils
.
isNotEmpty
(
classificationNames
))
{
for
(
String
classificationName
:
classificationNames
)
{
entityVertex
.
addListProperty
(
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
,
classificationName
);
}
}
String
propClsName
=
CLASSIFICATION_NAME_DELIMITER
+
StringUtils
.
join
(
classificationNames
,
CLASSIFICATION_NAME_DELIMITER
)
+
CLASSIFICATION_NAME_DELIMITER
;
entityVertex
.
setProperty
(
PROPAGATED_CLASSIFICATION_NAMES_KEY
,
propClsName
);
}
}
}
}
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
View file @
8127189c
...
@@ -81,11 +81,11 @@ import static org.apache.atlas.model.instance.EntityMutations.EntityOperation.PA
...
@@ -81,11 +81,11 @@ import static org.apache.atlas.model.instance.EntityMutations.EntityOperation.PA
import
static
org
.
apache
.
atlas
.
model
.
instance
.
EntityMutations
.
EntityOperation
.
UPDATE
;
import
static
org
.
apache
.
atlas
.
model
.
instance
.
EntityMutations
.
EntityOperation
.
UPDATE
;
import
static
org
.
apache
.
atlas
.
model
.
typedef
.
AtlasStructDef
.
AtlasAttributeDef
.
Cardinality
.
SET
;
import
static
org
.
apache
.
atlas
.
model
.
typedef
.
AtlasStructDef
.
AtlasAttributeDef
.
Cardinality
.
SET
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.*;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.*;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
DEFAULT_REMOVE_PROPAGATIONS_ON_ENTITY_DELETE
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getCollectionElementsUsingRelationship
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getCollectionElementsUsingRelationship
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getClassificationEdge
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getClassificationEdge
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getClassificationVertex
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getClassificationVertex
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getDefaultRemovePropagations
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getDefaultRemovePropagations
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getDelimitedClassificationNames
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getMapElementsProperty
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getMapElementsProperty
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getStatus
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getStatus
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getTraitLabel
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getTraitLabel
;
...
@@ -1512,7 +1512,7 @@ public class EntityGraphMapper {
...
@@ -1512,7 +1512,7 @@ public class EntityGraphMapper {
LOG
.
debug
(
"Adding classification [{}] to [{}] using edge label: [{}]"
,
classificationName
,
entityType
.
getTypeName
(),
getTraitLabel
(
classificationName
));
LOG
.
debug
(
"Adding classification [{}] to [{}] using edge label: [{}]"
,
classificationName
,
entityType
.
getTypeName
(),
getTraitLabel
(
classificationName
));
}
}
add
ClassificationAndTrait
Names
(
entityVertex
,
classificationName
);
add
ToClassification
Names
(
entityVertex
,
classificationName
);
// add a new AtlasVertex for the struct or trait instance
// add a new AtlasVertex for the struct or trait instance
AtlasVertex
classificationVertex
=
createClassificationVertex
(
classification
);
AtlasVertex
classificationVertex
=
createClassificationVertex
(
classification
);
...
@@ -1676,47 +1676,67 @@ public class EntityGraphMapper {
...
@@ -1676,47 +1676,67 @@ public class EntityGraphMapper {
traitNames
.
remove
(
classificationName
);
traitNames
.
remove
(
classificationName
);
updateClassificationAndTrait
Names
(
entityVertex
,
traitNames
);
setClassification
Names
(
entityVertex
,
traitNames
);
updateModificationMetadata
(
entityVertex
);
updateModificationMetadata
(
entityVertex
);
for
(
Map
.
Entry
<
AtlasVertex
,
List
<
AtlasClassification
>>
entry
:
removedClassifications
.
entrySet
())
{
for
(
Map
.
Entry
<
AtlasVertex
,
List
<
AtlasClassification
>>
entry
:
removedClassifications
.
entrySet
())
{
AtlasEntity
entity
=
updateClassificationText
(
entry
.
getKey
());
AtlasEntity
entity
=
updateClassificationText
(
entry
.
getKey
());
List
<
AtlasClassification
>
deletedClassificationNames
=
entry
.
getValue
();
List
<
AtlasClassification
>
deletedClassificationNames
=
entry
.
getValue
();
entityChangeNotifier
.
onClassificationDeletedFromEntity
(
entity
,
deletedClassificationNames
);
entityChangeNotifier
.
onClassificationDeletedFromEntity
(
entity
,
deletedClassificationNames
);
}
}
}
}
p
ublic
AtlasEntity
updateClassificationText
(
AtlasVertex
vertex
)
throws
AtlasBaseException
{
p
rivate
AtlasEntity
updateClassificationText
(
AtlasVertex
vertex
)
throws
AtlasBaseException
{
String
guid
=
GraphHelper
.
getGuid
(
vertex
);
String
guid
=
GraphHelper
.
getGuid
(
vertex
);
AtlasEntity
entity
=
instanceConverter
.
getAndCacheEntity
(
guid
);
AtlasEntity
entity
=
instanceConverter
.
getAndCacheEntity
(
guid
);
vertex
.
setProperty
(
CLASSIFICATION_TEXT_KEY
,
fullTextMapperV2
.
getClassificationTextForEntity
(
entity
));
vertex
.
setProperty
(
CLASSIFICATION_TEXT_KEY
,
fullTextMapperV2
.
getClassificationTextForEntity
(
entity
));
return
entity
;
return
entity
;
}
}
public
void
setClassificationNames
(
AtlasVertex
vertex
)
{
public
void
updateClassificationTextAndNames
(
AtlasVertex
vertex
)
throws
AtlasBaseException
{
List
<
String
>
clsNamesList
=
getTraitNames
(
vertex
,
false
);
String
guid
=
GraphHelper
.
getGuid
(
vertex
);
List
<
String
>
propClsNamesList
=
getTraitNames
(
vertex
,
true
);
AtlasEntity
entity
=
instanceConverter
.
getAndCacheEntity
(
guid
);
String
clsNames
=
StringUtils
.
join
(
clsNamesList
,
CLASSIFICATION_NAME_DELIMITER
);
String
propClsNames
=
StringUtils
.
join
(
propClsNamesList
,
CLASSIFICATION_NAME_DELIMITER
);
if
(
CollectionUtils
.
isEmpty
(
entity
.
getClassifications
()))
return
;
List
<
String
>
classificationNames
=
new
ArrayList
<>();
List
<
String
>
propagatedClassificationNames
=
new
ArrayList
<>();
for
(
AtlasClassification
classification
:
entity
.
getClassifications
())
{
if
(
isPropagatedClassification
(
classification
,
guid
))
{
propagatedClassificationNames
.
add
(
classification
.
getTypeName
());
}
else
{
classificationNames
.
add
(
classification
.
getTypeName
());
}
}
vertex
.
setProperty
(
CLASSIFICATION_NAMES_KEY
,
clsNames
);
vertex
.
setProperty
(
CLASSIFICATION_NAMES_KEY
,
getDelimitedClassificationNames
(
classificationNames
));
vertex
.
setProperty
(
PROPAGATED_CLASSIFICATION_NAMES_KEY
,
propClsNames
);
vertex
.
setProperty
(
PROPAGATED_CLASSIFICATION_NAMES_KEY
,
getDelimitedClassificationNames
(
propagatedClassificationNames
));
vertex
.
setProperty
(
CLASSIFICATION_TEXT_KEY
,
fullTextMapperV2
.
getClassificationTextForEntity
(
entity
));
}
}
private
void
addClassificationAndTraitNames
(
AtlasVertex
entityVertex
,
String
classificationName
)
{
private
boolean
isPropagatedClassification
(
AtlasClassification
classification
,
String
guid
)
{
String
classificationEntityGuid
=
classification
.
getEntityGuid
();
return
StringUtils
.
isNotEmpty
(
classificationEntityGuid
)
&&
!
StringUtils
.
equals
(
classificationEntityGuid
,
guid
);
}
private
void
addToClassificationNames
(
AtlasVertex
entityVertex
,
String
classificationName
)
{
AtlasGraphUtilsV2
.
addEncodedProperty
(
entityVertex
,
TRAIT_NAMES_PROPERTY_KEY
,
classificationName
);
AtlasGraphUtilsV2
.
addEncodedProperty
(
entityVertex
,
TRAIT_NAMES_PROPERTY_KEY
,
classificationName
);
String
clsNames
=
entityVertex
.
getProperty
(
CLASSIFICATION_NAMES_KEY
,
String
.
class
);
String
clsNames
=
entityVertex
.
getProperty
(
CLASSIFICATION_NAMES_KEY
,
String
.
class
);
clsNames
=
StringUtils
.
isEmpty
(
clsNames
)
?
classificationName
:
clsNames
+
CLASSIFICATION_NAME_DELIMITER
+
classificationName
;
clsNames
=
StringUtils
.
isEmpty
(
clsNames
)
?
CLASSIFICATION_NAME_DELIMITER
+
classificationName
:
clsNames
+
classificationName
;
clsNames
=
clsNames
+
CLASSIFICATION_NAME_DELIMITER
;
entityVertex
.
setProperty
(
CLASSIFICATION_NAMES_KEY
,
clsNames
);
entityVertex
.
setProperty
(
CLASSIFICATION_NAMES_KEY
,
clsNames
);
}
}
private
void
updateClassificationAndTrait
Names
(
AtlasVertex
entityVertex
,
List
<
String
>
traitNames
)
{
private
void
setClassification
Names
(
AtlasVertex
entityVertex
,
List
<
String
>
traitNames
)
{
if
(
entityVertex
!=
null
)
{
if
(
entityVertex
!=
null
)
{
entityVertex
.
removeProperty
(
TRAIT_NAMES_PROPERTY_KEY
);
entityVertex
.
removeProperty
(
TRAIT_NAMES_PROPERTY_KEY
);
entityVertex
.
removeProperty
(
CLASSIFICATION_NAMES_KEY
);
entityVertex
.
removeProperty
(
CLASSIFICATION_NAMES_KEY
);
...
@@ -1725,7 +1745,11 @@ public class EntityGraphMapper {
...
@@ -1725,7 +1745,11 @@ public class EntityGraphMapper {
AtlasGraphUtilsV2
.
addEncodedProperty
(
entityVertex
,
TRAIT_NAMES_PROPERTY_KEY
,
traitName
);
AtlasGraphUtilsV2
.
addEncodedProperty
(
entityVertex
,
TRAIT_NAMES_PROPERTY_KEY
,
traitName
);
}
}
entityVertex
.
setProperty
(
CLASSIFICATION_NAMES_KEY
,
StringUtils
.
join
(
traitNames
,
CLASSIFICATION_NAME_DELIMITER
));
String
clsNames
=
StringUtils
.
join
(
traitNames
,
CLASSIFICATION_NAME_DELIMITER
);
clsNames
=
StringUtils
.
isEmpty
(
clsNames
)
?
clsNames
:
CLASSIFICATION_NAME_DELIMITER
+
clsNames
+
CLASSIFICATION_NAME_DELIMITER
;
entityVertex
.
setProperty
(
CLASSIFICATION_NAMES_KEY
,
clsNames
);
}
}
}
}
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityStateChecker.java
View file @
8127189c
...
@@ -34,6 +34,7 @@ import org.apache.atlas.type.AtlasClassificationType;
...
@@ -34,6 +34,7 @@ import org.apache.atlas.type.AtlasClassificationType;
import
org.apache.atlas.type.AtlasEntityType
;
import
org.apache.atlas.type.AtlasEntityType
;
import
org.apache.atlas.type.AtlasTypeRegistry
;
import
org.apache.atlas.type.AtlasTypeRegistry
;
import
org.apache.commons.collections.CollectionUtils
;
import
org.apache.commons.collections.CollectionUtils
;
import
org.apache.commons.lang.StringUtils
;
import
org.slf4j.Logger
;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
org.slf4j.LoggerFactory
;
import
org.springframework.stereotype.Component
;
import
org.springframework.stereotype.Component
;
...
@@ -48,6 +49,7 @@ import java.util.List;
...
@@ -48,6 +49,7 @@ import java.util.List;
import
java.util.Map
;
import
java.util.Map
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_EDGE_IS_PROPAGATED_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
Constants
.
CLASSIFICATION_EDGE_IS_PROPAGATED_PROPERTY_KEY
;
import
static
org
.
apache
.
atlas
.
repository
.
graph
.
GraphHelper
.
getDelimitedClassificationNames
;
@Component
@Component
public
final
class
EntityStateChecker
{
public
final
class
EntityStateChecker
{
...
@@ -266,10 +268,13 @@ public final class EntityStateChecker {
...
@@ -266,10 +268,13 @@ public final class EntityStateChecker {
}
}
entityVertex
.
removeProperty
(
Constants
.
TRAIT_NAMES_PROPERTY_KEY
);
entityVertex
.
removeProperty
(
Constants
.
TRAIT_NAMES_PROPERTY_KEY
);
entityVertex
.
removeProperty
(
Constants
.
CLASSIFICATION_NAMES_KEY
);
for
(
String
classificationName
:
traitVertexNames
)
{
for
(
String
classificationName
:
traitVertexNames
)
{
AtlasGraphUtilsV2
.
addEncodedProperty
(
entityVertex
,
Constants
.
TRAIT_NAMES_PROPERTY_KEY
,
classificationName
);
AtlasGraphUtilsV2
.
addEncodedProperty
(
entityVertex
,
Constants
.
TRAIT_NAMES_PROPERTY_KEY
,
classificationName
);
}
}
entityVertex
.
setProperty
(
Constants
.
CLASSIFICATION_NAMES_KEY
,
getDelimitedClassificationNames
(
traitVertexNames
));
}
}
if
(
propagatedTraitNamesToAdd
!=
null
||
propagatedTraitNamesToRemove
!=
null
)
{
if
(
propagatedTraitNamesToAdd
!=
null
||
propagatedTraitNamesToRemove
!=
null
)
{
...
@@ -282,10 +287,13 @@ public final class EntityStateChecker {
...
@@ -282,10 +287,13 @@ public final class EntityStateChecker {
}
}
entityVertex
.
removeProperty
(
Constants
.
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
);
entityVertex
.
removeProperty
(
Constants
.
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
);
entityVertex
.
removeProperty
(
Constants
.
PROPAGATED_CLASSIFICATION_NAMES_KEY
);
for
(
String
classificationName
:
propagatedTraitVertexNames
)
{
for
(
String
classificationName
:
propagatedTraitVertexNames
)
{
AtlasGraphUtilsV2
.
addEncodedProperty
(
entityVertex
,
Constants
.
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
,
classificationName
);
AtlasGraphUtilsV2
.
addEncodedProperty
(
entityVertex
,
Constants
.
PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
,
classificationName
);
}
}
entityVertex
.
setProperty
(
Constants
.
PROPAGATED_CLASSIFICATION_NAMES_KEY
,
getDelimitedClassificationNames
(
propagatedTraitVertexNames
));
}
}
AtlasGraphUtilsV2
.
setEncodedProperty
(
entityVertex
,
Constants
.
MODIFICATION_TIMESTAMP_PROPERTY_KEY
,
RequestContext
.
get
().
getRequestTime
());
AtlasGraphUtilsV2
.
setEncodedProperty
(
entityVertex
,
Constants
.
MODIFICATION_TIMESTAMP_PROPERTY_KEY
,
RequestContext
.
get
().
getRequestTime
());
...
...
This diff is collapsed.
Click to expand it.
webapp/src/test/java/org/apache/atlas/web/adapters/TestEntityREST.java
View file @
8127189c
...
@@ -131,11 +131,12 @@ public class TestEntityREST {
...
@@ -131,11 +131,12 @@ public class TestEntityREST {
Assert
.
assertEquals
(
retrievedClassification
,
testClassification
);
Assert
.
assertEquals
(
retrievedClassification
,
testClassification
);
// For ATLAS-3327 to test internal properties are added properly.
// For ATLAS-3327 to test internal properties are added properly.
AtlasVertex
vertex
=
AtlasGraphUtilsV2
.
findByGuid
(
dbEntity
.
getGuid
());
AtlasVertex
vertex
=
AtlasGraphUtilsV2
.
findByGuid
(
dbEntity
.
getGuid
());
String
expectedClsName
=
Constants
.
CLASSIFICATION_NAME_DELIMITER
+
TestUtilsV2
.
CLASSIFICATION
+
Constants
.
CLASSIFICATION_NAME_DELIMITER
;
String
classificationNames
=
vertex
.
getProperty
(
Constants
.
CLASSIFICATION_NAMES_KEY
,
String
.
class
);
String
classificationNames
=
vertex
.
getProperty
(
Constants
.
CLASSIFICATION_NAMES_KEY
,
String
.
class
);
Assert
.
assertNotNull
(
classificationNames
);
Assert
.
assertNotNull
(
classificationNames
);
Assert
.
assertEquals
(
classificationNames
,
TestUtilsV2
.
CLASSIFICATION
);
Assert
.
assertEquals
(
classificationNames
,
expectedClsName
);
}
}
@Test
(
dependsOnMethods
=
"testGetEntityById"
)
@Test
(
dependsOnMethods
=
"testGetEntityById"
)
...
@@ -167,8 +168,9 @@ public class TestEntityREST {
...
@@ -167,8 +168,9 @@ public class TestEntityREST {
}
}
// For ATLAS-3327 to test internal properties are added properly.
// For ATLAS-3327 to test internal properties are added properly.
String
expectedClsNames
=
TestUtilsV2
.
CLASSIFICATION
+
"|"
+
TestUtilsV2
.
PHI
;
String
expectedClsNames
=
Constants
.
CLASSIFICATION_NAME_DELIMITER
+
TestUtilsV2
.
CLASSIFICATION
+
AtlasVertex
vertex
=
AtlasGraphUtilsV2
.
findByGuid
(
dbEntity
.
getGuid
());
Constants
.
CLASSIFICATION_NAME_DELIMITER
+
TestUtilsV2
.
PHI
+
Constants
.
CLASSIFICATION_NAME_DELIMITER
;
AtlasVertex
vertex
=
AtlasGraphUtilsV2
.
findByGuid
(
dbEntity
.
getGuid
());
String
classificationNames
=
vertex
.
getProperty
(
Constants
.
CLASSIFICATION_NAMES_KEY
,
String
.
class
);
String
classificationNames
=
vertex
.
getProperty
(
Constants
.
CLASSIFICATION_NAMES_KEY
,
String
.
class
);
Assert
.
assertNotNull
(
classificationNames
);
Assert
.
assertNotNull
(
classificationNames
);
...
@@ -204,8 +206,8 @@ public class TestEntityREST {
...
@@ -204,8 +206,8 @@ public class TestEntityREST {
deleteClassification
(
dbEntity
.
getGuid
(),
TestUtilsV2
.
PHI
);
deleteClassification
(
dbEntity
.
getGuid
(),
TestUtilsV2
.
PHI
);
expectedClsNames
=
TestUtilsV2
.
CLASSIFICATION
;
expectedClsNames
=
Constants
.
CLASSIFICATION_NAME_DELIMITER
+
TestUtilsV2
.
CLASSIFICATION
+
Constants
.
CLASSIFICATION_NAME_DELIMITER
;
vertex
=
AtlasGraphUtilsV2
.
findByGuid
(
dbEntity
.
getGuid
());
vertex
=
AtlasGraphUtilsV2
.
findByGuid
(
dbEntity
.
getGuid
());
classificationNames
=
vertex
.
getProperty
(
Constants
.
CLASSIFICATION_NAMES_KEY
,
String
.
class
);
classificationNames
=
vertex
.
getProperty
(
Constants
.
CLASSIFICATION_NAMES_KEY
,
String
.
class
);
Assert
.
assertNotNull
(
classificationNames
);
Assert
.
assertNotNull
(
classificationNames
);
...
@@ -257,7 +259,7 @@ public class TestEntityREST {
...
@@ -257,7 +259,7 @@ public class TestEntityREST {
Assert
.
assertEquals
(
classifications
,
retrievedClassificationsList
);
Assert
.
assertEquals
(
classifications
,
retrievedClassificationsList
);
String
expectedClsNames
=
TestUtilsV2
.
CLASSIFICATION
;
String
expectedClsNames
=
Constants
.
CLASSIFICATION_NAME_DELIMITER
+
TestUtilsV2
.
CLASSIFICATION
+
Constants
.
CLASSIFICATION_NAME_DELIMITER
;
AtlasVertex
vertex
=
AtlasGraphUtilsV2
.
findByGuid
(
dbEntity
.
getGuid
());
AtlasVertex
vertex
=
AtlasGraphUtilsV2
.
findByGuid
(
dbEntity
.
getGuid
());
String
classificationNames
=
vertex
.
getProperty
(
Constants
.
CLASSIFICATION_NAMES_KEY
,
String
.
class
);
String
classificationNames
=
vertex
.
getProperty
(
Constants
.
CLASSIFICATION_NAMES_KEY
,
String
.
class
);
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment