Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
atlas
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
dataplatform
atlas
Commits
3de30f55
Commit
3de30f55
authored
4 years ago
by
Ashutosh Mestry
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
ATLAS-3799: EntityConumer only adds entity GUIDs that are added when they were produced.
parent
214f2a76
master
No related merge requests found
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
20 deletions
+5
-20
EntityConsumer.java
...pository/store/graph/v2/bulkimport/pc/EntityConsumer.java
+5
-20
No files found.
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/pc/EntityConsumer.java
View file @
3de30f55
...
...
@@ -59,7 +59,7 @@ public class EntityConsumer extends WorkItemConsumer<AtlasEntity.AtlasEntityWith
private
final
EntityGraphRetriever
entityRetrieverBulk
;
private
List
<
AtlasEntity
.
AtlasEntityWithExtInfo
>
entityBuffer
=
new
ArrayList
<>();
private
List
<
EntityMutationResponse
>
localResults
=
new
ArrayList
<>();
private
List
<
String
>
localResults
=
new
ArrayList
<>();
public
EntityConsumer
(
AtlasTypeRegistry
typeRegistry
,
AtlasGraph
atlasGraph
,
AtlasEntityStore
entityStore
,
...
...
@@ -119,7 +119,7 @@ public class EntityConsumer extends WorkItemConsumer<AtlasEntity.AtlasEntityWith
private
void
importUsingBulkEntityStore
(
AtlasEntity
.
AtlasEntityWithExtInfo
entityWithExtInfo
)
throws
AtlasBaseException
{
EntityStream
oneEntityStream
=
new
AtlasEntityStreamForImport
(
entityWithExtInfo
,
null
);
EntityMutationResponse
result
=
entityStoreBulk
.
createOrUpdateForImportNoCommit
(
oneEntityStream
);
localResults
.
add
(
result
);
localResults
.
add
(
entityWithExtInfo
.
getEntity
().
getGuid
()
);
entityBuffer
.
add
(
entityWithExtInfo
);
}
...
...
@@ -133,9 +133,9 @@ public class EntityConsumer extends WorkItemConsumer<AtlasEntity.AtlasEntityWith
try
{
LOG
.
info
(
"Regular: EntityStore: {}: Starting..."
,
this
.
counter
.
get
());
AtlasEntityStreamForImport
oneEntityStream
=
new
AtlasEntityStreamForImport
(
entityWithExtInfo
,
null
);
EntityMutationResponse
result
=
this
.
entityStore
.
createOrUpdateForImportNoCommit
(
oneEntityStream
);
this
.
entityStore
.
createOrUpdateForImportNoCommit
(
oneEntityStream
);
atlasGraph
.
commit
();
localResults
.
add
(
result
);
localResults
.
add
(
entityWithExtInfo
.
getEntity
().
getGuid
()
);
dispatchResults
();
}
catch
(
Exception
e
)
{
atlasGraph
.
rollback
();
...
...
@@ -244,12 +244,7 @@ public class EntityConsumer extends WorkItemConsumer<AtlasEntity.AtlasEntityWith
}
private
void
dispatchResults
()
{
localResults
.
stream
().
forEach
(
x
->
{
addResultsFromResponse
(
x
.
getCreatedEntities
());
addResultsFromResponse
(
x
.
getUpdatedEntities
());
addResultsFromResponse
(
x
.
getDeletedEntities
());
});
localResults
.
stream
().
forEach
(
x
->
addResult
(
x
));
clear
();
}
...
...
@@ -261,16 +256,6 @@ public class EntityConsumer extends WorkItemConsumer<AtlasEntity.AtlasEntityWith
}
}
private
void
addResultsFromResponse
(
List
<
AtlasEntityHeader
>
entities
)
{
if
(
CollectionUtils
.
isEmpty
(
entities
))
{
return
;
}
for
(
AtlasEntityHeader
eh
:
entities
)
{
addResult
(
eh
.
getGuid
());
}
}
private
void
clear
()
{
localResults
.
clear
();
entityBuffer
.
clear
();
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment