Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
atlas
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
dataplatform
atlas
Commits
fae00825
Commit
fae00825
authored
7 years ago
by
apoorvnaik
Committed by
Madhan Neethiraj
7 years ago
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
ATLAS-2044: In-memory filtering for correctness after index query
parent
fa11d13a
Expand all
Hide whitespace changes
Inline
Side-by-side
Showing
5 changed files
with
13 additions
and
43 deletions
+13
-43
ClassificationSearchProcessor.java
...apache/atlas/discovery/ClassificationSearchProcessor.java
+6
-13
EntitySearchProcessor.java
...ava/org/apache/atlas/discovery/EntitySearchProcessor.java
+6
-17
FullTextSearchProcessor.java
...a/org/apache/atlas/discovery/FullTextSearchProcessor.java
+1
-13
SearchProcessor.java
...main/java/org/apache/atlas/discovery/SearchProcessor.java
+0
-0
SearchPredicateUtil.java
.../main/java/org/apache/atlas/util/SearchPredicateUtil.java
+0
-0
No files found.
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
View file @
fae00825
...
...
@@ -87,6 +87,8 @@ public class ClassificationSearchProcessor extends SearchProcessor {
indexQueryString
=
STRAY_ELIPSIS_PATTERN
.
matcher
(
indexQueryString
).
replaceAll
(
""
);
this
.
indexQuery
=
graph
.
indexQuery
(
Constants
.
VERTEX_INDEX
,
indexQueryString
);
constructInMemoryPredicate
(
classificationType
,
filterCriteria
,
indexAttributes
);
}
else
{
indexQuery
=
null
;
}
...
...
@@ -184,6 +186,9 @@ public class ClassificationSearchProcessor extends SearchProcessor {
getVertices
(
queryResult
,
classificationVertices
);
}
// Do in-memory filtering before the graph query
CollectionUtils
.
filter
(
classificationVertices
,
inMemoryPredicate
);
for
(
AtlasVertex
classificationVertex
:
classificationVertices
)
{
Iterable
<
AtlasEdge
>
edges
=
classificationVertex
.
getEdges
(
AtlasEdgeDirection
.
IN
);
...
...
@@ -208,19 +213,7 @@ public class ClassificationSearchProcessor extends SearchProcessor {
super
.
filter
(
entityVertices
);
for
(
AtlasVertex
entityVertex
:
entityVertices
)
{
resultIdx
++;
if
(
resultIdx
<=
startIdx
)
{
continue
;
}
ret
.
add
(
entityVertex
);
if
(
ret
.
size
()
==
limit
)
{
break
;
}
}
resultIdx
=
collectResultVertices
(
ret
,
startIdx
,
limit
,
resultIdx
,
entityVertices
);
}
}
finally
{
AtlasPerfTracer
.
log
(
perf
);
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
View file @
fae00825
...
...
@@ -71,6 +71,8 @@ public class EntitySearchProcessor extends SearchProcessor {
if
(
attrSearchByIndex
)
{
constructFilterQuery
(
indexQuery
,
entityType
,
filterCriteria
,
indexAttributes
);
constructInMemoryPredicate
(
entityType
,
filterCriteria
,
indexAttributes
);
}
else
{
graphAttributes
.
addAll
(
indexAttributes
);
}
...
...
@@ -165,11 +167,10 @@ public class EntitySearchProcessor extends SearchProcessor {
break
;
}
while
(
idxQueryResult
.
hasNext
())
{
AtlasVertex
vertex
=
idxQueryResult
.
next
().
getVertex
();
getVerticesFromIndexQueryResult
(
idxQueryResult
,
entityVertices
);
entityVertices
.
add
(
vertex
);
}
// Do in-memory filtering before the graph query
CollectionUtils
.
filter
(
entityVertices
,
inMemoryPredicate
);
if
(
graphQuery
!=
null
)
{
AtlasGraphQuery
guidQuery
=
context
.
getGraph
().
query
().
in
(
Constants
.
GUID_PROPERTY_KEY
,
getGuids
(
entityVertices
));
...
...
@@ -191,19 +192,7 @@ public class EntitySearchProcessor extends SearchProcessor {
super
.
filter
(
entityVertices
);
for
(
AtlasVertex
entityVertex
:
entityVertices
)
{
resultIdx
++;
if
(
resultIdx
<=
startIdx
)
{
continue
;
}
ret
.
add
(
entityVertex
);
if
(
ret
.
size
()
==
limit
)
{
break
;
}
}
resultIdx
=
collectResultVertices
(
ret
,
startIdx
,
limit
,
resultIdx
,
entityVertices
);
}
}
finally
{
AtlasPerfTracer
.
log
(
perf
);
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
View file @
fae00825
...
...
@@ -141,19 +141,7 @@ public class FullTextSearchProcessor extends SearchProcessor {
super
.
filter
(
entityVertices
);
for
(
AtlasVertex
entityVertex
:
entityVertices
)
{
resultIdx
++;
if
(
resultIdx
<=
startIdx
)
{
continue
;
}
ret
.
add
(
entityVertex
);
if
(
ret
.
size
()
==
limit
)
{
break
;
}
}
resultIdx
=
collectResultVertices
(
ret
,
startIdx
,
limit
,
resultIdx
,
entityVertices
);
}
}
finally
{
AtlasPerfTracer
.
log
(
perf
);
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
View file @
fae00825
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java
0 → 100644
View file @
fae00825
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment