Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
atlas
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
dataplatform
atlas
Commits
c08c9f38
Commit
c08c9f38
authored
May 06, 2020
by
chaitali borole
Committed by
nixonrodrigues
May 14, 2020
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
ATLAS-3783 : DSL query search should return results for both the relationship edge directions
Signed-off-by:
nixonrodrigues
<
nixon@apache.org
>
parent
1c034cba
Hide whitespace changes
Inline
Side-by-side
Showing
7 changed files
with
62 additions
and
5 deletions
+62
-5
SearchAdvanced.md
docs/src/documents/Search/SearchAdvanced.md
+12
-0
GremlinQueryComposer.java
...ain/java/org/apache/atlas/query/GremlinQueryComposer.java
+14
-2
IdentifierHelper.java
...rc/main/java/org/apache/atlas/query/IdentifierHelper.java
+8
-2
Lookup.java
repository/src/main/java/org/apache/atlas/query/Lookup.java
+3
-0
RegistryBasedLookup.java
...main/java/org/apache/atlas/query/RegistryBasedLookup.java
+16
-0
DSLQueriesTest.java
.../src/test/java/org/apache/atlas/query/DSLQueriesTest.java
+2
-0
GremlinQueryComposerTest.java
...java/org/apache/atlas/query/GremlinQueryComposerTest.java
+7
-1
No files found.
docs/src/documents/Search/SearchAdvanced.md
View file @
c08c9f38
...
@@ -98,6 +98,18 @@ Example: To retrieve _DB_ whose name starts with _R_ followed by has any 3 chara
...
@@ -98,6 +98,18 @@ Example: To retrieve _DB_ whose name starts with _R_ followed by has any 3 chara
{
`DB where name like "R???rt?*"`
}
{
`DB where name like "R???rt?*"`
}
</SyntaxHighlighter>
</SyntaxHighlighter>
Example: To find all the columns in a Table.
<SyntaxHighlighter
wrapLines=
{true}
language=
"html"
style=
{theme.dark}
>
{
`Column where table.name="sales_fact"`
}
</SyntaxHighlighter>
Example: To find all the Tables for a column.
<SyntaxHighlighter
wrapLines=
{true}
language=
"html"
style=
{theme.dark}
>
{
`Table where columns.name="sales"`
}
</SyntaxHighlighter>
### Using Date Literals
### Using Date Literals
Dates used in literals need to be specified using the ISO 8601 format.
Dates used in literals need to be specified using the ISO 8601 format.
...
...
repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java
View file @
c08c9f38
...
@@ -49,6 +49,8 @@ import java.util.stream.Stream;
...
@@ -49,6 +49,8 @@ import java.util.stream.Stream;
import
static
org
.
apache
.
atlas
.
model
.
discovery
.
SearchParameters
.
ALL_CLASSIFICATIONS
;
import
static
org
.
apache
.
atlas
.
model
.
discovery
.
SearchParameters
.
ALL_CLASSIFICATIONS
;
import
static
org
.
apache
.
atlas
.
model
.
discovery
.
SearchParameters
.
NO_CLASSIFICATIONS
;
import
static
org
.
apache
.
atlas
.
model
.
discovery
.
SearchParameters
.
NO_CLASSIFICATIONS
;
import
static
org
.
apache
.
atlas
.
type
.
AtlasStructType
.
AtlasAttribute
.
AtlasRelationshipEdgeDirection
.
IN
;
import
static
org
.
apache
.
atlas
.
type
.
AtlasStructType
.
AtlasAttribute
.
AtlasRelationshipEdgeDirection
.
OUT
;
public
class
GremlinQueryComposer
{
public
class
GremlinQueryComposer
{
private
static
final
Logger
LOG
=
LoggerFactory
.
getLogger
(
GremlinQueryComposer
.
class
);
private
static
final
Logger
LOG
=
LoggerFactory
.
getLogger
(
GremlinQueryComposer
.
class
);
...
@@ -214,7 +216,12 @@ public class GremlinQueryComposer {
...
@@ -214,7 +216,12 @@ public class GremlinQueryComposer {
if
(
org
!=
null
&&
org
.
isReferredType
())
{
if
(
org
!=
null
&&
org
.
isReferredType
())
{
add
(
GremlinClause
.
DEDUP
);
add
(
GremlinClause
.
DEDUP
);
add
(
GremlinClause
.
IN
,
org
.
getEdgeLabel
());
if
(
org
.
getEdgeDirection
()
!=
null
)
{
GremlinClause
gremlinClauseForEdgeLabel
=
org
.
getEdgeDirection
().
equals
(
IN
)
?
GremlinClause
.
OUT
:
GremlinClause
.
IN
;
add
(
gremlinClauseForEdgeLabel
,
org
.
getEdgeLabel
());
}
else
{
add
(
GremlinClause
.
OUT
,
org
.
getEdgeLabel
());
}
context
.
registerActive
(
currentType
);
context
.
registerActive
(
currentType
);
}
}
}
}
...
@@ -575,7 +582,12 @@ public class GremlinQueryComposer {
...
@@ -575,7 +582,12 @@ public class GremlinQueryComposer {
private
boolean
introduceType
(
IdentifierHelper
.
Info
ia
)
{
private
boolean
introduceType
(
IdentifierHelper
.
Info
ia
)
{
if
(
ia
.
isReferredType
())
{
if
(
ia
.
isReferredType
())
{
add
(
GremlinClause
.
OUT
,
ia
.
getEdgeLabel
());
if
(
ia
.
getEdgeDirection
()
!=
null
)
{
GremlinClause
gremlinClauseForEdgeLabel
=
ia
.
getEdgeDirection
().
equals
(
OUT
)
?
GremlinClause
.
OUT
:
GremlinClause
.
IN
;
add
(
gremlinClauseForEdgeLabel
,
ia
.
getEdgeLabel
());
}
else
{
add
(
GremlinClause
.
OUT
,
ia
.
getEdgeLabel
());
}
context
.
registerActive
(
ia
);
context
.
registerActive
(
ia
);
}
}
...
...
repository/src/main/java/org/apache/atlas/query/IdentifierHelper.java
View file @
c08c9f38
...
@@ -21,6 +21,7 @@ package org.apache.atlas.query;
...
@@ -21,6 +21,7 @@ package org.apache.atlas.query;
import
org.apache.atlas.AtlasErrorCode
;
import
org.apache.atlas.AtlasErrorCode
;
import
org.apache.atlas.exception.AtlasBaseException
;
import
org.apache.atlas.exception.AtlasBaseException
;
import
org.apache.atlas.type.AtlasBusinessMetadataType
;
import
org.apache.atlas.type.AtlasBusinessMetadataType
;
import
org.apache.atlas.type.AtlasStructType.AtlasAttribute.AtlasRelationshipEdgeDirection
;
import
org.apache.atlas.type.AtlasType
;
import
org.apache.atlas.type.AtlasType
;
import
org.apache.commons.lang.StringUtils
;
import
org.apache.commons.lang.StringUtils
;
...
@@ -126,6 +127,7 @@ public class IdentifierHelper {
...
@@ -126,6 +127,7 @@ public class IdentifierHelper {
private
String
typeName
;
private
String
typeName
;
private
String
attributeName
;
private
String
attributeName
;
private
boolean
isPrimitive
;
private
boolean
isPrimitive
;
private
AtlasRelationshipEdgeDirection
edgeDirection
;
private
String
edgeLabel
;
private
String
edgeLabel
;
private
boolean
introduceType
;
private
boolean
introduceType
;
private
boolean
hasSubtypes
;
private
boolean
hasSubtypes
;
...
@@ -157,7 +159,7 @@ public class IdentifierHelper {
...
@@ -157,7 +159,7 @@ public class IdentifierHelper {
updateSubTypes
(
lookup
,
context
);
updateSubTypes
(
lookup
,
context
);
}
}
}
catch
(
NullPointerException
ex
)
{
}
catch
(
NullPointerException
ex
)
{
context
.
getErrorList
().
add
(
ex
.
getMessage
()
);
context
.
getErrorList
().
add
(
"NullPointerException"
);
}
}
}
}
...
@@ -178,6 +180,7 @@ public class IdentifierHelper {
...
@@ -178,6 +180,7 @@ public class IdentifierHelper {
private
void
updateEdgeInfo
(
org
.
apache
.
atlas
.
query
.
Lookup
lookup
,
GremlinQueryComposer
.
Context
context
)
{
private
void
updateEdgeInfo
(
org
.
apache
.
atlas
.
query
.
Lookup
lookup
,
GremlinQueryComposer
.
Context
context
)
{
if
(!
isPrimitive
&&
!
isTrait
&&
typeName
!=
attributeName
)
{
if
(!
isPrimitive
&&
!
isTrait
&&
typeName
!=
attributeName
)
{
edgeDirection
=
lookup
.
getRelationshipEdgeDirection
(
context
,
attributeName
);
edgeLabel
=
lookup
.
getRelationshipEdgeLabel
(
context
,
attributeName
);
edgeLabel
=
lookup
.
getRelationshipEdgeLabel
(
context
,
attributeName
);
typeName
=
lookup
.
getTypeFromEdge
(
context
,
attributeName
);
typeName
=
lookup
.
getTypeFromEdge
(
context
,
attributeName
);
}
}
...
@@ -225,7 +228,6 @@ public class IdentifierHelper {
...
@@ -225,7 +228,6 @@ public class IdentifierHelper {
setIsDate
(
lookup
,
context
,
isPrimitive
,
attributeName
);
setIsDate
(
lookup
,
context
,
isPrimitive
,
attributeName
);
setIsNumeric
(
lookup
,
context
,
isPrimitive
,
attributeName
);
setIsNumeric
(
lookup
,
context
,
isPrimitive
,
attributeName
);
}
}
private
String
getDefaultQualifiedNameForSinglePartName
(
GremlinQueryComposer
.
Context
context
,
String
s
)
{
private
String
getDefaultQualifiedNameForSinglePartName
(
GremlinQueryComposer
.
Context
context
,
String
s
)
{
String
qn
=
context
.
getTypeNameFromAlias
(
s
);
String
qn
=
context
.
getTypeNameFromAlias
(
s
);
if
(
StringUtils
.
isEmpty
(
qn
)
&&
SelectClauseComposer
.
isKeyword
(
s
))
{
if
(
StringUtils
.
isEmpty
(
qn
)
&&
SelectClauseComposer
.
isKeyword
(
s
))
{
...
@@ -273,6 +275,10 @@ public class IdentifierHelper {
...
@@ -273,6 +275,10 @@ public class IdentifierHelper {
return
attributeName
;
return
attributeName
;
}
}
public
AtlasRelationshipEdgeDirection
getEdgeDirection
()
{
return
edgeDirection
;
}
public
String
getEdgeLabel
()
{
public
String
getEdgeLabel
()
{
return
edgeLabel
;
return
edgeLabel
;
}
}
...
...
repository/src/main/java/org/apache/atlas/query/Lookup.java
View file @
c08c9f38
...
@@ -20,6 +20,7 @@ package org.apache.atlas.query;
...
@@ -20,6 +20,7 @@ package org.apache.atlas.query;
import
org.apache.atlas.exception.AtlasBaseException
;
import
org.apache.atlas.exception.AtlasBaseException
;
import
org.apache.atlas.type.AtlasType
;
import
org.apache.atlas.type.AtlasType
;
import
org.apache.atlas.type.AtlasStructType.AtlasAttribute.AtlasRelationshipEdgeDirection
;
public
interface
Lookup
{
public
interface
Lookup
{
AtlasType
getType
(
String
typeName
)
throws
AtlasBaseException
;
AtlasType
getType
(
String
typeName
)
throws
AtlasBaseException
;
...
@@ -30,6 +31,8 @@ public interface Lookup {
...
@@ -30,6 +31,8 @@ public interface Lookup {
String
getRelationshipEdgeLabel
(
GremlinQueryComposer
.
Context
context
,
String
attributeName
);
String
getRelationshipEdgeLabel
(
GremlinQueryComposer
.
Context
context
,
String
attributeName
);
AtlasRelationshipEdgeDirection
getRelationshipEdgeDirection
(
GremlinQueryComposer
.
Context
context
,
String
attributeName
);
boolean
hasAttribute
(
GremlinQueryComposer
.
Context
context
,
String
typeName
);
boolean
hasAttribute
(
GremlinQueryComposer
.
Context
context
,
String
typeName
);
boolean
doesTypeHaveSubTypes
(
GremlinQueryComposer
.
Context
context
);
boolean
doesTypeHaveSubTypes
(
GremlinQueryComposer
.
Context
context
);
...
...
repository/src/main/java/org/apache/atlas/query/RegistryBasedLookup.java
View file @
c08c9f38
...
@@ -22,6 +22,7 @@ import org.apache.atlas.exception.AtlasBaseException;
...
@@ -22,6 +22,7 @@ import org.apache.atlas.exception.AtlasBaseException;
import
org.apache.atlas.model.TypeCategory
;
import
org.apache.atlas.model.TypeCategory
;
import
org.apache.atlas.model.typedef.AtlasBaseTypeDef
;
import
org.apache.atlas.model.typedef.AtlasBaseTypeDef
;
import
org.apache.atlas.type.*
;
import
org.apache.atlas.type.*
;
import
org.apache.atlas.type.AtlasStructType.AtlasAttribute.AtlasRelationshipEdgeDirection
;
import
org.apache.commons.lang.StringUtils
;
import
org.apache.commons.lang.StringUtils
;
import
java.util.*
;
import
java.util.*
;
...
@@ -122,6 +123,21 @@ class RegistryBasedLookup implements Lookup {
...
@@ -122,6 +123,21 @@ class RegistryBasedLookup implements Lookup {
}
}
@Override
@Override
public
AtlasRelationshipEdgeDirection
getRelationshipEdgeDirection
(
GremlinQueryComposer
.
Context
context
,
String
attributeName
)
{
AtlasEntityType
entityType
=
context
.
getActiveEntityType
();
AtlasStructType
.
AtlasAttribute
attribute
=
null
;
AtlasRelationshipEdgeDirection
ret
=
null
;
if
(
entityType
!=
null
)
{
attribute
=
entityType
.
getRelationshipAttribute
(
attributeName
,
null
);
if
(
attribute
!=
null
)
{
ret
=
attribute
.
getRelationshipEdgeDirection
();
}
}
return
ret
;
}
@Override
public
boolean
hasAttribute
(
GremlinQueryComposer
.
Context
context
,
String
typeName
)
{
public
boolean
hasAttribute
(
GremlinQueryComposer
.
Context
context
,
String
typeName
)
{
AtlasEntityType
entityType
=
context
.
getActiveEntityType
();
AtlasEntityType
entityType
=
context
.
getActiveEntityType
();
...
...
repository/src/test/java/org/apache/atlas/query/DSLQueriesTest.java
View file @
c08c9f38
...
@@ -205,6 +205,8 @@ public class DSLQueriesTest extends BasicTestSetup {
...
@@ -205,6 +205,8 @@ public class DSLQueriesTest extends BasicTestSetup {
@DataProvider
(
name
=
"basicProvider"
)
@DataProvider
(
name
=
"basicProvider"
)
private
Object
[][]
basicQueries
()
{
private
Object
[][]
basicQueries
()
{
return
new
Object
[][]{
return
new
Object
[][]{
{
"hive_column where table.name = \"sales_fact_daily_mv\""
,
4
},
{
"hive_table where columns.name = \"app_id\""
,
2
},
{
"from hive_db"
,
3
},
{
"from hive_db"
,
3
},
{
"hive_db"
,
3
},
{
"hive_db"
,
3
},
{
"hive_db as d select d"
,
3
},
{
"hive_db as d select d"
,
3
},
...
...
repository/src/test/java/org/apache/atlas/query/GremlinQueryComposerTest.java
View file @
c08c9f38
...
@@ -24,6 +24,7 @@ import org.apache.atlas.model.typedef.AtlasStructDef;
...
@@ -24,6 +24,7 @@ import org.apache.atlas.model.typedef.AtlasStructDef;
import
org.apache.atlas.query.antlr4.AtlasDSLParser
;
import
org.apache.atlas.query.antlr4.AtlasDSLParser
;
import
org.apache.atlas.type.AtlasEntityType
;
import
org.apache.atlas.type.AtlasEntityType
;
import
org.apache.atlas.type.AtlasStructType
;
import
org.apache.atlas.type.AtlasStructType
;
import
org.apache.atlas.type.AtlasStructType.AtlasAttribute.AtlasRelationshipEdgeDirection
;
import
org.apache.atlas.type.AtlasType
;
import
org.apache.atlas.type.AtlasType
;
import
org.apache.atlas.type.AtlasTypeRegistry
;
import
org.apache.atlas.type.AtlasTypeRegistry
;
import
org.apache.commons.lang.StringUtils
;
import
org.apache.commons.lang.StringUtils
;
...
@@ -36,7 +37,7 @@ import static org.mockito.Mockito.mock;
...
@@ -36,7 +37,7 @@ import static org.mockito.Mockito.mock;
import
static
org
.
mockito
.
Mockito
.
when
;
import
static
org
.
mockito
.
Mockito
.
when
;
import
static
org
.
testng
.
Assert
.
assertEquals
;
import
static
org
.
testng
.
Assert
.
assertEquals
;
import
static
org
.
testng
.
Assert
.
fail
;
import
static
org
.
testng
.
Assert
.
fail
;
import
static
org
.
apache
.
atlas
.
type
.
AtlasStructType
.
AtlasAttribute
.
AtlasRelationshipEdgeDirection
.
OUT
;
public
class
GremlinQueryComposerTest
{
public
class
GremlinQueryComposerTest
{
@Test
@Test
public
void
classification
()
{
public
void
classification
()
{
...
@@ -482,6 +483,11 @@ public class GremlinQueryComposerTest {
...
@@ -482,6 +483,11 @@ public class GremlinQueryComposerTest {
}
}
@Override
@Override
public
AtlasRelationshipEdgeDirection
getRelationshipEdgeDirection
(
GremlinQueryComposer
.
Context
context
,
String
attributeName
)
{
return
OUT
;
}
@Override
public
boolean
hasAttribute
(
GremlinQueryComposer
.
Context
context
,
String
attributeName
)
{
public
boolean
hasAttribute
(
GremlinQueryComposer
.
Context
context
,
String
attributeName
)
{
return
(
context
.
getActiveTypeName
().
equals
(
"Table"
)
&&
attributeName
.
equals
(
"db"
))
||
return
(
context
.
getActiveTypeName
().
equals
(
"Table"
)
&&
attributeName
.
equals
(
"db"
))
||
(
context
.
getActiveTypeName
().
equals
(
"Table"
)
&&
attributeName
.
equals
(
"columns"
))
||
(
context
.
getActiveTypeName
().
equals
(
"Table"
)
&&
attributeName
.
equals
(
"columns"
))
||
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment