Search.twiki 4.46 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
---+ Search

Atlas exposes search over the metadata in two ways:
   * Search using DSL
   * Full-text search

---++ Search DSL Grammar
The DSL exposes an SQL like query language for searching the metadata based on the type system.
The grammar for the DSL is below.

<verbatim>
queryWithPath: query ~ opt(WITHPATH)

query: rep1sep(singleQuery, opt(COMMA))

16
singleQuery: singleQrySrc ~ opt(loopExpression) ~ opt(selectClause) ~ opt(orderby) ~ opt(limitOffset)
17 18 19 20 21 22 23 24

singleQrySrc = FROM ~ fromSrc ~ opt(WHERE) ~ opt(expr ^? notIdExpression) |
        WHERE ~ (expr ^? notIdExpression) |
        expr ^? notIdExpression |
        fromSrc ~ opt(WHERE) ~ opt(expr ^? notIdExpression)

fromSrc: identifier ~ AS ~ alias | identifier

25 26 27 28 29 30 31 32
orderby: ORDERBY ~ order ~ opt (sortOrder) 
    
limitOffset: LIMIT ~ lmt ~ opt (offset)
    
offset: OFFSET ~ offsetValue 
    
sortOrder = ASC | DESC
    
33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
loopExpression: LOOP ~ (LPAREN ~> query <~ RPAREN) ~ opt(intConstant <~ TIMES) ~ opt(AS ~> alias)

selectClause: SELECT ~ rep1sep(selectExpression, COMMA)

selectExpression:  expr ~ opt(AS ~> alias)

expr:  compE ~ opt(rep(exprRight))

exprRight: (AND | OR) ~ compE

compE:
        arithE ~ (LT | LTE | EQ | NEQ | GT | GTE) ~ arithE |
            arithE ~ (ISA | IS) ~ ident  |
            arithE ~ HAS ~ ident  |
            arithE

arithE: multiE ~ opt(rep(arithERight))

arithERight: (PLUS | MINUS) ~ multiE

multiE: atomE ~ opt(rep(multiERight))

multiERight: (STAR | DIV) ~ atomE

atomE: literal | identifier | LPAREN ~> expr <~ RPAREN

identifier: rep1sep(ident, DOT)

alias: ident | stringLit

literal: booleanConstant |
        intConstant  |
        longConstant  |
        floatConstant |
        doubleConstant  |
        stringLit
</verbatim>

Grammar language:
{noformat}
opt(a)     => a is optional
~            => a combinator. 'a ~ b' means a followed by b
rep         => zero or more
rep1sep => one or more, separated by second arg.
{noformat}

Language Notes:
80
   * A *!SingleQuery* expression can be used to search for entities of a _Trait_ or _Class_.
81
 Entities can be filtered based on a 'Where Clause' and Entity Attributes can be retrieved based on a 'Select Clause'.
82
   * An Entity Graph can be traversed/joined by combining one or more !SingleQueries.
83 84 85 86
   * An attempt is made to make the expressions look SQL like by accepting keywords "SELECT",
 "FROM", and "WHERE"; but these are optional and users can simply think in terms of Entity Graph Traversals.
   * The transitive closure of an Entity relationship can be expressed via the _Loop_ expression. A
  _Loop_ expression can be any traversal (recursively a query) that represents a _Path_ that ends in an Entity of the same _Type_ as the starting Entity.
87
   * The _!WithPath_ clause can be used with transitive closure queries to retrieve the Path that
88 89
 connects the two related Entities. (We also provide a higher level interface for Closure Queries
  see scaladoc for 'org.apache.atlas.query.ClosureQuery')
90 91 92
   * ORDERBY is optional. Orderby clause should be specified in single quote ('). When order by clause is specified case insensitive sorting is done in ascending order.
    For sorting in descending order specify 'DESC' after order by clause. If no order by is specified then no default sorting is applied.
   * LIMIT is optional. It limits the maximum number of objects to be fetched starting from specified optional offset. If no offset is specified count starts from beginning.
93 94 95
   * There are couple of Predicate functions different from SQL:
      * _is_ or _isa_can be used to filter Entities that have a particular Trait.
      * _has_ can be used to filter Entities that have a value for a particular Attribute.
96 97
   * When querying for a space delimited multiple-word identifier, it need to be enclosed within
   backquote (`)
98 99 100 101 102

---+++ DSL Examples

   * from DB
   * DB where name="Reporting" select name, owner
103 104 105 106 107
   * DB where name="Reporting" select name, owner orderby 'name'
   * DB where name="Reporting" select name limit 10
   * DB where name="Reporting" select name, owner limit 10 offset 0
   * DB where name="Reporting" select name, owner orderby 'name' limit 10 offset 5
   * DB where name="Reporting" select name, owner orderby 'name' desc limit 10 offset 5
108
   * DB has name
109
   * DB is !JdbcAccess
110 111 112
   * Column where Column isa PII
   * Table where name="sales_fact", columns
   * Table where name="sales_fact", columns as column select column.name, column.dataType, column.comment
113
   * `Log Data`
114 115 116 117 118


---++ Full-text Search

Atlas also exposes a lucene style full-text search capability.