ExportAPI.md 7.21 KB
Newer Older
1 2
---
name: Export API
3
route: /ExportAPI
4 5 6 7 8 9 10 11 12
menu: Documentation
submenu: Import/Export
---

import  themen  from 'theme/styles/styled-colors';
import  * as theme  from 'react-syntax-highlighter/dist/esm/styles/hljs';
import SyntaxHighlighter from 'react-syntax-highlighter';

# Export API
13 14 15 16 17
The general approach is:
   * Consumer specifies the scope of data to be exported (details below).
   * The API if successful, will return the stream in the format specified.
   * Error will be returned on failure of the call.

18
See [here](#/ExportHDFSAPI) for details on exporting *hdfs_path* entities.
19

20 21
|**Title**|**Export API**|
| ------------ | ------------ |
22 23 24 25
| _Example_ | See Examples sections below. |
| _URL_ |_api/atlas/admin/export_ |
| _Method_ |_POST_ |
| _URL Parameters_ |_None_ |
26
| _Data Parameters_| The class _AtlasExportRequest_ is used to specify the items to export. The list of _AtlasObjectId_(s) allow for specifying the multiple items to export in a session. The _AtlasObjectId_ is a tuple of entity type, name of unique attribute, value of unique attribute. Several items can be specified. See examples below.|
27
| _Success Response_|File stream as _application/zip_.|
28
|_Error Response_|Errors that are handled within the system will be returned as _AtlasBaseException_. |
29 30 31
| _Notes_ | Consumer could choose to consume the output of the API by programmatically using _java.io.ByteOutputStream_ or by manually, save the contents of the stream to a file on the disk.|

__Method Signature__
32 33 34

<SyntaxHighlighter wrapLines={true} language="shell" style={theme.dark}>
{`@POST
35
@Path("/export")
36 37
@Consumes("application/json;charset=UTF-8")`}
</SyntaxHighlighter>
38

39
### Additional Options
40 41 42 43
It is possible to specify additional parameters for the _Export_ operation.

Current implementation has 2 options. Both are optional:

44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

* _matchType_ This option configures the approach used for fetching the starting entity. It has follow values:
    * _startsWith_ Search for an entity that is prefixed with the specified criteria.
    * _endsWith_ Search for an entity that is suffixed with the specified criteria.
    * _contains_ Search for an entity that has the specified criteria as a sub-string.
    *  _matches_ Search for an entity that is a regular expression match with the specified criteria.





* _fetchType_ This option configures the approach used for fetching entities. It has following values:
    * _FULL_: This fetches all the entities that are connected directly and indirectly to the starting entity. E.g. If a starting entity specified is a table, then this option will fetch the table, database and all the other tables within the database.
    * _CONNECTED_: This fetches all the etnties that are connected directly to the starting entity. E.g. If a starting entity specified is a table, then this option will fetch the table and the database entity only.
    *  _INCREMENTAL_: See [here](#/IncrementalExport) for details.


61 62 63

If no _matchType_ is specified, exact match is used. Which means, that the entire string is used in the search criteria.

64
Searching using _matchType_ applies for all types of entities. It is particularly useful for matching entities of type hdfs_path (see [here](#/ExportHDFSAPI)).
65 66 67 68 69

The _fetchType_ option defaults to _FULL_.

For complete example see section below.

70
### Contents of Exported ZIP File
71 72 73 74 75 76 77 78 79 80

The exported ZIP file has the following entries within it:
   * _atlas-export-result.json_:
      * Input filters: The scope of export.
      * File format: The format chosen for the export operation.
      * Metrics: The number of entity definitions, classifications and entities exported.
   * _atlas-typesdef.json_: Type definitions for the entities exported.
   * _atlas-export-order.json_: Order in which entities should be exported.
   * _{guid}.json_: Individual entities are exported with file names that correspond to their id.

81
### Examples
82
The _AtlasExportRequest_ below shows filters that attempt to export 2 databases in cluster cl1:
83 84 85

<SyntaxHighlighter wrapLines={true} language="json" style={theme.dark}>
{`{
86 87 88 89
    "itemsToExport": [
       { "typeName": "hive_db", "uniqueAttributes": { "qualifiedName": "accounts@cl1" } },
       { "typeName": "hive_db", "uniqueAttributes": { "qualifiedName": "hr@cl1" } }
    ]
90 91
}`}
</SyntaxHighlighter>
92

93
The _AtlasExportRequest_ below specifies the _fetchType_ as _FULL_. The _matchType_ option will fetch _accounts@cl1_.
94 95 96

<SyntaxHighlighter wrapLines={true} language="json" style={theme.dark}>
{`{
97
    "itemsToExport": [
98
       { "typeName": "hive_db", "uniqueAttributes": { "qualifiedName": "accounts@" } }
99
    ],
100
    "options": {
101 102 103
        "fetchType": "FULL",
        "matchType": "startsWith"
    }
104 105
}`}
</SyntaxHighlighter>
106

107 108 109 110 111 112 113 114 115 116 117 118 119
The _AtlasExportRequest_ below specifies the _guid_ instead of _uniqueAttribues_ to fetch _accounts@cl1_.

<SyntaxHighlighter wrapLines={true} language="json" style={theme.dark}>
{`{
    "itemsToExport": [
       { "typeName": "hive_db", "guid": "846c5e9c-3ac6-40ju-8289-fb0cebm64783" }
    ],
    "options": {
        "fetchType": "FULL",
    }
}`}
</SyntaxHighlighter>

120
The _AtlasExportRequest_ below specifies the _fetchType_ as _connected_. The _matchType_ option will fetch _accountsReceivable_, _accountsPayable_, etc present in the database.
121 122 123

<SyntaxHighlighter wrapLines={true} language="json" style={theme.dark}>
{`{
124
    "itemsToExport": [
125
       { "typeName": "hive_db", "uniqueAttributes": { "qualifiedName": "accounts" } }
126
    ],
127
    "options": {
128 129 130
        "fetchType": "CONNECTED",
        "matchType": "startsWith"
    }
131 132
}`}
</SyntaxHighlighter>
133

134
Below is the _AtlasExportResult_ JSON for the export of the _Sales_ DB present in the _QuickStart_.
135 136 137

The _metrics_ contains the number of types and entities exported as part of the operation.

138 139
<SyntaxHighlighter wrapLines={true} language="json" style={theme.dark}>
{`{
140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167
    "clientIpAddress": "10.0.2.15",
    "hostName": "10.0.2.2",
    "metrics": {
        "duration": 1415,
        "entitiesWithExtInfo": 12,
        "entity:DB_v1": 2,
        "entity:LoadProcess_v1": 2,
        "entity:Table_v1": 6,
        "entity:View_v1": 2,
        "typedef:Column_v1": 1,
        "typedef:DB_v1": 1,
        "typedef:LoadProcess_v1": 1,
        "typedef:StorageDesc_v1": 1,
        "typedef:Table_v1": 1,
        "typedef:View_v1": 1,
        "typedef:classification": 6
    },
    "operationStatus": "SUCCESS",
    "request": {
        "itemsToExport": [
            {
                "typeName": "DB_v1",
                "uniqueAttributes": {
                    "name": "Sales"
                }
            }
        ],
        "options": {
168
            "fetchType": "full"
169 170 171
        }
    },
    "userName": "admin"
172 173
}`}
</SyntaxHighlighter>
174

175
### CURL Calls
176
Below are sample CURL calls that demonstrate Export of _QuickStart_ database.
177

178 179
<SyntaxHighlighter wrapLines={true} language="shell" style={theme.dark}>
{`curl -X POST -u adminuser:password -H "Content-Type: application/json" -H "Cache-Control: no-cache" -d '{
180
    "itemsToExport": [
181 182 183
            { "typeName": "DB", "uniqueAttributes": { "name": "Sales" }},
            { "typeName": "DB", "uniqueAttributes": { "name": "Reporting" }},
            { "typeName": "DB", "uniqueAttributes": { "name": "Logging" }}
184
    ],
185
        "options": { "fetchType": "full" }
186 187
    }' "http://localhost:21000/api/atlas/admin/export" > quickStartDB.zip`}
</SyntaxHighlighter>