Commit 099324e8 by Umesh Padashetty Committed by Madhan Neethiraj

ATLAS-4045: fixed typos and other grammatical mistakes in export/import API documentation

parent 0eb01b0e
......@@ -23,7 +23,7 @@ See [here](#/ExportHDFSAPI) for details on exporting *hdfs_path* entities.
| _URL_ |_api/atlas/admin/export_ |
| _Method_ |_POST_ |
| _URL Parameters_ |_None_ |
| _Data Parameters_| The class _AtlasExportRequest_ is used to specify the items to export. The list of _AtlasObjectId_(s) allow for specifying the multiple items to export in a session. The _AtlasObjectId_ is a tuple of entity type, name of unique attribute, value of unique attribute. Several items can be specified. See examples below.|
| _Data Parameters_| The class _AtlasExportRequest_ is used to specify the items to export. The list of _AtlasObjectId_(s) allows for specifying the multiple items to export in a session. The _AtlasObjectId_ is a tuple of entity type, name of unique attribute, value of unique attribute. Several items can be specified. See examples below.|
| _Success Response_|File stream as _application/zip_.|
|_Error Response_|Errors that are handled within the system will be returned as _AtlasBaseException_. |
| _Notes_ | Consumer could choose to consume the output of the API by programmatically using _java.io.ByteOutputStream_ or by manually, save the contents of the stream to a file on the disk.|
......@@ -39,10 +39,10 @@ __Method Signature__
### Additional Options
It is possible to specify additional parameters for the _Export_ operation.
Current implementation has 2 options. Both are optional:
The current implementation has 2 options. Both are optional:
* _matchType_ This option configures the approach used for fetching the starting entity. It has follow values:
* _matchType_ This option configures the approach used for fetching the starting entity. It has the following values:
* _startsWith_ Search for an entity that is prefixed with the specified criteria.
* _endsWith_ Search for an entity that is suffixed with the specified criteria.
* _contains_ Search for an entity that has the specified criteria as a sub-string.
......@@ -52,20 +52,20 @@ Current implementation has 2 options. Both are optional:
* _fetchType_ This option configures the approach used for fetching entities. It has following values:
* _fetchType_ This option configures the approach used for fetching entities. It has the following values:
* _FULL_: This fetches all the entities that are connected directly and indirectly to the starting entity. E.g. If a starting entity specified is a table, then this option will fetch the table, database and all the other tables within the database.
* _CONNECTED_: This fetches all the etnties that are connected directly to the starting entity. E.g. If a starting entity specified is a table, then this option will fetch the table and the database entity only.
* _INCREMENTAL_: See [here](#/IncrementalExport) for details.
If no _matchType_ is specified, exact match is used. Which means, that the entire string is used in the search criteria.
If no _matchType_ is specified, an exact match is used. Which means, that the entire string is used in the search criteria.
Searching using _matchType_ applies for all types of entities. It is particularly useful for matching entities of type hdfs_path (see [here](#/ExportHDFSAPI)).
Searching using _matchType_ applies to all types of entities. It is particularly useful for matching entities of type hdfs_path (see [here](#/ExportHDFSAPI)).
The _fetchType_ option defaults to _FULL_.
For complete example see section below.
For a complete example see the section below.
### Contents of Exported ZIP File
......
......@@ -30,10 +30,10 @@ __Sample HDFS Setup__
### Export API Using matchType
To export entities that represent HDFS path, use the Export API using the _matchType_ option. Details can be found [here](#/ExportAPI).
To export entities that represent the HDFS path, use the Export API using the _matchType_ option. Details can be found [here](#/ExportAPI).
### Example Using CURL Calls
Below are sample CURL calls that performs export operation on the _Sample HDFS Setup_ shown above.
Below are sample CURL calls perform an export operation on the _Sample HDFS Setup_ shown above.
<SyntaxHighlighter wrapLines={true} language="shell" style={theme.dark}>
{`curl -X POST -u adminuser:password -H "Content-Type: application/json" -H "Cache-Control: no-cache" -d '{
......@@ -48,4 +48,4 @@ Below are sample CURL calls that performs export operation on the _Sample HDFS S
</SyntaxHighlighter>
### Automatic Creation of HDFS entities
Given that HDFS entity creation is a manual process. The Export API offers a mechanism for creation of requested HDFS entities.
Given that HDFS entity creation is a manual process. The Export API offers a mechanism for the creation of requested HDFS entities.
......@@ -13,7 +13,7 @@ import SyntaxHighlighter from 'react-syntax-highlighter';
#### Background
The new audits for Export and Import operations also have corresponding REST APIs to programatically fetch the audit entries.
The new audits for Export and Import operations also have corresponding REST APIs to programmatically fetch the audit entries.
#### REST APIs
......
......@@ -14,7 +14,7 @@ import SyntaxHighlighter from 'react-syntax-highlighter';
The general approach is:
* Consumer makes a ZIP file available for import operation. See details below for the 2 flavors of the API.
* The API if successful, will return the results of the operation.
* Error will be returned on failure of the call.
* Error will be returned on the failure of the call.
### Import ZIP File Using POST
......@@ -67,9 +67,9 @@ Please see [here](#/ImportAPIOptions) for the available options during import pr
__AtlasImportResult Response__
The API will return the results of the import operation in the format defined by the _AtlasImportResult_:
* _AtlasImportParameters_: This contains a collection of name value pair of the options that are applied during the import operation.
* _AtlasImportParameters_: This contains a collection of name-value pair of the options that are applied during the import operation.
* _Metrics_: Operation metrics. These include details on the number of types imported, number of entities imported, etc.
* _Processed Entities_: Contains list of GUIDs for the entities that were processed.
* _Processed Entities_: Contains a list of GUIDs for the entities that were processed.
* _Operation Status_: Overall status of the operation. Values are _SUCCESS_, PARTIAL_SUCCESS, _FAIL_.
### Examples Using CURL Calls
......@@ -93,7 +93,7 @@ The _request_ parameter is optional. If import has to be run without any options
</SyntaxHighlighter>
The call below performs Import of _QuickStart_ database using a ZIP file available on server.
The call below performs Import of _QuickStart_ database using a ZIP file available on the server.
<SyntaxHighlighter wrapLines={true} language="shell" style={theme.dark}>
{`curl -X POST -u adminuser:password -H "Cache-Control: no-cache" -d ./importOptions.json
......
......@@ -11,7 +11,7 @@ import SyntaxHighlighter from 'react-syntax-highlighter';
# Import API Options
Import API options are specified as _options_ JSON. Since the API accepts multi-part form data, it is possible to sepecify multipls input streams within the CURL call.
Import API options are specified as _options_ JSON. Since the API accepts multi-part form data, it is possible to specify multiple input streams within the CURL call.
### Examples Using CURL Calls
<SyntaxHighlighter wrapLines={true} language="shell" style={theme.dark}>
......@@ -35,7 +35,7 @@ To use the defaults, set the contents of _importOptions.json_ to:
### Options
Following options are supported for Import process:
* Specify transforms during import operation.
* Specify transforms during the import operation.
* Resume import by specifying starting entity guid.
* Optionally import type definition.
* Handling large imports.
......@@ -52,7 +52,7 @@ Right now these are the transforms that can be applied:
Example:
The example below applies couple of transforms to the the _qualifiedName_ attribute of hive_table. It converts the value to lower case, then searches for 'cl1', if found, replaces it with 'cl2'.
The example below applies a couple of transforms to the _qualifiedName_ attribute of hive_table. It converts the value to lower case, then searches for 'cl1', if found, replaces it with 'cl2'.
To use the option, set the contents of _importOptions.json_ to:
......@@ -70,7 +70,7 @@ Please refer to [ATLAS-1825](https://issues.apache.org/jira/browse/ATLAS-1825) f
When an import operation is in progress and the server goes down, it would be possible to resume import from the last successfully imported entity. This would allow the import to resume from where it left off.
Server-side logging is improved to display the detail of the last successfully imported entity, this includes the index within the import list and the entity's guid. Either can be used specify the point to resume import.
Server-side logging is improved to display the detail of the last successfully imported entity, this includes the index within the import list and the entity's guid. Either can be used to specify the point to resume import.
To use the option, set the contents of _importOptions.json_ to:
......@@ -103,19 +103,19 @@ Steps to use the behavior:
The output of Export has _atlas-typedef.json_ that contains the type definitions for the entities exported.
By default (that is if no options is specified), the type definitions are imported and applied to the system being imported to. The entity import is performed after this.
By default (that is if no options are specified), the type definitions are imported and applied to the system being imported to. The entity import is performed after this.
In some cases, you would not want to modify the type definitions. Import may be better off failing than the types be modified.
In some cases, you would not want to modify the type definitions. The import may be better off failing than the types be modified.
This option allows for optionally importing of type definition. The option is set to _true_ by default, which means that type definition is imported. With this option set to _false_, type definitions preseneraent in the source will not be imported. In case of mismatch between the entities being imported the types present in the system where the import is being performed, the operation will fail.
This option allows for optionally importing of the type definition. The option is set to _true_ by default, which means that type definition is imported. With this option set to _false_, type definitions present in the source will not be imported. In case of mismatch between the entities being imported the types present in the system where the import is being performed, the operation will fail.
Table below enumerates the conditions that get addressed as part of type definition import:
|**Condition**|**Action**|
|-------------|----------|
| Incoming type does not exist in target system | Type is created. |
|Type to be imported and type in target system are same | No change |
|Type to be imported and type in target system differ by some attributes| Target system type is updated to the attributes present in the source.<br /> It is possible that the target system will have attributes in addition to the one present in the source.<br /> In that case, the target system's type attributes will be an union of the attributes.<br /> Attributes in target system will not be deleted to match the source. <br />If the type of the attribute differ, import process will be aborted and exception logged.|
| Incoming type does not exist in the target system | Type is created. |
|Type to be imported and type in the target system are same | No change |
|Type to be imported and type in target system differ by some attributes| Target system type is updated to the attributes present in the source.<br /> It is possible that the target system will have attributes in addition to the one present in the source.<br /> In that case, the target system's type attributes will be a union of the attributes.<br /> Attributes in target system will not be deleted to match the source. <br />If the type of the attribute differ, the import process will be aborted and exception logged.|
To use the option, set the contents of _importOptions.json_ to:
......@@ -129,7 +129,7 @@ To use the option, set the contents of _importOptions.json_ to:
#### Specifying File to be Imported From Server Location
In scenario where the file to be imported is present at a location on the server, the _importfile_ API can be used. It behaves like the Import API.
In a scenario where the file to be imported is present at a location on the server, the _importfile_ API can be used. It behaves like the Import API.
To use the option, set the contents of _importOptions.json_ to:
......@@ -152,10 +152,10 @@ _CURL_
#### Handling Large Imports
By default, the Import Service stores all of the data in memory. This may be limiting for ZIPs containing large amount of data.
By default, the Import Service stores all of the data in memory. This may be limiting for ZIPs containing a large amount of data.
To configure temporary directory use the application property _atlas.import.temp.directory_. If this property is left blank, default in-memory implementation is used.
To configure the temporary directory use the application property _atlas.import.temp.directory_. If this property is left blank, the default in-memory implementation is used.
Please ensure that there is sufficient disk space available for the operation.
The contents of the directory created as backing store for the import operation will be erased after the operation is over.
The contents of the directory created as a backing store for the import operation will be erased after the operation is over.
......@@ -19,7 +19,7 @@ During Import Process, entity transforms are required to make changes to the ent
A transformation framework allows a mechanism to selectively transform an entity or specific attributes of that entity.
To achieve this, the framework, provides:
To achieve this, the framework provides:
* Way to set a condition that needs to be satisfied for a transformation to be applied.
* Action to be taken on the entity once the condition is met.
......@@ -28,14 +28,14 @@ The existing transformation frameworks allowed this to happen.
#### Reason for New Transformation Framework
While the existing framework provided the basic benefits of transformation framework, it did not have support for some of the commonly used Atlas types. Which meant that users of this framework would have to meticulously define transformations for every type they are working with. This can be tedious and potentially error prone.
While the existing framework provided the basic benefits of the transformation framework, it did not have support for some of the commonly used Atlas types. Which meant that users of this framework would have to meticulously define transformations for every type they are working with. This can be tedious and potentially error-prone.
The new framework addresses this problem by providing built-in transformations for some of the commonly used types. It can also be extended to accommodate new types.
#### Approach
The approach used by the new transformation framework creates a transformation by:
* Specifying a condition.
* Specifying action(s) to be taken if condition is met.
* Specifying action(s) to be taken if the condition is met.
##### Conditions
......@@ -55,12 +55,12 @@ HAS_VALUE | Entity attribute has value. |
##### Actions
|**Action Type** | *Description** |
-------------------|----------------------------------------------|
ADD_CLASSIFICATION | Add classifiction |
REPLACE_PREFIX | Replace value starting with another value. |
TO_LOWER | Convert value of an attribute to lower case. |
SET | Set the value of an attribute |
CLEAR | Clear value of an attribute |
-------------------|------------------------------------------------|
ADD_CLASSIFICATION | Add classification |
REPLACE_PREFIX | Replace value starting with another value. |
TO_LOWER | Convert a value of an attribute to lower case. |
SET | Set the value of an attribute |
CLEAR | Clear value of an attribute |
#### Built-in Transforms
......@@ -79,7 +79,7 @@ During import, hive_db entity whose _qualifiedName_ is _stocks@cl1_ will get the
}`}
</SyntaxHighlighter>
Every imported entity will get the classification by simply changing the condition. The __entity is special condition which matches entity.
Every imported entity will get the classification by simply changing the condition. The __entity is a special condition which matches entity.
<SyntaxHighlighter wrapLines={true} language="json" style={theme.dark}>
......@@ -93,7 +93,7 @@ Every imported entity will get the classification by simply changing the conditi
}`}
</SyntaxHighlighter>
To add classification to only the top-level entity (entity that is used as starting point for an export), use:
To add the classification to only the top-level entity (an entity that is used as a starting point for an export), use:
<SyntaxHighlighter wrapLines={true} language="json" style={theme.dark}>
{`{
......
......@@ -22,16 +22,16 @@ The release of 0.8.3 includes the following improvements to Export and Import AP
* New [ExportImportAudits](#/ExportImportAudits) for Export & Import operations.
### Background
The Import-Export APIs for Atlas facilitate transfer of data to and from a cluster that has Atlas provisioned.
The Import-Export APIs for Atlas facilitate the transfer of data to and from a cluster that has Atlas provisioned.
The APIs when integrated with backup and/or disaster recovery process will ensure participation of Atlas.
### Introduction
There are 2 broad categories viz. Export & Import. The details of the APIs are discussed below.
There are 2 broad categories viz. Export & Import. The details of the APIs are as discussed below.
The APIs are available only to _admin_ user.
Only a single import or export operation can be performed at a given time. The operations have a potential for generating large amount. They can also put pressure on resources. This restriction tries to alleviate this problem.
Only a single import or export operation can be performed at a given time. The operations have the potential for generating large amount of data. They can also put pressure on resources. This restriction tries to alleviate this problem.
For Import-Export APIs relating to HDFS path, can be found [here](#/ExportHDFSAPI).
......
......@@ -31,7 +31,7 @@ New _fetchType_ added to indicate incremental export. This option can be used wi
#### Getting Change Marker
The very first call to export with _fetchType_ set to _incremental_ should be made with _changeMarker_ set to 0. This will perform a full export. The _AtlasExportResult_ will have the _changeMarker_ set to a value. This is the value that should be used for subsequent call to Export.
The very first call to export with _fetchType_ set to _incremental_ should be made with _changeMarker_ set to 0. This will perform a full export. The _AtlasExportResult_ will have the _changeMarker_ set to a value. This is the value that should be used for a subsequent call to Export.
#### Skip Lineage Option
......@@ -39,5 +39,5 @@ Export can be performed by skipping lineage information. This avoids all lineage
#### Benefit of Incremental Export
The real benefit of incremental export comes when export is done with _skipLineage_ option set to _true_. This greatly improves performance when fetching entities that have changed since the last export.
The real benefit of incremental export comes when the export is done with _skipLineage_ option set to _true_. This greatly improves performance when fetching entities that have changed since the last export.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment