index.twiki 3.97 KB
Newer Older
1
---+ Data Governance and Metadata framework for Hadoop
2 3


4
---++ Overview
5

6
Atlas is a scalable and extensible set of core foundational governance services – enabling
7 8
enterprises to effectively and efficiently meet their compliance requirements within Hadoop and
allows integration with the whole enterprise data ecosystem.
9

10 11 12 13
Apache Atlas provides open metadata management and governance capabilities for organizations
to build a catalog of their data assets, classify and govern these assets and provide collaboration
capabilities around these data assets for data scientists, analysts and the data governance team.

14
---++ Features
15

16 17 18 19 20 21 22 23 24 25 26
---+++ Metadata types & instances
   * Pre-defined types for various Hadoop and non-Hadoop metadata
   * Ability to define new types for the metadata to be managed
   * Types can have primitive attributes, complex attributes, object references; can inherit from other types
   * Instances of types, called entities, capture metadata object details and their relationships
   * REST APIs to work with types and instances allow easier integration

---+++ Classification
   * Ability to dynamically create classifications - like PII, EXPIRES_ON, DATA_QUALITY, SENSITIVE
   * Classifications can include attributes - like expiry_date attribute in EXPIRES_ON classification
   * Entities can be associated with multiple classifications, enabling easier discovery and security enforcement
27
   * Propagation of classifications via lineage - automatically ensures that classifications follow the data as it goes through various processing
28

29 30 31
---+++ Lineage
   * Intuitive UI to view lineage of data as it moves through various processes
   * REST APIs to access and update lineage
32

33 34 35 36
---+++ Search/Discovery
   * Intuitive UI to search entities by type, classification, attribute value or free-text
   * Rich REST APIs to search by complex criteria
   * SQL like query language to search entities - Domain Specific Language (DSL)
37

38
---+++ Security & Data Masking
39 40
   * Fine grained security for metadata access, enabling controls on access to entity instances and operations like add/update/remove classifications
   * Integration with Apache Ranger enables authorization/data-masking on data access based on classifications associated with entities in Apache Atlas. For example:
41 42
      * who can access data classified as PII, SENSITIVE
      * customer-service users can only see last 4 digits of columns classified as NATIONAL_ID
43

44

45
---++ Getting Started
46

47
   * [[WhatsNew-1.0][What's new in Apache Atlas 1.0?]]
48 49
   * [[InstallationSteps][Build & Install]]
   * [[QuickStart][Quick Start]]
50

51

52 53
---++ Documentation

54 55
   * [[Architecture][High Level Architecture]]
   * [[TypeSystem][Type System]]
56 57
   * [[Search - Basic][Search: Basic]]
   * [[Search - Advanced][Search: Advanced]]
58
   * [[Glossary][Glossary]]
59
   * [[security][Security]]
60 61
   * [[Atlas-Authentication][Authentication]]
   * [[Atlas-Authorization-Model][Atlas Authorization Model]]
62 63
      * [[Atlas-Authorization-Simple-Authorizer][Steps to configure Atlas Simple Authorizer]]
      * [[Atlas-Authorization-Ranger-Authorizer][Steps to configure Atlas Ranger Authorizer]]
64
   * [[ClassificationPropagation][Classification Propagation]]
65
   * [[Configuration][Configuration]]
66
   * [[Notifications][Notifications]]
67
   * Hooks & Bridges
68 69 70 71
      * [[Hook-HBase][HBase Hook & Bridge]]
      * [[Hook-Hive][Hive Hook & Bridge]]
      * [[Hook-Sqoop][Sqoop Hook]]
      * [[Hook-Storm][Storm Hook]]
72
      * [[Bridge-Kafka][Kafka Bridge]]
73
   * [[HighAvailability][Fault Tolerance And High Availability Options]]
74
   * [[Migration-0.8-to-1.0][Migration from Apache Atlas 0.8]]
75

76 77
---++ API Documentation

78
   * <a href="api/v2/index.html">REST API Documentation</a>
79
   * [[Import-Export-API][Export & Import REST API Documentation]]
80
   * <a href="../api/rest.html">Legacy API Documentation</a>
81

82 83 84
---++ Developer Setup Documentation
   * [[EclipseSetup][Developer Setup: Eclipse]]

85 86
#LicenseInfo
---+ Licensing Information
87

88
Atlas is distributed under [[http://www.apache.org/licenses/][Apache License 2.0]].