Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
atlas
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
dataplatform
atlas
Commits
39c52b2b
Commit
39c52b2b
authored
9 years ago
by
Shwetha GS
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
ATLAS-79 Unique constraint is not honoured (shwethags)
parent
96059e0a
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
89 additions
and
64 deletions
+89
-64
release-log.txt
release-log.txt
+1
-0
Constants.java
.../src/main/java/org/apache/atlas/repository/Constants.java
+0
-4
GraphBackedSearchIndexer.java
...ache/atlas/repository/graph/GraphBackedSearchIndexer.java
+45
-60
EntityJerseyResourceIT.java
...rg/apache/atlas/web/resources/EntityJerseyResourceIT.java
+43
-0
No files found.
release-log.txt
View file @
39c52b2b
...
@@ -8,6 +8,7 @@ ATLAS-54 Rename configs in hive hook (shwethags)
...
@@ -8,6 +8,7 @@ ATLAS-54 Rename configs in hive hook (shwethags)
ATLAS-3 Mixed Index creation fails with Date types (suma.shivaprasad via shwethags)
ATLAS-3 Mixed Index creation fails with Date types (suma.shivaprasad via shwethags)
ALL CHANGES:
ALL CHANGES:
ATLAS-79 Unique constraint is not honoured (shwethags)
ATLAS-25 Fix Atlas on Java 8 (sandeep.samudrala via shwethags)
ATLAS-25 Fix Atlas on Java 8 (sandeep.samudrala via shwethags)
ATLAS-86 Jenkins build failing as of build #41 (shwethags)
ATLAS-86 Jenkins build failing as of build #41 (shwethags)
ATLAS-80 Support for variables in application properties (shwethags)
ATLAS-80 Support for variables in application properties (shwethags)
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/Constants.java
View file @
39c52b2b
...
@@ -26,19 +26,16 @@ public final class Constants {
...
@@ -26,19 +26,16 @@ public final class Constants {
public
static
final
String
INTERNAL_PROPERTY_KEY_PREFIX
=
"__"
;
public
static
final
String
INTERNAL_PROPERTY_KEY_PREFIX
=
"__"
;
public
static
final
String
GUID_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"guid"
;
public
static
final
String
GUID_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"guid"
;
public
static
final
String
GUID_INDEX
=
"guid_index"
;
/**
/**
* Entity type name property key.
* Entity type name property key.
*/
*/
public
static
final
String
ENTITY_TYPE_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"typeName"
;
public
static
final
String
ENTITY_TYPE_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"typeName"
;
public
static
final
String
ENTITY_TYPE_INDEX
=
"type_index"
;
/**
/**
* Entity type's super types property key.
* Entity type's super types property key.
*/
*/
public
static
final
String
SUPER_TYPES_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"superTypeNames"
;
public
static
final
String
SUPER_TYPES_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"superTypeNames"
;
public
static
final
String
SUPER_TYPES_INDEX
=
"super_types_index"
;
/**
/**
* Full-text for the entity for enabling full-text search.
* Full-text for the entity for enabling full-text search.
...
@@ -57,7 +54,6 @@ public final class Constants {
...
@@ -57,7 +54,6 @@ public final class Constants {
* Trait names property key and index name.
* Trait names property key and index name.
*/
*/
public
static
final
String
TRAIT_NAMES_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"traitNames"
;
public
static
final
String
TRAIT_NAMES_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"traitNames"
;
public
static
final
String
TRAIT_NAMES_INDEX
=
"trait_names_index"
;
public
static
final
String
VERSION_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"version"
;
public
static
final
String
VERSION_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"version"
;
public
static
final
String
TIMESTAMP_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"timestamp"
;
public
static
final
String
TIMESTAMP_PROPERTY_KEY
=
INTERNAL_PROPERTY_KEY_PREFIX
+
"timestamp"
;
...
...
This diff is collapsed.
Click to expand it.
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
View file @
39c52b2b
...
@@ -36,6 +36,7 @@ import org.apache.atlas.typesystem.types.AttributeInfo;
...
@@ -36,6 +36,7 @@ import org.apache.atlas.typesystem.types.AttributeInfo;
import
org.apache.atlas.typesystem.types.ClassType
;
import
org.apache.atlas.typesystem.types.ClassType
;
import
org.apache.atlas.typesystem.types.DataTypes
;
import
org.apache.atlas.typesystem.types.DataTypes
;
import
org.apache.atlas.typesystem.types.IDataType
;
import
org.apache.atlas.typesystem.types.IDataType
;
import
org.apache.atlas.typesystem.types.Multiplicity
;
import
org.apache.atlas.typesystem.types.StructType
;
import
org.apache.atlas.typesystem.types.StructType
;
import
org.apache.atlas.typesystem.types.TraitType
;
import
org.apache.atlas.typesystem.types.TraitType
;
import
org.slf4j.Logger
;
import
org.slf4j.Logger
;
...
@@ -44,8 +45,9 @@ import org.slf4j.LoggerFactory;
...
@@ -44,8 +45,9 @@ import org.slf4j.LoggerFactory;
import
javax.inject.Inject
;
import
javax.inject.Inject
;
import
java.math.BigDecimal
;
import
java.math.BigDecimal
;
import
java.math.BigInteger
;
import
java.math.BigInteger
;
import
java.util.ArrayList
;
import
java.util.Collection
;
import
java.util.Collection
;
import
java.util.
Date
;
import
java.util.
List
;
import
java.util.Map
;
import
java.util.Map
;
/**
/**
...
@@ -59,6 +61,12 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
...
@@ -59,6 +61,12 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
private
TitanManagement
management
;
private
TitanManagement
management
;
List
<
Class
>
MIXED_INDEX_EXCLUSIONS
=
new
ArrayList
()
{{
add
(
Boolean
.
class
);
add
(
BigDecimal
.
class
);
add
(
BigInteger
.
class
);
}};
@Inject
@Inject
public
GraphBackedSearchIndexer
(
GraphProvider
<
TitanGraph
>
graphProvider
)
throws
RepositoryException
{
public
GraphBackedSearchIndexer
(
GraphProvider
<
TitanGraph
>
graphProvider
)
throws
RepositoryException
{
...
@@ -74,7 +82,7 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
...
@@ -74,7 +82,7 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
* Initializes the indices for the graph - create indices for Global Vertex Keys
* Initializes the indices for the graph - create indices for Global Vertex Keys
*/
*/
private
void
initialize
()
{
private
void
initialize
()
{
if
(
management
.
containsPropertyKey
(
Constants
.
GUID
_PROPERTY_KEY
))
{
if
(
management
.
containsPropertyKey
(
Constants
.
VERTEX_TYPE
_PROPERTY_KEY
))
{
LOG
.
info
(
"Global indexes already exist for graph"
);
LOG
.
info
(
"Global indexes already exist for graph"
);
return
;
return
;
}
}
...
@@ -85,20 +93,17 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
...
@@ -85,20 +93,17 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
management
.
buildIndex
(
Constants
.
EDGE_INDEX
,
Edge
.
class
).
buildMixedIndex
(
Constants
.
BACKING_INDEX
);
management
.
buildIndex
(
Constants
.
EDGE_INDEX
,
Edge
.
class
).
buildMixedIndex
(
Constants
.
BACKING_INDEX
);
// create a composite index for guid as its unique
// create a composite index for guid as its unique
createComposite
Index
(
Constants
.
GUID_INDEX
,
Constants
.
GUID_PROPERTY_KEY
,
String
.
class
,
true
,
Cardinality
.
SINGLE
);
createComposite
AndMixedIndex
(
Constants
.
GUID_PROPERTY_KEY
,
String
.
class
,
true
,
Cardinality
.
SINGLE
,
true
);
// create a composite and mixed index for type since it can be combined with other keys
// create a composite and mixed index for type since it can be combined with other keys
createCompositeAndMixedIndex
(
Constants
.
ENTITY_TYPE_INDEX
,
Constants
.
ENTITY_TYPE_PROPERTY_KEY
,
String
.
class
,
createCompositeAndMixedIndex
(
Constants
.
ENTITY_TYPE_PROPERTY_KEY
,
String
.
class
,
false
,
Cardinality
.
SINGLE
,
true
);
false
,
Cardinality
.
SINGLE
);
// create a composite and mixed index for type since it can be combined with other keys
// create a composite and mixed index for type since it can be combined with other keys
createCompositeAndMixedIndex
(
Constants
.
SUPER_TYPES_INDEX
,
Constants
.
SUPER_TYPES_PROPERTY_KEY
,
String
.
class
,
createCompositeAndMixedIndex
(
Constants
.
SUPER_TYPES_PROPERTY_KEY
,
String
.
class
,
false
,
Cardinality
.
SET
,
true
);
false
,
Cardinality
.
SET
);
// create a composite and mixed index for traitNames since it can be combined with other
// create a composite and mixed index for traitNames since it can be combined with other
// keys. Traits must be a set and not a list.
// keys. Traits must be a set and not a list.
createCompositeAndMixedIndex
(
Constants
.
TRAIT_NAMES_INDEX
,
Constants
.
TRAIT_NAMES_PROPERTY_KEY
,
String
.
class
,
createCompositeAndMixedIndex
(
Constants
.
TRAIT_NAMES_PROPERTY_KEY
,
String
.
class
,
false
,
Cardinality
.
SET
,
true
);
false
,
Cardinality
.
SET
);
// Index for full text search
// Index for full text search
createFullTextIndex
();
createFullTextIndex
();
...
@@ -125,13 +130,10 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
...
@@ -125,13 +130,10 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
private
void
createTypeStoreIndexes
()
{
private
void
createTypeStoreIndexes
()
{
//Create unique index on typeName
//Create unique index on typeName
createCompositeIndex
(
Constants
.
TYPENAME_PROPERTY_KEY
,
Constants
.
TYPENAME_PROPERTY_KEY
,
String
.
class
,
true
,
createCompositeAndMixedIndex
(
Constants
.
TYPENAME_PROPERTY_KEY
,
String
.
class
,
true
,
Cardinality
.
SINGLE
,
true
);
Cardinality
.
SINGLE
);
//create index on vertex type
//create index on vertex type
createCompositeIndex
(
Constants
.
VERTEX_TYPE_PROPERTY_KEY
,
Constants
.
VERTEX_TYPE_PROPERTY_KEY
,
String
.
class
,
createCompositeAndMixedIndex
(
Constants
.
VERTEX_TYPE_PROPERTY_KEY
,
String
.
class
,
false
,
Cardinality
.
SINGLE
,
true
);
false
,
Cardinality
.
SINGLE
);
}
}
/**
/**
...
@@ -201,11 +203,14 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
...
@@ -201,11 +203,14 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
final
String
propertyName
=
typeName
+
"."
+
field
.
name
;
final
String
propertyName
=
typeName
+
"."
+
field
.
name
;
switch
(
field
.
dataType
().
getTypeCategory
())
{
switch
(
field
.
dataType
().
getTypeCategory
())
{
case
PRIMITIVE:
case
PRIMITIVE:
createVertexMixedIndex
(
propertyName
,
getPrimitiveClass
(
field
.
dataType
()));
Cardinality
cardinality
=
getCardinality
(
field
.
multiplicity
);
createCompositeAndMixedIndex
(
propertyName
,
getPrimitiveClass
(
field
.
dataType
()),
field
.
isUnique
,
cardinality
,
false
);
break
;
break
;
case
ENUM:
case
ENUM:
createVertexMixedIndex
(
propertyName
,
String
.
class
);
cardinality
=
getCardinality
(
field
.
multiplicity
);
createCompositeAndMixedIndex
(
propertyName
,
String
.
class
,
field
.
isUnique
,
cardinality
,
false
);
break
;
break
;
case
ARRAY:
case
ARRAY:
...
@@ -264,7 +269,7 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
...
@@ -264,7 +269,7 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
throw
new
IllegalArgumentException
(
"unknown data type "
+
dataType
);
throw
new
IllegalArgumentException
(
"unknown data type "
+
dataType
);
}
}
/*
private
Cardinality
getCardinality
(
Multiplicity
multiplicity
)
{
private
Cardinality
getCardinality
(
Multiplicity
multiplicity
)
{
if
(
multiplicity
==
Multiplicity
.
OPTIONAL
||
multiplicity
==
Multiplicity
.
REQUIRED
)
{
if
(
multiplicity
==
Multiplicity
.
OPTIONAL
||
multiplicity
==
Multiplicity
.
REQUIRED
)
{
return
Cardinality
.
SINGLE
;
return
Cardinality
.
SINGLE
;
...
@@ -277,66 +282,46 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
...
@@ -277,66 +282,46 @@ public class GraphBackedSearchIndexer implements SearchIndexer {
// todo - default to LIST as this is the most forgiving
// todo - default to LIST as this is the most forgiving
return
Cardinality
.
LIST
;
return
Cardinality
.
LIST
;
}
}
*/
private
void
createCompositeAndMixedIndex
(
String
indexName
,
String
propertyName
,
Class
propertyClass
,
boolean
isUnique
,
Cardinality
cardinality
)
{
createCompositeIndex
(
indexName
,
propertyName
,
propertyClass
,
isUnique
,
cardinality
);
createVertexMixedIndex
(
propertyName
,
propertyClass
);
}
private
PropertyKey
createCompositeIndex
(
String
indexName
,
String
propertyName
,
Class
propertyClass
,
boolean
isUnique
,
Cardinality
cardinality
)
{
PropertyKey
propertyKey
=
management
.
getPropertyKey
(
propertyName
);
if
(
propertyKey
==
null
)
{
propertyKey
=
management
.
makePropertyKey
(
propertyName
).
dataType
(
propertyClass
).
cardinality
(
cardinality
).
make
();
TitanManagement
.
IndexBuilder
indexBuilder
=
management
.
buildIndex
(
indexName
,
Vertex
.
class
).
addKey
(
propertyKey
);
if
(
isUnique
)
{
indexBuilder
=
indexBuilder
.
unique
();
}
indexBuilder
.
buildCompositeIndex
();
LOG
.
info
(
"Created index for property {} in composite index {}"
,
propertyName
,
indexName
);
}
return
propertyKey
;
}
private
PropertyKey
createVertexMixedIndex
(
String
propertyName
,
Class
propertyClass
)
{
private
PropertyKey
createCompositeAndMixedIndex
(
String
propertyName
,
Class
propertyClass
,
boolean
isUnique
,
Cardinality
cardinality
,
boolean
force
)
{
PropertyKey
propertyKey
=
management
.
getPropertyKey
(
propertyName
);
PropertyKey
propertyKey
=
management
.
getPropertyKey
(
propertyName
);
if
(
propertyKey
==
null
)
{
if
(
propertyKey
==
null
)
{
// ignored cardinality as Can only index single-valued property keys on vertices
propertyKey
=
management
.
makePropertyKey
(
propertyName
).
dataType
(
propertyClass
).
cardinality
(
cardinality
)
propertyKey
=
management
.
makePropertyKey
(
propertyName
).
dataType
(
propertyClass
)
.
make
();
.
make
();
if
(!
checkIfMixedIndexApplicable
(
propertyClass
))
{
if
(
checkIfMixedIndexApplicable
(
propertyClass
,
cardinality
))
{
LOG
.
debug
(
"Creating composite index for property {} of type {} "
,
propertyName
,
propertyClass
.
getName
());
//Use standard index as backing index only supports string, int and geo types
management
.
buildIndex
(
propertyName
,
Vertex
.
class
).
addKey
(
propertyKey
).
buildCompositeIndex
();
LOG
.
debug
(
"Created composite index for property {} of type {} "
,
propertyName
,
propertyClass
.
getName
());
}
else
{
//Use backing index
//Use backing index
LOG
.
debug
(
"Creating backing index for property {} of type {} "
,
propertyName
,
propertyClass
.
getName
());
LOG
.
debug
(
"Creating backing index for property {} of type {} "
,
propertyName
,
propertyClass
.
getName
());
TitanGraphIndex
vertexIndex
=
management
.
getGraphIndex
(
Constants
.
VERTEX_INDEX
);
TitanGraphIndex
vertexIndex
=
management
.
getGraphIndex
(
Constants
.
VERTEX_INDEX
);
management
.
addIndexKey
(
vertexIndex
,
propertyKey
);
management
.
addIndexKey
(
vertexIndex
,
propertyKey
);
LOG
.
debug
(
"Created backing index for property {} of type {} "
,
propertyName
,
propertyClass
.
getName
());
LOG
.
debug
(
"Created backing index for property {} of type {} "
,
propertyName
,
propertyClass
.
getName
());
}
}
LOG
.
info
(
"Created mixed vertex index for property {}"
,
propertyName
);
//Create mixed index only for meta properties and unique constraints:
//Unique can't be achieved with backing/mixed index
//Creating composite index for every attribute will bloat up the index
if
(
force
||
isUnique
)
{
LOG
.
debug
(
"Creating composite index for property {} of type {} "
,
propertyName
,
propertyClass
.
getName
());
TitanManagement
.
IndexBuilder
indexBuilder
=
management
.
buildIndex
(
propertyName
,
Vertex
.
class
).
addKey
(
propertyKey
);
if
(
isUnique
)
{
indexBuilder
.
unique
();
}
indexBuilder
.
buildCompositeIndex
();
LOG
.
debug
(
"Created composite index for property {} of type {} "
,
propertyName
,
propertyClass
.
getName
());
}
}
}
return
propertyKey
;
return
propertyKey
;
}
}
private
boolean
checkIfMixedIndexApplicable
(
Class
propertyClass
)
{
private
boolean
checkIfMixedIndexApplicable
(
Class
propertyClass
,
Cardinality
cardinality
)
{
if
(
propertyClass
==
Boolean
.
class
||
propertyClass
==
BigDecimal
.
class
||
propertyClass
==
BigInteger
.
class
)
{
return
!(
MIXED_INDEX_EXCLUSIONS
.
contains
(
propertyClass
)
||
cardinality
==
Cardinality
.
LIST
||
cardinality
==
return
false
;
Cardinality
.
SET
);
}
return
true
;
}
}
public
void
commit
()
throws
IndexException
{
public
void
commit
()
throws
IndexException
{
...
...
This diff is collapsed.
Click to expand it.
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java
View file @
39c52b2b
...
@@ -116,6 +116,49 @@ public class EntityJerseyResourceIT extends BaseResourceIT {
...
@@ -116,6 +116,49 @@ public class EntityJerseyResourceIT extends BaseResourceIT {
}
}
@Test
@Test
public
void
testUniqueAttribute
()
throws
Exception
{
//create type
String
typeName
=
"type"
+
randomString
();
HierarchicalTypeDefinition
<
ClassType
>
typeDefinition
=
TypesUtil
.
createClassTypeDef
(
typeName
,
ImmutableList
.<
String
>
of
(),
TypesUtil
.
createUniqueRequiredAttrDef
(
"name"
,
DataTypes
.
STRING_TYPE
));
TypesDef
typesDef
=
TypeUtils
.
getTypesDef
(
ImmutableList
.<
EnumTypeDefinition
>
of
(),
ImmutableList
.<
StructTypeDefinition
>
of
(),
ImmutableList
.<
HierarchicalTypeDefinition
<
TraitType
>>
of
(),
ImmutableList
.
of
(
typeDefinition
));
createType
(
typesDef
);
//create entity
String
name
=
"name"
+
randomString
();
Referenceable
referenceable
=
new
Referenceable
(
typeName
);
referenceable
.
set
(
"name"
,
name
);
createInstance
(
referenceable
);
//create entity with same name again - should fail
try
{
createInstance
(
referenceable
);
Assert
.
fail
(
"Expected exception"
);
}
catch
(
Exception
e
)
{
//expected exception
}
//create another type with same attribute - should allow
typeName
=
"type"
+
randomString
();
typeDefinition
=
TypesUtil
.
createClassTypeDef
(
typeName
,
ImmutableList
.<
String
>
of
(),
TypesUtil
.
createUniqueRequiredAttrDef
(
"name"
,
DataTypes
.
STRING_TYPE
));
typesDef
=
TypeUtils
.
getTypesDef
(
ImmutableList
.<
EnumTypeDefinition
>
of
(),
ImmutableList
.<
StructTypeDefinition
>
of
(),
ImmutableList
.<
HierarchicalTypeDefinition
<
TraitType
>>
of
(),
ImmutableList
.
of
(
typeDefinition
));
createType
(
typesDef
);
referenceable
=
new
Referenceable
(
typeName
);
referenceable
.
set
(
"name"
,
name
);
createInstance
(
referenceable
);
}
@Test
public
void
testSubmitEntityWithBadDateFormat
()
throws
Exception
{
public
void
testSubmitEntityWithBadDateFormat
()
throws
Exception
{
try
{
try
{
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment