Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
atlas
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
dataplatform
atlas
Commits
28991c52
Commit
28991c52
authored
May 13, 2016
by
Shwetha GS
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
ATLAS-645 FieldMapping.output() results in stack overflow when instances…
ATLAS-645 FieldMapping.output() results in stack overflow when instances reference each other (dkantor via shwethags)
parent
454feb47
Hide whitespace changes
Inline
Side-by-side
Showing
15 changed files
with
408 additions
and
70 deletions
+408
-70
release-log.txt
release-log.txt
+1
-0
ReferenceableInstance.java
...e/atlas/typesystem/persistence/ReferenceableInstance.java
+4
-1
StructInstance.java
...g/apache/atlas/typesystem/persistence/StructInstance.java
+2
-22
AbstractDataType.java
...a/org/apache/atlas/typesystem/types/AbstractDataType.java
+21
-1
AttributeInfo.java
...java/org/apache/atlas/typesystem/types/AttributeInfo.java
+27
-9
ClassType.java
...ain/java/org/apache/atlas/typesystem/types/ClassType.java
+3
-2
DataTypes.java
...ain/java/org/apache/atlas/typesystem/types/DataTypes.java
+2
-1
FieldMapping.java
.../java/org/apache/atlas/typesystem/types/FieldMapping.java
+55
-23
HierarchicalType.java
...a/org/apache/atlas/typesystem/types/HierarchicalType.java
+53
-2
IDataType.java
...ain/java/org/apache/atlas/typesystem/types/IDataType.java
+20
-1
Multiplicity.java
.../java/org/apache/atlas/typesystem/types/Multiplicity.java
+1
-2
StructType.java
...in/java/org/apache/atlas/typesystem/types/StructType.java
+61
-2
TraitType.java
...ain/java/org/apache/atlas/typesystem/types/TraitType.java
+3
-2
TypedStructHandler.java
...org/apache/atlas/typesystem/types/TypedStructHandler.java
+4
-2
FieldMappingTest.java
...a/org/apache/atlas/typesystem/types/FieldMappingTest.java
+151
-0
No files found.
release-log.txt
View file @
28991c52
...
@@ -20,6 +20,7 @@ ATLAS-409 Atlas will not import avro tables with schema read from a file (dosset
...
@@ -20,6 +20,7 @@ ATLAS-409 Atlas will not import avro tables with schema read from a file (dosset
ATLAS-379 Create sqoop and falcon metadata addons (venkatnrangan,bvellanki,sowmyaramesh via shwethags)
ATLAS-379 Create sqoop and falcon metadata addons (venkatnrangan,bvellanki,sowmyaramesh via shwethags)
ALL CHANGES:
ALL CHANGES:
ATLAS-645 FieldMapping.output() results in stack overflow when instances reference each other (dkantor via shwethags)
ATLAS-733 UI: "undefined" XHR request is made for every entity GET page request. (kevalbhatt18 via yhemanth)
ATLAS-733 UI: "undefined" XHR request is made for every entity GET page request. (kevalbhatt18 via yhemanth)
ATLAS-663,ATLAS-673 Install Setup: SOLR (tbeerbower via sumasai)
ATLAS-663,ATLAS-673 Install Setup: SOLR (tbeerbower via sumasai)
ATLAS-629 Kafka messages in ATLAS_HOOK might be lost in HA mode at the instant of failover. (yhemanth)
ATLAS-629 Kafka messages in ATLAS_HOOK might be lost in HA mode at the instant of failover. (yhemanth)
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/persistence/ReferenceableInstance.java
View file @
28991c52
...
@@ -20,7 +20,9 @@ package org.apache.atlas.typesystem.persistence;
...
@@ -20,7 +20,9 @@ package org.apache.atlas.typesystem.persistence;
import
com.google.common.collect.ImmutableList
;
import
com.google.common.collect.ImmutableList
;
import
com.google.common.collect.ImmutableMap
;
import
com.google.common.collect.ImmutableMap
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.typesystem.IReferenceableInstance
;
import
org.apache.atlas.typesystem.IStruct
;
import
org.apache.atlas.typesystem.IStruct
;
import
org.apache.atlas.typesystem.ITypedReferenceableInstance
;
import
org.apache.atlas.typesystem.ITypedReferenceableInstance
;
import
org.apache.atlas.typesystem.ITypedStruct
;
import
org.apache.atlas.typesystem.ITypedStruct
;
...
@@ -33,6 +35,7 @@ import java.math.BigDecimal;
...
@@ -33,6 +35,7 @@ import java.math.BigDecimal;
import
java.math.BigInteger
;
import
java.math.BigInteger
;
import
java.security.MessageDigest
;
import
java.security.MessageDigest
;
import
java.util.Date
;
import
java.util.Date
;
import
java.util.HashSet
;
/*
/*
* @todo handle names prefixed by traitName.
* @todo handle names prefixed by traitName.
...
@@ -89,7 +92,7 @@ public class ReferenceableInstance extends StructInstance implements ITypedRefer
...
@@ -89,7 +92,7 @@ public class ReferenceableInstance extends StructInstance implements ITypedRefer
StringBuilder
buf
=
new
StringBuilder
();
StringBuilder
buf
=
new
StringBuilder
();
String
prefix
=
""
;
String
prefix
=
""
;
fieldMapping
.
output
(
this
,
buf
,
prefix
);
fieldMapping
.
output
(
this
,
buf
,
prefix
,
new
HashSet
<
IReferenceableInstance
>()
);
return
buf
.
toString
();
return
buf
.
toString
();
}
catch
(
AtlasException
me
)
{
}
catch
(
AtlasException
me
)
{
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/persistence/StructInstance.java
View file @
28991c52
...
@@ -20,8 +20,8 @@ package org.apache.atlas.typesystem.persistence;
...
@@ -20,8 +20,8 @@ package org.apache.atlas.typesystem.persistence;
import
com.google.common.collect.ImmutableList
;
import
com.google.common.collect.ImmutableList
;
import
com.google.common.collect.ImmutableMap
;
import
com.google.common.collect.ImmutableMap
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.typesystem.IStruct
;
import
org.apache.atlas.typesystem.ITypedStruct
;
import
org.apache.atlas.typesystem.ITypedStruct
;
import
org.apache.atlas.typesystem.types.AttributeInfo
;
import
org.apache.atlas.typesystem.types.AttributeInfo
;
import
org.apache.atlas.typesystem.types.ClassType
;
import
org.apache.atlas.typesystem.types.ClassType
;
...
@@ -31,7 +31,6 @@ import org.apache.atlas.typesystem.types.EnumValue;
...
@@ -31,7 +31,6 @@ import org.apache.atlas.typesystem.types.EnumValue;
import
org.apache.atlas.typesystem.types.FieldMapping
;
import
org.apache.atlas.typesystem.types.FieldMapping
;
import
org.apache.atlas.typesystem.types.StructType
;
import
org.apache.atlas.typesystem.types.StructType
;
import
org.apache.atlas.typesystem.types.TypeSystem
;
import
org.apache.atlas.typesystem.types.TypeSystem
;
import
org.apache.atlas.typesystem.types.TypeUtils
;
import
org.apache.atlas.typesystem.types.ValueConversionException
;
import
org.apache.atlas.typesystem.types.ValueConversionException
;
import
org.apache.atlas.utils.MD5Utils
;
import
org.apache.atlas.utils.MD5Utils
;
...
@@ -724,32 +723,13 @@ public class StructInstance implements ITypedStruct {
...
@@ -724,32 +723,13 @@ public class StructInstance implements ITypedStruct {
strings
[
pos
]
=
val
;
strings
[
pos
]
=
val
;
}
}
public
void
output
(
IStruct
s
,
Appendable
buf
,
String
prefix
)
throws
AtlasException
{
TypeUtils
.
outputVal
(
"{"
,
buf
,
prefix
);
if
(
s
==
null
)
{
TypeUtils
.
outputVal
(
"<null>\n"
,
buf
,
""
);
return
;
}
TypeUtils
.
outputVal
(
"\n"
,
buf
,
""
);
String
fieldPrefix
=
prefix
+
"\t"
;
for
(
Map
.
Entry
<
String
,
AttributeInfo
>
e
:
fieldMapping
.
fields
.
entrySet
())
{
String
attrName
=
e
.
getKey
();
AttributeInfo
i
=
e
.
getValue
();
Object
aVal
=
s
.
get
(
attrName
);
TypeUtils
.
outputVal
(
attrName
+
" : "
,
buf
,
fieldPrefix
);
i
.
dataType
().
output
(
aVal
,
buf
,
""
);
TypeUtils
.
outputVal
(
"\n"
,
buf
,
""
);
}
TypeUtils
.
outputVal
(
"\n}\n"
,
buf
,
""
);
}
@Override
@Override
public
String
toString
()
{
public
String
toString
()
{
try
{
try
{
StringBuilder
buf
=
new
StringBuilder
();
StringBuilder
buf
=
new
StringBuilder
();
String
prefix
=
""
;
String
prefix
=
""
;
fieldMapping
.
output
(
this
,
buf
,
prefix
);
fieldMapping
.
output
(
this
,
buf
,
prefix
,
null
);
return
buf
.
toString
();
return
buf
.
toString
();
}
catch
(
AtlasException
me
)
{
}
catch
(
AtlasException
me
)
{
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/types/AbstractDataType.java
View file @
28991c52
...
@@ -22,7 +22,9 @@ import com.google.common.collect.ImmutableSortedMap;
...
@@ -22,7 +22,9 @@ import com.google.common.collect.ImmutableSortedMap;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.AtlasException
;
import
java.io.IOException
;
import
java.util.Map
;
import
java.util.Map
;
import
java.util.Set
;
abstract
class
AbstractDataType
<
T
>
implements
IDataType
<
T
>
{
abstract
class
AbstractDataType
<
T
>
implements
IDataType
<
T
>
{
...
@@ -44,7 +46,7 @@ abstract class AbstractDataType<T> implements IDataType<T> {
...
@@ -44,7 +46,7 @@ abstract class AbstractDataType<T> implements IDataType<T> {
}
}
@Override
@Override
public
void
output
(
T
val
,
Appendable
buf
,
String
prefix
)
throws
AtlasException
{
public
void
output
(
T
val
,
Appendable
buf
,
String
prefix
,
Set
<
T
>
inProcess
)
throws
AtlasException
{
if
(
val
instanceof
Map
)
{
if
(
val
instanceof
Map
)
{
ImmutableSortedMap
immutableSortedMap
=
ImmutableSortedMap
.
copyOf
((
Map
)
val
);
ImmutableSortedMap
immutableSortedMap
=
ImmutableSortedMap
.
copyOf
((
Map
)
val
);
TypeUtils
.
outputVal
(
val
==
null
?
"<null>"
:
immutableSortedMap
.
toString
(),
buf
,
prefix
);
TypeUtils
.
outputVal
(
val
==
null
?
"<null>"
:
immutableSortedMap
.
toString
(),
buf
,
prefix
);
...
@@ -53,6 +55,24 @@ abstract class AbstractDataType<T> implements IDataType<T> {
...
@@ -53,6 +55,24 @@ abstract class AbstractDataType<T> implements IDataType<T> {
}
}
}
}
@Override
public
void
output
(
Appendable
buf
,
Set
<
String
>
typesInProcess
)
throws
AtlasException
{
try
{
buf
.
append
(
toString
());
}
catch
(
IOException
e
)
{
throw
new
AtlasException
(
e
);
}
}
/* (non-Javadoc)
* @see java.lang.Object#toString()
*/
@Override
public
String
toString
()
{
return
"{name="
+
name
+
", description="
+
description
+
"}"
;
}
/**
/**
* Validate that current definition can be updated with the new definition
* Validate that current definition can be updated with the new definition
* @param newType
* @param newType
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/types/AttributeInfo.java
View file @
28991c52
...
@@ -22,7 +22,10 @@ import org.apache.atlas.AtlasException;
...
@@ -22,7 +22,10 @@ import org.apache.atlas.AtlasException;
import
org.codehaus.jettison.json.JSONException
;
import
org.codehaus.jettison.json.JSONException
;
import
org.codehaus.jettison.json.JSONObject
;
import
org.codehaus.jettison.json.JSONObject
;
import
java.io.IOException
;
import
java.util.HashSet
;
import
java.util.Map
;
import
java.util.Map
;
import
java.util.Set
;
public
class
AttributeInfo
{
public
class
AttributeInfo
{
public
final
String
name
;
public
final
String
name
;
...
@@ -60,15 +63,30 @@ public class AttributeInfo {
...
@@ -60,15 +63,30 @@ public class AttributeInfo {
@Override
@Override
public
String
toString
()
{
public
String
toString
()
{
return
"AttributeInfo{"
+
StringBuilder
buf
=
new
StringBuilder
();
"name='"
+
name
+
'\''
+
try
{
", dataType="
+
dataType
+
output
(
buf
,
new
HashSet
<
String
>());
", multiplicity="
+
multiplicity
+
}
catch
(
AtlasException
e
)
{
", isComposite="
+
isComposite
+
throw
new
RuntimeException
(
e
);
", isUnique="
+
isUnique
+
}
", isIndexable="
+
isIndexable
+
return
buf
.
toString
();
", reverseAttributeName='"
+
reverseAttributeName
+
'\''
+
}
'}'
;
public
void
output
(
Appendable
buf
,
Set
<
String
>
typesInProcess
)
throws
AtlasException
{
try
{
buf
.
append
(
"{name="
).
append
(
name
);
buf
.
append
(
", dataType="
);
dataType
.
output
(
buf
,
typesInProcess
);
buf
.
append
(
", multiplicity="
).
append
(
multiplicity
.
toString
());
buf
.
append
(
", isComposite="
).
append
(
Boolean
.
toString
(
isComposite
));
buf
.
append
(
", isUnique="
).
append
(
Boolean
.
toString
(
isUnique
));
buf
.
append
(
", isIndexable="
).
append
(
Boolean
.
toString
(
isIndexable
));
buf
.
append
(
", reverseAttributeName="
).
append
(
reverseAttributeName
);
buf
.
append
(
'}'
);
}
catch
(
IOException
e
)
{
throw
new
AtlasException
(
e
);
}
}
}
@Override
@Override
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/types/ClassType.java
View file @
28991c52
...
@@ -41,6 +41,7 @@ import java.security.MessageDigest;
...
@@ -41,6 +41,7 @@ import java.security.MessageDigest;
import
java.util.Date
;
import
java.util.Date
;
import
java.util.List
;
import
java.util.List
;
import
java.util.Map
;
import
java.util.Map
;
import
java.util.Set
;
public
class
ClassType
extends
HierarchicalType
<
ClassType
,
IReferenceableInstance
>
public
class
ClassType
extends
HierarchicalType
<
ClassType
,
IReferenceableInstance
>
implements
IConstructableType
<
IReferenceableInstance
,
ITypedReferenceableInstance
>
{
implements
IConstructableType
<
IReferenceableInstance
,
ITypedReferenceableInstance
>
{
...
@@ -207,8 +208,8 @@ public class ClassType extends HierarchicalType<ClassType, IReferenceableInstanc
...
@@ -207,8 +208,8 @@ public class ClassType extends HierarchicalType<ClassType, IReferenceableInstanc
}
}
@Override
@Override
public
void
output
(
IReferenceableInstance
s
,
Appendable
buf
,
String
prefix
)
throws
AtlasException
{
public
void
output
(
IReferenceableInstance
s
,
Appendable
buf
,
String
prefix
,
Set
<
IReferenceableInstance
>
inProcess
)
throws
AtlasException
{
fieldMapping
.
output
(
s
,
buf
,
prefix
);
fieldMapping
.
output
(
s
,
buf
,
prefix
,
inProcess
);
}
}
@Override
@Override
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java
View file @
28991c52
...
@@ -38,6 +38,7 @@ import java.util.Date;
...
@@ -38,6 +38,7 @@ import java.util.Date;
import
java.util.Iterator
;
import
java.util.Iterator
;
import
java.util.List
;
import
java.util.List
;
import
java.util.Map
;
import
java.util.Map
;
import
java.util.Set
;
public
class
DataTypes
{
public
class
DataTypes
{
...
@@ -425,7 +426,7 @@ public class DataTypes {
...
@@ -425,7 +426,7 @@ public class DataTypes {
}
}
@Override
@Override
public
void
output
(
Date
val
,
Appendable
buf
,
String
prefix
)
throws
AtlasException
{
public
void
output
(
Date
val
,
Appendable
buf
,
String
prefix
,
Set
<
Date
>
inProcess
)
throws
AtlasException
{
TypeUtils
.
outputVal
(
val
==
null
?
"<null>"
:
TypeSystem
.
getInstance
().
getDateFormat
().
format
(
val
),
buf
,
TypeUtils
.
outputVal
(
val
==
null
?
"<null>"
:
TypeSystem
.
getInstance
().
getDateFormat
().
format
(
val
),
buf
,
prefix
);
prefix
);
}
}
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/types/FieldMapping.java
View file @
28991c52
...
@@ -23,7 +23,9 @@ import org.apache.atlas.typesystem.IReferenceableInstance;
...
@@ -23,7 +23,9 @@ import org.apache.atlas.typesystem.IReferenceableInstance;
import
org.apache.atlas.typesystem.IStruct
;
import
org.apache.atlas.typesystem.IStruct
;
import
org.apache.atlas.typesystem.persistence.Id
;
import
org.apache.atlas.typesystem.persistence.Id
;
import
java.util.HashSet
;
import
java.util.Map
;
import
java.util.Map
;
import
java.util.Set
;
public
class
FieldMapping
{
public
class
FieldMapping
{
...
@@ -70,7 +72,7 @@ public class FieldMapping {
...
@@ -70,7 +72,7 @@ public class FieldMapping {
this
.
numReferenceables
=
numReferenceables
;
this
.
numReferenceables
=
numReferenceables
;
}
}
protected
void
outputFields
(
IStruct
s
,
Appendable
buf
,
String
fieldPrefix
)
throws
AtlasException
{
protected
void
outputFields
(
IStruct
s
,
Appendable
buf
,
String
fieldPrefix
,
Set
<?
extends
IStruct
>
inProcess
)
throws
AtlasException
{
for
(
Map
.
Entry
<
String
,
AttributeInfo
>
e
:
fields
.
entrySet
())
{
for
(
Map
.
Entry
<
String
,
AttributeInfo
>
e
:
fields
.
entrySet
())
{
String
attrName
=
e
.
getKey
();
String
attrName
=
e
.
getKey
();
AttributeInfo
i
=
e
.
getValue
();
AttributeInfo
i
=
e
.
getValue
();
...
@@ -79,52 +81,82 @@ public class FieldMapping {
...
@@ -79,52 +81,82 @@ public class FieldMapping {
if
(
aVal
!=
null
&&
aVal
instanceof
Id
)
{
if
(
aVal
!=
null
&&
aVal
instanceof
Id
)
{
TypeUtils
.
outputVal
(
aVal
.
toString
(),
buf
,
""
);
TypeUtils
.
outputVal
(
aVal
.
toString
(),
buf
,
""
);
}
else
{
}
else
{
i
.
dataType
().
output
(
aVal
,
buf
,
fieldPrefix
);
i
.
dataType
().
output
(
aVal
,
buf
,
fieldPrefix
,
inProcess
);
}
}
TypeUtils
.
outputVal
(
"\n"
,
buf
,
""
);
TypeUtils
.
outputVal
(
"\n"
,
buf
,
""
);
}
}
}
}
public
void
output
(
IStruct
s
,
Appendable
buf
,
String
prefix
)
throws
AtlasException
{
public
void
output
(
IStruct
s
,
Appendable
buf
,
String
prefix
,
Set
<
IStruct
>
inProcess
)
throws
AtlasException
{
if
(
s
==
null
)
{
if
(
s
==
null
)
{
TypeUtils
.
outputVal
(
"<null>\n"
,
buf
,
""
);
TypeUtils
.
outputVal
(
"<null>\n"
,
buf
,
""
);
return
;
return
;
}
}
TypeUtils
.
outputVal
(
"{"
,
buf
,
prefix
);
TypeUtils
.
outputVal
(
"\n"
,
buf
,
""
);
if
(
inProcess
==
null
)
{
String
fieldPrefix
=
prefix
+
"\t"
;
inProcess
=
new
HashSet
<>();
}
else
if
(
inProcess
.
contains
(
s
))
{
// Avoid infinite recursion when structs reference each other.
return
;
}
inProcess
.
add
(
s
);
try
{
TypeUtils
.
outputVal
(
"{"
,
buf
,
prefix
);
TypeUtils
.
outputVal
(
"\n"
,
buf
,
""
);
String
fieldPrefix
=
prefix
+
"\t"
;
outputFields
(
s
,
buf
,
fieldPrefix
);
outputFields
(
s
,
buf
,
fieldPrefix
,
inProcess
);
TypeUtils
.
outputVal
(
"}"
,
buf
,
prefix
);
TypeUtils
.
outputVal
(
"}"
,
buf
,
prefix
);
}
finally
{
inProcess
.
remove
(
s
);
}
}
}
public
void
output
(
IReferenceableInstance
s
,
Appendable
buf
,
String
prefix
)
throws
AtlasException
{
public
void
output
(
IReferenceableInstance
s
,
Appendable
buf
,
String
prefix
,
Set
<
IReferenceableInstance
>
inProcess
)
throws
AtlasException
{
if
(
s
==
null
)
{
if
(
s
==
null
)
{
TypeUtils
.
outputVal
(
"<null>\n"
,
buf
,
""
);
TypeUtils
.
outputVal
(
"<null>\n"
,
buf
,
""
);
return
;
return
;
}
}
TypeUtils
.
outputVal
(
"{"
,
buf
,
prefix
);
TypeUtils
.
outputVal
(
"\n"
,
buf
,
""
);
if
(
inProcess
==
null
)
{
String
fieldPrefix
=
prefix
+
"\t"
;
inProcess
=
new
HashSet
<>();
}
else
if
(
inProcess
.
contains
(
s
))
{
// Avoid infinite recursion when structs reference each other.
return
;
}
inProcess
.
add
(
s
);
TypeUtils
.
outputVal
(
"id : "
,
buf
,
fieldPrefix
);
try
{
TypeUtils
.
outputVal
(
s
.
getId
().
toString
(),
buf
,
""
);
TypeUtils
.
outputVal
(
"{"
,
buf
,
prefix
);
TypeUtils
.
outputVal
(
"\n"
,
buf
,
""
);
outputFields
(
s
,
buf
,
fieldPrefix
);
TypeUtils
.
outputVal
(
"\n"
,
buf
,
""
);
String
fieldPrefix
=
prefix
+
"\t"
;
TypeSystem
ts
=
TypeSystem
.
getInstance
();
TypeUtils
.
outputVal
(
"id : "
,
buf
,
fieldPrefix
);
TypeUtils
.
outputVal
(
s
.
getId
().
toString
(),
buf
,
""
);
TypeUtils
.
outputVal
(
"\n"
,
buf
,
""
);
for
(
String
sT
:
s
.
getTraits
())
{
outputFields
(
s
,
buf
,
fieldPrefix
,
inProcess
);
TraitType
tt
=
ts
.
getDataType
(
TraitType
.
class
,
sT
);
TypeUtils
.
outputVal
(
sT
+
" : "
,
buf
,
fieldPrefix
);
TypeSystem
ts
=
TypeSystem
.
getInstance
();
tt
.
output
(
s
.
getTrait
(
sT
),
buf
,
fieldPrefix
);
}
TypeUtils
.
outputVal
(
"}"
,
buf
,
prefix
);
for
(
String
sT
:
s
.
getTraits
())
{
TraitType
tt
=
ts
.
getDataType
(
TraitType
.
class
,
sT
);
TypeUtils
.
outputVal
(
sT
+
" : "
,
buf
,
fieldPrefix
);
tt
.
output
(
s
.
getTrait
(
sT
),
buf
,
fieldPrefix
,
null
);
}
TypeUtils
.
outputVal
(
"}"
,
buf
,
prefix
);
}
finally
{
inProcess
.
remove
(
s
);
}
}
}
}
}
typesystem/src/main/java/org/apache/atlas/typesystem/types/HierarchicalType.java
View file @
28991c52
...
@@ -21,12 +21,14 @@ package org.apache.atlas.typesystem.types;
...
@@ -21,12 +21,14 @@ package org.apache.atlas.typesystem.types;
import
com.google.common.collect.ImmutableList
;
import
com.google.common.collect.ImmutableList
;
import
com.google.common.collect.ImmutableMap
;
import
com.google.common.collect.ImmutableMap
;
import
com.google.common.collect.ImmutableSet
;
import
com.google.common.collect.ImmutableSet
;
import
com.google.common.collect.UnmodifiableIterator
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.typesystem.IStruct
;
import
org.apache.atlas.typesystem.IStruct
;
import
org.apache.atlas.typesystem.persistence.DownCastStructInstance
;
import
org.apache.atlas.typesystem.persistence.DownCastStructInstance
;
import
org.apache.atlas.typesystem.types.TypeUtils.Pair
;
import
org.apache.atlas.typesystem.types.TypeUtils.Pair
;
import
java.io.IOException
;
import
java.util.ArrayList
;
import
java.util.ArrayList
;
import
java.util.HashMap
;
import
java.util.HashMap
;
import
java.util.HashSet
;
import
java.util.HashSet
;
...
@@ -367,9 +369,58 @@ public abstract class HierarchicalType<ST extends HierarchicalType, T> extends A
...
@@ -367,9 +369,58 @@ public abstract class HierarchicalType<ST extends HierarchicalType, T> extends A
@Override
@Override
public
String
toString
()
{
public
String
toString
()
{
StringBuilder
buf
=
new
StringBuilder
();
try
{
output
(
buf
,
new
HashSet
<
String
>());
}
catch
(
AtlasException
e
)
{
throw
new
RuntimeException
(
e
);
}
return
buf
.
toString
();
}
@Override
public
void
output
(
Appendable
buf
,
Set
<
String
>
typesInProcess
)
throws
AtlasException
{
if
(
typesInProcess
==
null
)
{
typesInProcess
=
new
HashSet
<>();
}
else
if
(
typesInProcess
.
contains
(
name
))
{
// Avoid infinite recursion on bi-directional reference attributes.
try
{
buf
.
append
(
name
);
}
catch
(
IOException
e
)
{
throw
new
AtlasException
(
e
);
}
return
;
}
return
"[name="
+
name
+
", description="
+
description
+
typesInProcess
.
add
(
name
);
", superTypes="
+
superTypes
+
", immediateAttrs="
+
immediateAttrs
+
"]"
;
try
{
buf
.
append
(
getClass
().
getSimpleName
()).
append
(
'{'
);
buf
.
append
(
"name="
).
append
(
name
);
buf
.
append
(
", description="
).
append
(
description
);
buf
.
append
(
", superTypes="
).
append
(
superTypes
.
toString
());
buf
.
append
(
", immediateAttrs=["
);
UnmodifiableIterator
<
AttributeInfo
>
it
=
immediateAttrs
.
iterator
();
while
(
it
.
hasNext
())
{
AttributeInfo
attrInfo
=
it
.
next
();
attrInfo
.
output
(
buf
,
typesInProcess
);
if
(
it
.
hasNext
())
{
buf
.
append
(
", "
);
}
else
{
buf
.
append
(
']'
);
}
}
buf
.
append
(
"}"
);
}
catch
(
IOException
e
)
{
throw
new
AtlasException
(
e
);
}
finally
{
typesInProcess
.
remove
(
name
);
}
}
}
public
Set
<
String
>
getAllSuperTypeNames
()
{
public
Set
<
String
>
getAllSuperTypeNames
()
{
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/types/IDataType.java
View file @
28991c52
...
@@ -21,6 +21,7 @@ package org.apache.atlas.typesystem.types;
...
@@ -21,6 +21,7 @@ package org.apache.atlas.typesystem.types;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.AtlasException
;
import
java.security.MessageDigest
;
import
java.security.MessageDigest
;
import
java.util.Set
;
public
interface
IDataType
<
T
>
{
public
interface
IDataType
<
T
>
{
String
getName
();
String
getName
();
...
@@ -29,7 +30,25 @@ public interface IDataType<T> {
...
@@ -29,7 +30,25 @@ public interface IDataType<T> {
DataTypes
.
TypeCategory
getTypeCategory
();
DataTypes
.
TypeCategory
getTypeCategory
();
void
output
(
T
val
,
Appendable
buf
,
String
prefix
)
throws
AtlasException
;
/**
* Output a string representation of a value instance of this type.
*
* @param val
* @param buf
* @param prefix
* @param inProcess
* @throws AtlasException
*/
void
output
(
T
val
,
Appendable
buf
,
String
prefix
,
Set
<
T
>
inProcess
)
throws
AtlasException
;
/**
* Output a string representation of this type.
*
* @param buf
* @param typesInProcess
* @throws AtlasException
*/
void
output
(
Appendable
buf
,
Set
<
String
>
typesInProcess
)
throws
AtlasException
;
void
validateUpdate
(
IDataType
newType
)
throws
TypeUpdateException
;
void
validateUpdate
(
IDataType
newType
)
throws
TypeUpdateException
;
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/types/Multiplicity.java
View file @
28991c52
...
@@ -79,8 +79,7 @@ public final class Multiplicity {
...
@@ -79,8 +79,7 @@ public final class Multiplicity {
@Override
@Override
public
String
toString
()
{
public
String
toString
()
{
return
"Multiplicity{"
+
return
"{lower="
+
lower
+
"lower="
+
lower
+
", upper="
+
upper
+
", upper="
+
upper
+
", isUnique="
+
isUnique
+
", isUnique="
+
isUnique
+
'}'
;
'}'
;
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/types/StructType.java
View file @
28991c52
...
@@ -18,12 +18,16 @@
...
@@ -18,12 +18,16 @@
package
org
.
apache
.
atlas
.
typesystem
.
types
;
package
org
.
apache
.
atlas
.
typesystem
.
types
;
import
java.io.IOException
;
import
java.nio.charset.Charset
;
import
java.nio.charset.Charset
;
import
java.security.MessageDigest
;
import
java.security.MessageDigest
;
import
java.util.HashMap
;
import
java.util.HashMap
;
import
java.util.HashSet
;
import
java.util.Iterator
;
import
java.util.LinkedHashMap
;
import
java.util.LinkedHashMap
;
import
java.util.List
;
import
java.util.List
;
import
java.util.Map
;
import
java.util.Map
;
import
java.util.Set
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.typesystem.IStruct
;
import
org.apache.atlas.typesystem.IStruct
;
...
@@ -182,8 +186,63 @@ public class StructType extends AbstractDataType<IStruct> implements IConstructa
...
@@ -182,8 +186,63 @@ public class StructType extends AbstractDataType<IStruct> implements IConstructa
}
}
@Override
@Override
public
void
output
(
IStruct
s
,
Appendable
buf
,
String
prefix
)
throws
AtlasException
{
public
void
output
(
IStruct
s
,
Appendable
buf
,
String
prefix
,
Set
<
IStruct
>
inProcess
)
throws
AtlasException
{
handler
.
output
(
s
,
buf
,
prefix
);
handler
.
output
(
s
,
buf
,
prefix
,
inProcess
);
}
@Override
public
String
toString
()
{
StringBuilder
buf
=
new
StringBuilder
();
try
{
output
(
buf
,
new
HashSet
<
String
>());
}
catch
(
AtlasException
e
)
{
throw
new
RuntimeException
(
e
);
}
return
buf
.
toString
();
}
@Override
public
void
output
(
Appendable
buf
,
Set
<
String
>
typesInProcess
)
throws
AtlasException
{
if
(
typesInProcess
==
null
)
{
typesInProcess
=
new
HashSet
<>();
}
else
if
(
typesInProcess
.
contains
(
name
))
{
// Avoid infinite recursion on bi-directional reference attributes.
try
{
buf
.
append
(
name
);
}
catch
(
IOException
e
)
{
throw
new
AtlasException
(
e
);
}
return
;
}
typesInProcess
.
add
(
name
);
try
{
buf
.
append
(
getClass
().
getSimpleName
());
buf
.
append
(
"{name="
).
append
(
name
);
buf
.
append
(
", description="
).
append
(
description
);
buf
.
append
(
", fieldMapping.fields=["
);
Iterator
<
AttributeInfo
>
it
=
fieldMapping
.
fields
.
values
().
iterator
();
while
(
it
.
hasNext
())
{
AttributeInfo
attrInfo
=
it
.
next
();
attrInfo
.
output
(
buf
,
typesInProcess
);
if
(
it
.
hasNext
())
{
buf
.
append
(
", "
);
}
else
{
buf
.
append
(
']'
);
}
}
buf
.
append
(
"}"
);
}
catch
(
IOException
e
)
{
throw
new
AtlasException
(
e
);
}
finally
{
typesInProcess
.
remove
(
name
);
}
}
}
@Override
@Override
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/types/TraitType.java
View file @
28991c52
...
@@ -28,6 +28,7 @@ import java.nio.charset.Charset;
...
@@ -28,6 +28,7 @@ import java.nio.charset.Charset;
import
java.security.MessageDigest
;
import
java.security.MessageDigest
;
import
java.util.List
;
import
java.util.List
;
import
java.util.Map
;
import
java.util.Map
;
import
java.util.Set
;
public
class
TraitType
extends
HierarchicalType
<
TraitType
,
IStruct
>
public
class
TraitType
extends
HierarchicalType
<
TraitType
,
IStruct
>
implements
IConstructableType
<
IStruct
,
ITypedStruct
>
{
implements
IConstructableType
<
IStruct
,
ITypedStruct
>
{
...
@@ -63,8 +64,8 @@ public class TraitType extends HierarchicalType<TraitType, IStruct>
...
@@ -63,8 +64,8 @@ public class TraitType extends HierarchicalType<TraitType, IStruct>
}
}
@Override
@Override
public
void
output
(
IStruct
s
,
Appendable
buf
,
String
prefix
)
throws
AtlasException
{
public
void
output
(
IStruct
s
,
Appendable
buf
,
String
prefix
,
Set
<
IStruct
>
inProcess
)
throws
AtlasException
{
handler
.
output
(
s
,
buf
,
prefix
);
handler
.
output
(
s
,
buf
,
prefix
,
inProcess
);
}
}
@Override
@Override
...
...
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypedStructHandler.java
View file @
28991c52
...
@@ -20,6 +20,7 @@ package org.apache.atlas.typesystem.types;
...
@@ -20,6 +20,7 @@ package org.apache.atlas.typesystem.types;
import
com.google.common.collect.ImmutableList
;
import
com.google.common.collect.ImmutableList
;
import
com.google.common.collect.ImmutableMap
;
import
com.google.common.collect.ImmutableMap
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.AtlasException
;
import
org.apache.atlas.typesystem.IStruct
;
import
org.apache.atlas.typesystem.IStruct
;
import
org.apache.atlas.typesystem.ITypedStruct
;
import
org.apache.atlas.typesystem.ITypedStruct
;
...
@@ -32,6 +33,7 @@ import java.math.BigDecimal;
...
@@ -32,6 +33,7 @@ import java.math.BigDecimal;
import
java.math.BigInteger
;
import
java.math.BigInteger
;
import
java.util.Date
;
import
java.util.Date
;
import
java.util.Map
;
import
java.util.Map
;
import
java.util.Set
;
public
class
TypedStructHandler
{
public
class
TypedStructHandler
{
...
@@ -104,8 +106,8 @@ public class TypedStructHandler {
...
@@ -104,8 +106,8 @@ public class TypedStructHandler {
fieldMapping
.
numReferenceables
==
0
?
null
:
new
Id
[
fieldMapping
.
numReferenceables
]);
fieldMapping
.
numReferenceables
==
0
?
null
:
new
Id
[
fieldMapping
.
numReferenceables
]);
}
}
public
void
output
(
IStruct
s
,
Appendable
buf
,
String
prefix
)
throws
AtlasException
{
public
void
output
(
IStruct
s
,
Appendable
buf
,
String
prefix
,
Set
<
IStruct
>
inProcess
)
throws
AtlasException
{
fieldMapping
.
output
(
s
,
buf
,
prefix
);
fieldMapping
.
output
(
s
,
buf
,
prefix
,
inProcess
);
}
}
}
}
typesystem/src/test/java/org/apache/atlas/typesystem/types/FieldMappingTest.java
0 → 100644
View file @
28991c52
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org
.
apache
.
atlas
.
typesystem
.
types
;
import
java.util.HashSet
;
import
org.apache.atlas.typesystem.IReferenceableInstance
;
import
org.apache.atlas.typesystem.ITypedReferenceableInstance
;
import
org.apache.atlas.typesystem.ITypedStruct
;
import
org.apache.atlas.typesystem.TypesDef
;
import
org.apache.atlas.typesystem.types.AttributeDefinition
;
import
org.apache.atlas.typesystem.types.ClassType
;
import
org.apache.atlas.typesystem.types.EnumTypeDefinition
;
import
org.apache.atlas.typesystem.types.HierarchicalTypeDefinition
;
import
org.apache.atlas.typesystem.types.Multiplicity
;
import
org.apache.atlas.typesystem.types.StructTypeDefinition
;
import
org.apache.atlas.typesystem.types.TraitType
;
import
org.apache.atlas.typesystem.types.TypeSystem
;
import
org.apache.atlas.typesystem.types.utils.TypesUtil
;
import
org.testng.Assert
;
import
org.testng.annotations.BeforeTest
;
import
org.testng.annotations.Test
;
import
com.google.common.collect.ImmutableList
;
import
com.google.common.collect.ImmutableSet
;
/**
* Unit test for {@link FieldMapping}
*
*/
public
class
FieldMappingTest
{
@BeforeTest
public
void
beforeTest
()
throws
Exception
{
TypeSystem
typeSystem
=
TypeSystem
.
getInstance
();
typeSystem
.
reset
();
}
@Test
public
void
testOutputReferenceableInstance
()
throws
Exception
{
// ATLAS-645: verify that FieldMapping.output(IReferenceableInstance)
// does not infinitely recurse when ITypedReferenceableInstance's reference each other.
HierarchicalTypeDefinition
<
ClassType
>
valueDef
=
TypesUtil
.
createClassTypeDef
(
"Value"
,
ImmutableSet
.<
String
>
of
(),
new
AttributeDefinition
(
"owner"
,
"Owner"
,
Multiplicity
.
OPTIONAL
,
false
,
null
));
// Define class type with reference, where the value is a class reference to Value.
HierarchicalTypeDefinition
<
ClassType
>
ownerDef
=
TypesUtil
.
createClassTypeDef
(
"Owner"
,
ImmutableSet
.<
String
>
of
(),
new
AttributeDefinition
(
"value"
,
"Value"
,
Multiplicity
.
OPTIONAL
,
false
,
null
));
TypesDef
typesDef
=
TypesUtil
.
getTypesDef
(
ImmutableList
.<
EnumTypeDefinition
>
of
(),
ImmutableList
.<
StructTypeDefinition
>
of
(),
ImmutableList
.<
HierarchicalTypeDefinition
<
TraitType
>>
of
(),
ImmutableList
.
of
(
ownerDef
,
valueDef
));
TypeSystem
typeSystem
=
TypeSystem
.
getInstance
();
typeSystem
.
defineTypes
(
typesDef
);
ClassType
ownerType
=
typeSystem
.
getDataType
(
ClassType
.
class
,
"Owner"
);
// Prior to fix for ATLAS-645, this call would throw a StackOverflowError
try
{
ownerType
.
toString
();
}
catch
(
StackOverflowError
e
)
{
Assert
.
fail
(
"Infinite recursion in ClassType.toString() caused StackOverflowError"
);
}
ClassType
valueType
=
typeSystem
.
getDataType
(
ClassType
.
class
,
"Value"
);
// Create instances of Owner and Value that reference each other.
ITypedReferenceableInstance
ownerInstance
=
ownerType
.
createInstance
();
ITypedReferenceableInstance
valueInstance
=
valueType
.
createInstance
();
// Set Owner.value reference to Value instance.
ownerInstance
.
set
(
"value"
,
valueInstance
);
// Set Value.owner reference on Owner instance.
valueInstance
.
set
(
"owner"
,
ownerInstance
);
// Prior to fix for ATLAS-645, this call would throw a StackOverflowError
try
{
ownerInstance
.
fieldMapping
().
output
(
ownerInstance
,
new
StringBuilder
(),
""
,
new
HashSet
<
IReferenceableInstance
>());
}
catch
(
StackOverflowError
e
)
{
Assert
.
fail
(
"Infinite recursion in FieldMapping.output() caused StackOverflowError"
);
}
}
@Test
public
void
testOutputStruct
()
throws
Exception
{
// ATLAS-645: verify that FieldMapping.output(IStruct) does not infinitely recurse
// when an IStruct and ITypedReferenceableInstance reference each other.
HierarchicalTypeDefinition
<
ClassType
>
valueDef
=
TypesUtil
.
createClassTypeDef
(
"Value"
,
ImmutableSet
.<
String
>
of
(),
new
AttributeDefinition
(
"owner"
,
"Owner"
,
Multiplicity
.
OPTIONAL
,
false
,
null
));
// Define struct type with reference, where the value is a class reference to Value.
StructTypeDefinition
ownerDef
=
TypesUtil
.
createStructTypeDef
(
"Owner"
,
new
AttributeDefinition
(
"value"
,
"Value"
,
Multiplicity
.
OPTIONAL
,
false
,
null
));
TypesDef
typesDef
=
TypesUtil
.
getTypesDef
(
ImmutableList
.<
EnumTypeDefinition
>
of
(),
ImmutableList
.
of
(
ownerDef
),
ImmutableList
.<
HierarchicalTypeDefinition
<
TraitType
>>
of
(),
ImmutableList
.
of
(
valueDef
));
TypeSystem
typeSystem
=
TypeSystem
.
getInstance
();
typeSystem
.
reset
();
typeSystem
.
defineTypes
(
typesDef
);
StructType
ownerType
=
typeSystem
.
getDataType
(
StructType
.
class
,
"Owner"
);
ClassType
valueType
=
typeSystem
.
getDataType
(
ClassType
.
class
,
"Value"
);
// Prior to fix for ATLAS-645, this call would throw a StackOverflowError
try
{
ownerType
.
toString
();
}
catch
(
StackOverflowError
e
)
{
Assert
.
fail
(
"Infinite recursion in StructType.toString() caused StackOverflowError"
);
}
// Create instances of Owner and Value that reference each other.
ITypedStruct
ownerInstance
=
ownerType
.
createInstance
();
ITypedReferenceableInstance
valueInstance
=
valueType
.
createInstance
();
// Set Owner.value reference to Value instance.
ownerInstance
.
set
(
"value"
,
valueInstance
);
// Set Value.owner reference on Owner instance.
valueInstance
.
set
(
"owner"
,
ownerInstance
);
// Prior to fix for ATLAS-645, this call would throw a StackOverflowError
try
{
ownerInstance
.
fieldMapping
().
output
(
ownerInstance
,
new
StringBuilder
(),
""
,
null
);
}
catch
(
StackOverflowError
e
)
{
Assert
.
fail
(
"Infinite recursion in FieldMapping.output() caused StackOverflowError"
);
}
}
}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment