Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
atlas
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
dataplatform
atlas
Commits
6fddccd6
Commit
6fddccd6
authored
8 years ago
by
Suma Shivaprasad
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
ATLAS-1121 NPE while submitting topology in StormHook (ayubkhan via sumasai)
parent
ab95c1a7
master
No related merge requests found
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
66 additions
and
59 deletions
+66
-59
StormTopologyUtil.java
...n/java/org/apache/atlas/storm/hook/StormTopologyUtil.java
+65
-59
release-log.txt
release-log.txt
+1
-0
No files found.
addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormTopologyUtil.java
View file @
6fddccd6
...
@@ -36,6 +36,7 @@ import java.util.Set;
...
@@ -36,6 +36,7 @@ import java.util.Set;
* A storm topology utility class.
* A storm topology utility class.
*/
*/
public
final
class
StormTopologyUtil
{
public
final
class
StormTopologyUtil
{
public
static
final
Logger
LOG
=
org
.
slf4j
.
LoggerFactory
.
getLogger
(
StormTopologyUtil
.
class
);
private
StormTopologyUtil
()
{
private
StormTopologyUtil
()
{
}
}
...
@@ -136,74 +137,79 @@ public final class StormTopologyUtil {
...
@@ -136,74 +137,79 @@ public final class StormTopologyUtil {
Map
<
String
,
String
>
output
=
new
HashMap
<>();
Map
<
String
,
String
>
output
=
new
HashMap
<>();
if
(
objectsToSkip
.
add
(
instance
))
{
try
{
Class
clazz
=
instance
.
getClass
();
if
(
objectsToSkip
.
add
(
instance
))
{
for
(
Class
<?>
c
=
clazz
;
c
!=
null
;
c
=
c
.
getSuperclass
())
{
Class
clazz
=
instance
.
getClass
();
Field
[]
fields
=
c
.
getDeclaredFields
();
for
(
Class
<?>
c
=
clazz
;
c
!=
null
;
c
=
c
.
getSuperclass
())
{
for
(
Field
field
:
fields
)
{
Field
[]
fields
=
c
.
getDeclaredFields
();
if
(
java
.
lang
.
reflect
.
Modifier
.
isStatic
(
field
.
getModifiers
()))
{
for
(
Field
field
:
fields
)
{
continue
;
if
(
java
.
lang
.
reflect
.
Modifier
.
isStatic
(
field
.
getModifiers
()))
{
}
continue
;
}
String
key
;
if
(
prependClassName
)
{
key
=
String
.
format
(
"%s.%s"
,
clazz
.
getSimpleName
(),
field
.
getName
());
}
else
{
key
=
field
.
getName
();
}
boolean
accessible
=
field
.
isAccessible
();
String
key
;
if
(!
accessible
)
{
if
(
prependClassName
)
{
field
.
setAccessible
(
true
);
key
=
String
.
format
(
"%s.%s"
,
clazz
.
getSimpleName
(),
field
.
getName
());
}
}
else
{
Object
fieldVal
=
field
.
get
(
instance
);
key
=
field
.
getName
();
if
(
fieldVal
==
null
)
{
continue
;
}
else
if
(
fieldVal
.
getClass
().
isPrimitive
()
||
isWrapperType
(
fieldVal
.
getClass
()))
{
if
(
toString
(
fieldVal
,
false
).
isEmpty
())
continue
;
output
.
put
(
key
,
toString
(
fieldVal
,
false
));
}
else
if
(
isMapType
(
fieldVal
.
getClass
()))
{
//TODO: check if it makes more sense to just stick to json
// like structure instead of a flatten output.
Map
map
=
(
Map
)
fieldVal
;
for
(
Object
entry
:
map
.
entrySet
())
{
Object
mapKey
=
((
Map
.
Entry
)
entry
).
getKey
();
Object
mapVal
=
((
Map
.
Entry
)
entry
).
getValue
();
String
keyStr
=
getString
(
mapKey
,
false
,
objectsToSkip
);
String
valStr
=
getString
(
mapVal
,
false
,
objectsToSkip
);
if
((
valStr
==
null
)
||
(
valStr
.
isEmpty
()))
{
continue
;
}
else
{
output
.
put
(
String
.
format
(
"%s.%s"
,
key
,
keyStr
),
valStr
);
}
}
}
}
else
if
(
isCollectionType
(
fieldVal
.
getClass
()))
{
//TODO check if it makes more sense to just stick to
boolean
accessible
=
field
.
isAccessible
();
// json like structure instead of a flatten output.
if
(!
accessible
)
{
Collection
collection
=
(
Collection
)
fieldVal
;
field
.
setAccessible
(
true
);
if
(
collection
.
size
()
==
0
)
continue
;
String
outStr
=
""
;
for
(
Object
o
:
collection
)
{
outStr
+=
getString
(
o
,
false
,
objectsToSkip
)
+
","
;
}
}
if
(
outStr
.
length
()
>
0
)
{
Object
fieldVal
=
field
.
get
(
instance
);
outStr
=
outStr
.
substring
(
0
,
outStr
.
length
()
-
1
);
if
(
fieldVal
==
null
)
{
continue
;
}
else
if
(
fieldVal
.
getClass
().
isPrimitive
()
||
isWrapperType
(
fieldVal
.
getClass
()))
{
if
(
toString
(
fieldVal
,
false
).
isEmpty
())
continue
;
output
.
put
(
key
,
toString
(
fieldVal
,
false
));
}
else
if
(
isMapType
(
fieldVal
.
getClass
()))
{
//TODO: check if it makes more sense to just stick to json
// like structure instead of a flatten output.
Map
map
=
(
Map
)
fieldVal
;
for
(
Object
entry
:
map
.
entrySet
())
{
Object
mapKey
=
((
Map
.
Entry
)
entry
).
getKey
();
Object
mapVal
=
((
Map
.
Entry
)
entry
).
getValue
();
String
keyStr
=
getString
(
mapKey
,
false
,
objectsToSkip
);
String
valStr
=
getString
(
mapVal
,
false
,
objectsToSkip
);
if
((
valStr
==
null
)
||
(
valStr
.
isEmpty
()))
{
continue
;
}
else
{
output
.
put
(
String
.
format
(
"%s.%s"
,
key
,
keyStr
),
valStr
);
}
}
}
else
if
(
isCollectionType
(
fieldVal
.
getClass
()))
{
//TODO check if it makes more sense to just stick to
// json like structure instead of a flatten output.
Collection
collection
=
(
Collection
)
fieldVal
;
if
(
collection
.
size
()
==
0
)
continue
;
String
outStr
=
""
;
for
(
Object
o
:
collection
)
{
outStr
+=
getString
(
o
,
false
,
objectsToSkip
)
+
","
;
}
if
(
outStr
.
length
()
>
0
)
{
outStr
=
outStr
.
substring
(
0
,
outStr
.
length
()
-
1
);
}
output
.
put
(
key
,
String
.
format
(
"%s"
,
outStr
));
}
else
{
Map
<
String
,
String
>
nestedFieldValues
=
getFieldValues
(
fieldVal
,
false
,
objectsToSkip
);
for
(
Map
.
Entry
<
String
,
String
>
entry
:
nestedFieldValues
.
entrySet
())
{
output
.
put
(
String
.
format
(
"%s.%s"
,
key
,
entry
.
getKey
()),
entry
.
getValue
());
}
}
}
output
.
put
(
key
,
String
.
format
(
"%s"
,
outStr
));
if
(!
accessible
)
{
}
else
{
field
.
setAccessible
(
false
);
Map
<
String
,
String
>
nestedFieldValues
=
getFieldValues
(
fieldVal
,
false
,
objectsToSkip
);
for
(
Map
.
Entry
<
String
,
String
>
entry
:
nestedFieldValues
.
entrySet
())
{
output
.
put
(
String
.
format
(
"%s.%s"
,
key
,
entry
.
getKey
()),
entry
.
getValue
());
}
}
}
}
if
(!
accessible
)
{
field
.
setAccessible
(
false
);
}
}
}
}
}
}
}
catch
(
Exception
e
){
LOG
.
warn
(
"Exception while constructing topology"
,
e
);
}
return
output
;
return
output
;
}
}
...
...
This diff is collapsed.
Click to expand it.
release-log.txt
View file @
6fddccd6
...
@@ -6,6 +6,7 @@ INCOMPATIBLE CHANGES:
...
@@ -6,6 +6,7 @@ INCOMPATIBLE CHANGES:
ATLAS-1060 Add composite indexes for exact match performance improvements for all attributes (sumasai via shwethags)
ATLAS-1060 Add composite indexes for exact match performance improvements for all attributes (sumasai via shwethags)
ALL CHANGES:
ALL CHANGES:
ATLAS-1121 NPE while submitting topology in StormHook (ayubkhan via sumasai)
ATLAS-1119 Add retries for edge label creation (sumasai via shwethags)
ATLAS-1119 Add retries for edge label creation (sumasai via shwethags)
ATLAS-1111 Data loss is observed when atlas is restarted while hive_table metadata ingestion into kafka topic is in-progress(shwethags via sumasai)
ATLAS-1111 Data loss is observed when atlas is restarted while hive_table metadata ingestion into kafka topic is in-progress(shwethags via sumasai)
ATLAS-1115 Show Tag / Taxonomy Listing in sorted order (Kalyanikashikar via sumasai)
ATLAS-1115 Show Tag / Taxonomy Listing in sorted order (Kalyanikashikar via sumasai)
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment