Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
A
atlas
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
dataplatform
atlas
Commits
5ebd7070
Commit
5ebd7070
authored
6 years ago
by
Madhan Neethiraj
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
ATLAS-2492: updated Storm version to 1.2.0
parent
ec00aed1
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
29 additions
and
11 deletions
+29
-11
pom.xml
addons/storm-bridge-shim/pom.xml
+1
-1
pom.xml
addons/storm-bridge/pom.xml
+1
-1
StormAtlasHook.java
...main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
+27
-9
No files found.
addons/storm-bridge-shim/pom.xml
View file @
5ebd7070
...
...
@@ -31,7 +31,7 @@
<packaging>
jar
</packaging>
<properties>
<storm.version>
1.
0
.0
</storm.version>
<storm.version>
1.
2
.0
</storm.version>
</properties>
<dependencies>
...
...
This diff is collapsed.
Click to expand it.
addons/storm-bridge/pom.xml
View file @
5ebd7070
...
...
@@ -30,7 +30,7 @@
<packaging>
jar
</packaging>
<properties>
<storm.version>
1.
0
.0
</storm.version>
<storm.version>
1.
2
.0
</storm.version>
<hive.version>
1.2.1
</hive.version>
</properties>
...
...
This diff is collapsed.
Click to expand it.
addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
View file @
5ebd7070
...
...
@@ -186,31 +186,49 @@ public class StormAtlasHook extends AtlasHook implements ISubmitterHook {
Referenceable
dataSetReferenceable
;
// todo: need to redo this with a config driven approach
switch
(
name
)
{
case
"KafkaSpout"
:
case
"KafkaSpout"
:
{
String
topicName
=
config
.
get
(
"KafkaSpout.kafkaSpoutConfig.translator.topic"
);
String
uri
=
config
.
get
(
"KafkaSpout.kafkaSpoutConfig.kafkaProps.bootstrap.servers"
);
if
(
StringUtils
.
isEmpty
(
topicName
))
{
topicName
=
config
.
get
(
"KafkaSpout._spoutConfig.topic"
);
}
if
(
StringUtils
.
isEmpty
(
uri
))
{
uri
=
config
.
get
(
"KafkaSpout._spoutConfig.hosts.brokerZkStr"
);
}
dataSetReferenceable
=
new
Referenceable
(
StormDataTypes
.
KAFKA_TOPIC
.
getName
());
final
String
topicName
=
config
.
get
(
"KafkaSpout._spoutConfig.topic"
);
dataSetReferenceable
.
set
(
"topic"
,
topicName
);
dataSetReferenceable
.
set
(
"uri"
,
config
.
get
(
"KafkaSpout._spoutConfig.hosts.brokerZkStr"
));
dataSetReferenceable
.
set
(
"uri"
,
uri
);
if
(
StringUtils
.
isEmpty
(
topologyOwner
))
{
topologyOwner
=
ANONYMOUS_OWNER
;
}
dataSetReferenceable
.
set
(
AtlasClient
.
OWNER
,
topologyOwner
);
dataSetReferenceable
.
set
(
AtlasClient
.
REFERENCEABLE_ATTRIBUTE_NAME
,
getKafkaTopicQualifiedName
(
getClusterName
(
stormConf
),
topicName
));
dataSetReferenceable
.
set
(
AtlasClient
.
NAME
,
topicName
);
break
;
}
break
;
case
"HBaseBolt"
:
case
"HBaseBolt"
:
{
dataSetReferenceable
=
new
Referenceable
(
StormDataTypes
.
HBASE_TABLE
.
getName
());
final
String
hbaseTableName
=
config
.
get
(
"HBaseBolt.tableName"
);
dataSetReferenceable
.
set
(
"uri"
,
stormConf
.
get
(
"hbase.rootdir"
));
dataSetReferenceable
.
set
(
AtlasClient
.
NAME
,
hbaseTableName
);
String
uri
=
config
.
get
(
"hbase.rootdir"
);
if
(
StringUtils
.
isEmpty
(
uri
))
{
uri
=
hbaseTableName
;
}
dataSetReferenceable
.
set
(
"uri"
,
hbaseTableName
);
dataSetReferenceable
.
set
(
AtlasClient
.
NAME
,
uri
);
dataSetReferenceable
.
set
(
AtlasClient
.
OWNER
,
stormConf
.
get
(
"storm.kerberos.principal"
));
clusterName
=
extractComponentClusterName
(
HBaseConfiguration
.
create
(),
stormConf
);
//TODO - Hbase Namespace is hardcoded to 'default'. need to check how to get this or is it already part of tableName
dataSetReferenceable
.
set
(
AtlasClient
.
REFERENCEABLE_ATTRIBUTE_NAME
,
getHbaseTableQualifiedName
(
clusterName
,
HBASE_NAMESPACE_DEFAULT
,
hbaseTableName
));
break
;
}
break
;
case
"HdfsBolt"
:
dataSetReferenceable
=
new
Referenceable
(
HiveMetaStoreBridge
.
HDFS_PATH
);
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment