Migrating from Ab Inito to Nifi

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Migrating from Ab Inito to Nifi

Adunuthula, Seshu
Hello Folks,

Great project and a fantastic User Guide. Some questions:

  *   Would NiFi be the right tool to migrate Ab Initio flows that enable data pipelines for moving Data from Oracle to Teradata/Hadoop.
  *   If so do we have the right connectors Oracle/Teradata/Hadoop available or there would be interest in building such connectors/processors

Regards
Seshu Adunuthula

Reply | Threaded
Open this post in threaded view
|

Re: Migrating from Ab Inito to Nifi

Mark Payne
Seshu,


Thanks, it’s great to hear from you!


Currently, we do support integrating with HDFS, but at this time we don’t have the necessary processors to connect to Oracle or Teradata. There is a ticket https://issues.apache.org/jira/browse/NIFI-293 that is for executing arbitrary SQL queries via JDBC. I believe this would satisfy the need to integrate with both Oracle and Teradata.


Thanks

-Mark






From: Adunuthula, Seshu
Sent: ‎Friday‎, ‎March‎ ‎13‎, ‎2015 ‎1‎:‎00‎ ‎PM
To: [hidden email]





Hello Folks,

Great project and a fantastic User Guide. Some questions:

  *   Would NiFi be the right tool to migrate Ab Initio flows that enable data pipelines for moving Data from Oracle to Teradata/Hadoop.
  *   If so do we have the right connectors Oracle/Teradata/Hadoop available or there would be interest in building such connectors/processors

Regards
Seshu Adunuthula
Reply | Threaded
Open this post in threaded view
|

Re: Migrating from Ab Inito to Nifi

Adunuthula, Seshu
Mark,

Thanks for the quick response. I would suggest extending NIFI-2193 to
understand the Metadata of the SQL Data Sources (Oracle,Teradata,
PostgresQL, Hive etc) and allow for easy transformationsŠ

Regards
Seshu

On 3/13/15, 10:02 AM, "Mark Payne" <[hidden email]> wrote:

>Seshu,
>
>
>Thanks, it¹s great to hear from you!
>
>
>Currently, we do support integrating with HDFS, but at this time we don¹t
>have the necessary processors to connect to Oracle or Teradata. There is
>a ticket https://issues.apache.org/jira/browse/NIFI-293 that is for
>executing arbitrary SQL queries via JDBC. I believe this would satisfy
>the need to integrate with both Oracle and Teradata.
>
>
>Thanks
>
>-Mark
>
>
>
>
>
>
>From: Adunuthula, Seshu
>Sent: Friday, March 13, 2015 1:00 PM
>To: [hidden email]
>
>
>
>
>
>Hello Folks,
>
>Great project and a fantastic User Guide. Some questions:
>
>  *   Would NiFi be the right tool to migrate Ab Initio flows that enable
>data pipelines for moving Data from Oracle to Teradata/Hadoop.
>  *   If so do we have the right connectors Oracle/Teradata/Hadoop
>available or there would be interest in building such
>connectors/processors
>
>Regards
>Seshu Adunuthula

Reply | Threaded
Open this post in threaded view
|

Re: Migrating from Ab Inito to Nifi

Joe Witt
In reply to this post by Adunuthula, Seshu
Hello Seshu

To add to Mark's comments there are some open tickets for providing
processes to interact with rdbms style back-end interfaces.  We'd
definitely appreciate any contributions or ideas folks have there.

I don't have any experience with Ab Initio so don't have any
thoughtful commentary to add there.  But if your dataflows are of the
type that are conducive to automation which you want to be able to
observe and/or alter at runtime and for which you want detailed
tracking then NiFi may indeed be of value.  You can keep sharing your
ideas and providing further details about your use cases and we'll try
to assist as we can.  Contributions are of course welcome from
questions to ideas to code.

Thanks
Joe

On Fri, Mar 13, 2015 at 12:59 PM, Adunuthula, Seshu
<[hidden email]> wrote:

> Hello Folks,
>
> Great project and a fantastic User Guide. Some questions:
>
>   *   Would NiFi be the right tool to migrate Ab Initio flows that enable data pipelines for moving Data from Oracle to Teradata/Hadoop.
>   *   If so do we have the right connectors Oracle/Teradata/Hadoop available or there would be interest in building such connectors/processors
>
> Regards
> Seshu Adunuthula
>