develop custom processor

classic Classic list List threaded Threaded
15 messages Options
cpk
Reply | Threaded
Open this post in threaded view
|

develop custom processor

cpk
This post was updated on .
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

Mark Payne
Pavan,


The developer guide provides pretty extensive documentation on writing custom Processors. It is available in the ‘Developer’ link on the website (nifi.incubator.apache.org), or you can access it directly via http://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html


There’s also a Maven Archetype that is very useful. You can use it via:


mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=0.0.1-incubating


This will create a new maven module that contains a processor, a nar, and their parent.


Thanks

-Mark








From: chenduluru
Sent: ‎Tuesday‎, ‎February‎ ‎3‎, ‎2015 ‎10‎:‎52‎ ‎AM
To: [hidden email]





I want to write a custom processor which takes input as xml and change some
content in that xml and saved into some folder. Is there any example on
this?

I dont see any documentation for how to create custom processor?

Regards,
Pavan



--
View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/develop-custom-processor-tp558.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

Joe Witt
In reply to this post by cpk
Pavan,

Two parts to this:
1) How best to design the flow
  Best place to start for this is the overview [1] and user guide [2]

2) If any parts remain needing to be built then how to do so.
  Best place to start for this would be the dev guide [3] and looking at
the example processors links I sent previously.

For your description it is very possible the existing out of the box
processes do what you need.  But let's assume the xml transformation you
want to do requires something custom.  Your flow would be like this (for
example):

GetFile -> Custom XML Processor you build --> PutFile

[1] http://nifi.incubator.apache.org/docs/nifi-docs/overview.html
[2] http://nifi.incubator.apache.org/docs/nifi-docs/user-guide.html
[3] http://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html

If the documents as written and the examples provided leave specific gaps
please let us know.

Thanks
Joe



On Tue, Feb 3, 2015 at 10:42 AM, chenduluru <[hidden email]> wrote:

> I want to write a custom processor which takes input as xml and change some
> content in that xml and saved into some folder. Is there any example on
> this?
>
> I dont see any documentation for how to create custom processor?
>
> Regards,
> Pavan
>
>
>
> --
> View this message in context:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/develop-custom-processor-tp558.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive
> at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

Brandon DeVries
Pavan,

     As Joe said, if you're just looking for an excuse to try creating a
custom processor to learn, then definitely go for it.  However, it is
likely that the existing processors EvaluateXQuery or TransformXML (which
uses an XSLT stylesheet) can likely do what you're trying to accomplish.
If you'd first like to learn how to build a flow using standard components,
you could try that.  You may also be able to look at those processors for
examples when building your own.  Thanks.

Brandon

On Tue, Feb 3, 2015 at 10:57 AM, Joe Witt <[hidden email]> wrote:

> Pavan,
>
> Two parts to this:
> 1) How best to design the flow
>   Best place to start for this is the overview [1] and user guide [2]
>
> 2) If any parts remain needing to be built then how to do so.
>   Best place to start for this would be the dev guide [3] and looking at
> the example processors links I sent previously.
>
> For your description it is very possible the existing out of the box
> processes do what you need.  But let's assume the xml transformation you
> want to do requires something custom.  Your flow would be like this (for
> example):
>
> GetFile -> Custom XML Processor you build --> PutFile
>
> [1] http://nifi.incubator.apache.org/docs/nifi-docs/overview.html
> [2] http://nifi.incubator.apache.org/docs/nifi-docs/user-guide.html
> [3] http://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html
>
> If the documents as written and the examples provided leave specific gaps
> please let us know.
>
> Thanks
> Joe
>
>
>
> On Tue, Feb 3, 2015 at 10:42 AM, chenduluru <[hidden email]> wrote:
>
> > I want to write a custom processor which takes input as xml and change
> some
> > content in that xml and saved into some folder. Is there any example on
> > this?
> >
> > I dont see any documentation for how to create custom processor?
> >
> > Regards,
> > Pavan
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/develop-custom-processor-tp558.html
> > Sent from the Apache NiFi (incubating) Developer List mailing list
> archive
> > at Nabble.com.
> >
>
cpk
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

cpk
Thanks all for your response.

So If I write new java class which extends AbstractProcessor and build the project. I can see in the  processor list in UI?.
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

Joe Witt
If you follow the advice
http://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html#processor_api
on how to build processors then you will indeed see it automatically show
up in the UI.

Please subscribe your email address to the mailing list.  Having to
moderate each by hand will delay your ability to get feedback from the
community.

Thanks
Joe

On Tue, Feb 3, 2015 at 12:36 PM, chenduluru <[hidden email]> wrote:

> Thanks all for your response.
>
> So If I write new java class which extends AbstractProcessor and build the
> project. I can see in the  processor list in UI?.
>
>
>
> --
> View this message in context:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/develop-custom-processor-tp558p564.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive
> at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

Mark Payne
In reply to this post by cpk
NiFi has a construct of a NiFi Archive (NAR) for packaging processors (and other extensions) with their extensions.




For learning purposes, the easiest way to start is to add a Processor to the nifi-standard-processors module (/nifi/nifi-nar-assemblies/nifi-standard-bundle/nifi-standard-processors). If you are planning to develop a Processor that you would like to contribute back to Apache NiFi, then we should probably discuss where in the tree structure the Processor should go. If this is intended as a Processor for your own use that won’t be contributed back, it should not go in standard processors, but standard processors is still a good starting point for learning.




Then, in the nifi-standard-processors/src/main/resources/META-INF/org.apache.nifi.processor.Processor file, you will need to add a line with your fully-qualified Processor name. This is how the UI discovers the available processors.




Then you can rebuild nifi-standard-bundle and copy in nifi-standard-bundle/nifi-standard-nar/target/nifi-standard-nar-0.0.2-incubating.SNAPSHOT.nar to your lib/ directory. Or simply rebuild from the root nifi/ directory and you’re good to go.


Thanks

-Mark






From: chenduluru
Sent: ‎Tuesday‎, ‎February‎ ‎3‎, ‎2015 ‎12‎:‎46‎ ‎PM
To: [hidden email]





Thanks all for your response.

So If I write new java class which extends AbstractProcessor and build the
project. I can see in the  processor list in UI?.



--
View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/develop-custom-processor-tp558p564.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
cpk
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

cpk
This post was updated on .
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

Jennifer Barnabee
You can create a template of your flow, which will be an XML file. Then,
you can export/download that file to your local drive and re-import it
whenever you need to.

To do this, select whatever you want to make a template of (or select
nothing if you want to template *everything* you have on your graph). Click
the "Create Template" button in the center portion of the toolbar (to the
right of the red stop button). You will be prompted to name the template.

Next, go to the Template Management page by clicking the same type of
button (Templates) in the far-right portion of the toolbar (the fourth
button from the far right). Here, you will see the template that you saved
listed in the table. Click the download button in the row that your
template is listed on to download it to your local drive.  Whenever you
want to re-import a template, you go to this same Template Management page
and click "Browse" to find the template file on your local drive. Then,
once you've selected it, click "Import".

Let me know if you have any questions or issues. All of the above is
covered more completely in the User Guide here:
http://nifi.incubator.apache.org/docs/nifi-docs/user-guide.html#templates

Cheers,
Jenn


On Wed, Feb 4, 2015 at 4:25 AM, chenduluru <[hidden email]> wrote:

> Thanks for your response.
>
> Once I created data flow in UI I want to save it in some location and again
> want to import saved flow into nifi. How can I do this?
>
> When I rebuild project and run nifi my old flow is not available in nifi.
>
> I tried to save it in my local folder, it is getting saved as NiFi
> Flow1.htm. How can I import back to nifi?
>
>
>
> --
> View this message in context:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/develop-custom-processor-tp558p569.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive
> at Nabble.com.
>
cpk
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

cpk
I wrote one custom processor and its working fine but there is a issue with "onTrigger" method. Its getting run so many times until I stop the execution in UI.

Is there any way to stop the process after first execution of onTrigger method?
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

Mark Payne
In reply to this post by cpk
Hello,


You can configure how often the Processor is scheduled to run in the “Scheduling” tab of the Configuration dialog (right-click on processor and choose configure).


Thanks

-Mark






From: chenduluru
Sent: ‎Monday‎, ‎February‎ ‎9‎, ‎2015 ‎11‎:‎07‎ ‎AM
To: [hidden email]





I wrote one custom processor and its working fine but there is a issue with
"onTrigger" method. Its getting run so many times until I stop the execution
in UI.

Is there any way to stop the process after first execution of onTrigger
method?




--
View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/develop-custom-processor-tp558p642.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
cpk
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

cpk
This post was updated on .
CONTENTS DELETED
The author has deleted this message.
cpk
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

cpk
This post was updated on .
CONTENTS DELETED
The author has deleted this message.
cpk
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

cpk
This post was updated on .
In reply to this post by Mark Payne
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: develop custom processor

Joe Witt
Hello

Please subscribe to the mailing list.  We want to be helpful and
responsive.  However, by not subscribing it is requiring manual
approval of each e-mail sent.

To subscribe simply send an email to "[hidden email]"

Is the code available anywhere like Github or otherwise that we could
look at and provide feedback?

Thanks
Joe

On Mon, Feb 9, 2015 at 10:28 PM, chenduluru <[hidden email]> wrote:
> Hi
>
>
>
> --
> View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/develop-custom-processor-tp558p652.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.