Questions re: customizing a NiFi processor

[James McMahon]

Good evening. I am familiar with NiFi and have developed a number of
workflows using out -of-the-box processors. I now need to develop a custom
version of PublishAMQP that permits the use of attributes in expression
language to set specs for the AMQP target.

I am not very familiar with Maven, but have done some initial research and
hope I can bootstrap my effort following this link:
https://community.hortonworks.com/articles/4318/build-custom-nifi-processor.html

Lacking any experience developing custom processors, I'd like to understand
how I would establish in my maven project the source code for the existing
PublishAMQP. I'd like to modify that proven code base to allow the use of
expression language to set other configuration properties.

Can anyone help me with an example where source code for an existing
processor is brought in from a github resource or the like to use as my
starting point?

Thank you in advance for any assistance. -Jim

[Andy LoPresto-2]

Jim,

To be honest, I would recommend making those changes directly to the existing PublishAMQP processor, as unless I am missing something, that does not seem like a breaking change that would affect other users negatively. I’d suggest opening a Jira for the feature, cloning the GitHub repository (see the instructions in the Contributor Guide [1]), making the changes in your branch, building and testing, and then submitting a PR (or patch) for review and inclusion in the project. This will also require the least amount of code manipulation (example of a commit from Andre adding EL evaluation to a property descriptor [2]). 

If you do not wish to do that, my next recommendation would be to clone the project and create a new processor in the same Maven module (and add it to the MANIFEST) alongside PublishAMQP (call it PublishAMQPWithEL or something) and extend the existing PublishAMQP class and override just the specific behavior you are interested in. 

A possible, but in my opinion unnecessary, process would be to add a completely new NAR with the existing nifi-amqp-bundle/nifi-amqp-processors artifact listed as a dependency and perform your extension in your own NAR. 

Hope this helps. 

[1] https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide

[2] https://github.com/apache/nifi/commit/84f1fb395963614a2ff78a615c92fc79a52e2942

Andy LoPresto

[hidden email]

[hidden email]

PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Dec 2, 2017, at 3:43 PM, James McMahon <[hidden email]> wrote:

Good evening. I am familiar with NiFi and have developed a number of
workflows using out -of-the-box processors. I now need to develop a custom
version of PublishAMQP that permits the use of attributes in expression
language to set specs for the AMQP target.

I am not very familiar with Maven, but have done some initial research and
hope I can bootstrap my effort following this link:
https://community.hortonworks.com/articles/4318/build-custom-nifi-processor.html

Lacking any experience developing custom processors, I'd like to understand
how I would establish in my maven project the source code for the existing
PublishAMQP. I'd like to modify that proven code base to allow the use of
expression language to set other configuration properties.

Can anyone help me with an example where source code for an existing
processor is brought in from a github resource or the like to use as my
starting point?

Thank you in advance for any assistance. -Jim

[Mike Thomsen]

Jim,

I'd recommend just forking NiFi and updating it with a patch. Writing your
own processor for this sounds like massive overkill for what you need. If
you know how to use Git, you'll be fine. You can add me on GitHub as a
reviewer when you're done (I can at least do the initial pass for Andy or
someone else).

The easiest way to do the compliance check when you think you're done is to
have another terminal open to $NIFI_ROOT and run this:

mvn clean install -Pcontrib-check -pl :some-module-name

Note that ":" at the start. It's required for Maven to identify it as a
module name. That'll build only the module you want to test.

You can also run just your own tests with:

mvn test -Dtest=org.apache.nifi.x.y.z.MyTest

Sometimes that's necessary if other tests are broken for whatever reason
and they're not related to your code.

On Sat, Dec 2, 2017 at 6:43 PM, James McMahon <[hidden email]> wrote:

> Good evening. I am familiar with NiFi and have developed a number of
> workflows using out -of-the-box processors. I now need to develop a custom
> version of PublishAMQP that permits the use of attributes in expression
> language to set specs for the AMQP target.
>
> I am not very familiar with Maven, but have done some initial research and
> hope I can bootstrap my effort following this link:
> https://community.hortonworks.com/articles/4318/build-
> custom-nifi-processor.html
>
> Lacking any experience developing custom processors, I'd like to understand
> how I would establish in my maven project the source code for the existing
> PublishAMQP. I'd like to modify that proven code base to allow the use of
> expression language to set other configuration properties.
>
> Can anyone help me with an example where source code for an existing
> processor is brought in from a github resource or the like to use as my
> starting point?
>
> Thank you in advance for any assistance. -Jim
>