Re: Duplicated processors when using nifi processors dependency

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Duplicated processors when using nifi processors dependency

Christopher Currie-2
I’ve opened a PR on GitHub for this issue: https://github.com/apache/nifi/pull/2140

Let me know if there’s any other follow up I need to do; I’d love to see this get in for 1.4.0.

Christopher

On 2017-05-22 06:16, Bryan Bende <[hidden email]> wrote:

> Thanks James.>
>
> For the abstract AWS processors, I would go with one of the following...>
>
> 1) Another module under nifi-aws-bundle like:>
>
> nifi-aws-bunde>
>     nifi-aws-service-api>
>     nifi-aws-service-api-nar>
>     nifi-aws-abstract-processors (or some similar name)>
>     nifi-aws-processors>
>     nifi-aws-nar>
>
> 2) A new module under nifi-nar-bundles/nifi-extension-utils like nifi-aws-utils>
>
> Either way, the desired result is to have a JAR that someone can>
> depend on that only brings in the abstract processors.>
>
> We have been putting generic abstract processors into>
> nifi-nar-bundles/nifi-extension-utils /nifi-processor-utils, but since>
> this case depends on the AWS SDK, then we would be introducing that>
> dependency to all the NARs that depend on nifi-processor-utils, which>
> is why I was suggesting a new module.>
>
> Let me know if this doesn't make sense.>
>
>
> On Mon, May 22, 2017 at 1:24 AM, James Wing <[hidden email]> wrote:>
> > I created a JIRA for the nifi-aws-bundle structure,>
> > https://issues.apache.org/jira/browse/NIFI-3950.>
> >>
> > Is there a best practice for the abstract classes like Robert describes?>
> > Are they a better fit for the service API bundle or the processors bundle?>
> > Or does that become irrelevant once the NAR dependencies are aligned and>
> > they can be references as Java libraries?>
> >>
> > Thanks,>
> >>
> > James>
> >>
> > On Sun, May 21, 2017 at 10:28 AM, Bryan Bende <[hidden email]> wrote:>
> >>
> >> Currently the AWS NAR contains the processors, controller service API, and>
> >> controller service implementation, all in one NAR.>
> >>>
> >> Typically the controller service API should be in its own NAR, so there>
> >> should be something like:>
> >>>
> >> nifi-aws-bunde>
> >>         nifi-aws-service-api>
> >>         nifi-aws-service-api-nar>
> >>         nifi-aws-processors>
> >>         nifi-aws-nar>
> >>>
> >> The nifi-aws-service-api module would have the>
> >> AWSCredentialsProviderService interface. The nifi-aws-nar would have a NAR>
> >> dependency on nifi-aws-service-api-nar.>
> >>>
> >> Setting it up this way, you nifi-aws-utils project could have a provided>
> >> dependency on the nifi-aws-service-api, and then your custom NAR could have>
> >> a NAR dependency on nifi-aws-service-api-nar.>
> >>>
> >> There is some more information here about setting up controller services:>
> >>>
> >> https://cwiki.apache.org/confluence/display/NIFI/Maven+>
> >> Projects+for+Extensions#MavenProjectsforExtensions->
> >> LinkingProcessorsandControllerServices <https://cwiki.apache.org/>
> >> confluence/display/NIFI/Maven+Projects+for+Extensions#>
> >> MavenProjectsforExtensions-LinkingProcessorsandControllerServices>>
> >>>
> >> https://github.com/bbende/nifi-dependency-example <>
> >> https://github.com/bbende/nifi-dependency-example>>
> >>>
> >>>
> >> > On May 20, 2017, at 10:06 AM, Robert <[hidden email]> wrote:>
> >> >>
> >> > I tired to do move abstract aws processors from>
> >> > nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors to newly created>
> >> > nifi-aws-utils.>
> >> >>
> >> > Problem that I face now is with AWSCredentialsProviderControllerService>
> >> that>
> >> > imo should stay under nifi-aws-bundle, but it is required by>
> >> > AbstractAWSCredentialsProviderProcessor that should be moved to>
> >> > nifi-aws-utils as it is used by abstract aws processors.>
> >> >>
> >> > What is the preferred way to solve this?>
> >> > Should I created new 'nifi-aws-service-bundle' under nifi-nar-bundles and>
> >> > both nifi-aws-processors and nifi-aws-utils can depend on it?>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> > -->
> >> > View this message in context: http://apache-nifi-developer->
> >> list.39713.n7.nabble.com/Duplicated-processors-when-using-nifi-processors->
> >> dependency-tp15902p15937.html>
> >> > Sent from the Apache NiFi Developer List mailing list archive at>
> >> Nabble.com.>
> >>>
> >>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Duplicated processors when using nifi processors dependency

Christopher Currie
Sorry if this duplicates, I sent from the wrong address the first time.

I’ve opened a PR on GitHub for this issue: https://github.com/apache/
nifi/pull/2140

Let me know if there’s any other follow up I need to do; I’d love to see
this get in for 1.4.0.

Christopher


On 2017-05-22 06:16, Bryan Bende <[hidden email]> wrote:

Thanks James.>

For the abstract AWS processors, I would go with one of the following...>

1) Another module under nifi-aws-bundle like:>

nifi-aws-bunde>
   nifi-aws-service-api>
   nifi-aws-service-api-nar>
   nifi-aws-abstract-processors (or some similar name)>
   nifi-aws-processors>
   nifi-aws-nar>

2) A new module under nifi-nar-bundles/nifi-extension-utils like
nifi-aws-utils>

Either way, the desired result is to have a JAR that someone can>
depend on that only brings in the abstract processors.>

We have been putting generic abstract processors into>
nifi-nar-bundles/nifi-extension-utils /nifi-processor-utils, but since>
this case depends on the AWS SDK, then we would be introducing that>
dependency to all the NARs that depend on nifi-processor-utils, which>
is why I was suggesting a new module.>

Let me know if this doesn't make sense.>


On Mon, May 22, 2017 at 1:24 AM, James Wing <[hidden email]> wrote:>

I created a JIRA for the nifi-aws-bundle structure,>
https://issues.apache.org/jira/browse/NIFI-3950.>


Is there a best practice for the abstract classes like Robert describes?>
Are they a better fit for the service API bundle or the processors bundle?>
Or does that become irrelevant once the NAR dependencies are aligned and>
they can be references as Java libraries?>


Thanks,>


James>


On Sun, May 21, 2017 at 10:28 AM, Bryan Bende <[hidden email]> wrote:>


Currently the AWS NAR contains the processors, controller service API, and>
controller service implementation, all in one NAR.>


Typically the controller service API should be in its own NAR, so there>
should be something like:>


nifi-aws-bunde>
       nifi-aws-service-api>
       nifi-aws-service-api-nar>
       nifi-aws-processors>
       nifi-aws-nar>


The nifi-aws-service-api module would have the>
AWSCredentialsProviderService interface. The nifi-aws-nar would have a NAR>
dependency on nifi-aws-service-api-nar.>


Setting it up this way, you nifi-aws-utils project could have a provided>
dependency on the nifi-aws-service-api, and then your custom NAR could
have>
a NAR dependency on nifi-aws-service-api-nar.>


There is some more information here about setting up controller services:>


https://cwiki.apache.org/confluence/display/NIFI/Maven+>
Projects+for+Extensions#MavenProjectsforExtensions->
LinkingProcessorsandControllerServices <https://cwiki.apache.org/>
confluence/display/NIFI/Maven+Projects+for+Extensions#>
MavenProjectsforExtensions-LinkingProcessorsandControllerServices>>


https://github.com/bbende/nifi-dependency-example <>
https://github.com/bbende/nifi-dependency-example>>



On May 20, 2017, at 10:06 AM, Robert <[hidden email]> wrote:>


I tired to do move abstract aws processors from>
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors to newly created>
nifi-aws-utils.>


Problem that I face now is with AWSCredentialsProviderControllerService>

that>

imo should stay under nifi-aws-bundle, but it is required by>
AbstractAWSCredentialsProviderProcessor that should be moved to>
nifi-aws-utils as it is used by abstract aws processors.>


What is the preferred way to solve this?>
Should I created new 'nifi-aws-service-bundle' under nifi-nar-bundles and>
both nifi-aws-processors and nifi-aws-utils can depend on it?>






-->
View this message in context: http://apache-nifi-developer->

list.39713.n7.nabble.com/Duplicated-processors-when-using-nifi-processors->
dependency-tp15902p15937.html>

Sent from the Apache NiFi Developer List mailing list archive at>

Nabble.com.>