[GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

JPercivall
GitHub user bbende opened a pull request:

    https://github.com/apache/incubator-nifi/pull/11

    NIFI-272 - Create Maven Archetype for a Processor bundle

    Creating the nifi-maven-archetypes module and adding the nifi-processor-bundle-archetype.
   
    After running mvn clean install from nifi-maven-archetypes (or from nifi), you can test the archetype like this:
   
        mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=0.0.1-incubating-SNAPSHOT -DgroupId=org.apache.nifi -DartifactId=donothing -Dversion=1.0
   
    Using an artifactId of "donothing" will generate the following:
   
        donothing (artifactId of nifi-donothing-bundle)
        ├── nifi-donothing-nar
        │   └── pom.xml
        ├── nifi-donothing-processors
        │   ├── pom.xml
        │   └── src
        │       ├── main
        │       │   ├── java
        │       │   │   └── org
        │       │   │       └── apache
        │       │   │           └── nifi
        │       │   │               └── processors
        │       │   │                   └── MyProcessor.java
        │       │   └── resources
        │       │       ├── META-INF
        │       │       │   └── services
        │       │       │       └── org.apache.nifi.processor.Processor
        │       │       └── docs
        │       │           └── org.apache.nifi.processors.MyProcessor
        │       │               └── index.html
        │       └── test
        │           └── java
        │               └── org
        │                   └── apache
        │                       └── nifi
        │                           └── processors
        │                               └── MyProcessorTest.java
        └── pom.xml


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/bbende/incubator-nifi develop

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-nifi/pull/11.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #11
   
----
commit db2072ed8aa46bd8d157fe13387558f9ff7e44df
Author: bbende <[hidden email]>
Date:   2015-01-22T13:53:23Z

    Creating the nifi-maven-archetypes module and adding the nifi-processor-bundle-archetype.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

RE: [GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

Mark Payne
Bryan,
This is excellent. I pulled the patch and looked it over. Was looking at making just a few very minor tweaks to be a big more consistent with the code style that we typically use.
The only thing that I really would like to change -- and I dont know that we can -- is that the artifact id becomes the name of the directory that the project goes into. So if I want to create a processor bundle called "fire" for instance I'd want to call it "nifi-fire-bundle" for the root directory. So I'd enter that as the artifact id. But then we want the sub-modules to be named "nifi-fire-processors" and "nifi-fire-nar" without the "-bundle" part there. I don't see any way to do this in Maven.
I considered maybe offering an 'artifact-base-name' property so that the sub-modules could be named '${artifact-base-name}-processors' and '${artifact-base-name}-nar' so that the artifact base name in this example would be 'nifi-fire'.
Does anybody have a better approach for this?
Thanks-Mark

> From: [hidden email]
> To: [hidden email]
> Subject: [GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...
> Date: Thu, 22 Jan 2015 14:07:32 +0000
>
> GitHub user bbende opened a pull request:
>
>     https://github.com/apache/incubator-nifi/pull/11
>
>     NIFI-272 - Create Maven Archetype for a Processor bundle
>
>     Creating the nifi-maven-archetypes module and adding the nifi-processor-bundle-archetype.
>    
>     After running mvn clean install from nifi-maven-archetypes (or from nifi), you can test the archetype like this:
>    
>         mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=0.0.1-incubating-SNAPSHOT -DgroupId=org.apache.nifi -DartifactId=donothing -Dversion=1.0
>    
>     Using an artifactId of "donothing" will generate the following:
>    
>         donothing (artifactId of nifi-donothing-bundle)
>         ├── nifi-donothing-nar
>         │   └── pom.xml
>         ├── nifi-donothing-processors
>         │   ├── pom.xml
>         │   └── src
>         │       ├── main
>         │       │   ├── java
>         │       │   │   └── org
>         │       │   │       └── apache
>         │       │   │           └── nifi
>         │       │   │               └── processors
>         │       │   │                   └── MyProcessor.java
>         │       │   └── resources
>         │       │       ├── META-INF
>         │       │       │   └── services
>         │       │       │       └── org.apache.nifi.processor.Processor
>         │       │       └── docs
>         │       │           └── org.apache.nifi.processors.MyProcessor
>         │       │               └── index.html
>         │       └── test
>         │           └── java
>         │               └── org
>         │                   └── apache
>         │                       └── nifi
>         │                           └── processors
>         │                               └── MyProcessorTest.java
>         └── pom.xml
>
>
> You can merge this pull request into a Git repository by running:
>
>     $ git pull https://github.com/bbende/incubator-nifi develop
>
> Alternatively you can review and apply these changes as the patch at:
>
>     https://github.com/apache/incubator-nifi/pull/11.patch
>
> To close this pull request, make a commit to your master/trunk branch
> with (at least) the following in the commit message:
>
>     This closes #11
>    
> ----
> commit db2072ed8aa46bd8d157fe13387558f9ff7e44df
> Author: bbende <[hidden email]>
> Date:   2015-01-22T13:53:23Z
>
>     Creating the nifi-maven-archetypes module and adding the nifi-processor-bundle-archetype.
>
> ----
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at [hidden email] or file a JIRA ticket
> with INFRA.
> ---
     
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

JPercivall
In reply to this post by JPercivall
Github user bbende commented on the pull request:

    https://github.com/apache/incubator-nifi/pull/11#issuecomment-71037873
 
    Mark, I agree about the naming of the top level directory being annoying. I was thinking the same thing about having a second property so someone could do artifactId=donothing-bundle and artifactBaseName=donothing. I can play around with this more and see how it works and then submit an update.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

Re: [GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

Mike Drob
I'm not advocating either side in this, but do we want to explicitly
encourage use of the o.a.nifi package space? On one hand, it adds a degree
of consistency, on the other hand you are probably going to get lots of
interesting debugging questions about processors that have come out of the
community in the future.

Mike

On Thu, Jan 22, 2015 at 9:30 AM, bbende <[hidden email]> wrote:

> Github user bbende commented on the pull request:
>
>     https://github.com/apache/incubator-nifi/pull/11#issuecomment-71037873
>
>     Mark, I agree about the naming of the top level directory being
> annoying. I was thinking the same thing about having a second property so
> someone could do artifactId=donothing-bundle and
> artifactBaseName=donothing. I can play around with this more and see how it
> works and then submit an update.
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at [hidden email] or file a JIRA ticket
> with INFRA.
> ---
>
Reply | Threaded
Open this post in threaded view
|

Re: [GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

Bryan Bende
That is a good point, it may be possible to accept any package
name...Initially it was easier to force it to org.apache.nifi so that
META-INF/services/org.apache.nifi.processor.Processor and
docs/org.apache.nifi.processor.MyProcessor don't have to be dynamically
named, but it might be trivial to use the ${package} property provided to
name those files and directories appropriately, I just need to try it.

If that didn't work and we needed to force a package, we could also just
use some example package like "my.processor" (or anything not
org.apache.nifi) to avoid everyone using that by default.

As a starting point I'll see if the package can at least be dynamic, and
then go from there.

On Thu, Jan 22, 2015 at 10:47 AM, Mike Drob <[hidden email]> wrote:

> I'm not advocating either side in this, but do we want to explicitly
> encourage use of the o.a.nifi package space? On one hand, it adds a degree
> of consistency, on the other hand you are probably going to get lots of
> interesting debugging questions about processors that have come out of the
> community in the future.
>
> Mike
>
> On Thu, Jan 22, 2015 at 9:30 AM, bbende <[hidden email]> wrote:
>
> > Github user bbende commented on the pull request:
> >
> >
> https://github.com/apache/incubator-nifi/pull/11#issuecomment-71037873
> >
> >     Mark, I agree about the naming of the top level directory being
> > annoying. I was thinking the same thing about having a second property so
> > someone could do artifactId=donothing-bundle and
> > artifactBaseName=donothing. I can play around with this more and see how
> it
> > works and then submit an update.
> >
> >
> > ---
> > If your project is set up for it, you can reply to this email and have
> your
> > reply appear on GitHub as well. If your project does not have this
> feature
> > enabled and wishes so, or if the feature is enabled but not working,
> please
> > contact infrastructure at [hidden email] or file a JIRA
> ticket
> > with INFRA.
> > ---
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

Mark Payne
Good call. I agree with this. One note, though, Bryan the file named org.apache.nifi.processor.Processor will still keep that name because it's the fully qualified name of the Processor  interface. But the contents of that file will change.

Sent from my iPhone

> On Jan 22, 2015, at 12:22 PM, Bryan Bende <[hidden email]> wrote:
>
> That is a good point, it may be possible to accept any package
> name...Initially it was easier to force it to org.apache.nifi so that
> META-INF/services/org.apache.nifi.processor.Processor and
> docs/org.apache.nifi.processor.MyProcessor don't have to be dynamically
> named, but it might be trivial to use the ${package} property provided to
> name those files and directories appropriately, I just need to try it.
>
> If that didn't work and we needed to force a package, we could also just
> use some example package like "my.processor" (or anything not
> org.apache.nifi) to avoid everyone using that by default.
>
> As a starting point I'll see if the package can at least be dynamic, and
> then go from there.
>
>> On Thu, Jan 22, 2015 at 10:47 AM, Mike Drob <[hidden email]> wrote:
>>
>> I'm not advocating either side in this, but do we want to explicitly
>> encourage use of the o.a.nifi package space? On one hand, it adds a degree
>> of consistency, on the other hand you are probably going to get lots of
>> interesting debugging questions about processors that have come out of the
>> community in the future.
>>
>> Mike
>>
>>> On Thu, Jan 22, 2015 at 9:30 AM, bbende <[hidden email]> wrote:
>>>
>>> Github user bbende commented on the pull request:
>> https://github.com/apache/incubator-nifi/pull/11#issuecomment-71037873
>>>
>>>    Mark, I agree about the naming of the top level directory being
>>> annoying. I was thinking the same thing about having a second property so
>>> someone could do artifactId=donothing-bundle and
>>> artifactBaseName=donothing. I can play around with this more and see how
>> it
>>> works and then submit an update.
>>>
>>>
>>> ---
>>> If your project is set up for it, you can reply to this email and have
>> your
>>> reply appear on GitHub as well. If your project does not have this
>> feature
>>> enabled and wishes so, or if the feature is enabled but not working,
>> please
>>> contact infrastructure at [hidden email] or file a JIRA
>> ticket
>>> with INFRA.
>>> ---
>>
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

JPercivall
In reply to this post by JPercivall
Github user bbende commented on the pull request:

    https://github.com/apache/incubator-nifi/pull/11#issuecomment-71137813
 
    So I was able to address the two items we discussed:
   
    1. The artifactId will now be entered as the artifactId you want for the top-level artifact, so for example "nifi-donothing-bundle". An additional property called artifactBaseName will be specified that controls the sub-modules, in this example artifactBaseName would be set to "donothing" if you want to produce nifi-donothing-processors and nifi-donothing-nar.
   
      In order to get this to work I had to abandon the modules elements in the archetype descriptor in favor of using only filesets. The modules didn't seem to pick up the artifactBaseName property when doing replacement on the directories.
   
    2. The archetype will now respect the package entered during archetype generation.
   
    An updated example of using the archetype is the following:
   
        mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=0.0.1-incubating-SNAPSHOT -DgroupId=org.apache.nifi -DartifactId=nifi-donothing-bundle -DartifactBaseName=donothing -Dpackage=my.test.processors -Dversion=1.0


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

RE: [GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

Mark Payne
Bryan,
I pulled in your pull request, but I've not pushed it yet. I attached a patch to https://issues.apache.org/jira/browse/NIFI-272 to account for the mods that I made locally. Would like to have you review and see if you agree with the changes.
Basically, the the changes are:
* remove the <name> tag from poms and just use the default, as this is what I think we're moving to because so many of the artifacts are inconsistent with naming.* remove the default <properties> tag from the pom, as most artifacts in the NiFi baseline don't have it* Set default package name to ${groupId}.processors.${baseArtifactName}* Changed the default implementation of Processor's onTrigger to call session.get() and return if no FlowFile. This is the first few lines of probably 80+% of Processors.
We can change these changes if they're not desired. Otherwise, I will push the changes.
Thanks-Mark

> From: [hidden email]
> To: [hidden email]
> Subject: [GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...
> Date: Fri, 23 Jan 2015 02:23:42 +0000
>
> Github user bbende commented on the pull request:
>
>     https://github.com/apache/incubator-nifi/pull/11#issuecomment-71137813
>  
>     So I was able to address the two items we discussed:
>    
>     1. The artifactId will now be entered as the artifactId you want for the top-level artifact, so for example "nifi-donothing-bundle". An additional property called artifactBaseName will be specified that controls the sub-modules, in this example artifactBaseName would be set to "donothing" if you want to produce nifi-donothing-processors and nifi-donothing-nar.
>    
>       In order to get this to work I had to abandon the modules elements in the archetype descriptor in favor of using only filesets. The modules didn't seem to pick up the artifactBaseName property when doing replacement on the directories.
>    
>     2. The archetype will now respect the package entered during archetype generation.
>    
>     An updated example of using the archetype is the following:
>    
>         mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=0.0.1-incubating-SNAPSHOT -DgroupId=org.apache.nifi -DartifactId=nifi-donothing-bundle -DartifactBaseName=donothing -Dpackage=my.test.processors -Dversion=1.0
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at [hidden email] or file a JIRA ticket
> with INFRA.
> ---
     
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

JPercivall
In reply to this post by JPercivall
Github user bbende commented on the pull request:

    https://github.com/apache/incubator-nifi/pull/11#issuecomment-71244078
 
    Mark, those changes in the patch look good to me.
   
    I like the default value for the package property since it normally defaults to the groupId which is probably not what people would want.
   
    And of course you know best about the standards across other bundles/processors in terms of the name elements and onTrigger methods.
   
    Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

JPercivall
In reply to this post by JPercivall
Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-nifi/pull/11


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

RE: [GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...

Mark Payne
In reply to this post by JPercivall
Bryan,
Sounds good. This has been pushed to develop.
Thanks again for the contribution!
-Mark

> From: [hidden email]
> To: [hidden email]
> Subject: [GitHub] incubator-nifi pull request: NIFI-272 - Create Maven Archetype for...
> Date: Fri, 23 Jan 2015 18:52:44 +0000
>
> Github user bbende commented on the pull request:
>
>     https://github.com/apache/incubator-nifi/pull/11#issuecomment-71244078
>  
>     Mark, those changes in the patch look good to me.
>    
>     I like the default value for the package property since it normally defaults to the groupId which is probably not what people would want.
>    
>     And of course you know best about the standards across other bundles/processors in terms of the name elements and onTrigger methods.
>    
>     Thanks!
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at [hidden email] or file a JIRA ticket
> with INFRA.
> ---