NIFI-271 / refactored parent / checkstyle / rat / findbugs

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

NIFI-271 / refactored parent / checkstyle / rat / findbugs

Joe Witt
Hello All,

I believe NIFI-271 [1] is now resolved.  In it we:

1) completed the parent DRY refactoring identified by Benson a while ago

2) introduced checkstyle which is always running now.  Refactored code
and cleaned up 10s of thousands of violations.  This should help with
whitespace and other consistency issues that made reviews tougher.
Might be some IDE lessons learned folks need to share.

3) cleaned up RAT logic and pushed special cases to their specific
modules.  I'd like RAT to always run but we had to keep it in a
profile because it made build times way too long on some platforms.
If you want to run this just add '-Pcheck-licenses' when building.

4) Introduced a profile for convenient execution of findbugs.  Just
add '-Pfindbugs' when building. There are many outstanding issues so
it can't be made mandatory yet but we should work in that direction.
We can add exclusions/filters for violations we disagree with in a
module specific manner.

So to get up to date on the latest goodness:

git checkout develop
git pull
git clean -ffd
cd nifi-parent
mvn clean install
cd ../nifi-nar-maven-plugin
mvn clean install
cd ../nifi
mvn -T C1 clean install

Please advise if you run into any trouble.

[1] https://issues.apache.org/jira/browse/NIFI-271

Thanks
Joe
Reply | Threaded
Open this post in threaded view
|

Re: NIFI-271 / refactored parent / checkstyle / rat / findbugs

Dan Bress
Has anyone created eclipse/netbeans formatting rules that we can use so that "control-shift-f" and "alt-shift-f" format code that adheres to the checkstyle rules?

Dan Bress
Software Engineer
ONYX Consulting Services

________________________________________
From: Joe Witt <[hidden email]>
Sent: Tuesday, April 28, 2015 1:50 AM
To: [hidden email]
Subject: NIFI-271 / refactored parent / checkstyle / rat / findbugs

Hello All,

I believe NIFI-271 [1] is now resolved.  In it we:

1) completed the parent DRY refactoring identified by Benson a while ago

2) introduced checkstyle which is always running now.  Refactored code
and cleaned up 10s of thousands of violations.  This should help with
whitespace and other consistency issues that made reviews tougher.
Might be some IDE lessons learned folks need to share.

3) cleaned up RAT logic and pushed special cases to their specific
modules.  I'd like RAT to always run but we had to keep it in a
profile because it made build times way too long on some platforms.
If you want to run this just add '-Pcheck-licenses' when building.

4) Introduced a profile for convenient execution of findbugs.  Just
add '-Pfindbugs' when building. There are many outstanding issues so
it can't be made mandatory yet but we should work in that direction.
We can add exclusions/filters for violations we disagree with in a
module specific manner.

So to get up to date on the latest goodness:

git checkout develop
git pull
git clean -ffd
cd nifi-parent
mvn clean install
cd ../nifi-nar-maven-plugin
mvn clean install
cd ../nifi
mvn -T C1 clean install

Please advise if you run into any trouble.

[1] https://issues.apache.org/jira/browse/NIFI-271

Thanks
Joe
Reply | Threaded
Open this post in threaded view
|

Re: NIFI-271 / refactored parent / checkstyle / rat / findbugs

Joe Witt
Not that I am aware of.  Netbeans defaults I believe work well.  Will
defer to others for eclipse ideas.

On Tue, Apr 28, 2015 at 8:29 AM, Dan Bress <[hidden email]> wrote:

> Has anyone created eclipse/netbeans formatting rules that we can use so that "control-shift-f" and "alt-shift-f" format code that adheres to the checkstyle rules?
>
> Dan Bress
> Software Engineer
> ONYX Consulting Services
>
> ________________________________________
> From: Joe Witt <[hidden email]>
> Sent: Tuesday, April 28, 2015 1:50 AM
> To: [hidden email]
> Subject: NIFI-271 / refactored parent / checkstyle / rat / findbugs
>
> Hello All,
>
> I believe NIFI-271 [1] is now resolved.  In it we:
>
> 1) completed the parent DRY refactoring identified by Benson a while ago
>
> 2) introduced checkstyle which is always running now.  Refactored code
> and cleaned up 10s of thousands of violations.  This should help with
> whitespace and other consistency issues that made reviews tougher.
> Might be some IDE lessons learned folks need to share.
>
> 3) cleaned up RAT logic and pushed special cases to their specific
> modules.  I'd like RAT to always run but we had to keep it in a
> profile because it made build times way too long on some platforms.
> If you want to run this just add '-Pcheck-licenses' when building.
>
> 4) Introduced a profile for convenient execution of findbugs.  Just
> add '-Pfindbugs' when building. There are many outstanding issues so
> it can't be made mandatory yet but we should work in that direction.
> We can add exclusions/filters for violations we disagree with in a
> module specific manner.
>
> So to get up to date on the latest goodness:
>
> git checkout develop
> git pull
> git clean -ffd
> cd nifi-parent
> mvn clean install
> cd ../nifi-nar-maven-plugin
> mvn clean install
> cd ../nifi
> mvn -T C1 clean install
>
> Please advise if you run into any trouble.
>
> [1] https://issues.apache.org/jira/browse/NIFI-271
>
> Thanks
> Joe
Reply | Threaded
Open this post in threaded view
|

Re: NIFI-271 / refactored parent / checkstyle / rat / findbugs

Aldrin Piri
I think the creation of the IDE profiles is an important one to minimize
impedance as much as possible in getting contributions.

We should have profiles available that work out of the box as much as
possible for Eclipse, Netbeans, and IntelliJ.  If someone wants to start
tackling this, please feel free to create the associated ticket(s).
Otherwise, I'll look into doing so this evening.

On Tue, Apr 28, 2015 at 8:05 AM, Joe Witt <[hidden email]> wrote:

> Not that I am aware of.  Netbeans defaults I believe work well.  Will
> defer to others for eclipse ideas.
>
> On Tue, Apr 28, 2015 at 8:29 AM, Dan Bress <[hidden email]>
> wrote:
> > Has anyone created eclipse/netbeans formatting rules that we can use so
> that "control-shift-f" and "alt-shift-f" format code that adheres to the
> checkstyle rules?
> >
> > Dan Bress
> > Software Engineer
> > ONYX Consulting Services
> >
> > ________________________________________
> > From: Joe Witt <[hidden email]>
> > Sent: Tuesday, April 28, 2015 1:50 AM
> > To: [hidden email]
> > Subject: NIFI-271 / refactored parent / checkstyle / rat / findbugs
> >
> > Hello All,
> >
> > I believe NIFI-271 [1] is now resolved.  In it we:
> >
> > 1) completed the parent DRY refactoring identified by Benson a while ago
> >
> > 2) introduced checkstyle which is always running now.  Refactored code
> > and cleaned up 10s of thousands of violations.  This should help with
> > whitespace and other consistency issues that made reviews tougher.
> > Might be some IDE lessons learned folks need to share.
> >
> > 3) cleaned up RAT logic and pushed special cases to their specific
> > modules.  I'd like RAT to always run but we had to keep it in a
> > profile because it made build times way too long on some platforms.
> > If you want to run this just add '-Pcheck-licenses' when building.
> >
> > 4) Introduced a profile for convenient execution of findbugs.  Just
> > add '-Pfindbugs' when building. There are many outstanding issues so
> > it can't be made mandatory yet but we should work in that direction.
> > We can add exclusions/filters for violations we disagree with in a
> > module specific manner.
> >
> > So to get up to date on the latest goodness:
> >
> > git checkout develop
> > git pull
> > git clean -ffd
> > cd nifi-parent
> > mvn clean install
> > cd ../nifi-nar-maven-plugin
> > mvn clean install
> > cd ../nifi
> > mvn -T C1 clean install
> >
> > Please advise if you run into any trouble.
> >
> > [1] https://issues.apache.org/jira/browse/NIFI-271
> >
> > Thanks
> > Joe
>
Reply | Threaded
Open this post in threaded view
|

Re: NIFI-271 / refactored parent / checkstyle / rat / findbugs

BRANDON DEVRIES-2
All,

I created a ticket for this[1] and attached an initial eclipse profile.  After we get some wider input, opinions, should this end up in the developer section of the website?

Brandon

[1] https://issues.apache.org/jira/browse/NIFI-568 <https://issues.apache.org/jira/browse/NIFI-568>


> On Apr 28, 2015, at 3:01 PM, Aldrin Piri <[hidden email]> wrote:
>
>>> created eclipse/netbeans formatting rules

Reply | Threaded
Open this post in threaded view
|

Re: NIFI-271 / refactored parent / checkstyle / rat / findbugs

Mark Payne
Brandon,

I would vote yes, we will want the Eclipse formatter on the dev site. I
also have several Eclipse Code Templates that make writing processors
much easier that I would want to put there. Maybe some sort of
"Developer Resources" page.

Thanks
-Mark

------ Original Message ------
From: "BRANDON DEVRIES" <[hidden email]>
To: [hidden email]
Sent: 5/1/2015 10:25:25 AM
Subject: Re: NIFI-271 / refactored parent / checkstyle / rat / findbugs

>All,
>
>I created a ticket for this[1] and attached an initial eclipse profile.
>After we get some wider input, opinions, should this end up in the
>developer section of the website?
>
>Brandon
>
>[1] https://issues.apache.org/jira/browse/NIFI-568 
><https://issues.apache.org/jira/browse/NIFI-568>
>
>
>>  On Apr 28, 2015, at 3:01 PM, Aldrin Piri <[hidden email]>
>>wrote:
>>
>>>>  created eclipse/netbeans formatting rules
>
Reply | Threaded
Open this post in threaded view
|

Re: NIFI-271 / refactored parent / checkstyle / rat / findbugs

Aldrin Piri
In reply to this post by BRANDON DEVRIES-2
Brandon,

Thanks for kicking this off.  Yeah, that definitely seems workable.  I have
seen other Apache projects include such files as part of the repository in
a contrib/externals folder.

On Fri, May 1, 2015 at 10:25 AM, BRANDON DEVRIES <[hidden email]>
wrote:

> All,
>
> I created a ticket for this[1] and attached an initial eclipse profile.
> After we get some wider input, opinions, should this end up in the
> developer section of the website?
>
> Brandon
>
> [1] https://issues.apache.org/jira/browse/NIFI-568 <
> https://issues.apache.org/jira/browse/NIFI-568>
>
>
> > On Apr 28, 2015, at 3:01 PM, Aldrin Piri <[hidden email]> wrote:
> >
> >>> created eclipse/netbeans formatting rules
>
>
Reply | Threaded
Open this post in threaded view
|

Re: NIFI-271 / refactored parent / checkstyle / rat / findbugs

Joe Witt
How about putting it in externals then kink to those from the website?
On May 1, 2015 7:34 AM, "Aldrin Piri" <[hidden email]> wrote:

> Brandon,
>
> Thanks for kicking this off.  Yeah, that definitely seems workable.  I have
> seen other Apache projects include such files as part of the repository in
> a contrib/externals folder.
>
> On Fri, May 1, 2015 at 10:25 AM, BRANDON DEVRIES <
> [hidden email]>
> wrote:
>
> > All,
> >
> > I created a ticket for this[1] and attached an initial eclipse profile.
> > After we get some wider input, opinions, should this end up in the
> > developer section of the website?
> >
> > Brandon
> >
> > [1] https://issues.apache.org/jira/browse/NIFI-568 <
> > https://issues.apache.org/jira/browse/NIFI-568>
> >
> >
> > > On Apr 28, 2015, at 3:01 PM, Aldrin Piri <[hidden email]> wrote:
> > >
> > >>> created eclipse/netbeans formatting rules
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: NIFI-271 / refactored parent / checkstyle / rat / findbugs

Joe Witt
All,

As Mark was working the release today he cited too many unstable build
issues.  I had observed it myself.  Basically with checkstyle we were
seeing false positives and false negatives.  Believe it is related to
its not yet supporting multi-threaded builds, not sure.  In any event
since we already had to move RAT into a profile I just moved
checkstyle to be part of that profile as well.  So there is a
'contrib-check' profile now which replaced the 'check-licenses'
profile.  This 'contrib-check' profile runs both RAT and checkstyle on
the codebase.  It should be run in a single threaded build only.

The idea here then is this profile can be run as someone wishes to
contribute, by someone performing a review, and during the release
process to validate the items of the build are correct.

This resolves: https://issues.apache.org/jira/browse/NIFI-587

Thanks
Joe

On Fri, May 1, 2015 at 10:38 AM, Joe Witt <[hidden email]> wrote:

> How about putting it in externals then kink to those from the website?
>
> On May 1, 2015 7:34 AM, "Aldrin Piri" <[hidden email]> wrote:
>>
>> Brandon,
>>
>> Thanks for kicking this off.  Yeah, that definitely seems workable.  I
>> have
>> seen other Apache projects include such files as part of the repository in
>> a contrib/externals folder.
>>
>> On Fri, May 1, 2015 at 10:25 AM, BRANDON DEVRIES
>> <[hidden email]>
>> wrote:
>>
>> > All,
>> >
>> > I created a ticket for this[1] and attached an initial eclipse profile.
>> > After we get some wider input, opinions, should this end up in the
>> > developer section of the website?
>> >
>> > Brandon
>> >
>> > [1] https://issues.apache.org/jira/browse/NIFI-568 <
>> > https://issues.apache.org/jira/browse/NIFI-568>
>> >
>> >
>> > > On Apr 28, 2015, at 3:01 PM, Aldrin Piri <[hidden email]> wrote:
>> > >
>> > >>> created eclipse/netbeans formatting rules
>> >
>> >
Reply | Threaded
Open this post in threaded view
|

Re: NIFI-271 / refactored parent / checkstyle / rat / findbugs

Mark Payne
Thanks Joe. I'll be putting together another release in just a bit.

Thanks
-Mark

Sent from my iPhone

> On May 5, 2015, at 1:06 AM, Joe Witt <[hidden email]> wrote:
>
> All,
>
> As Mark was working the release today he cited too many unstable build
> issues.  I had observed it myself.  Basically with checkstyle we were
> seeing false positives and false negatives.  Believe it is related to
> its not yet supporting multi-threaded builds, not sure.  In any event
> since we already had to move RAT into a profile I just moved
> checkstyle to be part of that profile as well.  So there is a
> 'contrib-check' profile now which replaced the 'check-licenses'
> profile.  This 'contrib-check' profile runs both RAT and checkstyle on
> the codebase.  It should be run in a single threaded build only.
>
> The idea here then is this profile can be run as someone wishes to
> contribute, by someone performing a review, and during the release
> process to validate the items of the build are correct.
>
> This resolves: https://issues.apache.org/jira/browse/NIFI-587
>
> Thanks
> Joe
>
>> On Fri, May 1, 2015 at 10:38 AM, Joe Witt <[hidden email]> wrote:
>> How about putting it in externals then kink to those from the website?
>>
>>> On May 1, 2015 7:34 AM, "Aldrin Piri" <[hidden email]> wrote:
>>>
>>> Brandon,
>>>
>>> Thanks for kicking this off.  Yeah, that definitely seems workable.  I
>>> have
>>> seen other Apache projects include such files as part of the repository in
>>> a contrib/externals folder.
>>>
>>> On Fri, May 1, 2015 at 10:25 AM, BRANDON DEVRIES
>>> <[hidden email]>
>>> wrote:
>>>
>>>> All,
>>>>
>>>> I created a ticket for this[1] and attached an initial eclipse profile.
>>>> After we get some wider input, opinions, should this end up in the
>>>> developer section of the website?
>>>>
>>>> Brandon
>>>>
>>>> [1] https://issues.apache.org/jira/browse/NIFI-568 <
>>>> https://issues.apache.org/jira/browse/NIFI-568>
>>>>
>>>>
>>>>> On Apr 28, 2015, at 3:01 PM, Aldrin Piri <[hidden email]> wrote:
>>>>>
>>>>>>> created eclipse/netbeans formatting rules
>>>>
>>>>