[DISCUSS] NiFi 1.1.0 release

classic Classic list List threaded Threaded
44 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Matt Gilman
I have two items that I would like to wrap up prior to creating an RC for
1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
configuration. The work is already completed and I will be reviewing it
this today. Additionally, following recent interest on the mailing list,
I'd like to knock out NIFI-3020. This will allow an admin to configure a
strategy for user identity when logging in via LDAP. Specifically, it will
support usage of the DN (the default and current implementation) as well as
the username the user logged in as. I should be able to have a PR up for
this work later today.

Thanks!

Matt

[1] https://issues.apache.org/jira/browse/NIFI-2949
[2] https://issues.apache.org/jira/browse/NIFI-3020


On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:

> The code is within the twitter4j library itself.  I filed a request to
> twitter4jg.  The most likely case is we will need to submit a PR to them.
> However, I don't see this as something that should delay the release.  We
> can provide instructions for folks wanting to use the processor during the
> time we cannot make it available in a convenient manner.  I will provide a
> meaningful comment about this in release notes and pointers on what folks
> can do in the meantime.
>
> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:
>
> > I understand there was a discussion thread within the NiFi community for
> > this as well and I missed responding to that at that time. It just seems
> to
> > me like JSON processing is necessary for GetTwitter, which is incredibly
> > useful for demonstrating NiFi’s ability to read from a high volume stream
> > out of the box. With NIFI-3019 (Remove GetTwitter from default build), is
> > there any related effort to substitute an acceptable replacement JSON
> > library to restore this functionality?
> >
> > [1] https://issues.apache.org/jira/browse/NIFI-3019
> >
> > Andy LoPresto
> > [hidden email]
> > *[hidden email] <[hidden email]>*
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]> wrote:
> >
> > I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
> > NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
> > investigated some alternate TLS config options for the new version of the
> > client library.
> >
> > Is there any alternative to excluding the GetTwitter processor? Using
> > Johnzon [1] or the Android re-implementation [2] discussed in the mailing
> > list thread?
> >
> > [1] https://johnzon.apache.org/
> > [2] https://developer.android.com/reference/org/json/package-
> summary.html
> >
> >
> > Andy LoPresto
> > [hidden email]
> > *[hidden email] <[hidden email]>*
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
> >
> > Team
> >
> > Very happy to see that we are down to three items remaining tagged to
> > 1.1.0.  Solid effort over the recent weeks to close the gap including
> work
> > to get past the now category x Jason dependency we had.  The most notable
> > impact from that is the wildly popular GetTwitter processor, the fav new
> > nifi user and demo processor, can no longer be included in the default
> > build.  It is optionally available if users choose to build and use it
> but
> > we won't distribute binaries that have it.
> >
> > I see some review movement on some patch available but untagged items.
> >
> > I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
> > have any outstanding items?
> >
> > Thanks
> > Joe
> >
> > On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
> >
> > Ryan
> >
> > Not officially but I think we should try to close this thing out and
> > start a vote in the next week or two at most.
> >
> > I'm going through the tickets again now.  There is also a new issue of
> > the json-p license falling out of favor in Apache legal terms and
> > becoming Category-X.  Am looking into that now.
> >
> > Thanks
> > Joe
> >
> > On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]> wrote:
> >
> > Joe - Is there a target date for 1.1?
> >
> > On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:
> >
> > Team,
> >
> > Just an update on things with working toward an Apache NiFi 1.1.0
> > release.  There are still about 33 JIRAs there now and some are
> > awaiting review and are some are under active progress. Yet there is
> > good traction and progress. I think we should just stay vigilant with
> > what makes it in and keep working it down.  So let's please shoot for
> > a couple weeks from now.  If it is ready sooner I'll jump on it.
> >
> > Thanks
> > Joe
> >
> > On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
> >
> > Team,
> >
> > There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
> > avoiding putting more in there for now at least without a discussion.
> > Of the 31 JIRAs there the vast majority need review so we should be
> > able to close these down fairly quickly as long as we don't let the
> > list grow.
> >
> > Thanks
> > joe
> >
> > On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>
> >
> > wrote:
> >
> > Joe,
> >
> > Appreciate the offer it isn't my PR. I was just using it as an
> >
> > example.
> >
> > All
> >
> > mine are currently closed, which I greatly appreciate.
> >
> > Cheers,
> >
> > Edgardo
> >
> > On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
> >
> > Edgardo,
> >
> > You mentioned a PR from August. I'd be happy to help you work that
> > through review.
> >
> > Thanks
> > Joe
> >
> > On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
> >
> > [hidden email]
> >
> > <javascript:;>> wrote:
> >
> > I have agreed that at this point a release is important. My goal
> >
> > was
> >
> > try
> >
> > to
> >
> > squeeze in a much goodness as possible into the release, but the
> >
> > important
> >
> > bug fixes should come first. Getting 1.x into a state where the
> >
> > release
> >
> > notes don't say that it is geared toward developers and testers is
> >
> > really
> >
> > huge.
> >
> > I think Nifi is a great community otherwise I would participate in
> >
> > the
> >
> > mailing list, create Jira tickets and pull requests. I am only
> >
> > trying to
> >
> > strengthen the great thing that is going on here. We can always do
> >
> > better.
> >
> > I was not trying to put down this community only to participate and
> >
> > make
> >
> > it
> >
> > better. I think this conversation is an indication of how great
> >
> > this
> >
> > community is.
> >
> > Maybe I am being sensitive about this issue and trying to
> >
> > strengthen
> >
> > the
> >
> > nifi community even more, after coming from a conference where it
> >
> > was
> >
> > reported there was lots of excitement at first and now the
> >
> > participation
> >
> > in
> >
> > the community has really died down and they are struggling. I don't
> >
> > want
> >
> > to
> >
> > see that happen here.
> >
> > Cheers,
> >
> > Edgardo
> >
> >
> >
> >
> > On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
> >
> > <javascript:;>> wrote:
> >
> >
> > Edgardo,
> >
> > Thank you for your feedback. We hear your comments and as a
> >
> > committer I
> >
> > can
> >
> > share we are constantly looking to improve the PR process, having
> >
> > already
> >
> > taken many of the steps you suggest.
> >
> > However, it is important to notice that the number of PRs should
> >
> > not be
> >
> > seen as a metric of engagement by the development community: Most
> >
> > of us
> >
> > will submit PRs so that our work can be carefully reviewed by our
> >
> > peers
> >
> > and
> >
> > some of us will use JIRA patches to provide contributions.
> >
> > Having said that, it is true that some PRs may sit idle for a long
> >
> > time
> >
> > and
> >
> > we are working to improve this pipeline.
> >
> > It was therefore no coincidence that I  browsed most of the PRs
> >
> > performing
> >
> > a triage of items that have been superseded or diverged from the
> >
> > current
> >
> > code base.
> >
> > In fact, less than a month ago the dev team closed a number of
> >
> > stalled
> >
> > and
> >
> > superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
> >
> > Despite all the above, I think Joe has a point. The master
> >
> > contain a
> >
> > series
> >
> > of important bug fixes and suspect the community would benefit
> >
> > from
> >
> > a
> >
> > release sooner rather than later.
> >
> > Once again, thank you for your feedback and contribution. It is
> >
> > good to
> >
> > have you here.
> >
> > Andre
> >
> > On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
> >
> > [hidden email]
> >
> > <javascript:;>>
> >
> > wrote:
> >
> > Joe - You are correct I was mentioning the PRs that are
> >
> > currently
> >
> > open.
> >
> >
> > Regardless of how it happens reducing the count of open PRs I
> >
> > believe
> >
> > to
> >
> > be
> >
> > extremely important. Maybe I was hoping that the release could
> >
> > be
> >
> > a
> >
> > forcing
> >
> > function to make that happen. I believe that developers are more
> >
> > willing
> >
> > to
> >
> > contribute when they see that their PRs will actually be able
> >
> > accepted
> >
> > and
> >
> > merged into the code base. Having a low number of open PRs in
> >
> > progress
> >
> > is a
> >
> > great indication that the main nifi developers are fully engaged
> >
> > with
> >
> > the
> >
> > community.
> >
> > There are a few PRs that don't have any comments from committers
> >
> > at
> >
> > all.
> >
> > I
> >
> > found one from August in that state. If that was my PR I don't
> >
> > think I
> >
> > would be so willing to put another one in anytime soon. I do get
> >
> > that
> >
> > sometime PRs get stalled by the originator, if so maybe a rule
> >
> > about
> >
> > closing them after a certain amount of time or being taken over
> >
> > by a
> >
> > core
> >
> > contributor if they think it worthwhile.
> >
> > I would like to shoutout to James Wing on my last PR he was
> >
> > quick
> >
> > to
> >
> > review, provided great comments, testing, and even some
> >
> > additional
> >
> > code.
> >
> > It
> >
> > was a great PR experience.
> >
> > Cheers,
> >
> > Edgardo
> >
> >
> >
> > On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
> >
> > [hidden email] <javascript:;>.
> >
> > invalid> wrote:
> >
> > Joe, I think you misread. Edgardo is referring to the Pull
> >
> > Requests
> >
> > that
> >
> > are currently open, not the tickets assigned to the 1.1.0
> >
> > version.
> >
> >
> > I think these goals (releasing 1.1.0 and cutting down the PR
> >
> > count)
> >
> > should
> >
> > be two different efforts. Doing a thorough job reviewing
> >
> > takes a
> >
> > significant amount of time from both the reviewer and
> >
> > contributor.
> >
> > In
> >
> > order
> >
> > to cut it down significantly would take much longer than a
> >
> > couple
> >
> > days.
> >
> >
> > Also there has already been a lot of great new features and
> >
> > bug
> >
> > fixes
> >
> > contributed to the 1.X line and I don't think it's worth
> >
> > holding up
> >
> > a
> >
> > 1.1.0
> >
> > release for tickets not assigned to this fix version. As an
> >
> > added
> >
> > bonus
> >
> > though, I think many of the tickets tagged as 1.1.0 have PRs
> >
> > already
> >
> > open
> >
> > so closing those will make a large dent in the PR count.
> >
> >
> > Joe
> >
> > - - - - - -
> > Joseph Percivall
> > linkedin.com/in/Percivall
> > e: [hidden email] <javascript:;>
> >
> >
> >
> > On Thursday, October 13, 2016 3:58 PM, Joe Witt <
> >
> > [hidden email]
> >
> > <javascript:;>>
> >
> > wrote:
> >
> >
> >
> > There are less than 30 right now.  Many of the roughly 90+
> >
> > JIRAs
> >
> > opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
> >
> > or
> >
> > just
> >
> > had fix versions removed.
> >
> > We will need to have a push over the next bunch of days to
> >
> > deal
> >
> > with
> >
> > reviewing/merging/moving the remaining items.
> >
> > Thanks
> > Joe
> >
> >
> > On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
> >
> > [hidden email] <javascript:;>>
> >
> > wrote:
> >
> > Joe,
> >
> > There are 75 PRs currently open. Why not make a push over
> >
> > the
> >
> > next
> >
> > bunch
> >
> > of
> >
> > days to get them closed and then cut the release after that.
> >
> > Cheers,
> >
> > Edgardo
> >
> > On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
> >
> > [hidden email]
> >
> > <javascript:;>>
> >
> > wrote:
> >
> >
> > Team,
> >
> > There have been a ton of bugs fixed a few nice features.  I
> >
> > would
> >
> > like
> >
> > to move to get Apache NiFi 1.1.0 release going pretty much
> >
> > based
> >
> > on
> >
> > where we are now and plan to move most tickets to a new
> >
> > Apache
> >
> > NiFi
> >
> > 1.2.0 version.  We can try to get back on our roughly 6-8
> >
> > week
> >
> > release
> >
> > schedule and shoot for a mid to late Nov release for NiFi
> >
> > 1.2.0
> >
> > this
> >
> > way as well. Please advise if anyone has any other views on
> >
> > this. In
> >
> > the mean time I'll get the wheels in motion so you'll be
> >
> > seeing a
> >
> > lot
> >
> > of JIRA/issue updates to move version around.
> >
> > Thanks
> > Joe
> >
> > On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
> >
> > [hidden email]
> >
> > <javascript:;>>
> >
> > wrote:
> >
> > Sounds good Joe. I have no issue to you doing the rm'ing
> >
> > for
> >
> > it.
> >
> >
> > On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
> >
> > <javascript:;>> wrote:
> >
> >
> > Team,
> >
> > There are a lot of great fixes and improvements on the
> >
> > master
> >
> > line
> >
> > now
> >
> > and we're at a good time window to start pushing for a
> >
> > release.
> >
> > There
> >
> > are, however, about 90+ JIRAs assigned to 1.1.0 which
> >
> > are
> >
> > open.
> >
> > I'm
> >
> > going to go through them and remove fix versions where
> >
> > appropriate.
> >
> >
> > I'm happy to take on RM task for this release though if
> >
> > someone
> >
> > else
> >
> > would like to take that on please advise.
> >
> > Thanks
> > Joe
> >
> >
> >
> >
> >
> > --
> > Cheers,
> >
> > Edgardo
> >
> >
> >
> >
> >
> > --
> > Cheers,
> >
> > Edgardo
> >
> >
> >
> >
> >
> > --
> > Cheers,
> >
> > Edgardo
> >
> >
> >
> >
> > --
> > Cheers,
> >
> > Edgardo
> >
> > Sent from Gmail Mobile
> >
> >
> >
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Bryan Bende
I've noticed an issue with the per-instance class loading capability
introduced in NIFI-2909 where the additional classpath resources can get
incorrectly removed from the class loader.

I was able to reproduced this with a unit test and have a fix ready. I
believe this is important and needs to go in for the 1.1 release, going to
re-open NIFI-2909 and submit a PR shortly.

-Bryan

On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
wrote:

> I have two items that I would like to wrap up prior to creating an RC for
> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
> configuration. The work is already completed and I will be reviewing it
> this today. Additionally, following recent interest on the mailing list,
> I'd like to knock out NIFI-3020. This will allow an admin to configure a
> strategy for user identity when logging in via LDAP. Specifically, it will
> support usage of the DN (the default and current implementation) as well as
> the username the user logged in as. I should be able to have a PR up for
> this work later today.
>
> Thanks!
>
> Matt
>
> [1] https://issues.apache.org/jira/browse/NIFI-2949
> [2] https://issues.apache.org/jira/browse/NIFI-3020
>
>
> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
>
> > The code is within the twitter4j library itself.  I filed a request to
> > twitter4jg.  The most likely case is we will need to submit a PR to them.
> > However, I don't see this as something that should delay the release.  We
> > can provide instructions for folks wanting to use the processor during
> the
> > time we cannot make it available in a convenient manner.  I will provide
> a
> > meaningful comment about this in release notes and pointers on what folks
> > can do in the meantime.
> >
> > On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:
> >
> > > I understand there was a discussion thread within the NiFi community
> for
> > > this as well and I missed responding to that at that time. It just
> seems
> > to
> > > me like JSON processing is necessary for GetTwitter, which is
> incredibly
> > > useful for demonstrating NiFi’s ability to read from a high volume
> stream
> > > out of the box. With NIFI-3019 (Remove GetTwitter from default build),
> is
> > > there any related effort to substitute an acceptable replacement JSON
> > > library to restore this functionality?
> > >
> > > [1] https://issues.apache.org/jira/browse/NIFI-3019
> > >
> > > Andy LoPresto
> > > [hidden email]
> > > *[hidden email] <[hidden email]>*
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >
> > > On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
> wrote:
> > >
> > > I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
> > > NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
> > > investigated some alternate TLS config options for the new version of
> the
> > > client library.
> > >
> > > Is there any alternative to excluding the GetTwitter processor? Using
> > > Johnzon [1] or the Android re-implementation [2] discussed in the
> mailing
> > > list thread?
> > >
> > > [1] https://johnzon.apache.org/
> > > [2] https://developer.android.com/reference/org/json/package-
> > summary.html
> > >
> > >
> > > Andy LoPresto
> > > [hidden email]
> > > *[hidden email] <[hidden email]>*
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >
> > > On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
> > >
> > > Team
> > >
> > > Very happy to see that we are down to three items remaining tagged to
> > > 1.1.0.  Solid effort over the recent weeks to close the gap including
> > work
> > > to get past the now category x Jason dependency we had.  The most
> notable
> > > impact from that is the wildly popular GetTwitter processor, the fav
> new
> > > nifi user and demo processor, can no longer be included in the default
> > > build.  It is optionally available if users choose to build and use it
> > but
> > > we won't distribute binaries that have it.
> > >
> > > I see some review movement on some patch available but untagged items.
> > >
> > > I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
> > > have any outstanding items?
> > >
> > > Thanks
> > > Joe
> > >
> > > On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
> > >
> > > Ryan
> > >
> > > Not officially but I think we should try to close this thing out and
> > > start a vote in the next week or two at most.
> > >
> > > I'm going through the tickets again now.  There is also a new issue of
> > > the json-p license falling out of favor in Apache legal terms and
> > > becoming Category-X.  Am looking into that now.
> > >
> > > Thanks
> > > Joe
> > >
> > > On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
> wrote:
> > >
> > > Joe - Is there a target date for 1.1?
> > >
> > > On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:
> > >
> > > Team,
> > >
> > > Just an update on things with working toward an Apache NiFi 1.1.0
> > > release.  There are still about 33 JIRAs there now and some are
> > > awaiting review and are some are under active progress. Yet there is
> > > good traction and progress. I think we should just stay vigilant with
> > > what makes it in and keep working it down.  So let's please shoot for
> > > a couple weeks from now.  If it is ready sooner I'll jump on it.
> > >
> > > Thanks
> > > Joe
> > >
> > > On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
> > >
> > > Team,
> > >
> > > There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
> > > avoiding putting more in there for now at least without a discussion.
> > > Of the 31 JIRAs there the vast majority need review so we should be
> > > able to close these down fairly quickly as long as we don't let the
> > > list grow.
> > >
> > > Thanks
> > > joe
> > >
> > > On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>
> > >
> > > wrote:
> > >
> > > Joe,
> > >
> > > Appreciate the offer it isn't my PR. I was just using it as an
> > >
> > > example.
> > >
> > > All
> > >
> > > mine are currently closed, which I greatly appreciate.
> > >
> > > Cheers,
> > >
> > > Edgardo
> > >
> > > On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
> > >
> > > Edgardo,
> > >
> > > You mentioned a PR from August. I'd be happy to help you work that
> > > through review.
> > >
> > > Thanks
> > > Joe
> > >
> > > On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
> > >
> > > [hidden email]
> > >
> > > <javascript:;>> wrote:
> > >
> > > I have agreed that at this point a release is important. My goal
> > >
> > > was
> > >
> > > try
> > >
> > > to
> > >
> > > squeeze in a much goodness as possible into the release, but the
> > >
> > > important
> > >
> > > bug fixes should come first. Getting 1.x into a state where the
> > >
> > > release
> > >
> > > notes don't say that it is geared toward developers and testers is
> > >
> > > really
> > >
> > > huge.
> > >
> > > I think Nifi is a great community otherwise I would participate in
> > >
> > > the
> > >
> > > mailing list, create Jira tickets and pull requests. I am only
> > >
> > > trying to
> > >
> > > strengthen the great thing that is going on here. We can always do
> > >
> > > better.
> > >
> > > I was not trying to put down this community only to participate and
> > >
> > > make
> > >
> > > it
> > >
> > > better. I think this conversation is an indication of how great
> > >
> > > this
> > >
> > > community is.
> > >
> > > Maybe I am being sensitive about this issue and trying to
> > >
> > > strengthen
> > >
> > > the
> > >
> > > nifi community even more, after coming from a conference where it
> > >
> > > was
> > >
> > > reported there was lots of excitement at first and now the
> > >
> > > participation
> > >
> > > in
> > >
> > > the community has really died down and they are struggling. I don't
> > >
> > > want
> > >
> > > to
> > >
> > > see that happen here.
> > >
> > > Cheers,
> > >
> > > Edgardo
> > >
> > >
> > >
> > >
> > > On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
> > >
> > > <javascript:;>> wrote:
> > >
> > >
> > > Edgardo,
> > >
> > > Thank you for your feedback. We hear your comments and as a
> > >
> > > committer I
> > >
> > > can
> > >
> > > share we are constantly looking to improve the PR process, having
> > >
> > > already
> > >
> > > taken many of the steps you suggest.
> > >
> > > However, it is important to notice that the number of PRs should
> > >
> > > not be
> > >
> > > seen as a metric of engagement by the development community: Most
> > >
> > > of us
> > >
> > > will submit PRs so that our work can be carefully reviewed by our
> > >
> > > peers
> > >
> > > and
> > >
> > > some of us will use JIRA patches to provide contributions.
> > >
> > > Having said that, it is true that some PRs may sit idle for a long
> > >
> > > time
> > >
> > > and
> > >
> > > we are working to improve this pipeline.
> > >
> > > It was therefore no coincidence that I  browsed most of the PRs
> > >
> > > performing
> > >
> > > a triage of items that have been superseded or diverged from the
> > >
> > > current
> > >
> > > code base.
> > >
> > > In fact, less than a month ago the dev team closed a number of
> > >
> > > stalled
> > >
> > > and
> > >
> > > superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
> > >
> > > Despite all the above, I think Joe has a point. The master
> > >
> > > contain a
> > >
> > > series
> > >
> > > of important bug fixes and suspect the community would benefit
> > >
> > > from
> > >
> > > a
> > >
> > > release sooner rather than later.
> > >
> > > Once again, thank you for your feedback and contribution. It is
> > >
> > > good to
> > >
> > > have you here.
> > >
> > > Andre
> > >
> > > On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
> > >
> > > [hidden email]
> > >
> > > <javascript:;>>
> > >
> > > wrote:
> > >
> > > Joe - You are correct I was mentioning the PRs that are
> > >
> > > currently
> > >
> > > open.
> > >
> > >
> > > Regardless of how it happens reducing the count of open PRs I
> > >
> > > believe
> > >
> > > to
> > >
> > > be
> > >
> > > extremely important. Maybe I was hoping that the release could
> > >
> > > be
> > >
> > > a
> > >
> > > forcing
> > >
> > > function to make that happen. I believe that developers are more
> > >
> > > willing
> > >
> > > to
> > >
> > > contribute when they see that their PRs will actually be able
> > >
> > > accepted
> > >
> > > and
> > >
> > > merged into the code base. Having a low number of open PRs in
> > >
> > > progress
> > >
> > > is a
> > >
> > > great indication that the main nifi developers are fully engaged
> > >
> > > with
> > >
> > > the
> > >
> > > community.
> > >
> > > There are a few PRs that don't have any comments from committers
> > >
> > > at
> > >
> > > all.
> > >
> > > I
> > >
> > > found one from August in that state. If that was my PR I don't
> > >
> > > think I
> > >
> > > would be so willing to put another one in anytime soon. I do get
> > >
> > > that
> > >
> > > sometime PRs get stalled by the originator, if so maybe a rule
> > >
> > > about
> > >
> > > closing them after a certain amount of time or being taken over
> > >
> > > by a
> > >
> > > core
> > >
> > > contributor if they think it worthwhile.
> > >
> > > I would like to shoutout to James Wing on my last PR he was
> > >
> > > quick
> > >
> > > to
> > >
> > > review, provided great comments, testing, and even some
> > >
> > > additional
> > >
> > > code.
> > >
> > > It
> > >
> > > was a great PR experience.
> > >
> > > Cheers,
> > >
> > > Edgardo
> > >
> > >
> > >
> > > On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
> > >
> > > [hidden email] <javascript:;>.
> > >
> > > invalid> wrote:
> > >
> > > Joe, I think you misread. Edgardo is referring to the Pull
> > >
> > > Requests
> > >
> > > that
> > >
> > > are currently open, not the tickets assigned to the 1.1.0
> > >
> > > version.
> > >
> > >
> > > I think these goals (releasing 1.1.0 and cutting down the PR
> > >
> > > count)
> > >
> > > should
> > >
> > > be two different efforts. Doing a thorough job reviewing
> > >
> > > takes a
> > >
> > > significant amount of time from both the reviewer and
> > >
> > > contributor.
> > >
> > > In
> > >
> > > order
> > >
> > > to cut it down significantly would take much longer than a
> > >
> > > couple
> > >
> > > days.
> > >
> > >
> > > Also there has already been a lot of great new features and
> > >
> > > bug
> > >
> > > fixes
> > >
> > > contributed to the 1.X line and I don't think it's worth
> > >
> > > holding up
> > >
> > > a
> > >
> > > 1.1.0
> > >
> > > release for tickets not assigned to this fix version. As an
> > >
> > > added
> > >
> > > bonus
> > >
> > > though, I think many of the tickets tagged as 1.1.0 have PRs
> > >
> > > already
> > >
> > > open
> > >
> > > so closing those will make a large dent in the PR count.
> > >
> > >
> > > Joe
> > >
> > > - - - - - -
> > > Joseph Percivall
> > > linkedin.com/in/Percivall
> > > e: [hidden email] <javascript:;>
> > >
> > >
> > >
> > > On Thursday, October 13, 2016 3:58 PM, Joe Witt <
> > >
> > > [hidden email]
> > >
> > > <javascript:;>>
> > >
> > > wrote:
> > >
> > >
> > >
> > > There are less than 30 right now.  Many of the roughly 90+
> > >
> > > JIRAs
> > >
> > > opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
> > >
> > > or
> > >
> > > just
> > >
> > > had fix versions removed.
> > >
> > > We will need to have a push over the next bunch of days to
> > >
> > > deal
> > >
> > > with
> > >
> > > reviewing/merging/moving the remaining items.
> > >
> > > Thanks
> > > Joe
> > >
> > >
> > > On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
> > >
> > > [hidden email] <javascript:;>>
> > >
> > > wrote:
> > >
> > > Joe,
> > >
> > > There are 75 PRs currently open. Why not make a push over
> > >
> > > the
> > >
> > > next
> > >
> > > bunch
> > >
> > > of
> > >
> > > days to get them closed and then cut the release after that.
> > >
> > > Cheers,
> > >
> > > Edgardo
> > >
> > > On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
> > >
> > > [hidden email]
> > >
> > > <javascript:;>>
> > >
> > > wrote:
> > >
> > >
> > > Team,
> > >
> > > There have been a ton of bugs fixed a few nice features.  I
> > >
> > > would
> > >
> > > like
> > >
> > > to move to get Apache NiFi 1.1.0 release going pretty much
> > >
> > > based
> > >
> > > on
> > >
> > > where we are now and plan to move most tickets to a new
> > >
> > > Apache
> > >
> > > NiFi
> > >
> > > 1.2.0 version.  We can try to get back on our roughly 6-8
> > >
> > > week
> > >
> > > release
> > >
> > > schedule and shoot for a mid to late Nov release for NiFi
> > >
> > > 1.2.0
> > >
> > > this
> > >
> > > way as well. Please advise if anyone has any other views on
> > >
> > > this. In
> > >
> > > the mean time I'll get the wheels in motion so you'll be
> > >
> > > seeing a
> > >
> > > lot
> > >
> > > of JIRA/issue updates to move version around.
> > >
> > > Thanks
> > > Joe
> > >
> > > On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
> > >
> > > [hidden email]
> > >
> > > <javascript:;>>
> > >
> > > wrote:
> > >
> > > Sounds good Joe. I have no issue to you doing the rm'ing
> > >
> > > for
> > >
> > > it.
> > >
> > >
> > > On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
> > >
> > > <javascript:;>> wrote:
> > >
> > >
> > > Team,
> > >
> > > There are a lot of great fixes and improvements on the
> > >
> > > master
> > >
> > > line
> > >
> > > now
> > >
> > > and we're at a good time window to start pushing for a
> > >
> > > release.
> > >
> > > There
> > >
> > > are, however, about 90+ JIRAs assigned to 1.1.0 which
> > >
> > > are
> > >
> > > open.
> > >
> > > I'm
> > >
> > > going to go through them and remove fix versions where
> > >
> > > appropriate.
> > >
> > >
> > > I'm happy to take on RM task for this release though if
> > >
> > > someone
> > >
> > > else
> > >
> > > would like to take that on please advise.
> > >
> > > Thanks
> > > Joe
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Cheers,
> > >
> > > Edgardo
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Cheers,
> > >
> > > Edgardo
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Cheers,
> > >
> > > Edgardo
> > >
> > >
> > >
> > >
> > > --
> > > Cheers,
> > >
> > > Edgardo
> > >
> > > Sent from Gmail Mobile
> > >
> > >
> > >
> > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Joe Witt
Team

There is a thread on apache legal-discuss that might allow for a
graceperiod of continued usage of the json library.  Am going to keep
a close eye on this and if VP Legal approves we'll be able to keep the
twitter processors in which is definitely a good thing.  Will advise

Thanks
Joe

On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]> wrote:

> I've noticed an issue with the per-instance class loading capability
> introduced in NIFI-2909 where the additional classpath resources can get
> incorrectly removed from the class loader.
>
> I was able to reproduced this with a unit test and have a fix ready. I
> believe this is important and needs to go in for the 1.1 release, going to
> re-open NIFI-2909 and submit a PR shortly.
>
> -Bryan
>
> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
> wrote:
>
>> I have two items that I would like to wrap up prior to creating an RC for
>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
>> configuration. The work is already completed and I will be reviewing it
>> this today. Additionally, following recent interest on the mailing list,
>> I'd like to knock out NIFI-3020. This will allow an admin to configure a
>> strategy for user identity when logging in via LDAP. Specifically, it will
>> support usage of the DN (the default and current implementation) as well as
>> the username the user logged in as. I should be able to have a PR up for
>> this work later today.
>>
>> Thanks!
>>
>> Matt
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>>
>>
>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
>>
>> > The code is within the twitter4j library itself.  I filed a request to
>> > twitter4jg.  The most likely case is we will need to submit a PR to them.
>> > However, I don't see this as something that should delay the release.  We
>> > can provide instructions for folks wanting to use the processor during
>> the
>> > time we cannot make it available in a convenient manner.  I will provide
>> a
>> > meaningful comment about this in release notes and pointers on what folks
>> > can do in the meantime.
>> >
>> > On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:
>> >
>> > > I understand there was a discussion thread within the NiFi community
>> for
>> > > this as well and I missed responding to that at that time. It just
>> seems
>> > to
>> > > me like JSON processing is necessary for GetTwitter, which is
>> incredibly
>> > > useful for demonstrating NiFi’s ability to read from a high volume
>> stream
>> > > out of the box. With NIFI-3019 (Remove GetTwitter from default build),
>> is
>> > > there any related effort to substitute an acceptable replacement JSON
>> > > library to restore this functionality?
>> > >
>> > > [1] https://issues.apache.org/jira/browse/NIFI-3019
>> > >
>> > > Andy LoPresto
>> > > [hidden email]
>> > > *[hidden email] <[hidden email]>*
>> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> > >
>> > > On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>> wrote:
>> > >
>> > > I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
>> > > NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>> > > investigated some alternate TLS config options for the new version of
>> the
>> > > client library.
>> > >
>> > > Is there any alternative to excluding the GetTwitter processor? Using
>> > > Johnzon [1] or the Android re-implementation [2] discussed in the
>> mailing
>> > > list thread?
>> > >
>> > > [1] https://johnzon.apache.org/
>> > > [2] https://developer.android.com/reference/org/json/package-
>> > summary.html
>> > >
>> > >
>> > > Andy LoPresto
>> > > [hidden email]
>> > > *[hidden email] <[hidden email]>*
>> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> > >
>> > > On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>> > >
>> > > Team
>> > >
>> > > Very happy to see that we are down to three items remaining tagged to
>> > > 1.1.0.  Solid effort over the recent weeks to close the gap including
>> > work
>> > > to get past the now category x Jason dependency we had.  The most
>> notable
>> > > impact from that is the wildly popular GetTwitter processor, the fav
>> new
>> > > nifi user and demo processor, can no longer be included in the default
>> > > build.  It is optionally available if users choose to build and use it
>> > but
>> > > we won't distribute binaries that have it.
>> > >
>> > > I see some review movement on some patch available but untagged items.
>> > >
>> > > I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
>> > > have any outstanding items?
>> > >
>> > > Thanks
>> > > Joe
>> > >
>> > > On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>> > >
>> > > Ryan
>> > >
>> > > Not officially but I think we should try to close this thing out and
>> > > start a vote in the next week or two at most.
>> > >
>> > > I'm going through the tickets again now.  There is also a new issue of
>> > > the json-p license falling out of favor in Apache legal terms and
>> > > becoming Category-X.  Am looking into that now.
>> > >
>> > > Thanks
>> > > Joe
>> > >
>> > > On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>> wrote:
>> > >
>> > > Joe - Is there a target date for 1.1?
>> > >
>> > > On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:
>> > >
>> > > Team,
>> > >
>> > > Just an update on things with working toward an Apache NiFi 1.1.0
>> > > release.  There are still about 33 JIRAs there now and some are
>> > > awaiting review and are some are under active progress. Yet there is
>> > > good traction and progress. I think we should just stay vigilant with
>> > > what makes it in and keep working it down.  So let's please shoot for
>> > > a couple weeks from now.  If it is ready sooner I'll jump on it.
>> > >
>> > > Thanks
>> > > Joe
>> > >
>> > > On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
>> > >
>> > > Team,
>> > >
>> > > There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
>> > > avoiding putting more in there for now at least without a discussion.
>> > > Of the 31 JIRAs there the vast majority need review so we should be
>> > > able to close these down fairly quickly as long as we don't let the
>> > > list grow.
>> > >
>> > > Thanks
>> > > joe
>> > >
>> > > On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>
>> > >
>> > > wrote:
>> > >
>> > > Joe,
>> > >
>> > > Appreciate the offer it isn't my PR. I was just using it as an
>> > >
>> > > example.
>> > >
>> > > All
>> > >
>> > > mine are currently closed, which I greatly appreciate.
>> > >
>> > > Cheers,
>> > >
>> > > Edgardo
>> > >
>> > > On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>> > >
>> > > Edgardo,
>> > >
>> > > You mentioned a PR from August. I'd be happy to help you work that
>> > > through review.
>> > >
>> > > Thanks
>> > > Joe
>> > >
>> > > On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>> > >
>> > > [hidden email]
>> > >
>> > > <javascript:;>> wrote:
>> > >
>> > > I have agreed that at this point a release is important. My goal
>> > >
>> > > was
>> > >
>> > > try
>> > >
>> > > to
>> > >
>> > > squeeze in a much goodness as possible into the release, but the
>> > >
>> > > important
>> > >
>> > > bug fixes should come first. Getting 1.x into a state where the
>> > >
>> > > release
>> > >
>> > > notes don't say that it is geared toward developers and testers is
>> > >
>> > > really
>> > >
>> > > huge.
>> > >
>> > > I think Nifi is a great community otherwise I would participate in
>> > >
>> > > the
>> > >
>> > > mailing list, create Jira tickets and pull requests. I am only
>> > >
>> > > trying to
>> > >
>> > > strengthen the great thing that is going on here. We can always do
>> > >
>> > > better.
>> > >
>> > > I was not trying to put down this community only to participate and
>> > >
>> > > make
>> > >
>> > > it
>> > >
>> > > better. I think this conversation is an indication of how great
>> > >
>> > > this
>> > >
>> > > community is.
>> > >
>> > > Maybe I am being sensitive about this issue and trying to
>> > >
>> > > strengthen
>> > >
>> > > the
>> > >
>> > > nifi community even more, after coming from a conference where it
>> > >
>> > > was
>> > >
>> > > reported there was lots of excitement at first and now the
>> > >
>> > > participation
>> > >
>> > > in
>> > >
>> > > the community has really died down and they are struggling. I don't
>> > >
>> > > want
>> > >
>> > > to
>> > >
>> > > see that happen here.
>> > >
>> > > Cheers,
>> > >
>> > > Edgardo
>> > >
>> > >
>> > >
>> > >
>> > > On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>> > >
>> > > <javascript:;>> wrote:
>> > >
>> > >
>> > > Edgardo,
>> > >
>> > > Thank you for your feedback. We hear your comments and as a
>> > >
>> > > committer I
>> > >
>> > > can
>> > >
>> > > share we are constantly looking to improve the PR process, having
>> > >
>> > > already
>> > >
>> > > taken many of the steps you suggest.
>> > >
>> > > However, it is important to notice that the number of PRs should
>> > >
>> > > not be
>> > >
>> > > seen as a metric of engagement by the development community: Most
>> > >
>> > > of us
>> > >
>> > > will submit PRs so that our work can be carefully reviewed by our
>> > >
>> > > peers
>> > >
>> > > and
>> > >
>> > > some of us will use JIRA patches to provide contributions.
>> > >
>> > > Having said that, it is true that some PRs may sit idle for a long
>> > >
>> > > time
>> > >
>> > > and
>> > >
>> > > we are working to improve this pipeline.
>> > >
>> > > It was therefore no coincidence that I  browsed most of the PRs
>> > >
>> > > performing
>> > >
>> > > a triage of items that have been superseded or diverged from the
>> > >
>> > > current
>> > >
>> > > code base.
>> > >
>> > > In fact, less than a month ago the dev team closed a number of
>> > >
>> > > stalled
>> > >
>> > > and
>> > >
>> > > superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>> > >
>> > > Despite all the above, I think Joe has a point. The master
>> > >
>> > > contain a
>> > >
>> > > series
>> > >
>> > > of important bug fixes and suspect the community would benefit
>> > >
>> > > from
>> > >
>> > > a
>> > >
>> > > release sooner rather than later.
>> > >
>> > > Once again, thank you for your feedback and contribution. It is
>> > >
>> > > good to
>> > >
>> > > have you here.
>> > >
>> > > Andre
>> > >
>> > > On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>> > >
>> > > [hidden email]
>> > >
>> > > <javascript:;>>
>> > >
>> > > wrote:
>> > >
>> > > Joe - You are correct I was mentioning the PRs that are
>> > >
>> > > currently
>> > >
>> > > open.
>> > >
>> > >
>> > > Regardless of how it happens reducing the count of open PRs I
>> > >
>> > > believe
>> > >
>> > > to
>> > >
>> > > be
>> > >
>> > > extremely important. Maybe I was hoping that the release could
>> > >
>> > > be
>> > >
>> > > a
>> > >
>> > > forcing
>> > >
>> > > function to make that happen. I believe that developers are more
>> > >
>> > > willing
>> > >
>> > > to
>> > >
>> > > contribute when they see that their PRs will actually be able
>> > >
>> > > accepted
>> > >
>> > > and
>> > >
>> > > merged into the code base. Having a low number of open PRs in
>> > >
>> > > progress
>> > >
>> > > is a
>> > >
>> > > great indication that the main nifi developers are fully engaged
>> > >
>> > > with
>> > >
>> > > the
>> > >
>> > > community.
>> > >
>> > > There are a few PRs that don't have any comments from committers
>> > >
>> > > at
>> > >
>> > > all.
>> > >
>> > > I
>> > >
>> > > found one from August in that state. If that was my PR I don't
>> > >
>> > > think I
>> > >
>> > > would be so willing to put another one in anytime soon. I do get
>> > >
>> > > that
>> > >
>> > > sometime PRs get stalled by the originator, if so maybe a rule
>> > >
>> > > about
>> > >
>> > > closing them after a certain amount of time or being taken over
>> > >
>> > > by a
>> > >
>> > > core
>> > >
>> > > contributor if they think it worthwhile.
>> > >
>> > > I would like to shoutout to James Wing on my last PR he was
>> > >
>> > > quick
>> > >
>> > > to
>> > >
>> > > review, provided great comments, testing, and even some
>> > >
>> > > additional
>> > >
>> > > code.
>> > >
>> > > It
>> > >
>> > > was a great PR experience.
>> > >
>> > > Cheers,
>> > >
>> > > Edgardo
>> > >
>> > >
>> > >
>> > > On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>> > >
>> > > [hidden email] <javascript:;>.
>> > >
>> > > invalid> wrote:
>> > >
>> > > Joe, I think you misread. Edgardo is referring to the Pull
>> > >
>> > > Requests
>> > >
>> > > that
>> > >
>> > > are currently open, not the tickets assigned to the 1.1.0
>> > >
>> > > version.
>> > >
>> > >
>> > > I think these goals (releasing 1.1.0 and cutting down the PR
>> > >
>> > > count)
>> > >
>> > > should
>> > >
>> > > be two different efforts. Doing a thorough job reviewing
>> > >
>> > > takes a
>> > >
>> > > significant amount of time from both the reviewer and
>> > >
>> > > contributor.
>> > >
>> > > In
>> > >
>> > > order
>> > >
>> > > to cut it down significantly would take much longer than a
>> > >
>> > > couple
>> > >
>> > > days.
>> > >
>> > >
>> > > Also there has already been a lot of great new features and
>> > >
>> > > bug
>> > >
>> > > fixes
>> > >
>> > > contributed to the 1.X line and I don't think it's worth
>> > >
>> > > holding up
>> > >
>> > > a
>> > >
>> > > 1.1.0
>> > >
>> > > release for tickets not assigned to this fix version. As an
>> > >
>> > > added
>> > >
>> > > bonus
>> > >
>> > > though, I think many of the tickets tagged as 1.1.0 have PRs
>> > >
>> > > already
>> > >
>> > > open
>> > >
>> > > so closing those will make a large dent in the PR count.
>> > >
>> > >
>> > > Joe
>> > >
>> > > - - - - - -
>> > > Joseph Percivall
>> > > linkedin.com/in/Percivall
>> > > e: [hidden email] <javascript:;>
>> > >
>> > >
>> > >
>> > > On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>> > >
>> > > [hidden email]
>> > >
>> > > <javascript:;>>
>> > >
>> > > wrote:
>> > >
>> > >
>> > >
>> > > There are less than 30 right now.  Many of the roughly 90+
>> > >
>> > > JIRAs
>> > >
>> > > opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>> > >
>> > > or
>> > >
>> > > just
>> > >
>> > > had fix versions removed.
>> > >
>> > > We will need to have a push over the next bunch of days to
>> > >
>> > > deal
>> > >
>> > > with
>> > >
>> > > reviewing/merging/moving the remaining items.
>> > >
>> > > Thanks
>> > > Joe
>> > >
>> > >
>> > > On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>> > >
>> > > [hidden email] <javascript:;>>
>> > >
>> > > wrote:
>> > >
>> > > Joe,
>> > >
>> > > There are 75 PRs currently open. Why not make a push over
>> > >
>> > > the
>> > >
>> > > next
>> > >
>> > > bunch
>> > >
>> > > of
>> > >
>> > > days to get them closed and then cut the release after that.
>> > >
>> > > Cheers,
>> > >
>> > > Edgardo
>> > >
>> > > On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>> > >
>> > > [hidden email]
>> > >
>> > > <javascript:;>>
>> > >
>> > > wrote:
>> > >
>> > >
>> > > Team,
>> > >
>> > > There have been a ton of bugs fixed a few nice features.  I
>> > >
>> > > would
>> > >
>> > > like
>> > >
>> > > to move to get Apache NiFi 1.1.0 release going pretty much
>> > >
>> > > based
>> > >
>> > > on
>> > >
>> > > where we are now and plan to move most tickets to a new
>> > >
>> > > Apache
>> > >
>> > > NiFi
>> > >
>> > > 1.2.0 version.  We can try to get back on our roughly 6-8
>> > >
>> > > week
>> > >
>> > > release
>> > >
>> > > schedule and shoot for a mid to late Nov release for NiFi
>> > >
>> > > 1.2.0
>> > >
>> > > this
>> > >
>> > > way as well. Please advise if anyone has any other views on
>> > >
>> > > this. In
>> > >
>> > > the mean time I'll get the wheels in motion so you'll be
>> > >
>> > > seeing a
>> > >
>> > > lot
>> > >
>> > > of JIRA/issue updates to move version around.
>> > >
>> > > Thanks
>> > > Joe
>> > >
>> > > On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>> > >
>> > > [hidden email]
>> > >
>> > > <javascript:;>>
>> > >
>> > > wrote:
>> > >
>> > > Sounds good Joe. I have no issue to you doing the rm'ing
>> > >
>> > > for
>> > >
>> > > it.
>> > >
>> > >
>> > > On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>> > >
>> > > <javascript:;>> wrote:
>> > >
>> > >
>> > > Team,
>> > >
>> > > There are a lot of great fixes and improvements on the
>> > >
>> > > master
>> > >
>> > > line
>> > >
>> > > now
>> > >
>> > > and we're at a good time window to start pushing for a
>> > >
>> > > release.
>> > >
>> > > There
>> > >
>> > > are, however, about 90+ JIRAs assigned to 1.1.0 which
>> > >
>> > > are
>> > >
>> > > open.
>> > >
>> > > I'm
>> > >
>> > > going to go through them and remove fix versions where
>> > >
>> > > appropriate.
>> > >
>> > >
>> > > I'm happy to take on RM task for this release though if
>> > >
>> > > someone
>> > >
>> > > else
>> > >
>> > > would like to take that on please advise.
>> > >
>> > > Thanks
>> > > Joe
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > Cheers,
>> > >
>> > > Edgardo
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > Cheers,
>> > >
>> > > Edgardo
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > Cheers,
>> > >
>> > > Edgardo
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > Cheers,
>> > >
>> > > Edgardo
>> > >
>> > > Sent from Gmail Mobile
>> > >
>> > >
>> > >
>> > >
>> > >
>> >
>>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Andy LoPresto-2
Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been added to my plate for this release. Coordinated with Joe Witt and they should both be included. 


Andy LoPresto
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:

Team

There is a thread on apache legal-discuss that might allow for a
graceperiod of continued usage of the json library.  Am going to keep
a close eye on this and if VP Legal approves we'll be able to keep the
twitter processors in which is definitely a good thing.  Will advise

Thanks
Joe

On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]> wrote:
I've noticed an issue with the per-instance class loading capability
introduced in NIFI-2909 where the additional classpath resources can get
incorrectly removed from the class loader.

I was able to reproduced this with a unit test and have a fix ready. I
believe this is important and needs to go in for the 1.1 release, going to
re-open NIFI-2909 and submit a PR shortly.

-Bryan

On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
wrote:

I have two items that I would like to wrap up prior to creating an RC for
1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
configuration. The work is already completed and I will be reviewing it
this today. Additionally, following recent interest on the mailing list,
I'd like to knock out NIFI-3020. This will allow an admin to configure a
strategy for user identity when logging in via LDAP. Specifically, it will
support usage of the DN (the default and current implementation) as well as
the username the user logged in as. I should be able to have a PR up for
this work later today.

Thanks!

Matt

[1] https://issues.apache.org/jira/browse/NIFI-2949
[2] https://issues.apache.org/jira/browse/NIFI-3020


On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:

The code is within the twitter4j library itself.  I filed a request to
twitter4jg.  The most likely case is we will need to submit a PR to them.
However, I don't see this as something that should delay the release.  We
can provide instructions for folks wanting to use the processor during
the
time we cannot make it available in a convenient manner.  I will provide
a
meaningful comment about this in release notes and pointers on what folks
can do in the meantime.

On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:

I understand there was a discussion thread within the NiFi community
for
this as well and I missed responding to that at that time. It just
seems
to
me like JSON processing is necessary for GetTwitter, which is
incredibly
useful for demonstrating NiFi’s ability to read from a high volume
stream
out of the box. With NIFI-3019 (Remove GetTwitter from default build),
is
there any related effort to substitute an acceptable replacement JSON
library to restore this functionality?

[1] https://issues.apache.org/jira/browse/NIFI-3019

Andy LoPresto
[hidden email]
*[hidden email] <[hidden email]>*
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
wrote:

I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
investigated some alternate TLS config options for the new version of
the
client library.

Is there any alternative to excluding the GetTwitter processor? Using
Johnzon [1] or the Android re-implementation [2] discussed in the
mailing
list thread?

[1] https://johnzon.apache.org/
[2] https://developer.android.com/reference/org/json/package-
summary.html


Andy LoPresto
[hidden email]
*[hidden email] <[hidden email]>*
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:

Team

Very happy to see that we are down to three items remaining tagged to
1.1.0.  Solid effort over the recent weeks to close the gap including
work
to get past the now category x Jason dependency we had.  The most
notable
impact from that is the wildly popular GetTwitter processor, the fav
new
nifi user and demo processor, can no longer be included in the default
build.  It is optionally available if users choose to build and use it
but
we won't distribute binaries that have it.

I see some review movement on some patch available but untagged items.

I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
have any outstanding items?

Thanks
Joe

On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:

Ryan

Not officially but I think we should try to close this thing out and
start a vote in the next week or two at most.

I'm going through the tickets again now.  There is also a new issue of
the json-p license falling out of favor in Apache legal terms and
becoming Category-X.  Am looking into that now.

Thanks
Joe

On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
wrote:

Joe - Is there a target date for 1.1?

On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:

Team,

Just an update on things with working toward an Apache NiFi 1.1.0
release.  There are still about 33 JIRAs there now and some are
awaiting review and are some are under active progress. Yet there is
good traction and progress. I think we should just stay vigilant with
what makes it in and keep working it down.  So let's please shoot for
a couple weeks from now.  If it is ready sooner I'll jump on it.

Thanks
Joe

On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:

Team,

There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
avoiding putting more in there for now at least without a discussion.
Of the 31 JIRAs there the vast majority need review so we should be
able to close these down fairly quickly as long as we don't let the
list grow.

Thanks
joe

On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>

wrote:

Joe,

Appreciate the offer it isn't my PR. I was just using it as an

example.

All

mine are currently closed, which I greatly appreciate.

Cheers,

Edgardo

On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:

Edgardo,

You mentioned a PR from August. I'd be happy to help you work that
through review.

Thanks
Joe

On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <

[hidden email]

<javascript:;>> wrote:

I have agreed that at this point a release is important. My goal

was

try

to

squeeze in a much goodness as possible into the release, but the

important

bug fixes should come first. Getting 1.x into a state where the

release

notes don't say that it is geared toward developers and testers is

really

huge.

I think Nifi is a great community otherwise I would participate in

the

mailing list, create Jira tickets and pull requests. I am only

trying to

strengthen the great thing that is going on here. We can always do

better.

I was not trying to put down this community only to participate and

make

it

better. I think this conversation is an indication of how great

this

community is.

Maybe I am being sensitive about this issue and trying to

strengthen

the

nifi community even more, after coming from a conference where it

was

reported there was lots of excitement at first and now the

participation

in

the community has really died down and they are struggling. I don't

want

to

see that happen here.

Cheers,

Edgardo




On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]

<javascript:;>> wrote:


Edgardo,

Thank you for your feedback. We hear your comments and as a

committer I

can

share we are constantly looking to improve the PR process, having

already

taken many of the steps you suggest.

However, it is important to notice that the number of PRs should

not be

seen as a metric of engagement by the development community: Most

of us

will submit PRs so that our work can be carefully reviewed by our

peers

and

some of us will use JIRA patches to provide contributions.

Having said that, it is true that some PRs may sit idle for a long

time

and

we are working to improve this pipeline.

It was therefore no coincidence that I  browsed most of the PRs

performing

a triage of items that have been superseded or diverged from the

current

code base.

In fact, less than a month ago the dev team closed a number of

stalled

and

superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).

Despite all the above, I think Joe has a point. The master

contain a

series

of important bug fixes and suspect the community would benefit

from

a

release sooner rather than later.

Once again, thank you for your feedback and contribution. It is

good to

have you here.

Andre

On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <

[hidden email]

<javascript:;>>

wrote:

Joe - You are correct I was mentioning the PRs that are

currently

open.


Regardless of how it happens reducing the count of open PRs I

believe

to

be

extremely important. Maybe I was hoping that the release could

be

a

forcing

function to make that happen. I believe that developers are more

willing

to

contribute when they see that their PRs will actually be able

accepted

and

merged into the code base. Having a low number of open PRs in

progress

is a

great indication that the main nifi developers are fully engaged

with

the

community.

There are a few PRs that don't have any comments from committers

at

all.

I

found one from August in that state. If that was my PR I don't

think I

would be so willing to put another one in anytime soon. I do get

that

sometime PRs get stalled by the originator, if so maybe a rule

about

closing them after a certain amount of time or being taken over

by a

core

contributor if they think it worthwhile.

I would like to shoutout to James Wing on my last PR he was

quick

to

review, provided great comments, testing, and even some

additional

code.

It

was a great PR experience.

Cheers,

Edgardo



On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <

[hidden email] <javascript:;>.

invalid> wrote:

Joe, I think you misread. Edgardo is referring to the Pull

Requests

that

are currently open, not the tickets assigned to the 1.1.0

version.


I think these goals (releasing 1.1.0 and cutting down the PR

count)

should

be two different efforts. Doing a thorough job reviewing

takes a

significant amount of time from both the reviewer and

contributor.

In

order

to cut it down significantly would take much longer than a

couple

days.


Also there has already been a lot of great new features and

bug

fixes

contributed to the 1.X line and I don't think it's worth

holding up

a

1.1.0

release for tickets not assigned to this fix version. As an

added

bonus

though, I think many of the tickets tagged as 1.1.0 have PRs

already

open

so closing those will make a large dent in the PR count.


Joe

- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: [hidden email] <javascript:;>



On Thursday, October 13, 2016 3:58 PM, Joe Witt <

[hidden email]

<javascript:;>>

wrote:



There are less than 30 right now.  Many of the roughly 90+

JIRAs

opened on 1.1.0 were easily dispositioned to 1.2.0 or closed

or

just

had fix versions removed.

We will need to have a push over the next bunch of days to

deal

with

reviewing/merging/moving the remaining items.

Thanks
Joe


On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <

[hidden email] <javascript:;>>

wrote:

Joe,

There are 75 PRs currently open. Why not make a push over

the

next

bunch

of

days to get them closed and then cut the release after that.

Cheers,

Edgardo

On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <

[hidden email]

<javascript:;>>

wrote:


Team,

There have been a ton of bugs fixed a few nice features.  I

would

like

to move to get Apache NiFi 1.1.0 release going pretty much

based

on

where we are now and plan to move most tickets to a new

Apache

NiFi

1.2.0 version.  We can try to get back on our roughly 6-8

week

release

schedule and shoot for a mid to late Nov release for NiFi

1.2.0

this

way as well. Please advise if anyone has any other views on

this. In

the mean time I'll get the wheels in motion so you'll be

seeing a

lot

of JIRA/issue updates to move version around.

Thanks
Joe

On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <

[hidden email]

<javascript:;>>

wrote:

Sounds good Joe. I have no issue to you doing the rm'ing

for

it.


On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]

<javascript:;>> wrote:


Team,

There are a lot of great fixes and improvements on the

master

line

now

and we're at a good time window to start pushing for a

release.

There

are, however, about 90+ JIRAs assigned to 1.1.0 which

are

open.

I'm

going to go through them and remove fix versions where

appropriate.


I'm happy to take on RM task for this release though if

someone

else

would like to take that on please advise.

Thanks
Joe





--
Cheers,

Edgardo





--
Cheers,

Edgardo





--
Cheers,

Edgardo




--
Cheers,

Edgardo

Sent from Gmail Mobile









signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Andre
Andy,

Great to see NIFI-3050 implemented and certainly good news that NiFi 1.1.0
is set to include a number of security related improvements.



On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]> wrote:

> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been
> added to my plate for this release. Coordinated with Joe Witt and they
> should both be included.
>
> [1] https://issues.apache.org/jira/browse/NIFI-3050
> [2] https://issues.apache.org/jira/browse/NIFI-3051
>
> Andy LoPresto
> [hidden email]
> *[hidden email] <[hidden email]>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
>
> Team
>
> There is a thread on apache legal-discuss that might allow for a
> graceperiod of continued usage of the json library.  Am going to keep
> a close eye on this and if VP Legal approves we'll be able to keep the
> twitter processors in which is definitely a good thing.  Will advise
>
> Thanks
> Joe
>
> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]> wrote:
>
> I've noticed an issue with the per-instance class loading capability
> introduced in NIFI-2909 where the additional classpath resources can get
> incorrectly removed from the class loader.
>
> I was able to reproduced this with a unit test and have a fix ready. I
> believe this is important and needs to go in for the 1.1 release, going to
> re-open NIFI-2909 and submit a PR shortly.
>
> -Bryan
>
> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
> wrote:
>
> I have two items that I would like to wrap up prior to creating an RC for
> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
> configuration. The work is already completed and I will be reviewing it
> this today. Additionally, following recent interest on the mailing list,
> I'd like to knock out NIFI-3020. This will allow an admin to configure a
> strategy for user identity when logging in via LDAP. Specifically, it will
> support usage of the DN (the default and current implementation) as well as
> the username the user logged in as. I should be able to have a PR up for
> this work later today.
>
> Thanks!
>
> Matt
>
> [1] https://issues.apache.org/jira/browse/NIFI-2949
> [2] https://issues.apache.org/jira/browse/NIFI-3020
>
>
> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
>
> The code is within the twitter4j library itself.  I filed a request to
> twitter4jg.  The most likely case is we will need to submit a PR to them.
> However, I don't see this as something that should delay the release.  We
> can provide instructions for folks wanting to use the processor during
>
> the
>
> time we cannot make it available in a convenient manner.  I will provide
>
> a
>
> meaningful comment about this in release notes and pointers on what folks
> can do in the meantime.
>
> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:
>
> I understand there was a discussion thread within the NiFi community
>
> for
>
> this as well and I missed responding to that at that time. It just
>
> seems
>
> to
>
> me like JSON processing is necessary for GetTwitter, which is
>
> incredibly
>
> useful for demonstrating NiFi’s ability to read from a high volume
>
> stream
>
> out of the box. With NIFI-3019 (Remove GetTwitter from default build),
>
> is
>
> there any related effort to substitute an acceptable replacement JSON
> library to restore this functionality?
>
> [1] https://issues.apache.org/jira/browse/NIFI-3019
>
> Andy LoPresto
> [hidden email]
> *[hidden email] <[hidden email]>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>
> wrote:
>
>
> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
> investigated some alternate TLS config options for the new version of
>
> the
>
> client library.
>
> Is there any alternative to excluding the GetTwitter processor? Using
> Johnzon [1] or the Android re-implementation [2] discussed in the
>
> mailing
>
> list thread?
>
> [1] https://johnzon.apache.org/
> [2] https://developer.android.com/reference/org/json/package-
>
> summary.html
>
>
>
> Andy LoPresto
> [hidden email]
> *[hidden email] <[hidden email]>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>
> Team
>
> Very happy to see that we are down to three items remaining tagged to
> 1.1.0.  Solid effort over the recent weeks to close the gap including
>
> work
>
> to get past the now category x Jason dependency we had.  The most
>
> notable
>
> impact from that is the wildly popular GetTwitter processor, the fav
>
> new
>
> nifi user and demo processor, can no longer be included in the default
> build.  It is optionally available if users choose to build and use it
>
> but
>
> we won't distribute binaries that have it.
>
> I see some review movement on some patch available but untagged items.
>
> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
> have any outstanding items?
>
> Thanks
> Joe
>
> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>
> Ryan
>
> Not officially but I think we should try to close this thing out and
> start a vote in the next week or two at most.
>
> I'm going through the tickets again now.  There is also a new issue of
> the json-p license falling out of favor in Apache legal terms and
> becoming Category-X.  Am looking into that now.
>
> Thanks
> Joe
>
> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>
> wrote:
>
>
> Joe - Is there a target date for 1.1?
>
> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:
>
> Team,
>
> Just an update on things with working toward an Apache NiFi 1.1.0
> release.  There are still about 33 JIRAs there now and some are
> awaiting review and are some are under active progress. Yet there is
> good traction and progress. I think we should just stay vigilant with
> what makes it in and keep working it down.  So let's please shoot for
> a couple weeks from now.  If it is ready sooner I'll jump on it.
>
> Thanks
> Joe
>
> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
>
> Team,
>
> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
> avoiding putting more in there for now at least without a discussion.
> Of the 31 JIRAs there the vast majority need review so we should be
> able to close these down fairly quickly as long as we don't let the
> list grow.
>
> Thanks
> joe
>
> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>
>
> wrote:
>
> Joe,
>
> Appreciate the offer it isn't my PR. I was just using it as an
>
> example.
>
> All
>
> mine are currently closed, which I greatly appreciate.
>
> Cheers,
>
> Edgardo
>
> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>
> Edgardo,
>
> You mentioned a PR from August. I'd be happy to help you work that
> through review.
>
> Thanks
> Joe
>
> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>
> [hidden email]
>
> <javascript:;>> wrote:
>
> I have agreed that at this point a release is important. My goal
>
> was
>
> try
>
> to
>
> squeeze in a much goodness as possible into the release, but the
>
> important
>
> bug fixes should come first. Getting 1.x into a state where the
>
> release
>
> notes don't say that it is geared toward developers and testers is
>
> really
>
> huge.
>
> I think Nifi is a great community otherwise I would participate in
>
> the
>
> mailing list, create Jira tickets and pull requests. I am only
>
> trying to
>
> strengthen the great thing that is going on here. We can always do
>
> better.
>
> I was not trying to put down this community only to participate and
>
> make
>
> it
>
> better. I think this conversation is an indication of how great
>
> this
>
> community is.
>
> Maybe I am being sensitive about this issue and trying to
>
> strengthen
>
> the
>
> nifi community even more, after coming from a conference where it
>
> was
>
> reported there was lots of excitement at first and now the
>
> participation
>
> in
>
> the community has really died down and they are struggling. I don't
>
> want
>
> to
>
> see that happen here.
>
> Cheers,
>
> Edgardo
>
>
>
>
> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>
> <javascript:;>> wrote:
>
>
> Edgardo,
>
> Thank you for your feedback. We hear your comments and as a
>
> committer I
>
> can
>
> share we are constantly looking to improve the PR process, having
>
> already
>
> taken many of the steps you suggest.
>
> However, it is important to notice that the number of PRs should
>
> not be
>
> seen as a metric of engagement by the development community: Most
>
> of us
>
> will submit PRs so that our work can be carefully reviewed by our
>
> peers
>
> and
>
> some of us will use JIRA patches to provide contributions.
>
> Having said that, it is true that some PRs may sit idle for a long
>
> time
>
> and
>
> we are working to improve this pipeline.
>
> It was therefore no coincidence that I  browsed most of the PRs
>
> performing
>
> a triage of items that have been superseded or diverged from the
>
> current
>
> code base.
>
> In fact, less than a month ago the dev team closed a number of
>
> stalled
>
> and
>
> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>
> Despite all the above, I think Joe has a point. The master
>
> contain a
>
> series
>
> of important bug fixes and suspect the community would benefit
>
> from
>
> a
>
> release sooner rather than later.
>
> Once again, thank you for your feedback and contribution. It is
>
> good to
>
> have you here.
>
> Andre
>
> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>
> [hidden email]
>
> <javascript:;>>
>
> wrote:
>
> Joe - You are correct I was mentioning the PRs that are
>
> currently
>
> open.
>
>
> Regardless of how it happens reducing the count of open PRs I
>
> believe
>
> to
>
> be
>
> extremely important. Maybe I was hoping that the release could
>
> be
>
> a
>
> forcing
>
> function to make that happen. I believe that developers are more
>
> willing
>
> to
>
> contribute when they see that their PRs will actually be able
>
> accepted
>
> and
>
> merged into the code base. Having a low number of open PRs in
>
> progress
>
> is a
>
> great indication that the main nifi developers are fully engaged
>
> with
>
> the
>
> community.
>
> There are a few PRs that don't have any comments from committers
>
> at
>
> all.
>
> I
>
> found one from August in that state. If that was my PR I don't
>
> think I
>
> would be so willing to put another one in anytime soon. I do get
>
> that
>
> sometime PRs get stalled by the originator, if so maybe a rule
>
> about
>
> closing them after a certain amount of time or being taken over
>
> by a
>
> core
>
> contributor if they think it worthwhile.
>
> I would like to shoutout to James Wing on my last PR he was
>
> quick
>
> to
>
> review, provided great comments, testing, and even some
>
> additional
>
> code.
>
> It
>
> was a great PR experience.
>
> Cheers,
>
> Edgardo
>
>
>
> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>
> [hidden email] <javascript:;>.
>
> invalid> wrote:
>
> Joe, I think you misread. Edgardo is referring to the Pull
>
> Requests
>
> that
>
> are currently open, not the tickets assigned to the 1.1.0
>
> version.
>
>
> I think these goals (releasing 1.1.0 and cutting down the PR
>
> count)
>
> should
>
> be two different efforts. Doing a thorough job reviewing
>
> takes a
>
> significant amount of time from both the reviewer and
>
> contributor.
>
> In
>
> order
>
> to cut it down significantly would take much longer than a
>
> couple
>
> days.
>
>
> Also there has already been a lot of great new features and
>
> bug
>
> fixes
>
> contributed to the 1.X line and I don't think it's worth
>
> holding up
>
> a
>
> 1.1.0
>
> release for tickets not assigned to this fix version. As an
>
> added
>
> bonus
>
> though, I think many of the tickets tagged as 1.1.0 have PRs
>
> already
>
> open
>
> so closing those will make a large dent in the PR count.
>
>
> Joe
>
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: [hidden email] <javascript:;>
>
>
>
> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>
> [hidden email]
>
> <javascript:;>>
>
> wrote:
>
>
>
> There are less than 30 right now.  Many of the roughly 90+
>
> JIRAs
>
> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>
> or
>
> just
>
> had fix versions removed.
>
> We will need to have a push over the next bunch of days to
>
> deal
>
> with
>
> reviewing/merging/moving the remaining items.
>
> Thanks
> Joe
>
>
> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>
> [hidden email] <javascript:;>>
>
> wrote:
>
> Joe,
>
> There are 75 PRs currently open. Why not make a push over
>
> the
>
> next
>
> bunch
>
> of
>
> days to get them closed and then cut the release after that.
>
> Cheers,
>
> Edgardo
>
> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>
> [hidden email]
>
> <javascript:;>>
>
> wrote:
>
>
> Team,
>
> There have been a ton of bugs fixed a few nice features.  I
>
> would
>
> like
>
> to move to get Apache NiFi 1.1.0 release going pretty much
>
> based
>
> on
>
> where we are now and plan to move most tickets to a new
>
> Apache
>
> NiFi
>
> 1.2.0 version.  We can try to get back on our roughly 6-8
>
> week
>
> release
>
> schedule and shoot for a mid to late Nov release for NiFi
>
> 1.2.0
>
> this
>
> way as well. Please advise if anyone has any other views on
>
> this. In
>
> the mean time I'll get the wheels in motion so you'll be
>
> seeing a
>
> lot
>
> of JIRA/issue updates to move version around.
>
> Thanks
> Joe
>
> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>
> [hidden email]
>
> <javascript:;>>
>
> wrote:
>
> Sounds good Joe. I have no issue to you doing the rm'ing
>
> for
>
> it.
>
>
> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>
> <javascript:;>> wrote:
>
>
> Team,
>
> There are a lot of great fixes and improvements on the
>
> master
>
> line
>
> now
>
> and we're at a good time window to start pushing for a
>
> release.
>
> There
>
> are, however, about 90+ JIRAs assigned to 1.1.0 which
>
> are
>
> open.
>
> I'm
>
> going to go through them and remove fix versions where
>
> appropriate.
>
>
> I'm happy to take on RM task for this release though if
>
> someone
>
> else
>
> would like to take that on please advise.
>
> Thanks
> Joe
>
>
>
>
>
> --
> Cheers,
>
> Edgardo
>
>
>
>
>
> --
> Cheers,
>
> Edgardo
>
>
>
>
>
> --
> Cheers,
>
> Edgardo
>
>
>
>
> --
> Cheers,
>
> Edgardo
>
> Sent from Gmail Mobile
>
>
>
>
>
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Joe Witt
Team,

We appear to be very close.  Andy is working NIFI-3024 but otherwise
it is focus on testing.

I'm going to prep the RC and release notes now.  Unfortunately the
twitter changes for json.org will need to remain.  Consensus forming
on the legal-discuss thread regarding a grace period has been elusive
and we're already prepared to make the right steps so we'll just need
to take that on by being empathetic to the user base.

Thanks
Joe

On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:

> Andy,
>
> Great to see NIFI-3050 implemented and certainly good news that NiFi 1.1.0
> is set to include a number of security related improvements.
>
>
>
> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]> wrote:
>
>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been
>> added to my plate for this release. Coordinated with Joe Witt and they
>> should both be included.
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-3050
>> [2] https://issues.apache.org/jira/browse/NIFI-3051
>>
>> Andy LoPresto
>> [hidden email]
>> *[hidden email] <[hidden email]>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
>>
>> Team
>>
>> There is a thread on apache legal-discuss that might allow for a
>> graceperiod of continued usage of the json library.  Am going to keep
>> a close eye on this and if VP Legal approves we'll be able to keep the
>> twitter processors in which is definitely a good thing.  Will advise
>>
>> Thanks
>> Joe
>>
>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]> wrote:
>>
>> I've noticed an issue with the per-instance class loading capability
>> introduced in NIFI-2909 where the additional classpath resources can get
>> incorrectly removed from the class loader.
>>
>> I was able to reproduced this with a unit test and have a fix ready. I
>> believe this is important and needs to go in for the 1.1 release, going to
>> re-open NIFI-2909 and submit a PR shortly.
>>
>> -Bryan
>>
>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
>> wrote:
>>
>> I have two items that I would like to wrap up prior to creating an RC for
>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
>> configuration. The work is already completed and I will be reviewing it
>> this today. Additionally, following recent interest on the mailing list,
>> I'd like to knock out NIFI-3020. This will allow an admin to configure a
>> strategy for user identity when logging in via LDAP. Specifically, it will
>> support usage of the DN (the default and current implementation) as well as
>> the username the user logged in as. I should be able to have a PR up for
>> this work later today.
>>
>> Thanks!
>>
>> Matt
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>>
>>
>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
>>
>> The code is within the twitter4j library itself.  I filed a request to
>> twitter4jg.  The most likely case is we will need to submit a PR to them.
>> However, I don't see this as something that should delay the release.  We
>> can provide instructions for folks wanting to use the processor during
>>
>> the
>>
>> time we cannot make it available in a convenient manner.  I will provide
>>
>> a
>>
>> meaningful comment about this in release notes and pointers on what folks
>> can do in the meantime.
>>
>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:
>>
>> I understand there was a discussion thread within the NiFi community
>>
>> for
>>
>> this as well and I missed responding to that at that time. It just
>>
>> seems
>>
>> to
>>
>> me like JSON processing is necessary for GetTwitter, which is
>>
>> incredibly
>>
>> useful for demonstrating NiFi’s ability to read from a high volume
>>
>> stream
>>
>> out of the box. With NIFI-3019 (Remove GetTwitter from default build),
>>
>> is
>>
>> there any related effort to substitute an acceptable replacement JSON
>> library to restore this functionality?
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>>
>> Andy LoPresto
>> [hidden email]
>> *[hidden email] <[hidden email]>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>>
>> wrote:
>>
>>
>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>> investigated some alternate TLS config options for the new version of
>>
>> the
>>
>> client library.
>>
>> Is there any alternative to excluding the GetTwitter processor? Using
>> Johnzon [1] or the Android re-implementation [2] discussed in the
>>
>> mailing
>>
>> list thread?
>>
>> [1] https://johnzon.apache.org/
>> [2] https://developer.android.com/reference/org/json/package-
>>
>> summary.html
>>
>>
>>
>> Andy LoPresto
>> [hidden email]
>> *[hidden email] <[hidden email]>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>>
>> Team
>>
>> Very happy to see that we are down to three items remaining tagged to
>> 1.1.0.  Solid effort over the recent weeks to close the gap including
>>
>> work
>>
>> to get past the now category x Jason dependency we had.  The most
>>
>> notable
>>
>> impact from that is the wildly popular GetTwitter processor, the fav
>>
>> new
>>
>> nifi user and demo processor, can no longer be included in the default
>> build.  It is optionally available if users choose to build and use it
>>
>> but
>>
>> we won't distribute binaries that have it.
>>
>> I see some review movement on some patch available but untagged items.
>>
>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
>> have any outstanding items?
>>
>> Thanks
>> Joe
>>
>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>>
>> Ryan
>>
>> Not officially but I think we should try to close this thing out and
>> start a vote in the next week or two at most.
>>
>> I'm going through the tickets again now.  There is also a new issue of
>> the json-p license falling out of favor in Apache legal terms and
>> becoming Category-X.  Am looking into that now.
>>
>> Thanks
>> Joe
>>
>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>>
>> wrote:
>>
>>
>> Joe - Is there a target date for 1.1?
>>
>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:
>>
>> Team,
>>
>> Just an update on things with working toward an Apache NiFi 1.1.0
>> release.  There are still about 33 JIRAs there now and some are
>> awaiting review and are some are under active progress. Yet there is
>> good traction and progress. I think we should just stay vigilant with
>> what makes it in and keep working it down.  So let's please shoot for
>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>>
>> Thanks
>> Joe
>>
>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
>>
>> Team,
>>
>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
>> avoiding putting more in there for now at least without a discussion.
>> Of the 31 JIRAs there the vast majority need review so we should be
>> able to close these down fairly quickly as long as we don't let the
>> list grow.
>>
>> Thanks
>> joe
>>
>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>
>>
>> wrote:
>>
>> Joe,
>>
>> Appreciate the offer it isn't my PR. I was just using it as an
>>
>> example.
>>
>> All
>>
>> mine are currently closed, which I greatly appreciate.
>>
>> Cheers,
>>
>> Edgardo
>>
>> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>>
>> Edgardo,
>>
>> You mentioned a PR from August. I'd be happy to help you work that
>> through review.
>>
>> Thanks
>> Joe
>>
>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>>
>> [hidden email]
>>
>> <javascript:;>> wrote:
>>
>> I have agreed that at this point a release is important. My goal
>>
>> was
>>
>> try
>>
>> to
>>
>> squeeze in a much goodness as possible into the release, but the
>>
>> important
>>
>> bug fixes should come first. Getting 1.x into a state where the
>>
>> release
>>
>> notes don't say that it is geared toward developers and testers is
>>
>> really
>>
>> huge.
>>
>> I think Nifi is a great community otherwise I would participate in
>>
>> the
>>
>> mailing list, create Jira tickets and pull requests. I am only
>>
>> trying to
>>
>> strengthen the great thing that is going on here. We can always do
>>
>> better.
>>
>> I was not trying to put down this community only to participate and
>>
>> make
>>
>> it
>>
>> better. I think this conversation is an indication of how great
>>
>> this
>>
>> community is.
>>
>> Maybe I am being sensitive about this issue and trying to
>>
>> strengthen
>>
>> the
>>
>> nifi community even more, after coming from a conference where it
>>
>> was
>>
>> reported there was lots of excitement at first and now the
>>
>> participation
>>
>> in
>>
>> the community has really died down and they are struggling. I don't
>>
>> want
>>
>> to
>>
>> see that happen here.
>>
>> Cheers,
>>
>> Edgardo
>>
>>
>>
>>
>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>>
>> <javascript:;>> wrote:
>>
>>
>> Edgardo,
>>
>> Thank you for your feedback. We hear your comments and as a
>>
>> committer I
>>
>> can
>>
>> share we are constantly looking to improve the PR process, having
>>
>> already
>>
>> taken many of the steps you suggest.
>>
>> However, it is important to notice that the number of PRs should
>>
>> not be
>>
>> seen as a metric of engagement by the development community: Most
>>
>> of us
>>
>> will submit PRs so that our work can be carefully reviewed by our
>>
>> peers
>>
>> and
>>
>> some of us will use JIRA patches to provide contributions.
>>
>> Having said that, it is true that some PRs may sit idle for a long
>>
>> time
>>
>> and
>>
>> we are working to improve this pipeline.
>>
>> It was therefore no coincidence that I  browsed most of the PRs
>>
>> performing
>>
>> a triage of items that have been superseded or diverged from the
>>
>> current
>>
>> code base.
>>
>> In fact, less than a month ago the dev team closed a number of
>>
>> stalled
>>
>> and
>>
>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>>
>> Despite all the above, I think Joe has a point. The master
>>
>> contain a
>>
>> series
>>
>> of important bug fixes and suspect the community would benefit
>>
>> from
>>
>> a
>>
>> release sooner rather than later.
>>
>> Once again, thank you for your feedback and contribution. It is
>>
>> good to
>>
>> have you here.
>>
>> Andre
>>
>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>>
>> [hidden email]
>>
>> <javascript:;>>
>>
>> wrote:
>>
>> Joe - You are correct I was mentioning the PRs that are
>>
>> currently
>>
>> open.
>>
>>
>> Regardless of how it happens reducing the count of open PRs I
>>
>> believe
>>
>> to
>>
>> be
>>
>> extremely important. Maybe I was hoping that the release could
>>
>> be
>>
>> a
>>
>> forcing
>>
>> function to make that happen. I believe that developers are more
>>
>> willing
>>
>> to
>>
>> contribute when they see that their PRs will actually be able
>>
>> accepted
>>
>> and
>>
>> merged into the code base. Having a low number of open PRs in
>>
>> progress
>>
>> is a
>>
>> great indication that the main nifi developers are fully engaged
>>
>> with
>>
>> the
>>
>> community.
>>
>> There are a few PRs that don't have any comments from committers
>>
>> at
>>
>> all.
>>
>> I
>>
>> found one from August in that state. If that was my PR I don't
>>
>> think I
>>
>> would be so willing to put another one in anytime soon. I do get
>>
>> that
>>
>> sometime PRs get stalled by the originator, if so maybe a rule
>>
>> about
>>
>> closing them after a certain amount of time or being taken over
>>
>> by a
>>
>> core
>>
>> contributor if they think it worthwhile.
>>
>> I would like to shoutout to James Wing on my last PR he was
>>
>> quick
>>
>> to
>>
>> review, provided great comments, testing, and even some
>>
>> additional
>>
>> code.
>>
>> It
>>
>> was a great PR experience.
>>
>> Cheers,
>>
>> Edgardo
>>
>>
>>
>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>>
>> [hidden email] <javascript:;>.
>>
>> invalid> wrote:
>>
>> Joe, I think you misread. Edgardo is referring to the Pull
>>
>> Requests
>>
>> that
>>
>> are currently open, not the tickets assigned to the 1.1.0
>>
>> version.
>>
>>
>> I think these goals (releasing 1.1.0 and cutting down the PR
>>
>> count)
>>
>> should
>>
>> be two different efforts. Doing a thorough job reviewing
>>
>> takes a
>>
>> significant amount of time from both the reviewer and
>>
>> contributor.
>>
>> In
>>
>> order
>>
>> to cut it down significantly would take much longer than a
>>
>> couple
>>
>> days.
>>
>>
>> Also there has already been a lot of great new features and
>>
>> bug
>>
>> fixes
>>
>> contributed to the 1.X line and I don't think it's worth
>>
>> holding up
>>
>> a
>>
>> 1.1.0
>>
>> release for tickets not assigned to this fix version. As an
>>
>> added
>>
>> bonus
>>
>> though, I think many of the tickets tagged as 1.1.0 have PRs
>>
>> already
>>
>> open
>>
>> so closing those will make a large dent in the PR count.
>>
>>
>> Joe
>>
>> - - - - - -
>> Joseph Percivall
>> linkedin.com/in/Percivall
>> e: [hidden email] <javascript:;>
>>
>>
>>
>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>>
>> [hidden email]
>>
>> <javascript:;>>
>>
>> wrote:
>>
>>
>>
>> There are less than 30 right now.  Many of the roughly 90+
>>
>> JIRAs
>>
>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>>
>> or
>>
>> just
>>
>> had fix versions removed.
>>
>> We will need to have a push over the next bunch of days to
>>
>> deal
>>
>> with
>>
>> reviewing/merging/moving the remaining items.
>>
>> Thanks
>> Joe
>>
>>
>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>>
>> [hidden email] <javascript:;>>
>>
>> wrote:
>>
>> Joe,
>>
>> There are 75 PRs currently open. Why not make a push over
>>
>> the
>>
>> next
>>
>> bunch
>>
>> of
>>
>> days to get them closed and then cut the release after that.
>>
>> Cheers,
>>
>> Edgardo
>>
>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>>
>> [hidden email]
>>
>> <javascript:;>>
>>
>> wrote:
>>
>>
>> Team,
>>
>> There have been a ton of bugs fixed a few nice features.  I
>>
>> would
>>
>> like
>>
>> to move to get Apache NiFi 1.1.0 release going pretty much
>>
>> based
>>
>> on
>>
>> where we are now and plan to move most tickets to a new
>>
>> Apache
>>
>> NiFi
>>
>> 1.2.0 version.  We can try to get back on our roughly 6-8
>>
>> week
>>
>> release
>>
>> schedule and shoot for a mid to late Nov release for NiFi
>>
>> 1.2.0
>>
>> this
>>
>> way as well. Please advise if anyone has any other views on
>>
>> this. In
>>
>> the mean time I'll get the wheels in motion so you'll be
>>
>> seeing a
>>
>> lot
>>
>> of JIRA/issue updates to move version around.
>>
>> Thanks
>> Joe
>>
>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>>
>> [hidden email]
>>
>> <javascript:;>>
>>
>> wrote:
>>
>> Sounds good Joe. I have no issue to you doing the rm'ing
>>
>> for
>>
>> it.
>>
>>
>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>>
>> <javascript:;>> wrote:
>>
>>
>> Team,
>>
>> There are a lot of great fixes and improvements on the
>>
>> master
>>
>> line
>>
>> now
>>
>> and we're at a good time window to start pushing for a
>>
>> release.
>>
>> There
>>
>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>>
>> are
>>
>> open.
>>
>> I'm
>>
>> going to go through them and remove fix versions where
>>
>> appropriate.
>>
>>
>> I'm happy to take on RM task for this release though if
>>
>> someone
>>
>> else
>>
>> would like to take that on please advise.
>>
>> Thanks
>> Joe
>>
>>
>>
>>
>>
>> --
>> Cheers,
>>
>> Edgardo
>>
>>
>>
>>
>>
>> --
>> Cheers,
>>
>> Edgardo
>>
>>
>>
>>
>>
>> --
>> Cheers,
>>
>> Edgardo
>>
>>
>>
>>
>> --
>> Cheers,
>>
>> Edgardo
>>
>> Sent from Gmail Mobile
>>
>>
>>
>>
>>
>>
>>
>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Matt Burgess-2
Is there a good spot for us to put instructions on how to build the
Twitter processor and/or the Social Media NAR in the meantime? Maybe a
Wiki page or something simple to say "go to this directory, run this
Maven command, drop the NAR into your deployment..." ?

On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:

> Team,
>
> We appear to be very close.  Andy is working NIFI-3024 but otherwise
> it is focus on testing.
>
> I'm going to prep the RC and release notes now.  Unfortunately the
> twitter changes for json.org will need to remain.  Consensus forming
> on the legal-discuss thread regarding a grace period has been elusive
> and we're already prepared to make the right steps so we'll just need
> to take that on by being empathetic to the user base.
>
> Thanks
> Joe
>
> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
>> Andy,
>>
>> Great to see NIFI-3050 implemented and certainly good news that NiFi 1.1.0
>> is set to include a number of security related improvements.
>>
>>
>>
>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]> wrote:
>>
>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been
>>> added to my plate for this release. Coordinated with Joe Witt and they
>>> should both be included.
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
>>>
>>> Andy LoPresto
>>> [hidden email]
>>> *[hidden email] <[hidden email]>*
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
>>>
>>> Team
>>>
>>> There is a thread on apache legal-discuss that might allow for a
>>> graceperiod of continued usage of the json library.  Am going to keep
>>> a close eye on this and if VP Legal approves we'll be able to keep the
>>> twitter processors in which is definitely a good thing.  Will advise
>>>
>>> Thanks
>>> Joe
>>>
>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]> wrote:
>>>
>>> I've noticed an issue with the per-instance class loading capability
>>> introduced in NIFI-2909 where the additional classpath resources can get
>>> incorrectly removed from the class loader.
>>>
>>> I was able to reproduced this with a unit test and have a fix ready. I
>>> believe this is important and needs to go in for the 1.1 release, going to
>>> re-open NIFI-2909 and submit a PR shortly.
>>>
>>> -Bryan
>>>
>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
>>> wrote:
>>>
>>> I have two items that I would like to wrap up prior to creating an RC for
>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
>>> configuration. The work is already completed and I will be reviewing it
>>> this today. Additionally, following recent interest on the mailing list,
>>> I'd like to knock out NIFI-3020. This will allow an admin to configure a
>>> strategy for user identity when logging in via LDAP. Specifically, it will
>>> support usage of the DN (the default and current implementation) as well as
>>> the username the user logged in as. I should be able to have a PR up for
>>> this work later today.
>>>
>>> Thanks!
>>>
>>> Matt
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>>>
>>>
>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
>>>
>>> The code is within the twitter4j library itself.  I filed a request to
>>> twitter4jg.  The most likely case is we will need to submit a PR to them.
>>> However, I don't see this as something that should delay the release.  We
>>> can provide instructions for folks wanting to use the processor during
>>>
>>> the
>>>
>>> time we cannot make it available in a convenient manner.  I will provide
>>>
>>> a
>>>
>>> meaningful comment about this in release notes and pointers on what folks
>>> can do in the meantime.
>>>
>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:
>>>
>>> I understand there was a discussion thread within the NiFi community
>>>
>>> for
>>>
>>> this as well and I missed responding to that at that time. It just
>>>
>>> seems
>>>
>>> to
>>>
>>> me like JSON processing is necessary for GetTwitter, which is
>>>
>>> incredibly
>>>
>>> useful for demonstrating NiFi’s ability to read from a high volume
>>>
>>> stream
>>>
>>> out of the box. With NIFI-3019 (Remove GetTwitter from default build),
>>>
>>> is
>>>
>>> there any related effort to substitute an acceptable replacement JSON
>>> library to restore this functionality?
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>>>
>>> Andy LoPresto
>>> [hidden email]
>>> *[hidden email] <[hidden email]>*
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>>>
>>> wrote:
>>>
>>>
>>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>>> investigated some alternate TLS config options for the new version of
>>>
>>> the
>>>
>>> client library.
>>>
>>> Is there any alternative to excluding the GetTwitter processor? Using
>>> Johnzon [1] or the Android re-implementation [2] discussed in the
>>>
>>> mailing
>>>
>>> list thread?
>>>
>>> [1] https://johnzon.apache.org/
>>> [2] https://developer.android.com/reference/org/json/package-
>>>
>>> summary.html
>>>
>>>
>>>
>>> Andy LoPresto
>>> [hidden email]
>>> *[hidden email] <[hidden email]>*
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>>>
>>> Team
>>>
>>> Very happy to see that we are down to three items remaining tagged to
>>> 1.1.0.  Solid effort over the recent weeks to close the gap including
>>>
>>> work
>>>
>>> to get past the now category x Jason dependency we had.  The most
>>>
>>> notable
>>>
>>> impact from that is the wildly popular GetTwitter processor, the fav
>>>
>>> new
>>>
>>> nifi user and demo processor, can no longer be included in the default
>>> build.  It is optionally available if users choose to build and use it
>>>
>>> but
>>>
>>> we won't distribute binaries that have it.
>>>
>>> I see some review movement on some patch available but untagged items.
>>>
>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
>>> have any outstanding items?
>>>
>>> Thanks
>>> Joe
>>>
>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>>>
>>> Ryan
>>>
>>> Not officially but I think we should try to close this thing out and
>>> start a vote in the next week or two at most.
>>>
>>> I'm going through the tickets again now.  There is also a new issue of
>>> the json-p license falling out of favor in Apache legal terms and
>>> becoming Category-X.  Am looking into that now.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>>>
>>> wrote:
>>>
>>>
>>> Joe - Is there a target date for 1.1?
>>>
>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:
>>>
>>> Team,
>>>
>>> Just an update on things with working toward an Apache NiFi 1.1.0
>>> release.  There are still about 33 JIRAs there now and some are
>>> awaiting review and are some are under active progress. Yet there is
>>> good traction and progress. I think we should just stay vigilant with
>>> what makes it in and keep working it down.  So let's please shoot for
>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
>>>
>>> Team,
>>>
>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
>>> avoiding putting more in there for now at least without a discussion.
>>> Of the 31 JIRAs there the vast majority need review so we should be
>>> able to close these down fairly quickly as long as we don't let the
>>> list grow.
>>>
>>> Thanks
>>> joe
>>>
>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>
>>>
>>> wrote:
>>>
>>> Joe,
>>>
>>> Appreciate the offer it isn't my PR. I was just using it as an
>>>
>>> example.
>>>
>>> All
>>>
>>> mine are currently closed, which I greatly appreciate.
>>>
>>> Cheers,
>>>
>>> Edgardo
>>>
>>> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>>>
>>> Edgardo,
>>>
>>> You mentioned a PR from August. I'd be happy to help you work that
>>> through review.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>>>
>>> [hidden email]
>>>
>>> <javascript:;>> wrote:
>>>
>>> I have agreed that at this point a release is important. My goal
>>>
>>> was
>>>
>>> try
>>>
>>> to
>>>
>>> squeeze in a much goodness as possible into the release, but the
>>>
>>> important
>>>
>>> bug fixes should come first. Getting 1.x into a state where the
>>>
>>> release
>>>
>>> notes don't say that it is geared toward developers and testers is
>>>
>>> really
>>>
>>> huge.
>>>
>>> I think Nifi is a great community otherwise I would participate in
>>>
>>> the
>>>
>>> mailing list, create Jira tickets and pull requests. I am only
>>>
>>> trying to
>>>
>>> strengthen the great thing that is going on here. We can always do
>>>
>>> better.
>>>
>>> I was not trying to put down this community only to participate and
>>>
>>> make
>>>
>>> it
>>>
>>> better. I think this conversation is an indication of how great
>>>
>>> this
>>>
>>> community is.
>>>
>>> Maybe I am being sensitive about this issue and trying to
>>>
>>> strengthen
>>>
>>> the
>>>
>>> nifi community even more, after coming from a conference where it
>>>
>>> was
>>>
>>> reported there was lots of excitement at first and now the
>>>
>>> participation
>>>
>>> in
>>>
>>> the community has really died down and they are struggling. I don't
>>>
>>> want
>>>
>>> to
>>>
>>> see that happen here.
>>>
>>> Cheers,
>>>
>>> Edgardo
>>>
>>>
>>>
>>>
>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>>>
>>> <javascript:;>> wrote:
>>>
>>>
>>> Edgardo,
>>>
>>> Thank you for your feedback. We hear your comments and as a
>>>
>>> committer I
>>>
>>> can
>>>
>>> share we are constantly looking to improve the PR process, having
>>>
>>> already
>>>
>>> taken many of the steps you suggest.
>>>
>>> However, it is important to notice that the number of PRs should
>>>
>>> not be
>>>
>>> seen as a metric of engagement by the development community: Most
>>>
>>> of us
>>>
>>> will submit PRs so that our work can be carefully reviewed by our
>>>
>>> peers
>>>
>>> and
>>>
>>> some of us will use JIRA patches to provide contributions.
>>>
>>> Having said that, it is true that some PRs may sit idle for a long
>>>
>>> time
>>>
>>> and
>>>
>>> we are working to improve this pipeline.
>>>
>>> It was therefore no coincidence that I  browsed most of the PRs
>>>
>>> performing
>>>
>>> a triage of items that have been superseded or diverged from the
>>>
>>> current
>>>
>>> code base.
>>>
>>> In fact, less than a month ago the dev team closed a number of
>>>
>>> stalled
>>>
>>> and
>>>
>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>>>
>>> Despite all the above, I think Joe has a point. The master
>>>
>>> contain a
>>>
>>> series
>>>
>>> of important bug fixes and suspect the community would benefit
>>>
>>> from
>>>
>>> a
>>>
>>> release sooner rather than later.
>>>
>>> Once again, thank you for your feedback and contribution. It is
>>>
>>> good to
>>>
>>> have you here.
>>>
>>> Andre
>>>
>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>>>
>>> [hidden email]
>>>
>>> <javascript:;>>
>>>
>>> wrote:
>>>
>>> Joe - You are correct I was mentioning the PRs that are
>>>
>>> currently
>>>
>>> open.
>>>
>>>
>>> Regardless of how it happens reducing the count of open PRs I
>>>
>>> believe
>>>
>>> to
>>>
>>> be
>>>
>>> extremely important. Maybe I was hoping that the release could
>>>
>>> be
>>>
>>> a
>>>
>>> forcing
>>>
>>> function to make that happen. I believe that developers are more
>>>
>>> willing
>>>
>>> to
>>>
>>> contribute when they see that their PRs will actually be able
>>>
>>> accepted
>>>
>>> and
>>>
>>> merged into the code base. Having a low number of open PRs in
>>>
>>> progress
>>>
>>> is a
>>>
>>> great indication that the main nifi developers are fully engaged
>>>
>>> with
>>>
>>> the
>>>
>>> community.
>>>
>>> There are a few PRs that don't have any comments from committers
>>>
>>> at
>>>
>>> all.
>>>
>>> I
>>>
>>> found one from August in that state. If that was my PR I don't
>>>
>>> think I
>>>
>>> would be so willing to put another one in anytime soon. I do get
>>>
>>> that
>>>
>>> sometime PRs get stalled by the originator, if so maybe a rule
>>>
>>> about
>>>
>>> closing them after a certain amount of time or being taken over
>>>
>>> by a
>>>
>>> core
>>>
>>> contributor if they think it worthwhile.
>>>
>>> I would like to shoutout to James Wing on my last PR he was
>>>
>>> quick
>>>
>>> to
>>>
>>> review, provided great comments, testing, and even some
>>>
>>> additional
>>>
>>> code.
>>>
>>> It
>>>
>>> was a great PR experience.
>>>
>>> Cheers,
>>>
>>> Edgardo
>>>
>>>
>>>
>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>>>
>>> [hidden email] <javascript:;>.
>>>
>>> invalid> wrote:
>>>
>>> Joe, I think you misread. Edgardo is referring to the Pull
>>>
>>> Requests
>>>
>>> that
>>>
>>> are currently open, not the tickets assigned to the 1.1.0
>>>
>>> version.
>>>
>>>
>>> I think these goals (releasing 1.1.0 and cutting down the PR
>>>
>>> count)
>>>
>>> should
>>>
>>> be two different efforts. Doing a thorough job reviewing
>>>
>>> takes a
>>>
>>> significant amount of time from both the reviewer and
>>>
>>> contributor.
>>>
>>> In
>>>
>>> order
>>>
>>> to cut it down significantly would take much longer than a
>>>
>>> couple
>>>
>>> days.
>>>
>>>
>>> Also there has already been a lot of great new features and
>>>
>>> bug
>>>
>>> fixes
>>>
>>> contributed to the 1.X line and I don't think it's worth
>>>
>>> holding up
>>>
>>> a
>>>
>>> 1.1.0
>>>
>>> release for tickets not assigned to this fix version. As an
>>>
>>> added
>>>
>>> bonus
>>>
>>> though, I think many of the tickets tagged as 1.1.0 have PRs
>>>
>>> already
>>>
>>> open
>>>
>>> so closing those will make a large dent in the PR count.
>>>
>>>
>>> Joe
>>>
>>> - - - - - -
>>> Joseph Percivall
>>> linkedin.com/in/Percivall
>>> e: [hidden email] <javascript:;>
>>>
>>>
>>>
>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>>>
>>> [hidden email]
>>>
>>> <javascript:;>>
>>>
>>> wrote:
>>>
>>>
>>>
>>> There are less than 30 right now.  Many of the roughly 90+
>>>
>>> JIRAs
>>>
>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>>>
>>> or
>>>
>>> just
>>>
>>> had fix versions removed.
>>>
>>> We will need to have a push over the next bunch of days to
>>>
>>> deal
>>>
>>> with
>>>
>>> reviewing/merging/moving the remaining items.
>>>
>>> Thanks
>>> Joe
>>>
>>>
>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>>>
>>> [hidden email] <javascript:;>>
>>>
>>> wrote:
>>>
>>> Joe,
>>>
>>> There are 75 PRs currently open. Why not make a push over
>>>
>>> the
>>>
>>> next
>>>
>>> bunch
>>>
>>> of
>>>
>>> days to get them closed and then cut the release after that.
>>>
>>> Cheers,
>>>
>>> Edgardo
>>>
>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>>>
>>> [hidden email]
>>>
>>> <javascript:;>>
>>>
>>> wrote:
>>>
>>>
>>> Team,
>>>
>>> There have been a ton of bugs fixed a few nice features.  I
>>>
>>> would
>>>
>>> like
>>>
>>> to move to get Apache NiFi 1.1.0 release going pretty much
>>>
>>> based
>>>
>>> on
>>>
>>> where we are now and plan to move most tickets to a new
>>>
>>> Apache
>>>
>>> NiFi
>>>
>>> 1.2.0 version.  We can try to get back on our roughly 6-8
>>>
>>> week
>>>
>>> release
>>>
>>> schedule and shoot for a mid to late Nov release for NiFi
>>>
>>> 1.2.0
>>>
>>> this
>>>
>>> way as well. Please advise if anyone has any other views on
>>>
>>> this. In
>>>
>>> the mean time I'll get the wheels in motion so you'll be
>>>
>>> seeing a
>>>
>>> lot
>>>
>>> of JIRA/issue updates to move version around.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>>>
>>> [hidden email]
>>>
>>> <javascript:;>>
>>>
>>> wrote:
>>>
>>> Sounds good Joe. I have no issue to you doing the rm'ing
>>>
>>> for
>>>
>>> it.
>>>
>>>
>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>>>
>>> <javascript:;>> wrote:
>>>
>>>
>>> Team,
>>>
>>> There are a lot of great fixes and improvements on the
>>>
>>> master
>>>
>>> line
>>>
>>> now
>>>
>>> and we're at a good time window to start pushing for a
>>>
>>> release.
>>>
>>> There
>>>
>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>>>
>>> are
>>>
>>> open.
>>>
>>> I'm
>>>
>>> going to go through them and remove fix versions where
>>>
>>> appropriate.
>>>
>>>
>>> I'm happy to take on RM task for this release though if
>>>
>>> someone
>>>
>>> else
>>>
>>> would like to take that on please advise.
>>>
>>> Thanks
>>> Joe
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>>
>>> Edgardo
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>>
>>> Edgardo
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>>
>>> Edgardo
>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>>
>>> Edgardo
>>>
>>> Sent from Gmail Mobile
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Joe Witt
matt

i'll add a wiki page or set of instructions linked from the release notes.

all,

walked through the 250 or so JIRAs in the 1.1.0 release and pulled out
highlights.  The items noted are as follows.  Will likely reduce this
down further for the release notes but wanted to put this out in case
folks have things they think are really important to highlight.

- Core Improvements:
   - Performance: Session Migration
   - Stability: Cluster Management
   - Developer: Framework supports easy user driven classloader extension
   - Expression Language: Now supports base64 and hex encoded values
and Math functions
   - Repositories now support rollback
   - Faster startup due to more efficient state restoration algorithm
- UX Improvements:
   - Visual Backpressure Indicator
   - Introduced more colors to better highlight actions and components
   - Performance: Validate non-running components
   - Provenance graph image can be exported
   - Cron Scheduling for Primary node tasks now supported
- Updated versions
   - Azure Event Hub 0.9.0
   - Spark 2.0.1
   - Hadoop 2.7.x
- New/Improved Processors
   - new Fetch/Put Elastic Search 5.0
   - new ParseCEF to parse CEF formatted logs
   - improve ExtractEmail now supports TNEF files
   - new Validate CSV
   - improved Solr processors now support SSL and Kerberos
   - new Websocket client and server processors
- New Utility
   - Zookeeper Migrator (move from one zookeeper to another)
- Security
   - Restricted Processors
   - Site-to-site now supports port forwarding
   - Improved Policy Management UX
- Migration Notes:
   - Restricted Processors
   - Twitter Processor Removed

On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]> wrote:

> Is there a good spot for us to put instructions on how to build the
> Twitter processor and/or the Social Media NAR in the meantime? Maybe a
> Wiki page or something simple to say "go to this directory, run this
> Maven command, drop the NAR into your deployment..." ?
>
> On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:
>> Team,
>>
>> We appear to be very close.  Andy is working NIFI-3024 but otherwise
>> it is focus on testing.
>>
>> I'm going to prep the RC and release notes now.  Unfortunately the
>> twitter changes for json.org will need to remain.  Consensus forming
>> on the legal-discuss thread regarding a grace period has been elusive
>> and we're already prepared to make the right steps so we'll just need
>> to take that on by being empathetic to the user base.
>>
>> Thanks
>> Joe
>>
>> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
>>> Andy,
>>>
>>> Great to see NIFI-3050 implemented and certainly good news that NiFi 1.1.0
>>> is set to include a number of security related improvements.
>>>
>>>
>>>
>>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]> wrote:
>>>
>>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been
>>>> added to my plate for this release. Coordinated with Joe Witt and they
>>>> should both be included.
>>>>
>>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
>>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
>>>>
>>>> Andy LoPresto
>>>> [hidden email]
>>>> *[hidden email] <[hidden email]>*
>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>
>>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
>>>>
>>>> Team
>>>>
>>>> There is a thread on apache legal-discuss that might allow for a
>>>> graceperiod of continued usage of the json library.  Am going to keep
>>>> a close eye on this and if VP Legal approves we'll be able to keep the
>>>> twitter processors in which is definitely a good thing.  Will advise
>>>>
>>>> Thanks
>>>> Joe
>>>>
>>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]> wrote:
>>>>
>>>> I've noticed an issue with the per-instance class loading capability
>>>> introduced in NIFI-2909 where the additional classpath resources can get
>>>> incorrectly removed from the class loader.
>>>>
>>>> I was able to reproduced this with a unit test and have a fix ready. I
>>>> believe this is important and needs to go in for the 1.1 release, going to
>>>> re-open NIFI-2909 and submit a PR shortly.
>>>>
>>>> -Bryan
>>>>
>>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
>>>> wrote:
>>>>
>>>> I have two items that I would like to wrap up prior to creating an RC for
>>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
>>>> configuration. The work is already completed and I will be reviewing it
>>>> this today. Additionally, following recent interest on the mailing list,
>>>> I'd like to knock out NIFI-3020. This will allow an admin to configure a
>>>> strategy for user identity when logging in via LDAP. Specifically, it will
>>>> support usage of the DN (the default and current implementation) as well as
>>>> the username the user logged in as. I should be able to have a PR up for
>>>> this work later today.
>>>>
>>>> Thanks!
>>>>
>>>> Matt
>>>>
>>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>>>>
>>>>
>>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
>>>>
>>>> The code is within the twitter4j library itself.  I filed a request to
>>>> twitter4jg.  The most likely case is we will need to submit a PR to them.
>>>> However, I don't see this as something that should delay the release.  We
>>>> can provide instructions for folks wanting to use the processor during
>>>>
>>>> the
>>>>
>>>> time we cannot make it available in a convenient manner.  I will provide
>>>>
>>>> a
>>>>
>>>> meaningful comment about this in release notes and pointers on what folks
>>>> can do in the meantime.
>>>>
>>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:
>>>>
>>>> I understand there was a discussion thread within the NiFi community
>>>>
>>>> for
>>>>
>>>> this as well and I missed responding to that at that time. It just
>>>>
>>>> seems
>>>>
>>>> to
>>>>
>>>> me like JSON processing is necessary for GetTwitter, which is
>>>>
>>>> incredibly
>>>>
>>>> useful for demonstrating NiFi’s ability to read from a high volume
>>>>
>>>> stream
>>>>
>>>> out of the box. With NIFI-3019 (Remove GetTwitter from default build),
>>>>
>>>> is
>>>>
>>>> there any related effort to substitute an acceptable replacement JSON
>>>> library to restore this functionality?
>>>>
>>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>>>>
>>>> Andy LoPresto
>>>> [hidden email]
>>>> *[hidden email] <[hidden email]>*
>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>
>>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>>>>
>>>> wrote:
>>>>
>>>>
>>>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
>>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>>>> investigated some alternate TLS config options for the new version of
>>>>
>>>> the
>>>>
>>>> client library.
>>>>
>>>> Is there any alternative to excluding the GetTwitter processor? Using
>>>> Johnzon [1] or the Android re-implementation [2] discussed in the
>>>>
>>>> mailing
>>>>
>>>> list thread?
>>>>
>>>> [1] https://johnzon.apache.org/
>>>> [2] https://developer.android.com/reference/org/json/package-
>>>>
>>>> summary.html
>>>>
>>>>
>>>>
>>>> Andy LoPresto
>>>> [hidden email]
>>>> *[hidden email] <[hidden email]>*
>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>
>>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>>>>
>>>> Team
>>>>
>>>> Very happy to see that we are down to three items remaining tagged to
>>>> 1.1.0.  Solid effort over the recent weeks to close the gap including
>>>>
>>>> work
>>>>
>>>> to get past the now category x Jason dependency we had.  The most
>>>>
>>>> notable
>>>>
>>>> impact from that is the wildly popular GetTwitter processor, the fav
>>>>
>>>> new
>>>>
>>>> nifi user and demo processor, can no longer be included in the default
>>>> build.  It is optionally available if users choose to build and use it
>>>>
>>>> but
>>>>
>>>> we won't distribute binaries that have it.
>>>>
>>>> I see some review movement on some patch available but untagged items.
>>>>
>>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
>>>> have any outstanding items?
>>>>
>>>> Thanks
>>>> Joe
>>>>
>>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>>>>
>>>> Ryan
>>>>
>>>> Not officially but I think we should try to close this thing out and
>>>> start a vote in the next week or two at most.
>>>>
>>>> I'm going through the tickets again now.  There is also a new issue of
>>>> the json-p license falling out of favor in Apache legal terms and
>>>> becoming Category-X.  Am looking into that now.
>>>>
>>>> Thanks
>>>> Joe
>>>>
>>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>>>>
>>>> wrote:
>>>>
>>>>
>>>> Joe - Is there a target date for 1.1?
>>>>
>>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:
>>>>
>>>> Team,
>>>>
>>>> Just an update on things with working toward an Apache NiFi 1.1.0
>>>> release.  There are still about 33 JIRAs there now and some are
>>>> awaiting review and are some are under active progress. Yet there is
>>>> good traction and progress. I think we should just stay vigilant with
>>>> what makes it in and keep working it down.  So let's please shoot for
>>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>>>>
>>>> Thanks
>>>> Joe
>>>>
>>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
>>>>
>>>> Team,
>>>>
>>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
>>>> avoiding putting more in there for now at least without a discussion.
>>>> Of the 31 JIRAs there the vast majority need review so we should be
>>>> able to close these down fairly quickly as long as we don't let the
>>>> list grow.
>>>>
>>>> Thanks
>>>> joe
>>>>
>>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>
>>>>
>>>> wrote:
>>>>
>>>> Joe,
>>>>
>>>> Appreciate the offer it isn't my PR. I was just using it as an
>>>>
>>>> example.
>>>>
>>>> All
>>>>
>>>> mine are currently closed, which I greatly appreciate.
>>>>
>>>> Cheers,
>>>>
>>>> Edgardo
>>>>
>>>> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>>>>
>>>> Edgardo,
>>>>
>>>> You mentioned a PR from August. I'd be happy to help you work that
>>>> through review.
>>>>
>>>> Thanks
>>>> Joe
>>>>
>>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>>>>
>>>> [hidden email]
>>>>
>>>> <javascript:;>> wrote:
>>>>
>>>> I have agreed that at this point a release is important. My goal
>>>>
>>>> was
>>>>
>>>> try
>>>>
>>>> to
>>>>
>>>> squeeze in a much goodness as possible into the release, but the
>>>>
>>>> important
>>>>
>>>> bug fixes should come first. Getting 1.x into a state where the
>>>>
>>>> release
>>>>
>>>> notes don't say that it is geared toward developers and testers is
>>>>
>>>> really
>>>>
>>>> huge.
>>>>
>>>> I think Nifi is a great community otherwise I would participate in
>>>>
>>>> the
>>>>
>>>> mailing list, create Jira tickets and pull requests. I am only
>>>>
>>>> trying to
>>>>
>>>> strengthen the great thing that is going on here. We can always do
>>>>
>>>> better.
>>>>
>>>> I was not trying to put down this community only to participate and
>>>>
>>>> make
>>>>
>>>> it
>>>>
>>>> better. I think this conversation is an indication of how great
>>>>
>>>> this
>>>>
>>>> community is.
>>>>
>>>> Maybe I am being sensitive about this issue and trying to
>>>>
>>>> strengthen
>>>>
>>>> the
>>>>
>>>> nifi community even more, after coming from a conference where it
>>>>
>>>> was
>>>>
>>>> reported there was lots of excitement at first and now the
>>>>
>>>> participation
>>>>
>>>> in
>>>>
>>>> the community has really died down and they are struggling. I don't
>>>>
>>>> want
>>>>
>>>> to
>>>>
>>>> see that happen here.
>>>>
>>>> Cheers,
>>>>
>>>> Edgardo
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>>>>
>>>> <javascript:;>> wrote:
>>>>
>>>>
>>>> Edgardo,
>>>>
>>>> Thank you for your feedback. We hear your comments and as a
>>>>
>>>> committer I
>>>>
>>>> can
>>>>
>>>> share we are constantly looking to improve the PR process, having
>>>>
>>>> already
>>>>
>>>> taken many of the steps you suggest.
>>>>
>>>> However, it is important to notice that the number of PRs should
>>>>
>>>> not be
>>>>
>>>> seen as a metric of engagement by the development community: Most
>>>>
>>>> of us
>>>>
>>>> will submit PRs so that our work can be carefully reviewed by our
>>>>
>>>> peers
>>>>
>>>> and
>>>>
>>>> some of us will use JIRA patches to provide contributions.
>>>>
>>>> Having said that, it is true that some PRs may sit idle for a long
>>>>
>>>> time
>>>>
>>>> and
>>>>
>>>> we are working to improve this pipeline.
>>>>
>>>> It was therefore no coincidence that I  browsed most of the PRs
>>>>
>>>> performing
>>>>
>>>> a triage of items that have been superseded or diverged from the
>>>>
>>>> current
>>>>
>>>> code base.
>>>>
>>>> In fact, less than a month ago the dev team closed a number of
>>>>
>>>> stalled
>>>>
>>>> and
>>>>
>>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>>>>
>>>> Despite all the above, I think Joe has a point. The master
>>>>
>>>> contain a
>>>>
>>>> series
>>>>
>>>> of important bug fixes and suspect the community would benefit
>>>>
>>>> from
>>>>
>>>> a
>>>>
>>>> release sooner rather than later.
>>>>
>>>> Once again, thank you for your feedback and contribution. It is
>>>>
>>>> good to
>>>>
>>>> have you here.
>>>>
>>>> Andre
>>>>
>>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>>>>
>>>> [hidden email]
>>>>
>>>> <javascript:;>>
>>>>
>>>> wrote:
>>>>
>>>> Joe - You are correct I was mentioning the PRs that are
>>>>
>>>> currently
>>>>
>>>> open.
>>>>
>>>>
>>>> Regardless of how it happens reducing the count of open PRs I
>>>>
>>>> believe
>>>>
>>>> to
>>>>
>>>> be
>>>>
>>>> extremely important. Maybe I was hoping that the release could
>>>>
>>>> be
>>>>
>>>> a
>>>>
>>>> forcing
>>>>
>>>> function to make that happen. I believe that developers are more
>>>>
>>>> willing
>>>>
>>>> to
>>>>
>>>> contribute when they see that their PRs will actually be able
>>>>
>>>> accepted
>>>>
>>>> and
>>>>
>>>> merged into the code base. Having a low number of open PRs in
>>>>
>>>> progress
>>>>
>>>> is a
>>>>
>>>> great indication that the main nifi developers are fully engaged
>>>>
>>>> with
>>>>
>>>> the
>>>>
>>>> community.
>>>>
>>>> There are a few PRs that don't have any comments from committers
>>>>
>>>> at
>>>>
>>>> all.
>>>>
>>>> I
>>>>
>>>> found one from August in that state. If that was my PR I don't
>>>>
>>>> think I
>>>>
>>>> would be so willing to put another one in anytime soon. I do get
>>>>
>>>> that
>>>>
>>>> sometime PRs get stalled by the originator, if so maybe a rule
>>>>
>>>> about
>>>>
>>>> closing them after a certain amount of time or being taken over
>>>>
>>>> by a
>>>>
>>>> core
>>>>
>>>> contributor if they think it worthwhile.
>>>>
>>>> I would like to shoutout to James Wing on my last PR he was
>>>>
>>>> quick
>>>>
>>>> to
>>>>
>>>> review, provided great comments, testing, and even some
>>>>
>>>> additional
>>>>
>>>> code.
>>>>
>>>> It
>>>>
>>>> was a great PR experience.
>>>>
>>>> Cheers,
>>>>
>>>> Edgardo
>>>>
>>>>
>>>>
>>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>>>>
>>>> [hidden email] <javascript:;>.
>>>>
>>>> invalid> wrote:
>>>>
>>>> Joe, I think you misread. Edgardo is referring to the Pull
>>>>
>>>> Requests
>>>>
>>>> that
>>>>
>>>> are currently open, not the tickets assigned to the 1.1.0
>>>>
>>>> version.
>>>>
>>>>
>>>> I think these goals (releasing 1.1.0 and cutting down the PR
>>>>
>>>> count)
>>>>
>>>> should
>>>>
>>>> be two different efforts. Doing a thorough job reviewing
>>>>
>>>> takes a
>>>>
>>>> significant amount of time from both the reviewer and
>>>>
>>>> contributor.
>>>>
>>>> In
>>>>
>>>> order
>>>>
>>>> to cut it down significantly would take much longer than a
>>>>
>>>> couple
>>>>
>>>> days.
>>>>
>>>>
>>>> Also there has already been a lot of great new features and
>>>>
>>>> bug
>>>>
>>>> fixes
>>>>
>>>> contributed to the 1.X line and I don't think it's worth
>>>>
>>>> holding up
>>>>
>>>> a
>>>>
>>>> 1.1.0
>>>>
>>>> release for tickets not assigned to this fix version. As an
>>>>
>>>> added
>>>>
>>>> bonus
>>>>
>>>> though, I think many of the tickets tagged as 1.1.0 have PRs
>>>>
>>>> already
>>>>
>>>> open
>>>>
>>>> so closing those will make a large dent in the PR count.
>>>>
>>>>
>>>> Joe
>>>>
>>>> - - - - - -
>>>> Joseph Percivall
>>>> linkedin.com/in/Percivall
>>>> e: [hidden email] <javascript:;>
>>>>
>>>>
>>>>
>>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>>>>
>>>> [hidden email]
>>>>
>>>> <javascript:;>>
>>>>
>>>> wrote:
>>>>
>>>>
>>>>
>>>> There are less than 30 right now.  Many of the roughly 90+
>>>>
>>>> JIRAs
>>>>
>>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>>>>
>>>> or
>>>>
>>>> just
>>>>
>>>> had fix versions removed.
>>>>
>>>> We will need to have a push over the next bunch of days to
>>>>
>>>> deal
>>>>
>>>> with
>>>>
>>>> reviewing/merging/moving the remaining items.
>>>>
>>>> Thanks
>>>> Joe
>>>>
>>>>
>>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>>>>
>>>> [hidden email] <javascript:;>>
>>>>
>>>> wrote:
>>>>
>>>> Joe,
>>>>
>>>> There are 75 PRs currently open. Why not make a push over
>>>>
>>>> the
>>>>
>>>> next
>>>>
>>>> bunch
>>>>
>>>> of
>>>>
>>>> days to get them closed and then cut the release after that.
>>>>
>>>> Cheers,
>>>>
>>>> Edgardo
>>>>
>>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>>>>
>>>> [hidden email]
>>>>
>>>> <javascript:;>>
>>>>
>>>> wrote:
>>>>
>>>>
>>>> Team,
>>>>
>>>> There have been a ton of bugs fixed a few nice features.  I
>>>>
>>>> would
>>>>
>>>> like
>>>>
>>>> to move to get Apache NiFi 1.1.0 release going pretty much
>>>>
>>>> based
>>>>
>>>> on
>>>>
>>>> where we are now and plan to move most tickets to a new
>>>>
>>>> Apache
>>>>
>>>> NiFi
>>>>
>>>> 1.2.0 version.  We can try to get back on our roughly 6-8
>>>>
>>>> week
>>>>
>>>> release
>>>>
>>>> schedule and shoot for a mid to late Nov release for NiFi
>>>>
>>>> 1.2.0
>>>>
>>>> this
>>>>
>>>> way as well. Please advise if anyone has any other views on
>>>>
>>>> this. In
>>>>
>>>> the mean time I'll get the wheels in motion so you'll be
>>>>
>>>> seeing a
>>>>
>>>> lot
>>>>
>>>> of JIRA/issue updates to move version around.
>>>>
>>>> Thanks
>>>> Joe
>>>>
>>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>>>>
>>>> [hidden email]
>>>>
>>>> <javascript:;>>
>>>>
>>>> wrote:
>>>>
>>>> Sounds good Joe. I have no issue to you doing the rm'ing
>>>>
>>>> for
>>>>
>>>> it.
>>>>
>>>>
>>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>>>>
>>>> <javascript:;>> wrote:
>>>>
>>>>
>>>> Team,
>>>>
>>>> There are a lot of great fixes and improvements on the
>>>>
>>>> master
>>>>
>>>> line
>>>>
>>>> now
>>>>
>>>> and we're at a good time window to start pushing for a
>>>>
>>>> release.
>>>>
>>>> There
>>>>
>>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>>>>
>>>> are
>>>>
>>>> open.
>>>>
>>>> I'm
>>>>
>>>> going to go through them and remove fix versions where
>>>>
>>>> appropriate.
>>>>
>>>>
>>>> I'm happy to take on RM task for this release though if
>>>>
>>>> someone
>>>>
>>>> else
>>>>
>>>> would like to take that on please advise.
>>>>
>>>> Thanks
>>>> Joe
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Cheers,
>>>>
>>>> Edgardo
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Cheers,
>>>>
>>>> Edgardo
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Cheers,
>>>>
>>>> Edgardo
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Cheers,
>>>>
>>>> Edgardo
>>>>
>>>> Sent from Gmail Mobile
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Andy LoPresto-2
Important to call out in the security section that login-identity-providers.xml is now covered by encrypted config, and flow.xml.gz processor property encryption/nifi.sensitive.props.key value can now be migrated by the command-line tool (second part pending completion of NIFI-3024). 

Andy LoPresto
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Nov 21, 2016, at 9:30 PM, Joe Witt <[hidden email]> wrote:

matt

i'll add a wiki page or set of instructions linked from the release notes.

all,

walked through the 250 or so JIRAs in the 1.1.0 release and pulled out
highlights.  The items noted are as follows.  Will likely reduce this
down further for the release notes but wanted to put this out in case
folks have things they think are really important to highlight.

- Core Improvements:
  - Performance: Session Migration
  - Stability: Cluster Management
  - Developer: Framework supports easy user driven classloader extension
  - Expression Language: Now supports base64 and hex encoded values
and Math functions
  - Repositories now support rollback
  - Faster startup due to more efficient state restoration algorithm
- UX Improvements:
  - Visual Backpressure Indicator
  - Introduced more colors to better highlight actions and components
  - Performance: Validate non-running components
  - Provenance graph image can be exported
  - Cron Scheduling for Primary node tasks now supported
- Updated versions
  - Azure Event Hub 0.9.0
  - Spark 2.0.1
  - Hadoop 2.7.x
- New/Improved Processors
  - new Fetch/Put Elastic Search 5.0
  - new ParseCEF to parse CEF formatted logs
  - improve ExtractEmail now supports TNEF files
  - new Validate CSV
  - improved Solr processors now support SSL and Kerberos
  - new Websocket client and server processors
- New Utility
  - Zookeeper Migrator (move from one zookeeper to another)
- Security
  - Restricted Processors
  - Site-to-site now supports port forwarding
  - Improved Policy Management UX
- Migration Notes:
  - Restricted Processors
  - Twitter Processor Removed

On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]> wrote:
Is there a good spot for us to put instructions on how to build the
Twitter processor and/or the Social Media NAR in the meantime? Maybe a
Wiki page or something simple to say "go to this directory, run this
Maven command, drop the NAR into your deployment..." ?

On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:
Team,

We appear to be very close.  Andy is working NIFI-3024 but otherwise
it is focus on testing.

I'm going to prep the RC and release notes now.  Unfortunately the
twitter changes for json.org will need to remain.  Consensus forming
on the legal-discuss thread regarding a grace period has been elusive
and we're already prepared to make the right steps so we'll just need
to take that on by being empathetic to the user base.

Thanks
Joe

On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
Andy,

Great to see NIFI-3050 implemented and certainly good news that NiFi 1.1.0
is set to include a number of security related improvements.



On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]> wrote:

Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been
added to my plate for this release. Coordinated with Joe Witt and they
should both be included.

[1] https://issues.apache.org/jira/browse/NIFI-3050
[2] https://issues.apache.org/jira/browse/NIFI-3051

Andy LoPresto
[hidden email]
*[hidden email] <[hidden email]>*
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:

Team

There is a thread on apache legal-discuss that might allow for a
graceperiod of continued usage of the json library.  Am going to keep
a close eye on this and if VP Legal approves we'll be able to keep the
twitter processors in which is definitely a good thing.  Will advise

Thanks
Joe

On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]> wrote:

I've noticed an issue with the per-instance class loading capability
introduced in NIFI-2909 where the additional classpath resources can get
incorrectly removed from the class loader.

I was able to reproduced this with a unit test and have a fix ready. I
believe this is important and needs to go in for the 1.1 release, going to
re-open NIFI-2909 and submit a PR shortly.

-Bryan

On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
wrote:

I have two items that I would like to wrap up prior to creating an RC for
1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
configuration. The work is already completed and I will be reviewing it
this today. Additionally, following recent interest on the mailing list,
I'd like to knock out NIFI-3020. This will allow an admin to configure a
strategy for user identity when logging in via LDAP. Specifically, it will
support usage of the DN (the default and current implementation) as well as
the username the user logged in as. I should be able to have a PR up for
this work later today.

Thanks!

Matt

[1] https://issues.apache.org/jira/browse/NIFI-2949
[2] https://issues.apache.org/jira/browse/NIFI-3020


On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:

The code is within the twitter4j library itself.  I filed a request to
twitter4jg.  The most likely case is we will need to submit a PR to them.
However, I don't see this as something that should delay the release.  We
can provide instructions for folks wanting to use the processor during

the

time we cannot make it available in a convenient manner.  I will provide

a

meaningful comment about this in release notes and pointers on what folks
can do in the meantime.

On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:

I understand there was a discussion thread within the NiFi community

for

this as well and I missed responding to that at that time. It just

seems

to

me like JSON processing is necessary for GetTwitter, which is

incredibly

useful for demonstrating NiFi’s ability to read from a high volume

stream

out of the box. With NIFI-3019 (Remove GetTwitter from default build),

is

there any related effort to substitute an acceptable replacement JSON
library to restore this functionality?

[1] https://issues.apache.org/jira/browse/NIFI-3019

Andy LoPresto
[hidden email]
*[hidden email] <[hidden email]>*
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>

wrote:


I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
investigated some alternate TLS config options for the new version of

the

client library.

Is there any alternative to excluding the GetTwitter processor? Using
Johnzon [1] or the Android re-implementation [2] discussed in the

mailing

list thread?

[1] https://johnzon.apache.org/
[2] https://developer.android.com/reference/org/json/package-

summary.html



Andy LoPresto
[hidden email]
*[hidden email] <[hidden email]>*
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:

Team

Very happy to see that we are down to three items remaining tagged to
1.1.0.  Solid effort over the recent weeks to close the gap including

work

to get past the now category x Jason dependency we had.  The most

notable

impact from that is the wildly popular GetTwitter processor, the fav

new

nifi user and demo processor, can no longer be included in the default
build.  It is optionally available if users choose to build and use it

but

we won't distribute binaries that have it.

I see some review movement on some patch available but untagged items.

I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
have any outstanding items?

Thanks
Joe

On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:

Ryan

Not officially but I think we should try to close this thing out and
start a vote in the next week or two at most.

I'm going through the tickets again now.  There is also a new issue of
the json-p license falling out of favor in Apache legal terms and
becoming Category-X.  Am looking into that now.

Thanks
Joe

On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>

wrote:


Joe - Is there a target date for 1.1?

On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:

Team,

Just an update on things with working toward an Apache NiFi 1.1.0
release.  There are still about 33 JIRAs there now and some are
awaiting review and are some are under active progress. Yet there is
good traction and progress. I think we should just stay vigilant with
what makes it in and keep working it down.  So let's please shoot for
a couple weeks from now.  If it is ready sooner I'll jump on it.

Thanks
Joe

On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:

Team,

There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
avoiding putting more in there for now at least without a discussion.
Of the 31 JIRAs there the vast majority need review so we should be
able to close these down fairly quickly as long as we don't let the
list grow.

Thanks
joe

On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>

wrote:

Joe,

Appreciate the offer it isn't my PR. I was just using it as an

example.

All

mine are currently closed, which I greatly appreciate.

Cheers,

Edgardo

On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:

Edgardo,

You mentioned a PR from August. I'd be happy to help you work that
through review.

Thanks
Joe

On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <

[hidden email]

<javascript:;>> wrote:

I have agreed that at this point a release is important. My goal

was

try

to

squeeze in a much goodness as possible into the release, but the

important

bug fixes should come first. Getting 1.x into a state where the

release

notes don't say that it is geared toward developers and testers is

really

huge.

I think Nifi is a great community otherwise I would participate in

the

mailing list, create Jira tickets and pull requests. I am only

trying to

strengthen the great thing that is going on here. We can always do

better.

I was not trying to put down this community only to participate and

make

it

better. I think this conversation is an indication of how great

this

community is.

Maybe I am being sensitive about this issue and trying to

strengthen

the

nifi community even more, after coming from a conference where it

was

reported there was lots of excitement at first and now the

participation

in

the community has really died down and they are struggling. I don't

want

to

see that happen here.

Cheers,

Edgardo




On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]

<javascript:;>> wrote:


Edgardo,

Thank you for your feedback. We hear your comments and as a

committer I

can

share we are constantly looking to improve the PR process, having

already

taken many of the steps you suggest.

However, it is important to notice that the number of PRs should

not be

seen as a metric of engagement by the development community: Most

of us

will submit PRs so that our work can be carefully reviewed by our

peers

and

some of us will use JIRA patches to provide contributions.

Having said that, it is true that some PRs may sit idle for a long

time

and

we are working to improve this pipeline.

It was therefore no coincidence that I  browsed most of the PRs

performing

a triage of items that have been superseded or diverged from the

current

code base.

In fact, less than a month ago the dev team closed a number of

stalled

and

superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).

Despite all the above, I think Joe has a point. The master

contain a

series

of important bug fixes and suspect the community would benefit

from

a

release sooner rather than later.

Once again, thank you for your feedback and contribution. It is

good to

have you here.

Andre

On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <

[hidden email]

<javascript:;>>

wrote:

Joe - You are correct I was mentioning the PRs that are

currently

open.


Regardless of how it happens reducing the count of open PRs I

believe

to

be

extremely important. Maybe I was hoping that the release could

be

a

forcing

function to make that happen. I believe that developers are more

willing

to

contribute when they see that their PRs will actually be able

accepted

and

merged into the code base. Having a low number of open PRs in

progress

is a

great indication that the main nifi developers are fully engaged

with

the

community.

There are a few PRs that don't have any comments from committers

at

all.

I

found one from August in that state. If that was my PR I don't

think I

would be so willing to put another one in anytime soon. I do get

that

sometime PRs get stalled by the originator, if so maybe a rule

about

closing them after a certain amount of time or being taken over

by a

core

contributor if they think it worthwhile.

I would like to shoutout to James Wing on my last PR he was

quick

to

review, provided great comments, testing, and even some

additional

code.

It

was a great PR experience.

Cheers,

Edgardo



On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <

[hidden email] <javascript:;>.

invalid> wrote:

Joe, I think you misread. Edgardo is referring to the Pull

Requests

that

are currently open, not the tickets assigned to the 1.1.0

version.


I think these goals (releasing 1.1.0 and cutting down the PR

count)

should

be two different efforts. Doing a thorough job reviewing

takes a

significant amount of time from both the reviewer and

contributor.

In

order

to cut it down significantly would take much longer than a

couple

days.


Also there has already been a lot of great new features and

bug

fixes

contributed to the 1.X line and I don't think it's worth

holding up

a

1.1.0

release for tickets not assigned to this fix version. As an

added

bonus

though, I think many of the tickets tagged as 1.1.0 have PRs

already

open

so closing those will make a large dent in the PR count.


Joe

- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: [hidden email] <javascript:;>



On Thursday, October 13, 2016 3:58 PM, Joe Witt <

[hidden email]

<javascript:;>>

wrote:



There are less than 30 right now.  Many of the roughly 90+

JIRAs

opened on 1.1.0 were easily dispositioned to 1.2.0 or closed

or

just

had fix versions removed.

We will need to have a push over the next bunch of days to

deal

with

reviewing/merging/moving the remaining items.

Thanks
Joe


On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <

[hidden email] <javascript:;>>

wrote:

Joe,

There are 75 PRs currently open. Why not make a push over

the

next

bunch

of

days to get them closed and then cut the release after that.

Cheers,

Edgardo

On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <

[hidden email]

<javascript:;>>

wrote:


Team,

There have been a ton of bugs fixed a few nice features.  I

would

like

to move to get Apache NiFi 1.1.0 release going pretty much

based

on

where we are now and plan to move most tickets to a new

Apache

NiFi

1.2.0 version.  We can try to get back on our roughly 6-8

week

release

schedule and shoot for a mid to late Nov release for NiFi

1.2.0

this

way as well. Please advise if anyone has any other views on

this. In

the mean time I'll get the wheels in motion so you'll be

seeing a

lot

of JIRA/issue updates to move version around.

Thanks
Joe

On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <

[hidden email]

<javascript:;>>

wrote:

Sounds good Joe. I have no issue to you doing the rm'ing

for

it.


On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]

<javascript:;>> wrote:


Team,

There are a lot of great fixes and improvements on the

master

line

now

and we're at a good time window to start pushing for a

release.

There

are, however, about 90+ JIRAs assigned to 1.1.0 which

are

open.

I'm

going to go through them and remove fix versions where

appropriate.


I'm happy to take on RM task for this release though if

someone

else

would like to take that on please advise.

Thanks
Joe





--
Cheers,

Edgardo





--
Cheers,

Edgardo





--
Cheers,

Edgardo




--
Cheers,

Edgardo

Sent from Gmail Mobile











signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Joe Gresock
I'm biased, but I think the new Query / Scroll Elasticsearch processors are
pretty big ;)

On Tue, Nov 22, 2016 at 9:02 AM, Andy LoPresto <[hidden email]> wrote:

> Important to call out in the security section that
> login-identity-providers.xml is now covered by encrypted config, and
> flow.xml.gz processor property encryption/nifi.sensitive.props.key value
> can now be migrated by the command-line tool (second part pending
> completion of NIFI-3024).
>
> Andy LoPresto
> [hidden email]
> *[hidden email] <[hidden email]>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Nov 21, 2016, at 9:30 PM, Joe Witt <[hidden email]> wrote:
>
> matt
>
> i'll add a wiki page or set of instructions linked from the release notes.
>
> all,
>
> walked through the 250 or so JIRAs in the 1.1.0 release and pulled out
> highlights.  The items noted are as follows.  Will likely reduce this
> down further for the release notes but wanted to put this out in case
> folks have things they think are really important to highlight.
>
> - Core Improvements:
>   - Performance: Session Migration
>   - Stability: Cluster Management
>   - Developer: Framework supports easy user driven classloader extension
>   - Expression Language: Now supports base64 and hex encoded values
> and Math functions
>   - Repositories now support rollback
>   - Faster startup due to more efficient state restoration algorithm
> - UX Improvements:
>   - Visual Backpressure Indicator
>   - Introduced more colors to better highlight actions and components
>   - Performance: Validate non-running components
>   - Provenance graph image can be exported
>   - Cron Scheduling for Primary node tasks now supported
> - Updated versions
>   - Azure Event Hub 0.9.0
>   - Spark 2.0.1
>   - Hadoop 2.7.x
> - New/Improved Processors
>   - new Fetch/Put Elastic Search 5.0
>   - new ParseCEF to parse CEF formatted logs
>   - improve ExtractEmail now supports TNEF files
>   - new Validate CSV
>   - improved Solr processors now support SSL and Kerberos
>   - new Websocket client and server processors
> - New Utility
>   - Zookeeper Migrator (move from one zookeeper to another)
> - Security
>   - Restricted Processors
>   - Site-to-site now supports port forwarding
>   - Improved Policy Management UX
> - Migration Notes:
>   - Restricted Processors
>   - Twitter Processor Removed
>
> On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]>
> wrote:
>
> Is there a good spot for us to put instructions on how to build the
> Twitter processor and/or the Social Media NAR in the meantime? Maybe a
> Wiki page or something simple to say "go to this directory, run this
> Maven command, drop the NAR into your deployment..." ?
>
> On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:
>
> Team,
>
> We appear to be very close.  Andy is working NIFI-3024 but otherwise
> it is focus on testing.
>
> I'm going to prep the RC and release notes now.  Unfortunately the
> twitter changes for json.org will need to remain.  Consensus forming
> on the legal-discuss thread regarding a grace period has been elusive
> and we're already prepared to make the right steps so we'll just need
> to take that on by being empathetic to the user base.
>
> Thanks
> Joe
>
> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
>
> Andy,
>
> Great to see NIFI-3050 implemented and certainly good news that NiFi 1.1.0
> is set to include a number of security related improvements.
>
>
>
> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]>
> wrote:
>
> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been
> added to my plate for this release. Coordinated with Joe Witt and they
> should both be included.
>
> [1] https://issues.apache.org/jira/browse/NIFI-3050
> [2] https://issues.apache.org/jira/browse/NIFI-3051
>
> Andy LoPresto
> [hidden email]
> *[hidden email] <[hidden email]>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
>
> Team
>
> There is a thread on apache legal-discuss that might allow for a
> graceperiod of continued usage of the json library.  Am going to keep
> a close eye on this and if VP Legal approves we'll be able to keep the
> twitter processors in which is definitely a good thing.  Will advise
>
> Thanks
> Joe
>
> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]> wrote:
>
> I've noticed an issue with the per-instance class loading capability
> introduced in NIFI-2909 where the additional classpath resources can get
> incorrectly removed from the class loader.
>
> I was able to reproduced this with a unit test and have a fix ready. I
> believe this is important and needs to go in for the 1.1 release, going to
> re-open NIFI-2909 and submit a PR shortly.
>
> -Bryan
>
> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
> wrote:
>
> I have two items that I would like to wrap up prior to creating an RC for
> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
> configuration. The work is already completed and I will be reviewing it
> this today. Additionally, following recent interest on the mailing list,
> I'd like to knock out NIFI-3020. This will allow an admin to configure a
> strategy for user identity when logging in via LDAP. Specifically, it will
> support usage of the DN (the default and current implementation) as well as
> the username the user logged in as. I should be able to have a PR up for
> this work later today.
>
> Thanks!
>
> Matt
>
> [1] https://issues.apache.org/jira/browse/NIFI-2949
> [2] https://issues.apache.org/jira/browse/NIFI-3020
>
>
> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
>
> The code is within the twitter4j library itself.  I filed a request to
> twitter4jg.  The most likely case is we will need to submit a PR to them.
> However, I don't see this as something that should delay the release.  We
> can provide instructions for folks wanting to use the processor during
>
> the
>
> time we cannot make it available in a convenient manner.  I will provide
>
> a
>
> meaningful comment about this in release notes and pointers on what folks
> can do in the meantime.
>
> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:
>
> I understand there was a discussion thread within the NiFi community
>
> for
>
> this as well and I missed responding to that at that time. It just
>
> seems
>
> to
>
> me like JSON processing is necessary for GetTwitter, which is
>
> incredibly
>
> useful for demonstrating NiFi’s ability to read from a high volume
>
> stream
>
> out of the box. With NIFI-3019 (Remove GetTwitter from default build),
>
> is
>
> there any related effort to substitute an acceptable replacement JSON
> library to restore this functionality?
>
> [1] https://issues.apache.org/jira/browse/NIFI-3019
>
> Andy LoPresto
> [hidden email]
> *[hidden email] <[hidden email]>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>
> wrote:
>
>
> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
> investigated some alternate TLS config options for the new version of
>
> the
>
> client library.
>
> Is there any alternative to excluding the GetTwitter processor? Using
> Johnzon [1] or the Android re-implementation [2] discussed in the
>
> mailing
>
> list thread?
>
> [1] https://johnzon.apache.org/
> [2] https://developer.android.com/reference/org/json/package-
>
> summary.html
>
>
>
> Andy LoPresto
> [hidden email]
> *[hidden email] <[hidden email]>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>
> Team
>
> Very happy to see that we are down to three items remaining tagged to
> 1.1.0.  Solid effort over the recent weeks to close the gap including
>
> work
>
> to get past the now category x Jason dependency we had.  The most
>
> notable
>
> impact from that is the wildly popular GetTwitter processor, the fav
>
> new
>
> nifi user and demo processor, can no longer be included in the default
> build.  It is optionally available if users choose to build and use it
>
> but
>
> we won't distribute binaries that have it.
>
> I see some review movement on some patch available but untagged items.
>
> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
> have any outstanding items?
>
> Thanks
> Joe
>
> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>
> Ryan
>
> Not officially but I think we should try to close this thing out and
> start a vote in the next week or two at most.
>
> I'm going through the tickets again now.  There is also a new issue of
> the json-p license falling out of favor in Apache legal terms and
> becoming Category-X.  Am looking into that now.
>
> Thanks
> Joe
>
> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>
> wrote:
>
>
> Joe - Is there a target date for 1.1?
>
> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:
>
> Team,
>
> Just an update on things with working toward an Apache NiFi 1.1.0
> release.  There are still about 33 JIRAs there now and some are
> awaiting review and are some are under active progress. Yet there is
> good traction and progress. I think we should just stay vigilant with
> what makes it in and keep working it down.  So let's please shoot for
> a couple weeks from now.  If it is ready sooner I'll jump on it.
>
> Thanks
> Joe
>
> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
>
> Team,
>
> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
> avoiding putting more in there for now at least without a discussion.
> Of the 31 JIRAs there the vast majority need review so we should be
> able to close these down fairly quickly as long as we don't let the
> list grow.
>
> Thanks
> joe
>
> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>
>
> wrote:
>
> Joe,
>
> Appreciate the offer it isn't my PR. I was just using it as an
>
> example.
>
> All
>
> mine are currently closed, which I greatly appreciate.
>
> Cheers,
>
> Edgardo
>
> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>
> Edgardo,
>
> You mentioned a PR from August. I'd be happy to help you work that
> through review.
>
> Thanks
> Joe
>
> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>
> [hidden email]
>
> <javascript:;>> wrote:
>
> I have agreed that at this point a release is important. My goal
>
> was
>
> try
>
> to
>
> squeeze in a much goodness as possible into the release, but the
>
> important
>
> bug fixes should come first. Getting 1.x into a state where the
>
> release
>
> notes don't say that it is geared toward developers and testers is
>
> really
>
> huge.
>
> I think Nifi is a great community otherwise I would participate in
>
> the
>
> mailing list, create Jira tickets and pull requests. I am only
>
> trying to
>
> strengthen the great thing that is going on here. We can always do
>
> better.
>
> I was not trying to put down this community only to participate and
>
> make
>
> it
>
> better. I think this conversation is an indication of how great
>
> this
>
> community is.
>
> Maybe I am being sensitive about this issue and trying to
>
> strengthen
>
> the
>
> nifi community even more, after coming from a conference where it
>
> was
>
> reported there was lots of excitement at first and now the
>
> participation
>
> in
>
> the community has really died down and they are struggling. I don't
>
> want
>
> to
>
> see that happen here.
>
> Cheers,
>
> Edgardo
>
>
>
>
> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>
> <javascript:;>> wrote:
>
>
> Edgardo,
>
> Thank you for your feedback. We hear your comments and as a
>
> committer I
>
> can
>
> share we are constantly looking to improve the PR process, having
>
> already
>
> taken many of the steps you suggest.
>
> However, it is important to notice that the number of PRs should
>
> not be
>
> seen as a metric of engagement by the development community: Most
>
> of us
>
> will submit PRs so that our work can be carefully reviewed by our
>
> peers
>
> and
>
> some of us will use JIRA patches to provide contributions.
>
> Having said that, it is true that some PRs may sit idle for a long
>
> time
>
> and
>
> we are working to improve this pipeline.
>
> It was therefore no coincidence that I  browsed most of the PRs
>
> performing
>
> a triage of items that have been superseded or diverged from the
>
> current
>
> code base.
>
> In fact, less than a month ago the dev team closed a number of
>
> stalled
>
> and
>
> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>
> Despite all the above, I think Joe has a point. The master
>
> contain a
>
> series
>
> of important bug fixes and suspect the community would benefit
>
> from
>
> a
>
> release sooner rather than later.
>
> Once again, thank you for your feedback and contribution. It is
>
> good to
>
> have you here.
>
> Andre
>
> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>
> [hidden email]
>
> <javascript:;>>
>
> wrote:
>
> Joe - You are correct I was mentioning the PRs that are
>
> currently
>
> open.
>
>
> Regardless of how it happens reducing the count of open PRs I
>
> believe
>
> to
>
> be
>
> extremely important. Maybe I was hoping that the release could
>
> be
>
> a
>
> forcing
>
> function to make that happen. I believe that developers are more
>
> willing
>
> to
>
> contribute when they see that their PRs will actually be able
>
> accepted
>
> and
>
> merged into the code base. Having a low number of open PRs in
>
> progress
>
> is a
>
> great indication that the main nifi developers are fully engaged
>
> with
>
> the
>
> community.
>
> There are a few PRs that don't have any comments from committers
>
> at
>
> all.
>
> I
>
> found one from August in that state. If that was my PR I don't
>
> think I
>
> would be so willing to put another one in anytime soon. I do get
>
> that
>
> sometime PRs get stalled by the originator, if so maybe a rule
>
> about
>
> closing them after a certain amount of time or being taken over
>
> by a
>
> core
>
> contributor if they think it worthwhile.
>
> I would like to shoutout to James Wing on my last PR he was
>
> quick
>
> to
>
> review, provided great comments, testing, and even some
>
> additional
>
> code.
>
> It
>
> was a great PR experience.
>
> Cheers,
>
> Edgardo
>
>
>
> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>
> [hidden email] <javascript:;>.
>
> invalid> wrote:
>
> Joe, I think you misread. Edgardo is referring to the Pull
>
> Requests
>
> that
>
> are currently open, not the tickets assigned to the 1.1.0
>
> version.
>
>
> I think these goals (releasing 1.1.0 and cutting down the PR
>
> count)
>
> should
>
> be two different efforts. Doing a thorough job reviewing
>
> takes a
>
> significant amount of time from both the reviewer and
>
> contributor.
>
> In
>
> order
>
> to cut it down significantly would take much longer than a
>
> couple
>
> days.
>
>
> Also there has already been a lot of great new features and
>
> bug
>
> fixes
>
> contributed to the 1.X line and I don't think it's worth
>
> holding up
>
> a
>
> 1.1.0
>
> release for tickets not assigned to this fix version. As an
>
> added
>
> bonus
>
> though, I think many of the tickets tagged as 1.1.0 have PRs
>
> already
>
> open
>
> so closing those will make a large dent in the PR count.
>
>
> Joe
>
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: [hidden email] <javascript:;>
>
>
>
> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>
> [hidden email]
>
> <javascript:;>>
>
> wrote:
>
>
>
> There are less than 30 right now.  Many of the roughly 90+
>
> JIRAs
>
> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>
> or
>
> just
>
> had fix versions removed.
>
> We will need to have a push over the next bunch of days to
>
> deal
>
> with
>
> reviewing/merging/moving the remaining items.
>
> Thanks
> Joe
>
>
> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>
> [hidden email] <javascript:;>>
>
> wrote:
>
> Joe,
>
> There are 75 PRs currently open. Why not make a push over
>
> the
>
> next
>
> bunch
>
> of
>
> days to get them closed and then cut the release after that.
>
> Cheers,
>
> Edgardo
>
> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>
> [hidden email]
>
> <javascript:;>>
>
> wrote:
>
>
> Team,
>
> There have been a ton of bugs fixed a few nice features.  I
>
> would
>
> like
>
> to move to get Apache NiFi 1.1.0 release going pretty much
>
> based
>
> on
>
> where we are now and plan to move most tickets to a new
>
> Apache
>
> NiFi
>
> 1.2.0 version.  We can try to get back on our roughly 6-8
>
> week
>
> release
>
> schedule and shoot for a mid to late Nov release for NiFi
>
> 1.2.0
>
> this
>
> way as well. Please advise if anyone has any other views on
>
> this. In
>
> the mean time I'll get the wheels in motion so you'll be
>
> seeing a
>
> lot
>
> of JIRA/issue updates to move version around.
>
> Thanks
> Joe
>
> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>
> [hidden email]
>
> <javascript:;>>
>
> wrote:
>
> Sounds good Joe. I have no issue to you doing the rm'ing
>
> for
>
> it.
>
>
> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>
> <javascript:;>> wrote:
>
>
> Team,
>
> There are a lot of great fixes and improvements on the
>
> master
>
> line
>
> now
>
> and we're at a good time window to start pushing for a
>
> release.
>
> There
>
> are, however, about 90+ JIRAs assigned to 1.1.0 which
>
> are
>
> open.
>
> I'm
>
> going to go through them and remove fix versions where
>
> appropriate.
>
>
> I'm happy to take on RM task for this release though if
>
> someone
>
> else
>
> would like to take that on please advise.
>
> Thanks
> Joe
>
>
>
>
>
> --
> Cheers,
>
> Edgardo
>
>
>
>
>
> --
> Cheers,
>
> Edgardo
>
>
>
>
>
> --
> Cheers,
>
> Edgardo
>
>
>
>
> --
> Cheers,
>
> Edgardo
>
> Sent from Gmail Mobile
>
>
>
>
>
>
>
>
>
>
>


--
I know what it is to be in need, and I know what it is to have plenty.  I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want.  I can do
all this through him who gives me strength.    *-Philippians 4:12-13*
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Joe Witt
thanks Joe and Andy.  Have those on the list now.

On Tue, Nov 22, 2016 at 5:09 AM, Joe Gresock <[hidden email]> wrote:

> I'm biased, but I think the new Query / Scroll Elasticsearch processors are
> pretty big ;)
>
> On Tue, Nov 22, 2016 at 9:02 AM, Andy LoPresto <[hidden email]> wrote:
>
>> Important to call out in the security section that
>> login-identity-providers.xml is now covered by encrypted config, and
>> flow.xml.gz processor property encryption/nifi.sensitive.props.key value
>> can now be migrated by the command-line tool (second part pending
>> completion of NIFI-3024).
>>
>> Andy LoPresto
>> [hidden email]
>> *[hidden email] <[hidden email]>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Nov 21, 2016, at 9:30 PM, Joe Witt <[hidden email]> wrote:
>>
>> matt
>>
>> i'll add a wiki page or set of instructions linked from the release notes.
>>
>> all,
>>
>> walked through the 250 or so JIRAs in the 1.1.0 release and pulled out
>> highlights.  The items noted are as follows.  Will likely reduce this
>> down further for the release notes but wanted to put this out in case
>> folks have things they think are really important to highlight.
>>
>> - Core Improvements:
>>   - Performance: Session Migration
>>   - Stability: Cluster Management
>>   - Developer: Framework supports easy user driven classloader extension
>>   - Expression Language: Now supports base64 and hex encoded values
>> and Math functions
>>   - Repositories now support rollback
>>   - Faster startup due to more efficient state restoration algorithm
>> - UX Improvements:
>>   - Visual Backpressure Indicator
>>   - Introduced more colors to better highlight actions and components
>>   - Performance: Validate non-running components
>>   - Provenance graph image can be exported
>>   - Cron Scheduling for Primary node tasks now supported
>> - Updated versions
>>   - Azure Event Hub 0.9.0
>>   - Spark 2.0.1
>>   - Hadoop 2.7.x
>> - New/Improved Processors
>>   - new Fetch/Put Elastic Search 5.0
>>   - new ParseCEF to parse CEF formatted logs
>>   - improve ExtractEmail now supports TNEF files
>>   - new Validate CSV
>>   - improved Solr processors now support SSL and Kerberos
>>   - new Websocket client and server processors
>> - New Utility
>>   - Zookeeper Migrator (move from one zookeeper to another)
>> - Security
>>   - Restricted Processors
>>   - Site-to-site now supports port forwarding
>>   - Improved Policy Management UX
>> - Migration Notes:
>>   - Restricted Processors
>>   - Twitter Processor Removed
>>
>> On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]>
>> wrote:
>>
>> Is there a good spot for us to put instructions on how to build the
>> Twitter processor and/or the Social Media NAR in the meantime? Maybe a
>> Wiki page or something simple to say "go to this directory, run this
>> Maven command, drop the NAR into your deployment..." ?
>>
>> On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:
>>
>> Team,
>>
>> We appear to be very close.  Andy is working NIFI-3024 but otherwise
>> it is focus on testing.
>>
>> I'm going to prep the RC and release notes now.  Unfortunately the
>> twitter changes for json.org will need to remain.  Consensus forming
>> on the legal-discuss thread regarding a grace period has been elusive
>> and we're already prepared to make the right steps so we'll just need
>> to take that on by being empathetic to the user base.
>>
>> Thanks
>> Joe
>>
>> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
>>
>> Andy,
>>
>> Great to see NIFI-3050 implemented and certainly good news that NiFi 1.1.0
>> is set to include a number of security related improvements.
>>
>>
>>
>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]>
>> wrote:
>>
>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been
>> added to my plate for this release. Coordinated with Joe Witt and they
>> should both be included.
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-3050
>> [2] https://issues.apache.org/jira/browse/NIFI-3051
>>
>> Andy LoPresto
>> [hidden email]
>> *[hidden email] <[hidden email]>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
>>
>> Team
>>
>> There is a thread on apache legal-discuss that might allow for a
>> graceperiod of continued usage of the json library.  Am going to keep
>> a close eye on this and if VP Legal approves we'll be able to keep the
>> twitter processors in which is definitely a good thing.  Will advise
>>
>> Thanks
>> Joe
>>
>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]> wrote:
>>
>> I've noticed an issue with the per-instance class loading capability
>> introduced in NIFI-2909 where the additional classpath resources can get
>> incorrectly removed from the class loader.
>>
>> I was able to reproduced this with a unit test and have a fix ready. I
>> believe this is important and needs to go in for the 1.1 release, going to
>> re-open NIFI-2909 and submit a PR shortly.
>>
>> -Bryan
>>
>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
>> wrote:
>>
>> I have two items that I would like to wrap up prior to creating an RC for
>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
>> configuration. The work is already completed and I will be reviewing it
>> this today. Additionally, following recent interest on the mailing list,
>> I'd like to knock out NIFI-3020. This will allow an admin to configure a
>> strategy for user identity when logging in via LDAP. Specifically, it will
>> support usage of the DN (the default and current implementation) as well as
>> the username the user logged in as. I should be able to have a PR up for
>> this work later today.
>>
>> Thanks!
>>
>> Matt
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>>
>>
>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
>>
>> The code is within the twitter4j library itself.  I filed a request to
>> twitter4jg.  The most likely case is we will need to submit a PR to them.
>> However, I don't see this as something that should delay the release.  We
>> can provide instructions for folks wanting to use the processor during
>>
>> the
>>
>> time we cannot make it available in a convenient manner.  I will provide
>>
>> a
>>
>> meaningful comment about this in release notes and pointers on what folks
>> can do in the meantime.
>>
>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:
>>
>> I understand there was a discussion thread within the NiFi community
>>
>> for
>>
>> this as well and I missed responding to that at that time. It just
>>
>> seems
>>
>> to
>>
>> me like JSON processing is necessary for GetTwitter, which is
>>
>> incredibly
>>
>> useful for demonstrating NiFi’s ability to read from a high volume
>>
>> stream
>>
>> out of the box. With NIFI-3019 (Remove GetTwitter from default build),
>>
>> is
>>
>> there any related effort to substitute an acceptable replacement JSON
>> library to restore this functionality?
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>>
>> Andy LoPresto
>> [hidden email]
>> *[hidden email] <[hidden email]>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>>
>> wrote:
>>
>>
>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>> investigated some alternate TLS config options for the new version of
>>
>> the
>>
>> client library.
>>
>> Is there any alternative to excluding the GetTwitter processor? Using
>> Johnzon [1] or the Android re-implementation [2] discussed in the
>>
>> mailing
>>
>> list thread?
>>
>> [1] https://johnzon.apache.org/
>> [2] https://developer.android.com/reference/org/json/package-
>>
>> summary.html
>>
>>
>>
>> Andy LoPresto
>> [hidden email]
>> *[hidden email] <[hidden email]>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>>
>> Team
>>
>> Very happy to see that we are down to three items remaining tagged to
>> 1.1.0.  Solid effort over the recent weeks to close the gap including
>>
>> work
>>
>> to get past the now category x Jason dependency we had.  The most
>>
>> notable
>>
>> impact from that is the wildly popular GetTwitter processor, the fav
>>
>> new
>>
>> nifi user and demo processor, can no longer be included in the default
>> build.  It is optionally available if users choose to build and use it
>>
>> but
>>
>> we won't distribute binaries that have it.
>>
>> I see some review movement on some patch available but untagged items.
>>
>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
>> have any outstanding items?
>>
>> Thanks
>> Joe
>>
>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>>
>> Ryan
>>
>> Not officially but I think we should try to close this thing out and
>> start a vote in the next week or two at most.
>>
>> I'm going through the tickets again now.  There is also a new issue of
>> the json-p license falling out of favor in Apache legal terms and
>> becoming Category-X.  Am looking into that now.
>>
>> Thanks
>> Joe
>>
>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>>
>> wrote:
>>
>>
>> Joe - Is there a target date for 1.1?
>>
>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:
>>
>> Team,
>>
>> Just an update on things with working toward an Apache NiFi 1.1.0
>> release.  There are still about 33 JIRAs there now and some are
>> awaiting review and are some are under active progress. Yet there is
>> good traction and progress. I think we should just stay vigilant with
>> what makes it in and keep working it down.  So let's please shoot for
>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>>
>> Thanks
>> Joe
>>
>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
>>
>> Team,
>>
>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
>> avoiding putting more in there for now at least without a discussion.
>> Of the 31 JIRAs there the vast majority need review so we should be
>> able to close these down fairly quickly as long as we don't let the
>> list grow.
>>
>> Thanks
>> joe
>>
>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>
>>
>> wrote:
>>
>> Joe,
>>
>> Appreciate the offer it isn't my PR. I was just using it as an
>>
>> example.
>>
>> All
>>
>> mine are currently closed, which I greatly appreciate.
>>
>> Cheers,
>>
>> Edgardo
>>
>> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>>
>> Edgardo,
>>
>> You mentioned a PR from August. I'd be happy to help you work that
>> through review.
>>
>> Thanks
>> Joe
>>
>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>>
>> [hidden email]
>>
>> <javascript:;>> wrote:
>>
>> I have agreed that at this point a release is important. My goal
>>
>> was
>>
>> try
>>
>> to
>>
>> squeeze in a much goodness as possible into the release, but the
>>
>> important
>>
>> bug fixes should come first. Getting 1.x into a state where the
>>
>> release
>>
>> notes don't say that it is geared toward developers and testers is
>>
>> really
>>
>> huge.
>>
>> I think Nifi is a great community otherwise I would participate in
>>
>> the
>>
>> mailing list, create Jira tickets and pull requests. I am only
>>
>> trying to
>>
>> strengthen the great thing that is going on here. We can always do
>>
>> better.
>>
>> I was not trying to put down this community only to participate and
>>
>> make
>>
>> it
>>
>> better. I think this conversation is an indication of how great
>>
>> this
>>
>> community is.
>>
>> Maybe I am being sensitive about this issue and trying to
>>
>> strengthen
>>
>> the
>>
>> nifi community even more, after coming from a conference where it
>>
>> was
>>
>> reported there was lots of excitement at first and now the
>>
>> participation
>>
>> in
>>
>> the community has really died down and they are struggling. I don't
>>
>> want
>>
>> to
>>
>> see that happen here.
>>
>> Cheers,
>>
>> Edgardo
>>
>>
>>
>>
>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>>
>> <javascript:;>> wrote:
>>
>>
>> Edgardo,
>>
>> Thank you for your feedback. We hear your comments and as a
>>
>> committer I
>>
>> can
>>
>> share we are constantly looking to improve the PR process, having
>>
>> already
>>
>> taken many of the steps you suggest.
>>
>> However, it is important to notice that the number of PRs should
>>
>> not be
>>
>> seen as a metric of engagement by the development community: Most
>>
>> of us
>>
>> will submit PRs so that our work can be carefully reviewed by our
>>
>> peers
>>
>> and
>>
>> some of us will use JIRA patches to provide contributions.
>>
>> Having said that, it is true that some PRs may sit idle for a long
>>
>> time
>>
>> and
>>
>> we are working to improve this pipeline.
>>
>> It was therefore no coincidence that I  browsed most of the PRs
>>
>> performing
>>
>> a triage of items that have been superseded or diverged from the
>>
>> current
>>
>> code base.
>>
>> In fact, less than a month ago the dev team closed a number of
>>
>> stalled
>>
>> and
>>
>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>>
>> Despite all the above, I think Joe has a point. The master
>>
>> contain a
>>
>> series
>>
>> of important bug fixes and suspect the community would benefit
>>
>> from
>>
>> a
>>
>> release sooner rather than later.
>>
>> Once again, thank you for your feedback and contribution. It is
>>
>> good to
>>
>> have you here.
>>
>> Andre
>>
>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>>
>> [hidden email]
>>
>> <javascript:;>>
>>
>> wrote:
>>
>> Joe - You are correct I was mentioning the PRs that are
>>
>> currently
>>
>> open.
>>
>>
>> Regardless of how it happens reducing the count of open PRs I
>>
>> believe
>>
>> to
>>
>> be
>>
>> extremely important. Maybe I was hoping that the release could
>>
>> be
>>
>> a
>>
>> forcing
>>
>> function to make that happen. I believe that developers are more
>>
>> willing
>>
>> to
>>
>> contribute when they see that their PRs will actually be able
>>
>> accepted
>>
>> and
>>
>> merged into the code base. Having a low number of open PRs in
>>
>> progress
>>
>> is a
>>
>> great indication that the main nifi developers are fully engaged
>>
>> with
>>
>> the
>>
>> community.
>>
>> There are a few PRs that don't have any comments from committers
>>
>> at
>>
>> all.
>>
>> I
>>
>> found one from August in that state. If that was my PR I don't
>>
>> think I
>>
>> would be so willing to put another one in anytime soon. I do get
>>
>> that
>>
>> sometime PRs get stalled by the originator, if so maybe a rule
>>
>> about
>>
>> closing them after a certain amount of time or being taken over
>>
>> by a
>>
>> core
>>
>> contributor if they think it worthwhile.
>>
>> I would like to shoutout to James Wing on my last PR he was
>>
>> quick
>>
>> to
>>
>> review, provided great comments, testing, and even some
>>
>> additional
>>
>> code.
>>
>> It
>>
>> was a great PR experience.
>>
>> Cheers,
>>
>> Edgardo
>>
>>
>>
>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>>
>> [hidden email] <javascript:;>.
>>
>> invalid> wrote:
>>
>> Joe, I think you misread. Edgardo is referring to the Pull
>>
>> Requests
>>
>> that
>>
>> are currently open, not the tickets assigned to the 1.1.0
>>
>> version.
>>
>>
>> I think these goals (releasing 1.1.0 and cutting down the PR
>>
>> count)
>>
>> should
>>
>> be two different efforts. Doing a thorough job reviewing
>>
>> takes a
>>
>> significant amount of time from both the reviewer and
>>
>> contributor.
>>
>> In
>>
>> order
>>
>> to cut it down significantly would take much longer than a
>>
>> couple
>>
>> days.
>>
>>
>> Also there has already been a lot of great new features and
>>
>> bug
>>
>> fixes
>>
>> contributed to the 1.X line and I don't think it's worth
>>
>> holding up
>>
>> a
>>
>> 1.1.0
>>
>> release for tickets not assigned to this fix version. As an
>>
>> added
>>
>> bonus
>>
>> though, I think many of the tickets tagged as 1.1.0 have PRs
>>
>> already
>>
>> open
>>
>> so closing those will make a large dent in the PR count.
>>
>>
>> Joe
>>
>> - - - - - -
>> Joseph Percivall
>> linkedin.com/in/Percivall
>> e: [hidden email] <javascript:;>
>>
>>
>>
>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>>
>> [hidden email]
>>
>> <javascript:;>>
>>
>> wrote:
>>
>>
>>
>> There are less than 30 right now.  Many of the roughly 90+
>>
>> JIRAs
>>
>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>>
>> or
>>
>> just
>>
>> had fix versions removed.
>>
>> We will need to have a push over the next bunch of days to
>>
>> deal
>>
>> with
>>
>> reviewing/merging/moving the remaining items.
>>
>> Thanks
>> Joe
>>
>>
>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>>
>> [hidden email] <javascript:;>>
>>
>> wrote:
>>
>> Joe,
>>
>> There are 75 PRs currently open. Why not make a push over
>>
>> the
>>
>> next
>>
>> bunch
>>
>> of
>>
>> days to get them closed and then cut the release after that.
>>
>> Cheers,
>>
>> Edgardo
>>
>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>>
>> [hidden email]
>>
>> <javascript:;>>
>>
>> wrote:
>>
>>
>> Team,
>>
>> There have been a ton of bugs fixed a few nice features.  I
>>
>> would
>>
>> like
>>
>> to move to get Apache NiFi 1.1.0 release going pretty much
>>
>> based
>>
>> on
>>
>> where we are now and plan to move most tickets to a new
>>
>> Apache
>>
>> NiFi
>>
>> 1.2.0 version.  We can try to get back on our roughly 6-8
>>
>> week
>>
>> release
>>
>> schedule and shoot for a mid to late Nov release for NiFi
>>
>> 1.2.0
>>
>> this
>>
>> way as well. Please advise if anyone has any other views on
>>
>> this. In
>>
>> the mean time I'll get the wheels in motion so you'll be
>>
>> seeing a
>>
>> lot
>>
>> of JIRA/issue updates to move version around.
>>
>> Thanks
>> Joe
>>
>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>>
>> [hidden email]
>>
>> <javascript:;>>
>>
>> wrote:
>>
>> Sounds good Joe. I have no issue to you doing the rm'ing
>>
>> for
>>
>> it.
>>
>>
>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>>
>> <javascript:;>> wrote:
>>
>>
>> Team,
>>
>> There are a lot of great fixes and improvements on the
>>
>> master
>>
>> line
>>
>> now
>>
>> and we're at a good time window to start pushing for a
>>
>> release.
>>
>> There
>>
>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>>
>> are
>>
>> open.
>>
>> I'm
>>
>> going to go through them and remove fix versions where
>>
>> appropriate.
>>
>>
>> I'm happy to take on RM task for this release though if
>>
>> someone
>>
>> else
>>
>> would like to take that on please advise.
>>
>> Thanks
>> Joe
>>
>>
>>
>>
>>
>> --
>> Cheers,
>>
>> Edgardo
>>
>>
>>
>>
>>
>> --
>> Cheers,
>>
>> Edgardo
>>
>>
>>
>>
>>
>> --
>> Cheers,
>>
>> Edgardo
>>
>>
>>
>>
>> --
>> Cheers,
>>
>> Edgardo
>>
>> Sent from Gmail Mobile
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> --
> I know what it is to be in need, and I know what it is to have plenty.  I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want.  I can do
> all this through him who gives me strength.    *-Philippians 4:12-13*
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Matt Burgess-2
I've reviewed NIFI-3064 and it's good to go, would like to add that one as well.

Thanks,
Matt

On Tue, Nov 22, 2016 at 10:51 AM, Joe Witt <[hidden email]> wrote:

> thanks Joe and Andy.  Have those on the list now.
>
> On Tue, Nov 22, 2016 at 5:09 AM, Joe Gresock <[hidden email]> wrote:
>> I'm biased, but I think the new Query / Scroll Elasticsearch processors are
>> pretty big ;)
>>
>> On Tue, Nov 22, 2016 at 9:02 AM, Andy LoPresto <[hidden email]> wrote:
>>
>>> Important to call out in the security section that
>>> login-identity-providers.xml is now covered by encrypted config, and
>>> flow.xml.gz processor property encryption/nifi.sensitive.props.key value
>>> can now be migrated by the command-line tool (second part pending
>>> completion of NIFI-3024).
>>>
>>> Andy LoPresto
>>> [hidden email]
>>> *[hidden email] <[hidden email]>*
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>> On Nov 21, 2016, at 9:30 PM, Joe Witt <[hidden email]> wrote:
>>>
>>> matt
>>>
>>> i'll add a wiki page or set of instructions linked from the release notes.
>>>
>>> all,
>>>
>>> walked through the 250 or so JIRAs in the 1.1.0 release and pulled out
>>> highlights.  The items noted are as follows.  Will likely reduce this
>>> down further for the release notes but wanted to put this out in case
>>> folks have things they think are really important to highlight.
>>>
>>> - Core Improvements:
>>>   - Performance: Session Migration
>>>   - Stability: Cluster Management
>>>   - Developer: Framework supports easy user driven classloader extension
>>>   - Expression Language: Now supports base64 and hex encoded values
>>> and Math functions
>>>   - Repositories now support rollback
>>>   - Faster startup due to more efficient state restoration algorithm
>>> - UX Improvements:
>>>   - Visual Backpressure Indicator
>>>   - Introduced more colors to better highlight actions and components
>>>   - Performance: Validate non-running components
>>>   - Provenance graph image can be exported
>>>   - Cron Scheduling for Primary node tasks now supported
>>> - Updated versions
>>>   - Azure Event Hub 0.9.0
>>>   - Spark 2.0.1
>>>   - Hadoop 2.7.x
>>> - New/Improved Processors
>>>   - new Fetch/Put Elastic Search 5.0
>>>   - new ParseCEF to parse CEF formatted logs
>>>   - improve ExtractEmail now supports TNEF files
>>>   - new Validate CSV
>>>   - improved Solr processors now support SSL and Kerberos
>>>   - new Websocket client and server processors
>>> - New Utility
>>>   - Zookeeper Migrator (move from one zookeeper to another)
>>> - Security
>>>   - Restricted Processors
>>>   - Site-to-site now supports port forwarding
>>>   - Improved Policy Management UX
>>> - Migration Notes:
>>>   - Restricted Processors
>>>   - Twitter Processor Removed
>>>
>>> On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]>
>>> wrote:
>>>
>>> Is there a good spot for us to put instructions on how to build the
>>> Twitter processor and/or the Social Media NAR in the meantime? Maybe a
>>> Wiki page or something simple to say "go to this directory, run this
>>> Maven command, drop the NAR into your deployment..." ?
>>>
>>> On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:
>>>
>>> Team,
>>>
>>> We appear to be very close.  Andy is working NIFI-3024 but otherwise
>>> it is focus on testing.
>>>
>>> I'm going to prep the RC and release notes now.  Unfortunately the
>>> twitter changes for json.org will need to remain.  Consensus forming
>>> on the legal-discuss thread regarding a grace period has been elusive
>>> and we're already prepared to make the right steps so we'll just need
>>> to take that on by being empathetic to the user base.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
>>>
>>> Andy,
>>>
>>> Great to see NIFI-3050 implemented and certainly good news that NiFi 1.1.0
>>> is set to include a number of security related improvements.
>>>
>>>
>>>
>>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]>
>>> wrote:
>>>
>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been
>>> added to my plate for this release. Coordinated with Joe Witt and they
>>> should both be included.
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
>>>
>>> Andy LoPresto
>>> [hidden email]
>>> *[hidden email] <[hidden email]>*
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
>>>
>>> Team
>>>
>>> There is a thread on apache legal-discuss that might allow for a
>>> graceperiod of continued usage of the json library.  Am going to keep
>>> a close eye on this and if VP Legal approves we'll be able to keep the
>>> twitter processors in which is definitely a good thing.  Will advise
>>>
>>> Thanks
>>> Joe
>>>
>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]> wrote:
>>>
>>> I've noticed an issue with the per-instance class loading capability
>>> introduced in NIFI-2909 where the additional classpath resources can get
>>> incorrectly removed from the class loader.
>>>
>>> I was able to reproduced this with a unit test and have a fix ready. I
>>> believe this is important and needs to go in for the 1.1 release, going to
>>> re-open NIFI-2909 and submit a PR shortly.
>>>
>>> -Bryan
>>>
>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]>
>>> wrote:
>>>
>>> I have two items that I would like to wrap up prior to creating an RC for
>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
>>> configuration. The work is already completed and I will be reviewing it
>>> this today. Additionally, following recent interest on the mailing list,
>>> I'd like to knock out NIFI-3020. This will allow an admin to configure a
>>> strategy for user identity when logging in via LDAP. Specifically, it will
>>> support usage of the DN (the default and current implementation) as well as
>>> the username the user logged in as. I should be able to have a PR up for
>>> this work later today.
>>>
>>> Thanks!
>>>
>>> Matt
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>>>
>>>
>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
>>>
>>> The code is within the twitter4j library itself.  I filed a request to
>>> twitter4jg.  The most likely case is we will need to submit a PR to them.
>>> However, I don't see this as something that should delay the release.  We
>>> can provide instructions for folks wanting to use the processor during
>>>
>>> the
>>>
>>> time we cannot make it available in a convenient manner.  I will provide
>>>
>>> a
>>>
>>> meaningful comment about this in release notes and pointers on what folks
>>> can do in the meantime.
>>>
>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]> wrote:
>>>
>>> I understand there was a discussion thread within the NiFi community
>>>
>>> for
>>>
>>> this as well and I missed responding to that at that time. It just
>>>
>>> seems
>>>
>>> to
>>>
>>> me like JSON processing is necessary for GetTwitter, which is
>>>
>>> incredibly
>>>
>>> useful for demonstrating NiFi’s ability to read from a high volume
>>>
>>> stream
>>>
>>> out of the box. With NIFI-3019 (Remove GetTwitter from default build),
>>>
>>> is
>>>
>>> there any related effort to substitute an acceptable replacement JSON
>>> library to restore this functionality?
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>>>
>>> Andy LoPresto
>>> [hidden email]
>>> *[hidden email] <[hidden email]>*
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>>>
>>> wrote:
>>>
>>>
>>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>>> investigated some alternate TLS config options for the new version of
>>>
>>> the
>>>
>>> client library.
>>>
>>> Is there any alternative to excluding the GetTwitter processor? Using
>>> Johnzon [1] or the Android re-implementation [2] discussed in the
>>>
>>> mailing
>>>
>>> list thread?
>>>
>>> [1] https://johnzon.apache.org/
>>> [2] https://developer.android.com/reference/org/json/package-
>>>
>>> summary.html
>>>
>>>
>>>
>>> Andy LoPresto
>>> [hidden email]
>>> *[hidden email] <[hidden email]>*
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>>>
>>> Team
>>>
>>> Very happy to see that we are down to three items remaining tagged to
>>> 1.1.0.  Solid effort over the recent weeks to close the gap including
>>>
>>> work
>>>
>>> to get past the now category x Jason dependency we had.  The most
>>>
>>> notable
>>>
>>> impact from that is the wildly popular GetTwitter processor, the fav
>>>
>>> new
>>>
>>> nifi user and demo processor, can no longer be included in the default
>>> build.  It is optionally available if users choose to build and use it
>>>
>>> but
>>>
>>> we won't distribute binaries that have it.
>>>
>>> I see some review movement on some patch available but untagged items.
>>>
>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
>>> have any outstanding items?
>>>
>>> Thanks
>>> Joe
>>>
>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>>>
>>> Ryan
>>>
>>> Not officially but I think we should try to close this thing out and
>>> start a vote in the next week or two at most.
>>>
>>> I'm going through the tickets again now.  There is also a new issue of
>>> the json-p license falling out of favor in Apache legal terms and
>>> becoming Category-X.  Am looking into that now.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>>>
>>> wrote:
>>>
>>>
>>> Joe - Is there a target date for 1.1?
>>>
>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]> wrote:
>>>
>>> Team,
>>>
>>> Just an update on things with working toward an Apache NiFi 1.1.0
>>> release.  There are still about 33 JIRAs there now and some are
>>> awaiting review and are some are under active progress. Yet there is
>>> good traction and progress. I think we should just stay vigilant with
>>> what makes it in and keep working it down.  So let's please shoot for
>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
>>>
>>> Team,
>>>
>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
>>> avoiding putting more in there for now at least without a discussion.
>>> Of the 31 JIRAs there the vast majority need review so we should be
>>> able to close these down fairly quickly as long as we don't let the
>>> list grow.
>>>
>>> Thanks
>>> joe
>>>
>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]>
>>>
>>> wrote:
>>>
>>> Joe,
>>>
>>> Appreciate the offer it isn't my PR. I was just using it as an
>>>
>>> example.
>>>
>>> All
>>>
>>> mine are currently closed, which I greatly appreciate.
>>>
>>> Cheers,
>>>
>>> Edgardo
>>>
>>> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>>>
>>> Edgardo,
>>>
>>> You mentioned a PR from August. I'd be happy to help you work that
>>> through review.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>>>
>>> [hidden email]
>>>
>>> <javascript:;>> wrote:
>>>
>>> I have agreed that at this point a release is important. My goal
>>>
>>> was
>>>
>>> try
>>>
>>> to
>>>
>>> squeeze in a much goodness as possible into the release, but the
>>>
>>> important
>>>
>>> bug fixes should come first. Getting 1.x into a state where the
>>>
>>> release
>>>
>>> notes don't say that it is geared toward developers and testers is
>>>
>>> really
>>>
>>> huge.
>>>
>>> I think Nifi is a great community otherwise I would participate in
>>>
>>> the
>>>
>>> mailing list, create Jira tickets and pull requests. I am only
>>>
>>> trying to
>>>
>>> strengthen the great thing that is going on here. We can always do
>>>
>>> better.
>>>
>>> I was not trying to put down this community only to participate and
>>>
>>> make
>>>
>>> it
>>>
>>> better. I think this conversation is an indication of how great
>>>
>>> this
>>>
>>> community is.
>>>
>>> Maybe I am being sensitive about this issue and trying to
>>>
>>> strengthen
>>>
>>> the
>>>
>>> nifi community even more, after coming from a conference where it
>>>
>>> was
>>>
>>> reported there was lots of excitement at first and now the
>>>
>>> participation
>>>
>>> in
>>>
>>> the community has really died down and they are struggling. I don't
>>>
>>> want
>>>
>>> to
>>>
>>> see that happen here.
>>>
>>> Cheers,
>>>
>>> Edgardo
>>>
>>>
>>>
>>>
>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>>>
>>> <javascript:;>> wrote:
>>>
>>>
>>> Edgardo,
>>>
>>> Thank you for your feedback. We hear your comments and as a
>>>
>>> committer I
>>>
>>> can
>>>
>>> share we are constantly looking to improve the PR process, having
>>>
>>> already
>>>
>>> taken many of the steps you suggest.
>>>
>>> However, it is important to notice that the number of PRs should
>>>
>>> not be
>>>
>>> seen as a metric of engagement by the development community: Most
>>>
>>> of us
>>>
>>> will submit PRs so that our work can be carefully reviewed by our
>>>
>>> peers
>>>
>>> and
>>>
>>> some of us will use JIRA patches to provide contributions.
>>>
>>> Having said that, it is true that some PRs may sit idle for a long
>>>
>>> time
>>>
>>> and
>>>
>>> we are working to improve this pipeline.
>>>
>>> It was therefore no coincidence that I  browsed most of the PRs
>>>
>>> performing
>>>
>>> a triage of items that have been superseded or diverged from the
>>>
>>> current
>>>
>>> code base.
>>>
>>> In fact, less than a month ago the dev team closed a number of
>>>
>>> stalled
>>>
>>> and
>>>
>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>>>
>>> Despite all the above, I think Joe has a point. The master
>>>
>>> contain a
>>>
>>> series
>>>
>>> of important bug fixes and suspect the community would benefit
>>>
>>> from
>>>
>>> a
>>>
>>> release sooner rather than later.
>>>
>>> Once again, thank you for your feedback and contribution. It is
>>>
>>> good to
>>>
>>> have you here.
>>>
>>> Andre
>>>
>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>>>
>>> [hidden email]
>>>
>>> <javascript:;>>
>>>
>>> wrote:
>>>
>>> Joe - You are correct I was mentioning the PRs that are
>>>
>>> currently
>>>
>>> open.
>>>
>>>
>>> Regardless of how it happens reducing the count of open PRs I
>>>
>>> believe
>>>
>>> to
>>>
>>> be
>>>
>>> extremely important. Maybe I was hoping that the release could
>>>
>>> be
>>>
>>> a
>>>
>>> forcing
>>>
>>> function to make that happen. I believe that developers are more
>>>
>>> willing
>>>
>>> to
>>>
>>> contribute when they see that their PRs will actually be able
>>>
>>> accepted
>>>
>>> and
>>>
>>> merged into the code base. Having a low number of open PRs in
>>>
>>> progress
>>>
>>> is a
>>>
>>> great indication that the main nifi developers are fully engaged
>>>
>>> with
>>>
>>> the
>>>
>>> community.
>>>
>>> There are a few PRs that don't have any comments from committers
>>>
>>> at
>>>
>>> all.
>>>
>>> I
>>>
>>> found one from August in that state. If that was my PR I don't
>>>
>>> think I
>>>
>>> would be so willing to put another one in anytime soon. I do get
>>>
>>> that
>>>
>>> sometime PRs get stalled by the originator, if so maybe a rule
>>>
>>> about
>>>
>>> closing them after a certain amount of time or being taken over
>>>
>>> by a
>>>
>>> core
>>>
>>> contributor if they think it worthwhile.
>>>
>>> I would like to shoutout to James Wing on my last PR he was
>>>
>>> quick
>>>
>>> to
>>>
>>> review, provided great comments, testing, and even some
>>>
>>> additional
>>>
>>> code.
>>>
>>> It
>>>
>>> was a great PR experience.
>>>
>>> Cheers,
>>>
>>> Edgardo
>>>
>>>
>>>
>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>>>
>>> [hidden email] <javascript:;>.
>>>
>>> invalid> wrote:
>>>
>>> Joe, I think you misread. Edgardo is referring to the Pull
>>>
>>> Requests
>>>
>>> that
>>>
>>> are currently open, not the tickets assigned to the 1.1.0
>>>
>>> version.
>>>
>>>
>>> I think these goals (releasing 1.1.0 and cutting down the PR
>>>
>>> count)
>>>
>>> should
>>>
>>> be two different efforts. Doing a thorough job reviewing
>>>
>>> takes a
>>>
>>> significant amount of time from both the reviewer and
>>>
>>> contributor.
>>>
>>> In
>>>
>>> order
>>>
>>> to cut it down significantly would take much longer than a
>>>
>>> couple
>>>
>>> days.
>>>
>>>
>>> Also there has already been a lot of great new features and
>>>
>>> bug
>>>
>>> fixes
>>>
>>> contributed to the 1.X line and I don't think it's worth
>>>
>>> holding up
>>>
>>> a
>>>
>>> 1.1.0
>>>
>>> release for tickets not assigned to this fix version. As an
>>>
>>> added
>>>
>>> bonus
>>>
>>> though, I think many of the tickets tagged as 1.1.0 have PRs
>>>
>>> already
>>>
>>> open
>>>
>>> so closing those will make a large dent in the PR count.
>>>
>>>
>>> Joe
>>>
>>> - - - - - -
>>> Joseph Percivall
>>> linkedin.com/in/Percivall
>>> e: [hidden email] <javascript:;>
>>>
>>>
>>>
>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>>>
>>> [hidden email]
>>>
>>> <javascript:;>>
>>>
>>> wrote:
>>>
>>>
>>>
>>> There are less than 30 right now.  Many of the roughly 90+
>>>
>>> JIRAs
>>>
>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>>>
>>> or
>>>
>>> just
>>>
>>> had fix versions removed.
>>>
>>> We will need to have a push over the next bunch of days to
>>>
>>> deal
>>>
>>> with
>>>
>>> reviewing/merging/moving the remaining items.
>>>
>>> Thanks
>>> Joe
>>>
>>>
>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>>>
>>> [hidden email] <javascript:;>>
>>>
>>> wrote:
>>>
>>> Joe,
>>>
>>> There are 75 PRs currently open. Why not make a push over
>>>
>>> the
>>>
>>> next
>>>
>>> bunch
>>>
>>> of
>>>
>>> days to get them closed and then cut the release after that.
>>>
>>> Cheers,
>>>
>>> Edgardo
>>>
>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>>>
>>> [hidden email]
>>>
>>> <javascript:;>>
>>>
>>> wrote:
>>>
>>>
>>> Team,
>>>
>>> There have been a ton of bugs fixed a few nice features.  I
>>>
>>> would
>>>
>>> like
>>>
>>> to move to get Apache NiFi 1.1.0 release going pretty much
>>>
>>> based
>>>
>>> on
>>>
>>> where we are now and plan to move most tickets to a new
>>>
>>> Apache
>>>
>>> NiFi
>>>
>>> 1.2.0 version.  We can try to get back on our roughly 6-8
>>>
>>> week
>>>
>>> release
>>>
>>> schedule and shoot for a mid to late Nov release for NiFi
>>>
>>> 1.2.0
>>>
>>> this
>>>
>>> way as well. Please advise if anyone has any other views on
>>>
>>> this. In
>>>
>>> the mean time I'll get the wheels in motion so you'll be
>>>
>>> seeing a
>>>
>>> lot
>>>
>>> of JIRA/issue updates to move version around.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>>>
>>> [hidden email]
>>>
>>> <javascript:;>>
>>>
>>> wrote:
>>>
>>> Sounds good Joe. I have no issue to you doing the rm'ing
>>>
>>> for
>>>
>>> it.
>>>
>>>
>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>>>
>>> <javascript:;>> wrote:
>>>
>>>
>>> Team,
>>>
>>> There are a lot of great fixes and improvements on the
>>>
>>> master
>>>
>>> line
>>>
>>> now
>>>
>>> and we're at a good time window to start pushing for a
>>>
>>> release.
>>>
>>> There
>>>
>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>>>
>>> are
>>>
>>> open.
>>>
>>> I'm
>>>
>>> going to go through them and remove fix versions where
>>>
>>> appropriate.
>>>
>>>
>>> I'm happy to take on RM task for this release though if
>>>
>>> someone
>>>
>>> else
>>>
>>> would like to take that on please advise.
>>>
>>> Thanks
>>> Joe
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>>
>>> Edgardo
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>>
>>> Edgardo
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>>
>>> Edgardo
>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>>
>>> Edgardo
>>>
>>> Sent from Gmail Mobile
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> I know what it is to be in need, and I know what it is to have plenty.  I
>> have learned the secret of being content in any and every situation,
>> whether well fed or hungry, whether living in plenty or in want.  I can do
>> all this through him who gives me strength.    *-Philippians 4:12-13*
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

James Wing
In reply to this post by Joe Witt
Going through the tickets, it seems like quite a release.  A few more
things for your list:

GenerateFlowFile updated to support literal/expression content and
attributes

AWS-related:
* New processors PutCloudWatchMetric, PutKinesisStream
* Updated processors PutS3Object (content type, signer options), ListS3
(performance, versions)
* Added support for AWS assume role credentials with proxy


Thanks,

James

On Mon, Nov 21, 2016 at 9:30 PM, Joe Witt <[hidden email]> wrote:

> matt
>
> i'll add a wiki page or set of instructions linked from the release notes.
>
> all,
>
> walked through the 250 or so JIRAs in the 1.1.0 release and pulled out
> highlights.  The items noted are as follows.  Will likely reduce this
> down further for the release notes but wanted to put this out in case
> folks have things they think are really important to highlight.
>
> - Core Improvements:
>    - Performance: Session Migration
>    - Stability: Cluster Management
>    - Developer: Framework supports easy user driven classloader extension
>    - Expression Language: Now supports base64 and hex encoded values
> and Math functions
>    - Repositories now support rollback
>    - Faster startup due to more efficient state restoration algorithm
> - UX Improvements:
>    - Visual Backpressure Indicator
>    - Introduced more colors to better highlight actions and components
>    - Performance: Validate non-running components
>    - Provenance graph image can be exported
>    - Cron Scheduling for Primary node tasks now supported
> - Updated versions
>    - Azure Event Hub 0.9.0
>    - Spark 2.0.1
>    - Hadoop 2.7.x
> - New/Improved Processors
>    - new Fetch/Put Elastic Search 5.0
>    - new ParseCEF to parse CEF formatted logs
>    - improve ExtractEmail now supports TNEF files
>    - new Validate CSV
>    - improved Solr processors now support SSL and Kerberos
>    - new Websocket client and server processors
> - New Utility
>    - Zookeeper Migrator (move from one zookeeper to another)
> - Security
>    - Restricted Processors
>    - Site-to-site now supports port forwarding
>    - Improved Policy Management UX
> - Migration Notes:
>    - Restricted Processors
>    - Twitter Processor Removed
>
> On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]>
> wrote:
> > Is there a good spot for us to put instructions on how to build the
> > Twitter processor and/or the Social Media NAR in the meantime? Maybe a
> > Wiki page or something simple to say "go to this directory, run this
> > Maven command, drop the NAR into your deployment..." ?
> >
> > On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:
> >> Team,
> >>
> >> We appear to be very close.  Andy is working NIFI-3024 but otherwise
> >> it is focus on testing.
> >>
> >> I'm going to prep the RC and release notes now.  Unfortunately the
> >> twitter changes for json.org will need to remain.  Consensus forming
> >> on the legal-discuss thread regarding a grace period has been elusive
> >> and we're already prepared to make the right steps so we'll just need
> >> to take that on by being empathetic to the user base.
> >>
> >> Thanks
> >> Joe
> >>
> >> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
> >>> Andy,
> >>>
> >>> Great to see NIFI-3050 implemented and certainly good news that NiFi
> 1.1.0
> >>> is set to include a number of security related improvements.
> >>>
> >>>
> >>>
> >>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]>
> wrote:
> >>>
> >>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have
> been
> >>>> added to my plate for this release. Coordinated with Joe Witt and they
> >>>> should both be included.
> >>>>
> >>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
> >>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
> >>>>
> >>>> Andy LoPresto
> >>>> [hidden email]
> >>>> *[hidden email] <[hidden email]>*
> >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>>>
> >>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
> >>>>
> >>>> Team
> >>>>
> >>>> There is a thread on apache legal-discuss that might allow for a
> >>>> graceperiod of continued usage of the json library.  Am going to keep
> >>>> a close eye on this and if VP Legal approves we'll be able to keep the
> >>>> twitter processors in which is definitely a good thing.  Will advise
> >>>>
> >>>> Thanks
> >>>> Joe
> >>>>
> >>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]>
> wrote:
> >>>>
> >>>> I've noticed an issue with the per-instance class loading capability
> >>>> introduced in NIFI-2909 where the additional classpath resources can
> get
> >>>> incorrectly removed from the class loader.
> >>>>
> >>>> I was able to reproduced this with a unit test and have a fix ready. I
> >>>> believe this is important and needs to go in for the 1.1 release,
> going to
> >>>> re-open NIFI-2909 and submit a PR shortly.
> >>>>
> >>>> -Bryan
> >>>>
> >>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]
> >
> >>>> wrote:
> >>>>
> >>>> I have two items that I would like to wrap up prior to creating an RC
> for
> >>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group
> port
> >>>> configuration. The work is already completed and I will be reviewing
> it
> >>>> this today. Additionally, following recent interest on the mailing
> list,
> >>>> I'd like to knock out NIFI-3020. This will allow an admin to
> configure a
> >>>> strategy for user identity when logging in via LDAP. Specifically, it
> will
> >>>> support usage of the DN (the default and current implementation) as
> well as
> >>>> the username the user logged in as. I should be able to have a PR up
> for
> >>>> this work later today.
> >>>>
> >>>> Thanks!
> >>>>
> >>>> Matt
> >>>>
> >>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
> >>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
> >>>>
> >>>>
> >>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
> >>>>
> >>>> The code is within the twitter4j library itself.  I filed a request to
> >>>> twitter4jg.  The most likely case is we will need to submit a PR to
> them.
> >>>> However, I don't see this as something that should delay the
> release.  We
> >>>> can provide instructions for folks wanting to use the processor during
> >>>>
> >>>> the
> >>>>
> >>>> time we cannot make it available in a convenient manner.  I will
> provide
> >>>>
> >>>> a
> >>>>
> >>>> meaningful comment about this in release notes and pointers on what
> folks
> >>>> can do in the meantime.
> >>>>
> >>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]>
> wrote:
> >>>>
> >>>> I understand there was a discussion thread within the NiFi community
> >>>>
> >>>> for
> >>>>
> >>>> this as well and I missed responding to that at that time. It just
> >>>>
> >>>> seems
> >>>>
> >>>> to
> >>>>
> >>>> me like JSON processing is necessary for GetTwitter, which is
> >>>>
> >>>> incredibly
> >>>>
> >>>> useful for demonstrating NiFi’s ability to read from a high volume
> >>>>
> >>>> stream
> >>>>
> >>>> out of the box. With NIFI-3019 (Remove GetTwitter from default build),
> >>>>
> >>>> is
> >>>>
> >>>> there any related effort to substitute an acceptable replacement JSON
> >>>> library to restore this functionality?
> >>>>
> >>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
> >>>>
> >>>> Andy LoPresto
> >>>> [hidden email]
> >>>> *[hidden email] <[hidden email]>*
> >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>>>
> >>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
> >>>>
> >>>> wrote:
> >>>>
> >>>>
> >>>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
> >>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
> >>>> investigated some alternate TLS config options for the new version of
> >>>>
> >>>> the
> >>>>
> >>>> client library.
> >>>>
> >>>> Is there any alternative to excluding the GetTwitter processor? Using
> >>>> Johnzon [1] or the Android re-implementation [2] discussed in the
> >>>>
> >>>> mailing
> >>>>
> >>>> list thread?
> >>>>
> >>>> [1] https://johnzon.apache.org/
> >>>> [2] https://developer.android.com/reference/org/json/package-
> >>>>
> >>>> summary.html
> >>>>
> >>>>
> >>>>
> >>>> Andy LoPresto
> >>>> [hidden email]
> >>>> *[hidden email] <[hidden email]>*
> >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>>>
> >>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
> >>>>
> >>>> Team
> >>>>
> >>>> Very happy to see that we are down to three items remaining tagged to
> >>>> 1.1.0.  Solid effort over the recent weeks to close the gap including
> >>>>
> >>>> work
> >>>>
> >>>> to get past the now category x Jason dependency we had.  The most
> >>>>
> >>>> notable
> >>>>
> >>>> impact from that is the wildly popular GetTwitter processor, the fav
> >>>>
> >>>> new
> >>>>
> >>>> nifi user and demo processor, can no longer be included in the default
> >>>> build.  It is optionally available if users choose to build and use it
> >>>>
> >>>> but
> >>>>
> >>>> we won't distribute binaries that have it.
> >>>>
> >>>> I see some review movement on some patch available but untagged items.
> >>>>
> >>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri.
> Anyone
> >>>> have any outstanding items?
> >>>>
> >>>> Thanks
> >>>> Joe
> >>>>
> >>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
> >>>>
> >>>> Ryan
> >>>>
> >>>> Not officially but I think we should try to close this thing out and
> >>>> start a vote in the next week or two at most.
> >>>>
> >>>> I'm going through the tickets again now.  There is also a new issue of
> >>>> the json-p license falling out of favor in Apache legal terms and
> >>>> becoming Category-X.  Am looking into that now.
> >>>>
> >>>> Thanks
> >>>> Joe
> >>>>
> >>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
> >>>>
> >>>> wrote:
> >>>>
> >>>>
> >>>> Joe - Is there a target date for 1.1?
> >>>>
> >>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]>
> wrote:
> >>>>
> >>>> Team,
> >>>>
> >>>> Just an update on things with working toward an Apache NiFi 1.1.0
> >>>> release.  There are still about 33 JIRAs there now and some are
> >>>> awaiting review and are some are under active progress. Yet there is
> >>>> good traction and progress. I think we should just stay vigilant with
> >>>> what makes it in and keep working it down.  So let's please shoot for
> >>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
> >>>>
> >>>> Thanks
> >>>> Joe
> >>>>
> >>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
> >>>>
> >>>> Team,
> >>>>
> >>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
> >>>> avoiding putting more in there for now at least without a discussion.
> >>>> Of the 31 JIRAs there the vast majority need review so we should be
> >>>> able to close these down fairly quickly as long as we don't let the
> >>>> list grow.
> >>>>
> >>>> Thanks
> >>>> joe
> >>>>
> >>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]
> >
> >>>>
> >>>> wrote:
> >>>>
> >>>> Joe,
> >>>>
> >>>> Appreciate the offer it isn't my PR. I was just using it as an
> >>>>
> >>>> example.
> >>>>
> >>>> All
> >>>>
> >>>> mine are currently closed, which I greatly appreciate.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Edgardo
> >>>>
> >>>> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
> >>>>
> >>>> Edgardo,
> >>>>
> >>>> You mentioned a PR from August. I'd be happy to help you work that
> >>>> through review.
> >>>>
> >>>> Thanks
> >>>> Joe
> >>>>
> >>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
> >>>>
> >>>> [hidden email]
> >>>>
> >>>> <javascript:;>> wrote:
> >>>>
> >>>> I have agreed that at this point a release is important. My goal
> >>>>
> >>>> was
> >>>>
> >>>> try
> >>>>
> >>>> to
> >>>>
> >>>> squeeze in a much goodness as possible into the release, but the
> >>>>
> >>>> important
> >>>>
> >>>> bug fixes should come first. Getting 1.x into a state where the
> >>>>
> >>>> release
> >>>>
> >>>> notes don't say that it is geared toward developers and testers is
> >>>>
> >>>> really
> >>>>
> >>>> huge.
> >>>>
> >>>> I think Nifi is a great community otherwise I would participate in
> >>>>
> >>>> the
> >>>>
> >>>> mailing list, create Jira tickets and pull requests. I am only
> >>>>
> >>>> trying to
> >>>>
> >>>> strengthen the great thing that is going on here. We can always do
> >>>>
> >>>> better.
> >>>>
> >>>> I was not trying to put down this community only to participate and
> >>>>
> >>>> make
> >>>>
> >>>> it
> >>>>
> >>>> better. I think this conversation is an indication of how great
> >>>>
> >>>> this
> >>>>
> >>>> community is.
> >>>>
> >>>> Maybe I am being sensitive about this issue and trying to
> >>>>
> >>>> strengthen
> >>>>
> >>>> the
> >>>>
> >>>> nifi community even more, after coming from a conference where it
> >>>>
> >>>> was
> >>>>
> >>>> reported there was lots of excitement at first and now the
> >>>>
> >>>> participation
> >>>>
> >>>> in
> >>>>
> >>>> the community has really died down and they are struggling. I don't
> >>>>
> >>>> want
> >>>>
> >>>> to
> >>>>
> >>>> see that happen here.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Edgardo
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
> >>>>
> >>>> <javascript:;>> wrote:
> >>>>
> >>>>
> >>>> Edgardo,
> >>>>
> >>>> Thank you for your feedback. We hear your comments and as a
> >>>>
> >>>> committer I
> >>>>
> >>>> can
> >>>>
> >>>> share we are constantly looking to improve the PR process, having
> >>>>
> >>>> already
> >>>>
> >>>> taken many of the steps you suggest.
> >>>>
> >>>> However, it is important to notice that the number of PRs should
> >>>>
> >>>> not be
> >>>>
> >>>> seen as a metric of engagement by the development community: Most
> >>>>
> >>>> of us
> >>>>
> >>>> will submit PRs so that our work can be carefully reviewed by our
> >>>>
> >>>> peers
> >>>>
> >>>> and
> >>>>
> >>>> some of us will use JIRA patches to provide contributions.
> >>>>
> >>>> Having said that, it is true that some PRs may sit idle for a long
> >>>>
> >>>> time
> >>>>
> >>>> and
> >>>>
> >>>> we are working to improve this pipeline.
> >>>>
> >>>> It was therefore no coincidence that I  browsed most of the PRs
> >>>>
> >>>> performing
> >>>>
> >>>> a triage of items that have been superseded or diverged from the
> >>>>
> >>>> current
> >>>>
> >>>> code base.
> >>>>
> >>>> In fact, less than a month ago the dev team closed a number of
> >>>>
> >>>> stalled
> >>>>
> >>>> and
> >>>>
> >>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
> >>>>
> >>>> Despite all the above, I think Joe has a point. The master
> >>>>
> >>>> contain a
> >>>>
> >>>> series
> >>>>
> >>>> of important bug fixes and suspect the community would benefit
> >>>>
> >>>> from
> >>>>
> >>>> a
> >>>>
> >>>> release sooner rather than later.
> >>>>
> >>>> Once again, thank you for your feedback and contribution. It is
> >>>>
> >>>> good to
> >>>>
> >>>> have you here.
> >>>>
> >>>> Andre
> >>>>
> >>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
> >>>>
> >>>> [hidden email]
> >>>>
> >>>> <javascript:;>>
> >>>>
> >>>> wrote:
> >>>>
> >>>> Joe - You are correct I was mentioning the PRs that are
> >>>>
> >>>> currently
> >>>>
> >>>> open.
> >>>>
> >>>>
> >>>> Regardless of how it happens reducing the count of open PRs I
> >>>>
> >>>> believe
> >>>>
> >>>> to
> >>>>
> >>>> be
> >>>>
> >>>> extremely important. Maybe I was hoping that the release could
> >>>>
> >>>> be
> >>>>
> >>>> a
> >>>>
> >>>> forcing
> >>>>
> >>>> function to make that happen. I believe that developers are more
> >>>>
> >>>> willing
> >>>>
> >>>> to
> >>>>
> >>>> contribute when they see that their PRs will actually be able
> >>>>
> >>>> accepted
> >>>>
> >>>> and
> >>>>
> >>>> merged into the code base. Having a low number of open PRs in
> >>>>
> >>>> progress
> >>>>
> >>>> is a
> >>>>
> >>>> great indication that the main nifi developers are fully engaged
> >>>>
> >>>> with
> >>>>
> >>>> the
> >>>>
> >>>> community.
> >>>>
> >>>> There are a few PRs that don't have any comments from committers
> >>>>
> >>>> at
> >>>>
> >>>> all.
> >>>>
> >>>> I
> >>>>
> >>>> found one from August in that state. If that was my PR I don't
> >>>>
> >>>> think I
> >>>>
> >>>> would be so willing to put another one in anytime soon. I do get
> >>>>
> >>>> that
> >>>>
> >>>> sometime PRs get stalled by the originator, if so maybe a rule
> >>>>
> >>>> about
> >>>>
> >>>> closing them after a certain amount of time or being taken over
> >>>>
> >>>> by a
> >>>>
> >>>> core
> >>>>
> >>>> contributor if they think it worthwhile.
> >>>>
> >>>> I would like to shoutout to James Wing on my last PR he was
> >>>>
> >>>> quick
> >>>>
> >>>> to
> >>>>
> >>>> review, provided great comments, testing, and even some
> >>>>
> >>>> additional
> >>>>
> >>>> code.
> >>>>
> >>>> It
> >>>>
> >>>> was a great PR experience.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Edgardo
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
> >>>>
> >>>> [hidden email] <javascript:;>.
> >>>>
> >>>> invalid> wrote:
> >>>>
> >>>> Joe, I think you misread. Edgardo is referring to the Pull
> >>>>
> >>>> Requests
> >>>>
> >>>> that
> >>>>
> >>>> are currently open, not the tickets assigned to the 1.1.0
> >>>>
> >>>> version.
> >>>>
> >>>>
> >>>> I think these goals (releasing 1.1.0 and cutting down the PR
> >>>>
> >>>> count)
> >>>>
> >>>> should
> >>>>
> >>>> be two different efforts. Doing a thorough job reviewing
> >>>>
> >>>> takes a
> >>>>
> >>>> significant amount of time from both the reviewer and
> >>>>
> >>>> contributor.
> >>>>
> >>>> In
> >>>>
> >>>> order
> >>>>
> >>>> to cut it down significantly would take much longer than a
> >>>>
> >>>> couple
> >>>>
> >>>> days.
> >>>>
> >>>>
> >>>> Also there has already been a lot of great new features and
> >>>>
> >>>> bug
> >>>>
> >>>> fixes
> >>>>
> >>>> contributed to the 1.X line and I don't think it's worth
> >>>>
> >>>> holding up
> >>>>
> >>>> a
> >>>>
> >>>> 1.1.0
> >>>>
> >>>> release for tickets not assigned to this fix version. As an
> >>>>
> >>>> added
> >>>>
> >>>> bonus
> >>>>
> >>>> though, I think many of the tickets tagged as 1.1.0 have PRs
> >>>>
> >>>> already
> >>>>
> >>>> open
> >>>>
> >>>> so closing those will make a large dent in the PR count.
> >>>>
> >>>>
> >>>> Joe
> >>>>
> >>>> - - - - - -
> >>>> Joseph Percivall
> >>>> linkedin.com/in/Percivall
> >>>> e: [hidden email] <javascript:;>
> >>>>
> >>>>
> >>>>
> >>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
> >>>>
> >>>> [hidden email]
> >>>>
> >>>> <javascript:;>>
> >>>>
> >>>> wrote:
> >>>>
> >>>>
> >>>>
> >>>> There are less than 30 right now.  Many of the roughly 90+
> >>>>
> >>>> JIRAs
> >>>>
> >>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
> >>>>
> >>>> or
> >>>>
> >>>> just
> >>>>
> >>>> had fix versions removed.
> >>>>
> >>>> We will need to have a push over the next bunch of days to
> >>>>
> >>>> deal
> >>>>
> >>>> with
> >>>>
> >>>> reviewing/merging/moving the remaining items.
> >>>>
> >>>> Thanks
> >>>> Joe
> >>>>
> >>>>
> >>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
> >>>>
> >>>> [hidden email] <javascript:;>>
> >>>>
> >>>> wrote:
> >>>>
> >>>> Joe,
> >>>>
> >>>> There are 75 PRs currently open. Why not make a push over
> >>>>
> >>>> the
> >>>>
> >>>> next
> >>>>
> >>>> bunch
> >>>>
> >>>> of
> >>>>
> >>>> days to get them closed and then cut the release after that.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Edgardo
> >>>>
> >>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
> >>>>
> >>>> [hidden email]
> >>>>
> >>>> <javascript:;>>
> >>>>
> >>>> wrote:
> >>>>
> >>>>
> >>>> Team,
> >>>>
> >>>> There have been a ton of bugs fixed a few nice features.  I
> >>>>
> >>>> would
> >>>>
> >>>> like
> >>>>
> >>>> to move to get Apache NiFi 1.1.0 release going pretty much
> >>>>
> >>>> based
> >>>>
> >>>> on
> >>>>
> >>>> where we are now and plan to move most tickets to a new
> >>>>
> >>>> Apache
> >>>>
> >>>> NiFi
> >>>>
> >>>> 1.2.0 version.  We can try to get back on our roughly 6-8
> >>>>
> >>>> week
> >>>>
> >>>> release
> >>>>
> >>>> schedule and shoot for a mid to late Nov release for NiFi
> >>>>
> >>>> 1.2.0
> >>>>
> >>>> this
> >>>>
> >>>> way as well. Please advise if anyone has any other views on
> >>>>
> >>>> this. In
> >>>>
> >>>> the mean time I'll get the wheels in motion so you'll be
> >>>>
> >>>> seeing a
> >>>>
> >>>> lot
> >>>>
> >>>> of JIRA/issue updates to move version around.
> >>>>
> >>>> Thanks
> >>>> Joe
> >>>>
> >>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
> >>>>
> >>>> [hidden email]
> >>>>
> >>>> <javascript:;>>
> >>>>
> >>>> wrote:
> >>>>
> >>>> Sounds good Joe. I have no issue to you doing the rm'ing
> >>>>
> >>>> for
> >>>>
> >>>> it.
> >>>>
> >>>>
> >>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
> >>>>
> >>>> <javascript:;>> wrote:
> >>>>
> >>>>
> >>>> Team,
> >>>>
> >>>> There are a lot of great fixes and improvements on the
> >>>>
> >>>> master
> >>>>
> >>>> line
> >>>>
> >>>> now
> >>>>
> >>>> and we're at a good time window to start pushing for a
> >>>>
> >>>> release.
> >>>>
> >>>> There
> >>>>
> >>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
> >>>>
> >>>> are
> >>>>
> >>>> open.
> >>>>
> >>>> I'm
> >>>>
> >>>> going to go through them and remove fix versions where
> >>>>
> >>>> appropriate.
> >>>>
> >>>>
> >>>> I'm happy to take on RM task for this release though if
> >>>>
> >>>> someone
> >>>>
> >>>> else
> >>>>
> >>>> would like to take that on please advise.
> >>>>
> >>>> Thanks
> >>>> Joe
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Cheers,
> >>>>
> >>>> Edgardo
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Cheers,
> >>>>
> >>>> Edgardo
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Cheers,
> >>>>
> >>>> Edgardo
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Cheers,
> >>>>
> >>>> Edgardo
> >>>>
> >>>> Sent from Gmail Mobile
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Joe Witt
Thanks James.  Will add the AWS stuff now.  The generate flow file
I'll leave off since it is mostly a developer/debug tool unless you
disagree.

Thanks
Joe

On Tue, Nov 22, 2016 at 12:08 PM, James Wing <[hidden email]> wrote:

> Going through the tickets, it seems like quite a release.  A few more
> things for your list:
>
> GenerateFlowFile updated to support literal/expression content and
> attributes
>
> AWS-related:
> * New processors PutCloudWatchMetric, PutKinesisStream
> * Updated processors PutS3Object (content type, signer options), ListS3
> (performance, versions)
> * Added support for AWS assume role credentials with proxy
>
>
> Thanks,
>
> James
>
> On Mon, Nov 21, 2016 at 9:30 PM, Joe Witt <[hidden email]> wrote:
>
>> matt
>>
>> i'll add a wiki page or set of instructions linked from the release notes.
>>
>> all,
>>
>> walked through the 250 or so JIRAs in the 1.1.0 release and pulled out
>> highlights.  The items noted are as follows.  Will likely reduce this
>> down further for the release notes but wanted to put this out in case
>> folks have things they think are really important to highlight.
>>
>> - Core Improvements:
>>    - Performance: Session Migration
>>    - Stability: Cluster Management
>>    - Developer: Framework supports easy user driven classloader extension
>>    - Expression Language: Now supports base64 and hex encoded values
>> and Math functions
>>    - Repositories now support rollback
>>    - Faster startup due to more efficient state restoration algorithm
>> - UX Improvements:
>>    - Visual Backpressure Indicator
>>    - Introduced more colors to better highlight actions and components
>>    - Performance: Validate non-running components
>>    - Provenance graph image can be exported
>>    - Cron Scheduling for Primary node tasks now supported
>> - Updated versions
>>    - Azure Event Hub 0.9.0
>>    - Spark 2.0.1
>>    - Hadoop 2.7.x
>> - New/Improved Processors
>>    - new Fetch/Put Elastic Search 5.0
>>    - new ParseCEF to parse CEF formatted logs
>>    - improve ExtractEmail now supports TNEF files
>>    - new Validate CSV
>>    - improved Solr processors now support SSL and Kerberos
>>    - new Websocket client and server processors
>> - New Utility
>>    - Zookeeper Migrator (move from one zookeeper to another)
>> - Security
>>    - Restricted Processors
>>    - Site-to-site now supports port forwarding
>>    - Improved Policy Management UX
>> - Migration Notes:
>>    - Restricted Processors
>>    - Twitter Processor Removed
>>
>> On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]>
>> wrote:
>> > Is there a good spot for us to put instructions on how to build the
>> > Twitter processor and/or the Social Media NAR in the meantime? Maybe a
>> > Wiki page or something simple to say "go to this directory, run this
>> > Maven command, drop the NAR into your deployment..." ?
>> >
>> > On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:
>> >> Team,
>> >>
>> >> We appear to be very close.  Andy is working NIFI-3024 but otherwise
>> >> it is focus on testing.
>> >>
>> >> I'm going to prep the RC and release notes now.  Unfortunately the
>> >> twitter changes for json.org will need to remain.  Consensus forming
>> >> on the legal-discuss thread regarding a grace period has been elusive
>> >> and we're already prepared to make the right steps so we'll just need
>> >> to take that on by being empathetic to the user base.
>> >>
>> >> Thanks
>> >> Joe
>> >>
>> >> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
>> >>> Andy,
>> >>>
>> >>> Great to see NIFI-3050 implemented and certainly good news that NiFi
>> 1.1.0
>> >>> is set to include a number of security related improvements.
>> >>>
>> >>>
>> >>>
>> >>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]>
>> wrote:
>> >>>
>> >>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have
>> been
>> >>>> added to my plate for this release. Coordinated with Joe Witt and they
>> >>>> should both be included.
>> >>>>
>> >>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
>> >>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
>> >>>>
>> >>>> Andy LoPresto
>> >>>> [hidden email]
>> >>>> *[hidden email] <[hidden email]>*
>> >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> >>>>
>> >>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
>> >>>>
>> >>>> Team
>> >>>>
>> >>>> There is a thread on apache legal-discuss that might allow for a
>> >>>> graceperiod of continued usage of the json library.  Am going to keep
>> >>>> a close eye on this and if VP Legal approves we'll be able to keep the
>> >>>> twitter processors in which is definitely a good thing.  Will advise
>> >>>>
>> >>>> Thanks
>> >>>> Joe
>> >>>>
>> >>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]>
>> wrote:
>> >>>>
>> >>>> I've noticed an issue with the per-instance class loading capability
>> >>>> introduced in NIFI-2909 where the additional classpath resources can
>> get
>> >>>> incorrectly removed from the class loader.
>> >>>>
>> >>>> I was able to reproduced this with a unit test and have a fix ready. I
>> >>>> believe this is important and needs to go in for the 1.1 release,
>> going to
>> >>>> re-open NIFI-2909 and submit a PR shortly.
>> >>>>
>> >>>> -Bryan
>> >>>>
>> >>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <[hidden email]
>> >
>> >>>> wrote:
>> >>>>
>> >>>> I have two items that I would like to wrap up prior to creating an RC
>> for
>> >>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group
>> port
>> >>>> configuration. The work is already completed and I will be reviewing
>> it
>> >>>> this today. Additionally, following recent interest on the mailing
>> list,
>> >>>> I'd like to knock out NIFI-3020. This will allow an admin to
>> configure a
>> >>>> strategy for user identity when logging in via LDAP. Specifically, it
>> will
>> >>>> support usage of the DN (the default and current implementation) as
>> well as
>> >>>> the username the user logged in as. I should be able to have a PR up
>> for
>> >>>> this work later today.
>> >>>>
>> >>>> Thanks!
>> >>>>
>> >>>> Matt
>> >>>>
>> >>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>> >>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>> >>>>
>> >>>>
>> >>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]> wrote:
>> >>>>
>> >>>> The code is within the twitter4j library itself.  I filed a request to
>> >>>> twitter4jg.  The most likely case is we will need to submit a PR to
>> them.
>> >>>> However, I don't see this as something that should delay the
>> release.  We
>> >>>> can provide instructions for folks wanting to use the processor during
>> >>>>
>> >>>> the
>> >>>>
>> >>>> time we cannot make it available in a convenient manner.  I will
>> provide
>> >>>>
>> >>>> a
>> >>>>
>> >>>> meaningful comment about this in release notes and pointers on what
>> folks
>> >>>> can do in the meantime.
>> >>>>
>> >>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]>
>> wrote:
>> >>>>
>> >>>> I understand there was a discussion thread within the NiFi community
>> >>>>
>> >>>> for
>> >>>>
>> >>>> this as well and I missed responding to that at that time. It just
>> >>>>
>> >>>> seems
>> >>>>
>> >>>> to
>> >>>>
>> >>>> me like JSON processing is necessary for GetTwitter, which is
>> >>>>
>> >>>> incredibly
>> >>>>
>> >>>> useful for demonstrating NiFi’s ability to read from a high volume
>> >>>>
>> >>>> stream
>> >>>>
>> >>>> out of the box. With NIFI-3019 (Remove GetTwitter from default build),
>> >>>>
>> >>>> is
>> >>>>
>> >>>> there any related effort to substitute an acceptable replacement JSON
>> >>>> library to restore this functionality?
>> >>>>
>> >>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>> >>>>
>> >>>> Andy LoPresto
>> >>>> [hidden email]
>> >>>> *[hidden email] <[hidden email]>*
>> >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> >>>>
>> >>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>> >>>>
>> >>>> wrote:
>> >>>>
>> >>>>
>> >>>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and
>> >>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>> >>>> investigated some alternate TLS config options for the new version of
>> >>>>
>> >>>> the
>> >>>>
>> >>>> client library.
>> >>>>
>> >>>> Is there any alternative to excluding the GetTwitter processor? Using
>> >>>> Johnzon [1] or the Android re-implementation [2] discussed in the
>> >>>>
>> >>>> mailing
>> >>>>
>> >>>> list thread?
>> >>>>
>> >>>> [1] https://johnzon.apache.org/
>> >>>> [2] https://developer.android.com/reference/org/json/package-
>> >>>>
>> >>>> summary.html
>> >>>>
>> >>>>
>> >>>>
>> >>>> Andy LoPresto
>> >>>> [hidden email]
>> >>>> *[hidden email] <[hidden email]>*
>> >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> >>>>
>> >>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>> >>>>
>> >>>> Team
>> >>>>
>> >>>> Very happy to see that we are down to three items remaining tagged to
>> >>>> 1.1.0.  Solid effort over the recent weeks to close the gap including
>> >>>>
>> >>>> work
>> >>>>
>> >>>> to get past the now category x Jason dependency we had.  The most
>> >>>>
>> >>>> notable
>> >>>>
>> >>>> impact from that is the wildly popular GetTwitter processor, the fav
>> >>>>
>> >>>> new
>> >>>>
>> >>>> nifi user and demo processor, can no longer be included in the default
>> >>>> build.  It is optionally available if users choose to build and use it
>> >>>>
>> >>>> but
>> >>>>
>> >>>> we won't distribute binaries that have it.
>> >>>>
>> >>>> I see some review movement on some patch available but untagged items.
>> >>>>
>> >>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri.
>> Anyone
>> >>>> have any outstanding items?
>> >>>>
>> >>>> Thanks
>> >>>> Joe
>> >>>>
>> >>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>> >>>>
>> >>>> Ryan
>> >>>>
>> >>>> Not officially but I think we should try to close this thing out and
>> >>>> start a vote in the next week or two at most.
>> >>>>
>> >>>> I'm going through the tickets again now.  There is also a new issue of
>> >>>> the json-p license falling out of favor in Apache legal terms and
>> >>>> becoming Category-X.  Am looking into that now.
>> >>>>
>> >>>> Thanks
>> >>>> Joe
>> >>>>
>> >>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>> >>>>
>> >>>> wrote:
>> >>>>
>> >>>>
>> >>>> Joe - Is there a target date for 1.1?
>> >>>>
>> >>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]>
>> wrote:
>> >>>>
>> >>>> Team,
>> >>>>
>> >>>> Just an update on things with working toward an Apache NiFi 1.1.0
>> >>>> release.  There are still about 33 JIRAs there now and some are
>> >>>> awaiting review and are some are under active progress. Yet there is
>> >>>> good traction and progress. I think we should just stay vigilant with
>> >>>> what makes it in and keep working it down.  So let's please shoot for
>> >>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>> >>>>
>> >>>> Thanks
>> >>>> Joe
>> >>>>
>> >>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]> wrote:
>> >>>>
>> >>>> Team,
>> >>>>
>> >>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
>> >>>> avoiding putting more in there for now at least without a discussion.
>> >>>> Of the 31 JIRAs there the vast majority need review so we should be
>> >>>> able to close these down fairly quickly as long as we don't let the
>> >>>> list grow.
>> >>>>
>> >>>> Thanks
>> >>>> joe
>> >>>>
>> >>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <[hidden email]
>> >
>> >>>>
>> >>>> wrote:
>> >>>>
>> >>>> Joe,
>> >>>>
>> >>>> Appreciate the offer it isn't my PR. I was just using it as an
>> >>>>
>> >>>> example.
>> >>>>
>> >>>> All
>> >>>>
>> >>>> mine are currently closed, which I greatly appreciate.
>> >>>>
>> >>>> Cheers,
>> >>>>
>> >>>> Edgardo
>> >>>>
>> >>>> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>> >>>>
>> >>>> Edgardo,
>> >>>>
>> >>>> You mentioned a PR from August. I'd be happy to help you work that
>> >>>> through review.
>> >>>>
>> >>>> Thanks
>> >>>> Joe
>> >>>>
>> >>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>> >>>>
>> >>>> [hidden email]
>> >>>>
>> >>>> <javascript:;>> wrote:
>> >>>>
>> >>>> I have agreed that at this point a release is important. My goal
>> >>>>
>> >>>> was
>> >>>>
>> >>>> try
>> >>>>
>> >>>> to
>> >>>>
>> >>>> squeeze in a much goodness as possible into the release, but the
>> >>>>
>> >>>> important
>> >>>>
>> >>>> bug fixes should come first. Getting 1.x into a state where the
>> >>>>
>> >>>> release
>> >>>>
>> >>>> notes don't say that it is geared toward developers and testers is
>> >>>>
>> >>>> really
>> >>>>
>> >>>> huge.
>> >>>>
>> >>>> I think Nifi is a great community otherwise I would participate in
>> >>>>
>> >>>> the
>> >>>>
>> >>>> mailing list, create Jira tickets and pull requests. I am only
>> >>>>
>> >>>> trying to
>> >>>>
>> >>>> strengthen the great thing that is going on here. We can always do
>> >>>>
>> >>>> better.
>> >>>>
>> >>>> I was not trying to put down this community only to participate and
>> >>>>
>> >>>> make
>> >>>>
>> >>>> it
>> >>>>
>> >>>> better. I think this conversation is an indication of how great
>> >>>>
>> >>>> this
>> >>>>
>> >>>> community is.
>> >>>>
>> >>>> Maybe I am being sensitive about this issue and trying to
>> >>>>
>> >>>> strengthen
>> >>>>
>> >>>> the
>> >>>>
>> >>>> nifi community even more, after coming from a conference where it
>> >>>>
>> >>>> was
>> >>>>
>> >>>> reported there was lots of excitement at first and now the
>> >>>>
>> >>>> participation
>> >>>>
>> >>>> in
>> >>>>
>> >>>> the community has really died down and they are struggling. I don't
>> >>>>
>> >>>> want
>> >>>>
>> >>>> to
>> >>>>
>> >>>> see that happen here.
>> >>>>
>> >>>> Cheers,
>> >>>>
>> >>>> Edgardo
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>> >>>>
>> >>>> <javascript:;>> wrote:
>> >>>>
>> >>>>
>> >>>> Edgardo,
>> >>>>
>> >>>> Thank you for your feedback. We hear your comments and as a
>> >>>>
>> >>>> committer I
>> >>>>
>> >>>> can
>> >>>>
>> >>>> share we are constantly looking to improve the PR process, having
>> >>>>
>> >>>> already
>> >>>>
>> >>>> taken many of the steps you suggest.
>> >>>>
>> >>>> However, it is important to notice that the number of PRs should
>> >>>>
>> >>>> not be
>> >>>>
>> >>>> seen as a metric of engagement by the development community: Most
>> >>>>
>> >>>> of us
>> >>>>
>> >>>> will submit PRs so that our work can be carefully reviewed by our
>> >>>>
>> >>>> peers
>> >>>>
>> >>>> and
>> >>>>
>> >>>> some of us will use JIRA patches to provide contributions.
>> >>>>
>> >>>> Having said that, it is true that some PRs may sit idle for a long
>> >>>>
>> >>>> time
>> >>>>
>> >>>> and
>> >>>>
>> >>>> we are working to improve this pipeline.
>> >>>>
>> >>>> It was therefore no coincidence that I  browsed most of the PRs
>> >>>>
>> >>>> performing
>> >>>>
>> >>>> a triage of items that have been superseded or diverged from the
>> >>>>
>> >>>> current
>> >>>>
>> >>>> code base.
>> >>>>
>> >>>> In fact, less than a month ago the dev team closed a number of
>> >>>>
>> >>>> stalled
>> >>>>
>> >>>> and
>> >>>>
>> >>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>> >>>>
>> >>>> Despite all the above, I think Joe has a point. The master
>> >>>>
>> >>>> contain a
>> >>>>
>> >>>> series
>> >>>>
>> >>>> of important bug fixes and suspect the community would benefit
>> >>>>
>> >>>> from
>> >>>>
>> >>>> a
>> >>>>
>> >>>> release sooner rather than later.
>> >>>>
>> >>>> Once again, thank you for your feedback and contribution. It is
>> >>>>
>> >>>> good to
>> >>>>
>> >>>> have you here.
>> >>>>
>> >>>> Andre
>> >>>>
>> >>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>> >>>>
>> >>>> [hidden email]
>> >>>>
>> >>>> <javascript:;>>
>> >>>>
>> >>>> wrote:
>> >>>>
>> >>>> Joe - You are correct I was mentioning the PRs that are
>> >>>>
>> >>>> currently
>> >>>>
>> >>>> open.
>> >>>>
>> >>>>
>> >>>> Regardless of how it happens reducing the count of open PRs I
>> >>>>
>> >>>> believe
>> >>>>
>> >>>> to
>> >>>>
>> >>>> be
>> >>>>
>> >>>> extremely important. Maybe I was hoping that the release could
>> >>>>
>> >>>> be
>> >>>>
>> >>>> a
>> >>>>
>> >>>> forcing
>> >>>>
>> >>>> function to make that happen. I believe that developers are more
>> >>>>
>> >>>> willing
>> >>>>
>> >>>> to
>> >>>>
>> >>>> contribute when they see that their PRs will actually be able
>> >>>>
>> >>>> accepted
>> >>>>
>> >>>> and
>> >>>>
>> >>>> merged into the code base. Having a low number of open PRs in
>> >>>>
>> >>>> progress
>> >>>>
>> >>>> is a
>> >>>>
>> >>>> great indication that the main nifi developers are fully engaged
>> >>>>
>> >>>> with
>> >>>>
>> >>>> the
>> >>>>
>> >>>> community.
>> >>>>
>> >>>> There are a few PRs that don't have any comments from committers
>> >>>>
>> >>>> at
>> >>>>
>> >>>> all.
>> >>>>
>> >>>> I
>> >>>>
>> >>>> found one from August in that state. If that was my PR I don't
>> >>>>
>> >>>> think I
>> >>>>
>> >>>> would be so willing to put another one in anytime soon. I do get
>> >>>>
>> >>>> that
>> >>>>
>> >>>> sometime PRs get stalled by the originator, if so maybe a rule
>> >>>>
>> >>>> about
>> >>>>
>> >>>> closing them after a certain amount of time or being taken over
>> >>>>
>> >>>> by a
>> >>>>
>> >>>> core
>> >>>>
>> >>>> contributor if they think it worthwhile.
>> >>>>
>> >>>> I would like to shoutout to James Wing on my last PR he was
>> >>>>
>> >>>> quick
>> >>>>
>> >>>> to
>> >>>>
>> >>>> review, provided great comments, testing, and even some
>> >>>>
>> >>>> additional
>> >>>>
>> >>>> code.
>> >>>>
>> >>>> It
>> >>>>
>> >>>> was a great PR experience.
>> >>>>
>> >>>> Cheers,
>> >>>>
>> >>>> Edgardo
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>> >>>>
>> >>>> [hidden email] <javascript:;>.
>> >>>>
>> >>>> invalid> wrote:
>> >>>>
>> >>>> Joe, I think you misread. Edgardo is referring to the Pull
>> >>>>
>> >>>> Requests
>> >>>>
>> >>>> that
>> >>>>
>> >>>> are currently open, not the tickets assigned to the 1.1.0
>> >>>>
>> >>>> version.
>> >>>>
>> >>>>
>> >>>> I think these goals (releasing 1.1.0 and cutting down the PR
>> >>>>
>> >>>> count)
>> >>>>
>> >>>> should
>> >>>>
>> >>>> be two different efforts. Doing a thorough job reviewing
>> >>>>
>> >>>> takes a
>> >>>>
>> >>>> significant amount of time from both the reviewer and
>> >>>>
>> >>>> contributor.
>> >>>>
>> >>>> In
>> >>>>
>> >>>> order
>> >>>>
>> >>>> to cut it down significantly would take much longer than a
>> >>>>
>> >>>> couple
>> >>>>
>> >>>> days.
>> >>>>
>> >>>>
>> >>>> Also there has already been a lot of great new features and
>> >>>>
>> >>>> bug
>> >>>>
>> >>>> fixes
>> >>>>
>> >>>> contributed to the 1.X line and I don't think it's worth
>> >>>>
>> >>>> holding up
>> >>>>
>> >>>> a
>> >>>>
>> >>>> 1.1.0
>> >>>>
>> >>>> release for tickets not assigned to this fix version. As an
>> >>>>
>> >>>> added
>> >>>>
>> >>>> bonus
>> >>>>
>> >>>> though, I think many of the tickets tagged as 1.1.0 have PRs
>> >>>>
>> >>>> already
>> >>>>
>> >>>> open
>> >>>>
>> >>>> so closing those will make a large dent in the PR count.
>> >>>>
>> >>>>
>> >>>> Joe
>> >>>>
>> >>>> - - - - - -
>> >>>> Joseph Percivall
>> >>>> linkedin.com/in/Percivall
>> >>>> e: [hidden email] <javascript:;>
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>> >>>>
>> >>>> [hidden email]
>> >>>>
>> >>>> <javascript:;>>
>> >>>>
>> >>>> wrote:
>> >>>>
>> >>>>
>> >>>>
>> >>>> There are less than 30 right now.  Many of the roughly 90+
>> >>>>
>> >>>> JIRAs
>> >>>>
>> >>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>> >>>>
>> >>>> or
>> >>>>
>> >>>> just
>> >>>>
>> >>>> had fix versions removed.
>> >>>>
>> >>>> We will need to have a push over the next bunch of days to
>> >>>>
>> >>>> deal
>> >>>>
>> >>>> with
>> >>>>
>> >>>> reviewing/merging/moving the remaining items.
>> >>>>
>> >>>> Thanks
>> >>>> Joe
>> >>>>
>> >>>>
>> >>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>> >>>>
>> >>>> [hidden email] <javascript:;>>
>> >>>>
>> >>>> wrote:
>> >>>>
>> >>>> Joe,
>> >>>>
>> >>>> There are 75 PRs currently open. Why not make a push over
>> >>>>
>> >>>> the
>> >>>>
>> >>>> next
>> >>>>
>> >>>> bunch
>> >>>>
>> >>>> of
>> >>>>
>> >>>> days to get them closed and then cut the release after that.
>> >>>>
>> >>>> Cheers,
>> >>>>
>> >>>> Edgardo
>> >>>>
>> >>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>> >>>>
>> >>>> [hidden email]
>> >>>>
>> >>>> <javascript:;>>
>> >>>>
>> >>>> wrote:
>> >>>>
>> >>>>
>> >>>> Team,
>> >>>>
>> >>>> There have been a ton of bugs fixed a few nice features.  I
>> >>>>
>> >>>> would
>> >>>>
>> >>>> like
>> >>>>
>> >>>> to move to get Apache NiFi 1.1.0 release going pretty much
>> >>>>
>> >>>> based
>> >>>>
>> >>>> on
>> >>>>
>> >>>> where we are now and plan to move most tickets to a new
>> >>>>
>> >>>> Apache
>> >>>>
>> >>>> NiFi
>> >>>>
>> >>>> 1.2.0 version.  We can try to get back on our roughly 6-8
>> >>>>
>> >>>> week
>> >>>>
>> >>>> release
>> >>>>
>> >>>> schedule and shoot for a mid to late Nov release for NiFi
>> >>>>
>> >>>> 1.2.0
>> >>>>
>> >>>> this
>> >>>>
>> >>>> way as well. Please advise if anyone has any other views on
>> >>>>
>> >>>> this. In
>> >>>>
>> >>>> the mean time I'll get the wheels in motion so you'll be
>> >>>>
>> >>>> seeing a
>> >>>>
>> >>>> lot
>> >>>>
>> >>>> of JIRA/issue updates to move version around.
>> >>>>
>> >>>> Thanks
>> >>>> Joe
>> >>>>
>> >>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>> >>>>
>> >>>> [hidden email]
>> >>>>
>> >>>> <javascript:;>>
>> >>>>
>> >>>> wrote:
>> >>>>
>> >>>> Sounds good Joe. I have no issue to you doing the rm'ing
>> >>>>
>> >>>> for
>> >>>>
>> >>>> it.
>> >>>>
>> >>>>
>> >>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>> >>>>
>> >>>> <javascript:;>> wrote:
>> >>>>
>> >>>>
>> >>>> Team,
>> >>>>
>> >>>> There are a lot of great fixes and improvements on the
>> >>>>
>> >>>> master
>> >>>>
>> >>>> line
>> >>>>
>> >>>> now
>> >>>>
>> >>>> and we're at a good time window to start pushing for a
>> >>>>
>> >>>> release.
>> >>>>
>> >>>> There
>> >>>>
>> >>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>> >>>>
>> >>>> are
>> >>>>
>> >>>> open.
>> >>>>
>> >>>> I'm
>> >>>>
>> >>>> going to go through them and remove fix versions where
>> >>>>
>> >>>> appropriate.
>> >>>>
>> >>>>
>> >>>> I'm happy to take on RM task for this release though if
>> >>>>
>> >>>> someone
>> >>>>
>> >>>> else
>> >>>>
>> >>>> would like to take that on please advise.
>> >>>>
>> >>>> Thanks
>> >>>> Joe
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Cheers,
>> >>>>
>> >>>> Edgardo
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Cheers,
>> >>>>
>> >>>> Edgardo
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Cheers,
>> >>>>
>> >>>> Edgardo
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Cheers,
>> >>>>
>> >>>> Edgardo
>> >>>>
>> >>>> Sent from Gmail Mobile
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Michael Moser
In reply to this post by James Wing
After seeing "Repositories now support rollback" in the release notes and
reading NIFI-2854 [1], I have a question.

Are repositories created using NiFi 1.0.0 compatible with NiFi 1.1.0
software?  This is the goal that the ticket seems to indicate with 1.1.0
onward, but it's not clear whether 1.0.0 -> 1.1.0 is included.

Thanks,
-- Mike

[1] - https://issues.apache.org/jira/browse/NIFI-2854

On Tue, Nov 22, 2016 at 12:08 PM, James Wing <[hidden email]> wrote:

> Going through the tickets, it seems like quite a release.  A few more
> things for your list:
>
> GenerateFlowFile updated to support literal/expression content and
> attributes
>
> AWS-related:
> * New processors PutCloudWatchMetric, PutKinesisStream
> * Updated processors PutS3Object (content type, signer options), ListS3
> (performance, versions)
> * Added support for AWS assume role credentials with proxy
>
>
> Thanks,
>
> James
>
> On Mon, Nov 21, 2016 at 9:30 PM, Joe Witt <[hidden email]> wrote:
>
> > matt
> >
> > i'll add a wiki page or set of instructions linked from the release
> notes.
> >
> > all,
> >
> > walked through the 250 or so JIRAs in the 1.1.0 release and pulled out
> > highlights.  The items noted are as follows.  Will likely reduce this
> > down further for the release notes but wanted to put this out in case
> > folks have things they think are really important to highlight.
> >
> > - Core Improvements:
> >    - Performance: Session Migration
> >    - Stability: Cluster Management
> >    - Developer: Framework supports easy user driven classloader extension
> >    - Expression Language: Now supports base64 and hex encoded values
> > and Math functions
> >    - Repositories now support rollback
> >    - Faster startup due to more efficient state restoration algorithm
> > - UX Improvements:
> >    - Visual Backpressure Indicator
> >    - Introduced more colors to better highlight actions and components
> >    - Performance: Validate non-running components
> >    - Provenance graph image can be exported
> >    - Cron Scheduling for Primary node tasks now supported
> > - Updated versions
> >    - Azure Event Hub 0.9.0
> >    - Spark 2.0.1
> >    - Hadoop 2.7.x
> > - New/Improved Processors
> >    - new Fetch/Put Elastic Search 5.0
> >    - new ParseCEF to parse CEF formatted logs
> >    - improve ExtractEmail now supports TNEF files
> >    - new Validate CSV
> >    - improved Solr processors now support SSL and Kerberos
> >    - new Websocket client and server processors
> > - New Utility
> >    - Zookeeper Migrator (move from one zookeeper to another)
> > - Security
> >    - Restricted Processors
> >    - Site-to-site now supports port forwarding
> >    - Improved Policy Management UX
> > - Migration Notes:
> >    - Restricted Processors
> >    - Twitter Processor Removed
> >
> > On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]>
> > wrote:
> > > Is there a good spot for us to put instructions on how to build the
> > > Twitter processor and/or the Social Media NAR in the meantime? Maybe a
> > > Wiki page or something simple to say "go to this directory, run this
> > > Maven command, drop the NAR into your deployment..." ?
> > >
> > > On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:
> > >> Team,
> > >>
> > >> We appear to be very close.  Andy is working NIFI-3024 but otherwise
> > >> it is focus on testing.
> > >>
> > >> I'm going to prep the RC and release notes now.  Unfortunately the
> > >> twitter changes for json.org will need to remain.  Consensus forming
> > >> on the legal-discuss thread regarding a grace period has been elusive
> > >> and we're already prepared to make the right steps so we'll just need
> > >> to take that on by being empathetic to the user base.
> > >>
> > >> Thanks
> > >> Joe
> > >>
> > >> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
> > >>> Andy,
> > >>>
> > >>> Great to see NIFI-3050 implemented and certainly good news that NiFi
> > 1.1.0
> > >>> is set to include a number of security related improvements.
> > >>>
> > >>>
> > >>>
> > >>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]
> >
> > wrote:
> > >>>
> > >>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have
> > been
> > >>>> added to my plate for this release. Coordinated with Joe Witt and
> they
> > >>>> should both be included.
> > >>>>
> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
> > >>>>
> > >>>> Andy LoPresto
> > >>>> [hidden email]
> > >>>> *[hidden email] <[hidden email]>*
> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >>>>
> > >>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
> > >>>>
> > >>>> Team
> > >>>>
> > >>>> There is a thread on apache legal-discuss that might allow for a
> > >>>> graceperiod of continued usage of the json library.  Am going to
> keep
> > >>>> a close eye on this and if VP Legal approves we'll be able to keep
> the
> > >>>> twitter processors in which is definitely a good thing.  Will advise
> > >>>>
> > >>>> Thanks
> > >>>> Joe
> > >>>>
> > >>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]>
> > wrote:
> > >>>>
> > >>>> I've noticed an issue with the per-instance class loading capability
> > >>>> introduced in NIFI-2909 where the additional classpath resources can
> > get
> > >>>> incorrectly removed from the class loader.
> > >>>>
> > >>>> I was able to reproduced this with a unit test and have a fix
> ready. I
> > >>>> believe this is important and needs to go in for the 1.1 release,
> > going to
> > >>>> re-open NIFI-2909 and submit a PR shortly.
> > >>>>
> > >>>> -Bryan
> > >>>>
> > >>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <
> [hidden email]
> > >
> > >>>> wrote:
> > >>>>
> > >>>> I have two items that I would like to wrap up prior to creating an
> RC
> > for
> > >>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process
> Group
> > port
> > >>>> configuration. The work is already completed and I will be reviewing
> > it
> > >>>> this today. Additionally, following recent interest on the mailing
> > list,
> > >>>> I'd like to knock out NIFI-3020. This will allow an admin to
> > configure a
> > >>>> strategy for user identity when logging in via LDAP. Specifically,
> it
> > will
> > >>>> support usage of the DN (the default and current implementation) as
> > well as
> > >>>> the username the user logged in as. I should be able to have a PR up
> > for
> > >>>> this work later today.
> > >>>>
> > >>>> Thanks!
> > >>>>
> > >>>> Matt
> > >>>>
> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
> > >>>>
> > >>>>
> > >>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]>
> wrote:
> > >>>>
> > >>>> The code is within the twitter4j library itself.  I filed a request
> to
> > >>>> twitter4jg.  The most likely case is we will need to submit a PR to
> > them.
> > >>>> However, I don't see this as something that should delay the
> > release.  We
> > >>>> can provide instructions for folks wanting to use the processor
> during
> > >>>>
> > >>>> the
> > >>>>
> > >>>> time we cannot make it available in a convenient manner.  I will
> > provide
> > >>>>
> > >>>> a
> > >>>>
> > >>>> meaningful comment about this in release notes and pointers on what
> > folks
> > >>>> can do in the meantime.
> > >>>>
> > >>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]>
> > wrote:
> > >>>>
> > >>>> I understand there was a discussion thread within the NiFi community
> > >>>>
> > >>>> for
> > >>>>
> > >>>> this as well and I missed responding to that at that time. It just
> > >>>>
> > >>>> seems
> > >>>>
> > >>>> to
> > >>>>
> > >>>> me like JSON processing is necessary for GetTwitter, which is
> > >>>>
> > >>>> incredibly
> > >>>>
> > >>>> useful for demonstrating NiFi’s ability to read from a high volume
> > >>>>
> > >>>> stream
> > >>>>
> > >>>> out of the box. With NIFI-3019 (Remove GetTwitter from default
> build),
> > >>>>
> > >>>> is
> > >>>>
> > >>>> there any related effort to substitute an acceptable replacement
> JSON
> > >>>> library to restore this functionality?
> > >>>>
> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
> > >>>>
> > >>>> Andy LoPresto
> > >>>> [hidden email]
> > >>>> *[hidden email] <[hidden email]>*
> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >>>>
> > >>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
> > >>>>
> > >>>> wrote:
> > >>>>
> > >>>>
> > >>>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655,
> and
> > >>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
> > >>>> investigated some alternate TLS config options for the new version
> of
> > >>>>
> > >>>> the
> > >>>>
> > >>>> client library.
> > >>>>
> > >>>> Is there any alternative to excluding the GetTwitter processor?
> Using
> > >>>> Johnzon [1] or the Android re-implementation [2] discussed in the
> > >>>>
> > >>>> mailing
> > >>>>
> > >>>> list thread?
> > >>>>
> > >>>> [1] https://johnzon.apache.org/
> > >>>> [2] https://developer.android.com/reference/org/json/package-
> > >>>>
> > >>>> summary.html
> > >>>>
> > >>>>
> > >>>>
> > >>>> Andy LoPresto
> > >>>> [hidden email]
> > >>>> *[hidden email] <[hidden email]>*
> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >>>>
> > >>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
> > >>>>
> > >>>> Team
> > >>>>
> > >>>> Very happy to see that we are down to three items remaining tagged
> to
> > >>>> 1.1.0.  Solid effort over the recent weeks to close the gap
> including
> > >>>>
> > >>>> work
> > >>>>
> > >>>> to get past the now category x Jason dependency we had.  The most
> > >>>>
> > >>>> notable
> > >>>>
> > >>>> impact from that is the wildly popular GetTwitter processor, the fav
> > >>>>
> > >>>> new
> > >>>>
> > >>>> nifi user and demo processor, can no longer be included in the
> default
> > >>>> build.  It is optionally available if users choose to build and use
> it
> > >>>>
> > >>>> but
> > >>>>
> > >>>> we won't distribute binaries that have it.
> > >>>>
> > >>>> I see some review movement on some patch available but untagged
> items.
> > >>>>
> > >>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri.
> > Anyone
> > >>>> have any outstanding items?
> > >>>>
> > >>>> Thanks
> > >>>> Joe
> > >>>>
> > >>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
> > >>>>
> > >>>> Ryan
> > >>>>
> > >>>> Not officially but I think we should try to close this thing out and
> > >>>> start a vote in the next week or two at most.
> > >>>>
> > >>>> I'm going through the tickets again now.  There is also a new issue
> of
> > >>>> the json-p license falling out of favor in Apache legal terms and
> > >>>> becoming Category-X.  Am looking into that now.
> > >>>>
> > >>>> Thanks
> > >>>> Joe
> > >>>>
> > >>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
> > >>>>
> > >>>> wrote:
> > >>>>
> > >>>>
> > >>>> Joe - Is there a target date for 1.1?
> > >>>>
> > >>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]>
> > wrote:
> > >>>>
> > >>>> Team,
> > >>>>
> > >>>> Just an update on things with working toward an Apache NiFi 1.1.0
> > >>>> release.  There are still about 33 JIRAs there now and some are
> > >>>> awaiting review and are some are under active progress. Yet there is
> > >>>> good traction and progress. I think we should just stay vigilant
> with
> > >>>> what makes it in and keep working it down.  So let's please shoot
> for
> > >>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
> > >>>>
> > >>>> Thanks
> > >>>> Joe
> > >>>>
> > >>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]>
> wrote:
> > >>>>
> > >>>> Team,
> > >>>>
> > >>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.
> Let's
> > >>>> avoiding putting more in there for now at least without a
> discussion.
> > >>>> Of the 31 JIRAs there the vast majority need review so we should be
> > >>>> able to close these down fairly quickly as long as we don't let the
> > >>>> list grow.
> > >>>>
> > >>>> Thanks
> > >>>> joe
> > >>>>
> > >>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <
> [hidden email]
> > >
> > >>>>
> > >>>> wrote:
> > >>>>
> > >>>> Joe,
> > >>>>
> > >>>> Appreciate the offer it isn't my PR. I was just using it as an
> > >>>>
> > >>>> example.
> > >>>>
> > >>>> All
> > >>>>
> > >>>> mine are currently closed, which I greatly appreciate.
> > >>>>
> > >>>> Cheers,
> > >>>>
> > >>>> Edgardo
> > >>>>
> > >>>> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
> > >>>>
> > >>>> Edgardo,
> > >>>>
> > >>>> You mentioned a PR from August. I'd be happy to help you work that
> > >>>> through review.
> > >>>>
> > >>>> Thanks
> > >>>> Joe
> > >>>>
> > >>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
> > >>>>
> > >>>> [hidden email]
> > >>>>
> > >>>> <javascript:;>> wrote:
> > >>>>
> > >>>> I have agreed that at this point a release is important. My goal
> > >>>>
> > >>>> was
> > >>>>
> > >>>> try
> > >>>>
> > >>>> to
> > >>>>
> > >>>> squeeze in a much goodness as possible into the release, but the
> > >>>>
> > >>>> important
> > >>>>
> > >>>> bug fixes should come first. Getting 1.x into a state where the
> > >>>>
> > >>>> release
> > >>>>
> > >>>> notes don't say that it is geared toward developers and testers is
> > >>>>
> > >>>> really
> > >>>>
> > >>>> huge.
> > >>>>
> > >>>> I think Nifi is a great community otherwise I would participate in
> > >>>>
> > >>>> the
> > >>>>
> > >>>> mailing list, create Jira tickets and pull requests. I am only
> > >>>>
> > >>>> trying to
> > >>>>
> > >>>> strengthen the great thing that is going on here. We can always do
> > >>>>
> > >>>> better.
> > >>>>
> > >>>> I was not trying to put down this community only to participate and
> > >>>>
> > >>>> make
> > >>>>
> > >>>> it
> > >>>>
> > >>>> better. I think this conversation is an indication of how great
> > >>>>
> > >>>> this
> > >>>>
> > >>>> community is.
> > >>>>
> > >>>> Maybe I am being sensitive about this issue and trying to
> > >>>>
> > >>>> strengthen
> > >>>>
> > >>>> the
> > >>>>
> > >>>> nifi community even more, after coming from a conference where it
> > >>>>
> > >>>> was
> > >>>>
> > >>>> reported there was lots of excitement at first and now the
> > >>>>
> > >>>> participation
> > >>>>
> > >>>> in
> > >>>>
> > >>>> the community has really died down and they are struggling. I don't
> > >>>>
> > >>>> want
> > >>>>
> > >>>> to
> > >>>>
> > >>>> see that happen here.
> > >>>>
> > >>>> Cheers,
> > >>>>
> > >>>> Edgardo
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
> > >>>>
> > >>>> <javascript:;>> wrote:
> > >>>>
> > >>>>
> > >>>> Edgardo,
> > >>>>
> > >>>> Thank you for your feedback. We hear your comments and as a
> > >>>>
> > >>>> committer I
> > >>>>
> > >>>> can
> > >>>>
> > >>>> share we are constantly looking to improve the PR process, having
> > >>>>
> > >>>> already
> > >>>>
> > >>>> taken many of the steps you suggest.
> > >>>>
> > >>>> However, it is important to notice that the number of PRs should
> > >>>>
> > >>>> not be
> > >>>>
> > >>>> seen as a metric of engagement by the development community: Most
> > >>>>
> > >>>> of us
> > >>>>
> > >>>> will submit PRs so that our work can be carefully reviewed by our
> > >>>>
> > >>>> peers
> > >>>>
> > >>>> and
> > >>>>
> > >>>> some of us will use JIRA patches to provide contributions.
> > >>>>
> > >>>> Having said that, it is true that some PRs may sit idle for a long
> > >>>>
> > >>>> time
> > >>>>
> > >>>> and
> > >>>>
> > >>>> we are working to improve this pipeline.
> > >>>>
> > >>>> It was therefore no coincidence that I  browsed most of the PRs
> > >>>>
> > >>>> performing
> > >>>>
> > >>>> a triage of items that have been superseded or diverged from the
> > >>>>
> > >>>> current
> > >>>>
> > >>>> code base.
> > >>>>
> > >>>> In fact, less than a month ago the dev team closed a number of
> > >>>>
> > >>>> stalled
> > >>>>
> > >>>> and
> > >>>>
> > >>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
> > >>>>
> > >>>> Despite all the above, I think Joe has a point. The master
> > >>>>
> > >>>> contain a
> > >>>>
> > >>>> series
> > >>>>
> > >>>> of important bug fixes and suspect the community would benefit
> > >>>>
> > >>>> from
> > >>>>
> > >>>> a
> > >>>>
> > >>>> release sooner rather than later.
> > >>>>
> > >>>> Once again, thank you for your feedback and contribution. It is
> > >>>>
> > >>>> good to
> > >>>>
> > >>>> have you here.
> > >>>>
> > >>>> Andre
> > >>>>
> > >>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
> > >>>>
> > >>>> [hidden email]
> > >>>>
> > >>>> <javascript:;>>
> > >>>>
> > >>>> wrote:
> > >>>>
> > >>>> Joe - You are correct I was mentioning the PRs that are
> > >>>>
> > >>>> currently
> > >>>>
> > >>>> open.
> > >>>>
> > >>>>
> > >>>> Regardless of how it happens reducing the count of open PRs I
> > >>>>
> > >>>> believe
> > >>>>
> > >>>> to
> > >>>>
> > >>>> be
> > >>>>
> > >>>> extremely important. Maybe I was hoping that the release could
> > >>>>
> > >>>> be
> > >>>>
> > >>>> a
> > >>>>
> > >>>> forcing
> > >>>>
> > >>>> function to make that happen. I believe that developers are more
> > >>>>
> > >>>> willing
> > >>>>
> > >>>> to
> > >>>>
> > >>>> contribute when they see that their PRs will actually be able
> > >>>>
> > >>>> accepted
> > >>>>
> > >>>> and
> > >>>>
> > >>>> merged into the code base. Having a low number of open PRs in
> > >>>>
> > >>>> progress
> > >>>>
> > >>>> is a
> > >>>>
> > >>>> great indication that the main nifi developers are fully engaged
> > >>>>
> > >>>> with
> > >>>>
> > >>>> the
> > >>>>
> > >>>> community.
> > >>>>
> > >>>> There are a few PRs that don't have any comments from committers
> > >>>>
> > >>>> at
> > >>>>
> > >>>> all.
> > >>>>
> > >>>> I
> > >>>>
> > >>>> found one from August in that state. If that was my PR I don't
> > >>>>
> > >>>> think I
> > >>>>
> > >>>> would be so willing to put another one in anytime soon. I do get
> > >>>>
> > >>>> that
> > >>>>
> > >>>> sometime PRs get stalled by the originator, if so maybe a rule
> > >>>>
> > >>>> about
> > >>>>
> > >>>> closing them after a certain amount of time or being taken over
> > >>>>
> > >>>> by a
> > >>>>
> > >>>> core
> > >>>>
> > >>>> contributor if they think it worthwhile.
> > >>>>
> > >>>> I would like to shoutout to James Wing on my last PR he was
> > >>>>
> > >>>> quick
> > >>>>
> > >>>> to
> > >>>>
> > >>>> review, provided great comments, testing, and even some
> > >>>>
> > >>>> additional
> > >>>>
> > >>>> code.
> > >>>>
> > >>>> It
> > >>>>
> > >>>> was a great PR experience.
> > >>>>
> > >>>> Cheers,
> > >>>>
> > >>>> Edgardo
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
> > >>>>
> > >>>> [hidden email] <javascript:;>.
> > >>>>
> > >>>> invalid> wrote:
> > >>>>
> > >>>> Joe, I think you misread. Edgardo is referring to the Pull
> > >>>>
> > >>>> Requests
> > >>>>
> > >>>> that
> > >>>>
> > >>>> are currently open, not the tickets assigned to the 1.1.0
> > >>>>
> > >>>> version.
> > >>>>
> > >>>>
> > >>>> I think these goals (releasing 1.1.0 and cutting down the PR
> > >>>>
> > >>>> count)
> > >>>>
> > >>>> should
> > >>>>
> > >>>> be two different efforts. Doing a thorough job reviewing
> > >>>>
> > >>>> takes a
> > >>>>
> > >>>> significant amount of time from both the reviewer and
> > >>>>
> > >>>> contributor.
> > >>>>
> > >>>> In
> > >>>>
> > >>>> order
> > >>>>
> > >>>> to cut it down significantly would take much longer than a
> > >>>>
> > >>>> couple
> > >>>>
> > >>>> days.
> > >>>>
> > >>>>
> > >>>> Also there has already been a lot of great new features and
> > >>>>
> > >>>> bug
> > >>>>
> > >>>> fixes
> > >>>>
> > >>>> contributed to the 1.X line and I don't think it's worth
> > >>>>
> > >>>> holding up
> > >>>>
> > >>>> a
> > >>>>
> > >>>> 1.1.0
> > >>>>
> > >>>> release for tickets not assigned to this fix version. As an
> > >>>>
> > >>>> added
> > >>>>
> > >>>> bonus
> > >>>>
> > >>>> though, I think many of the tickets tagged as 1.1.0 have PRs
> > >>>>
> > >>>> already
> > >>>>
> > >>>> open
> > >>>>
> > >>>> so closing those will make a large dent in the PR count.
> > >>>>
> > >>>>
> > >>>> Joe
> > >>>>
> > >>>> - - - - - -
> > >>>> Joseph Percivall
> > >>>> linkedin.com/in/Percivall
> > >>>> e: [hidden email] <javascript:;>
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
> > >>>>
> > >>>> [hidden email]
> > >>>>
> > >>>> <javascript:;>>
> > >>>>
> > >>>> wrote:
> > >>>>
> > >>>>
> > >>>>
> > >>>> There are less than 30 right now.  Many of the roughly 90+
> > >>>>
> > >>>> JIRAs
> > >>>>
> > >>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
> > >>>>
> > >>>> or
> > >>>>
> > >>>> just
> > >>>>
> > >>>> had fix versions removed.
> > >>>>
> > >>>> We will need to have a push over the next bunch of days to
> > >>>>
> > >>>> deal
> > >>>>
> > >>>> with
> > >>>>
> > >>>> reviewing/merging/moving the remaining items.
> > >>>>
> > >>>> Thanks
> > >>>> Joe
> > >>>>
> > >>>>
> > >>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
> > >>>>
> > >>>> [hidden email] <javascript:;>>
> > >>>>
> > >>>> wrote:
> > >>>>
> > >>>> Joe,
> > >>>>
> > >>>> There are 75 PRs currently open. Why not make a push over
> > >>>>
> > >>>> the
> > >>>>
> > >>>> next
> > >>>>
> > >>>> bunch
> > >>>>
> > >>>> of
> > >>>>
> > >>>> days to get them closed and then cut the release after that.
> > >>>>
> > >>>> Cheers,
> > >>>>
> > >>>> Edgardo
> > >>>>
> > >>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
> > >>>>
> > >>>> [hidden email]
> > >>>>
> > >>>> <javascript:;>>
> > >>>>
> > >>>> wrote:
> > >>>>
> > >>>>
> > >>>> Team,
> > >>>>
> > >>>> There have been a ton of bugs fixed a few nice features.  I
> > >>>>
> > >>>> would
> > >>>>
> > >>>> like
> > >>>>
> > >>>> to move to get Apache NiFi 1.1.0 release going pretty much
> > >>>>
> > >>>> based
> > >>>>
> > >>>> on
> > >>>>
> > >>>> where we are now and plan to move most tickets to a new
> > >>>>
> > >>>> Apache
> > >>>>
> > >>>> NiFi
> > >>>>
> > >>>> 1.2.0 version.  We can try to get back on our roughly 6-8
> > >>>>
> > >>>> week
> > >>>>
> > >>>> release
> > >>>>
> > >>>> schedule and shoot for a mid to late Nov release for NiFi
> > >>>>
> > >>>> 1.2.0
> > >>>>
> > >>>> this
> > >>>>
> > >>>> way as well. Please advise if anyone has any other views on
> > >>>>
> > >>>> this. In
> > >>>>
> > >>>> the mean time I'll get the wheels in motion so you'll be
> > >>>>
> > >>>> seeing a
> > >>>>
> > >>>> lot
> > >>>>
> > >>>> of JIRA/issue updates to move version around.
> > >>>>
> > >>>> Thanks
> > >>>> Joe
> > >>>>
> > >>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
> > >>>>
> > >>>> [hidden email]
> > >>>>
> > >>>> <javascript:;>>
> > >>>>
> > >>>> wrote:
> > >>>>
> > >>>> Sounds good Joe. I have no issue to you doing the rm'ing
> > >>>>
> > >>>> for
> > >>>>
> > >>>> it.
> > >>>>
> > >>>>
> > >>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
> > >>>>
> > >>>> <javascript:;>> wrote:
> > >>>>
> > >>>>
> > >>>> Team,
> > >>>>
> > >>>> There are a lot of great fixes and improvements on the
> > >>>>
> > >>>> master
> > >>>>
> > >>>> line
> > >>>>
> > >>>> now
> > >>>>
> > >>>> and we're at a good time window to start pushing for a
> > >>>>
> > >>>> release.
> > >>>>
> > >>>> There
> > >>>>
> > >>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
> > >>>>
> > >>>> are
> > >>>>
> > >>>> open.
> > >>>>
> > >>>> I'm
> > >>>>
> > >>>> going to go through them and remove fix versions where
> > >>>>
> > >>>> appropriate.
> > >>>>
> > >>>>
> > >>>> I'm happy to take on RM task for this release though if
> > >>>>
> > >>>> someone
> > >>>>
> > >>>> else
> > >>>>
> > >>>> would like to take that on please advise.
> > >>>>
> > >>>> Thanks
> > >>>> Joe
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Cheers,
> > >>>>
> > >>>> Edgardo
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Cheers,
> > >>>>
> > >>>> Edgardo
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Cheers,
> > >>>>
> > >>>> Edgardo
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Cheers,
> > >>>>
> > >>>> Edgardo
> > >>>>
> > >>>> Sent from Gmail Mobile
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Joe Witt
Mike - yeah good catch and good question.  It does support the old
format.  We've always been pretty good about being diligent to allow
folks to upgrade and it honor existing state and in many cases
configurations and it would automatically port them over.  What has
always been far more problematic is 'rollback' where people ran on
newer configurations but could not then go back to old framework code.
That is what NIFI-2854 tackles at least as far as the
content/prov/flowfile repositories go.  Now, the code and
serialization is done in such a way that older version can simply
ignore what never versions encoded if they don't understand it but
they should be able to continue on.

I just tried out a 1.0.0 flow with data queued up.  Upgraded to a
latest NiFi 1.1.0-SNAPSHOT.  Moved the repos over.  And it came up
perfectly with all the queue data ready to roll.

Thanks
Joe

On Tue, Nov 22, 2016 at 12:16 PM, Michael Moser <[hidden email]> wrote:

> After seeing "Repositories now support rollback" in the release notes and
> reading NIFI-2854 [1], I have a question.
>
> Are repositories created using NiFi 1.0.0 compatible with NiFi 1.1.0
> software?  This is the goal that the ticket seems to indicate with 1.1.0
> onward, but it's not clear whether 1.0.0 -> 1.1.0 is included.
>
> Thanks,
> -- Mike
>
> [1] - https://issues.apache.org/jira/browse/NIFI-2854
>
> On Tue, Nov 22, 2016 at 12:08 PM, James Wing <[hidden email]> wrote:
>
>> Going through the tickets, it seems like quite a release.  A few more
>> things for your list:
>>
>> GenerateFlowFile updated to support literal/expression content and
>> attributes
>>
>> AWS-related:
>> * New processors PutCloudWatchMetric, PutKinesisStream
>> * Updated processors PutS3Object (content type, signer options), ListS3
>> (performance, versions)
>> * Added support for AWS assume role credentials with proxy
>>
>>
>> Thanks,
>>
>> James
>>
>> On Mon, Nov 21, 2016 at 9:30 PM, Joe Witt <[hidden email]> wrote:
>>
>> > matt
>> >
>> > i'll add a wiki page or set of instructions linked from the release
>> notes.
>> >
>> > all,
>> >
>> > walked through the 250 or so JIRAs in the 1.1.0 release and pulled out
>> > highlights.  The items noted are as follows.  Will likely reduce this
>> > down further for the release notes but wanted to put this out in case
>> > folks have things they think are really important to highlight.
>> >
>> > - Core Improvements:
>> >    - Performance: Session Migration
>> >    - Stability: Cluster Management
>> >    - Developer: Framework supports easy user driven classloader extension
>> >    - Expression Language: Now supports base64 and hex encoded values
>> > and Math functions
>> >    - Repositories now support rollback
>> >    - Faster startup due to more efficient state restoration algorithm
>> > - UX Improvements:
>> >    - Visual Backpressure Indicator
>> >    - Introduced more colors to better highlight actions and components
>> >    - Performance: Validate non-running components
>> >    - Provenance graph image can be exported
>> >    - Cron Scheduling for Primary node tasks now supported
>> > - Updated versions
>> >    - Azure Event Hub 0.9.0
>> >    - Spark 2.0.1
>> >    - Hadoop 2.7.x
>> > - New/Improved Processors
>> >    - new Fetch/Put Elastic Search 5.0
>> >    - new ParseCEF to parse CEF formatted logs
>> >    - improve ExtractEmail now supports TNEF files
>> >    - new Validate CSV
>> >    - improved Solr processors now support SSL and Kerberos
>> >    - new Websocket client and server processors
>> > - New Utility
>> >    - Zookeeper Migrator (move from one zookeeper to another)
>> > - Security
>> >    - Restricted Processors
>> >    - Site-to-site now supports port forwarding
>> >    - Improved Policy Management UX
>> > - Migration Notes:
>> >    - Restricted Processors
>> >    - Twitter Processor Removed
>> >
>> > On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]>
>> > wrote:
>> > > Is there a good spot for us to put instructions on how to build the
>> > > Twitter processor and/or the Social Media NAR in the meantime? Maybe a
>> > > Wiki page or something simple to say "go to this directory, run this
>> > > Maven command, drop the NAR into your deployment..." ?
>> > >
>> > > On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:
>> > >> Team,
>> > >>
>> > >> We appear to be very close.  Andy is working NIFI-3024 but otherwise
>> > >> it is focus on testing.
>> > >>
>> > >> I'm going to prep the RC and release notes now.  Unfortunately the
>> > >> twitter changes for json.org will need to remain.  Consensus forming
>> > >> on the legal-discuss thread regarding a grace period has been elusive
>> > >> and we're already prepared to make the right steps so we'll just need
>> > >> to take that on by being empathetic to the user base.
>> > >>
>> > >> Thanks
>> > >> Joe
>> > >>
>> > >> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
>> > >>> Andy,
>> > >>>
>> > >>> Great to see NIFI-3050 implemented and certainly good news that NiFi
>> > 1.1.0
>> > >>> is set to include a number of security related improvements.
>> > >>>
>> > >>>
>> > >>>
>> > >>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]
>> >
>> > wrote:
>> > >>>
>> > >>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have
>> > been
>> > >>>> added to my plate for this release. Coordinated with Joe Witt and
>> they
>> > >>>> should both be included.
>> > >>>>
>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
>> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
>> > >>>>
>> > >>>> Andy LoPresto
>> > >>>> [hidden email]
>> > >>>> *[hidden email] <[hidden email]>*
>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> > >>>>
>> > >>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
>> > >>>>
>> > >>>> Team
>> > >>>>
>> > >>>> There is a thread on apache legal-discuss that might allow for a
>> > >>>> graceperiod of continued usage of the json library.  Am going to
>> keep
>> > >>>> a close eye on this and if VP Legal approves we'll be able to keep
>> the
>> > >>>> twitter processors in which is definitely a good thing.  Will advise
>> > >>>>
>> > >>>> Thanks
>> > >>>> Joe
>> > >>>>
>> > >>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]>
>> > wrote:
>> > >>>>
>> > >>>> I've noticed an issue with the per-instance class loading capability
>> > >>>> introduced in NIFI-2909 where the additional classpath resources can
>> > get
>> > >>>> incorrectly removed from the class loader.
>> > >>>>
>> > >>>> I was able to reproduced this with a unit test and have a fix
>> ready. I
>> > >>>> believe this is important and needs to go in for the 1.1 release,
>> > going to
>> > >>>> re-open NIFI-2909 and submit a PR shortly.
>> > >>>>
>> > >>>> -Bryan
>> > >>>>
>> > >>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <
>> [hidden email]
>> > >
>> > >>>> wrote:
>> > >>>>
>> > >>>> I have two items that I would like to wrap up prior to creating an
>> RC
>> > for
>> > >>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process
>> Group
>> > port
>> > >>>> configuration. The work is already completed and I will be reviewing
>> > it
>> > >>>> this today. Additionally, following recent interest on the mailing
>> > list,
>> > >>>> I'd like to knock out NIFI-3020. This will allow an admin to
>> > configure a
>> > >>>> strategy for user identity when logging in via LDAP. Specifically,
>> it
>> > will
>> > >>>> support usage of the DN (the default and current implementation) as
>> > well as
>> > >>>> the username the user logged in as. I should be able to have a PR up
>> > for
>> > >>>> this work later today.
>> > >>>>
>> > >>>> Thanks!
>> > >>>>
>> > >>>> Matt
>> > >>>>
>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>> > >>>>
>> > >>>>
>> > >>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]>
>> wrote:
>> > >>>>
>> > >>>> The code is within the twitter4j library itself.  I filed a request
>> to
>> > >>>> twitter4jg.  The most likely case is we will need to submit a PR to
>> > them.
>> > >>>> However, I don't see this as something that should delay the
>> > release.  We
>> > >>>> can provide instructions for folks wanting to use the processor
>> during
>> > >>>>
>> > >>>> the
>> > >>>>
>> > >>>> time we cannot make it available in a convenient manner.  I will
>> > provide
>> > >>>>
>> > >>>> a
>> > >>>>
>> > >>>> meaningful comment about this in release notes and pointers on what
>> > folks
>> > >>>> can do in the meantime.
>> > >>>>
>> > >>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]>
>> > wrote:
>> > >>>>
>> > >>>> I understand there was a discussion thread within the NiFi community
>> > >>>>
>> > >>>> for
>> > >>>>
>> > >>>> this as well and I missed responding to that at that time. It just
>> > >>>>
>> > >>>> seems
>> > >>>>
>> > >>>> to
>> > >>>>
>> > >>>> me like JSON processing is necessary for GetTwitter, which is
>> > >>>>
>> > >>>> incredibly
>> > >>>>
>> > >>>> useful for demonstrating NiFi’s ability to read from a high volume
>> > >>>>
>> > >>>> stream
>> > >>>>
>> > >>>> out of the box. With NIFI-3019 (Remove GetTwitter from default
>> build),
>> > >>>>
>> > >>>> is
>> > >>>>
>> > >>>> there any related effort to substitute an acceptable replacement
>> JSON
>> > >>>> library to restore this functionality?
>> > >>>>
>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>> > >>>>
>> > >>>> Andy LoPresto
>> > >>>> [hidden email]
>> > >>>> *[hidden email] <[hidden email]>*
>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> > >>>>
>> > >>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>> > >>>>
>> > >>>> wrote:
>> > >>>>
>> > >>>>
>> > >>>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655,
>> and
>> > >>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>> > >>>> investigated some alternate TLS config options for the new version
>> of
>> > >>>>
>> > >>>> the
>> > >>>>
>> > >>>> client library.
>> > >>>>
>> > >>>> Is there any alternative to excluding the GetTwitter processor?
>> Using
>> > >>>> Johnzon [1] or the Android re-implementation [2] discussed in the
>> > >>>>
>> > >>>> mailing
>> > >>>>
>> > >>>> list thread?
>> > >>>>
>> > >>>> [1] https://johnzon.apache.org/
>> > >>>> [2] https://developer.android.com/reference/org/json/package-
>> > >>>>
>> > >>>> summary.html
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> Andy LoPresto
>> > >>>> [hidden email]
>> > >>>> *[hidden email] <[hidden email]>*
>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> > >>>>
>> > >>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>> > >>>>
>> > >>>> Team
>> > >>>>
>> > >>>> Very happy to see that we are down to three items remaining tagged
>> to
>> > >>>> 1.1.0.  Solid effort over the recent weeks to close the gap
>> including
>> > >>>>
>> > >>>> work
>> > >>>>
>> > >>>> to get past the now category x Jason dependency we had.  The most
>> > >>>>
>> > >>>> notable
>> > >>>>
>> > >>>> impact from that is the wildly popular GetTwitter processor, the fav
>> > >>>>
>> > >>>> new
>> > >>>>
>> > >>>> nifi user and demo processor, can no longer be included in the
>> default
>> > >>>> build.  It is optionally available if users choose to build and use
>> it
>> > >>>>
>> > >>>> but
>> > >>>>
>> > >>>> we won't distribute binaries that have it.
>> > >>>>
>> > >>>> I see some review movement on some patch available but untagged
>> items.
>> > >>>>
>> > >>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri.
>> > Anyone
>> > >>>> have any outstanding items?
>> > >>>>
>> > >>>> Thanks
>> > >>>> Joe
>> > >>>>
>> > >>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>> > >>>>
>> > >>>> Ryan
>> > >>>>
>> > >>>> Not officially but I think we should try to close this thing out and
>> > >>>> start a vote in the next week or two at most.
>> > >>>>
>> > >>>> I'm going through the tickets again now.  There is also a new issue
>> of
>> > >>>> the json-p license falling out of favor in Apache legal terms and
>> > >>>> becoming Category-X.  Am looking into that now.
>> > >>>>
>> > >>>> Thanks
>> > >>>> Joe
>> > >>>>
>> > >>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>> > >>>>
>> > >>>> wrote:
>> > >>>>
>> > >>>>
>> > >>>> Joe - Is there a target date for 1.1?
>> > >>>>
>> > >>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]>
>> > wrote:
>> > >>>>
>> > >>>> Team,
>> > >>>>
>> > >>>> Just an update on things with working toward an Apache NiFi 1.1.0
>> > >>>> release.  There are still about 33 JIRAs there now and some are
>> > >>>> awaiting review and are some are under active progress. Yet there is
>> > >>>> good traction and progress. I think we should just stay vigilant
>> with
>> > >>>> what makes it in and keep working it down.  So let's please shoot
>> for
>> > >>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>> > >>>>
>> > >>>> Thanks
>> > >>>> Joe
>> > >>>>
>> > >>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]>
>> wrote:
>> > >>>>
>> > >>>> Team,
>> > >>>>
>> > >>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.
>> Let's
>> > >>>> avoiding putting more in there for now at least without a
>> discussion.
>> > >>>> Of the 31 JIRAs there the vast majority need review so we should be
>> > >>>> able to close these down fairly quickly as long as we don't let the
>> > >>>> list grow.
>> > >>>>
>> > >>>> Thanks
>> > >>>> joe
>> > >>>>
>> > >>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <
>> [hidden email]
>> > >
>> > >>>>
>> > >>>> wrote:
>> > >>>>
>> > >>>> Joe,
>> > >>>>
>> > >>>> Appreciate the offer it isn't my PR. I was just using it as an
>> > >>>>
>> > >>>> example.
>> > >>>>
>> > >>>> All
>> > >>>>
>> > >>>> mine are currently closed, which I greatly appreciate.
>> > >>>>
>> > >>>> Cheers,
>> > >>>>
>> > >>>> Edgardo
>> > >>>>
>> > >>>> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>> > >>>>
>> > >>>> Edgardo,
>> > >>>>
>> > >>>> You mentioned a PR from August. I'd be happy to help you work that
>> > >>>> through review.
>> > >>>>
>> > >>>> Thanks
>> > >>>> Joe
>> > >>>>
>> > >>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>> > >>>>
>> > >>>> [hidden email]
>> > >>>>
>> > >>>> <javascript:;>> wrote:
>> > >>>>
>> > >>>> I have agreed that at this point a release is important. My goal
>> > >>>>
>> > >>>> was
>> > >>>>
>> > >>>> try
>> > >>>>
>> > >>>> to
>> > >>>>
>> > >>>> squeeze in a much goodness as possible into the release, but the
>> > >>>>
>> > >>>> important
>> > >>>>
>> > >>>> bug fixes should come first. Getting 1.x into a state where the
>> > >>>>
>> > >>>> release
>> > >>>>
>> > >>>> notes don't say that it is geared toward developers and testers is
>> > >>>>
>> > >>>> really
>> > >>>>
>> > >>>> huge.
>> > >>>>
>> > >>>> I think Nifi is a great community otherwise I would participate in
>> > >>>>
>> > >>>> the
>> > >>>>
>> > >>>> mailing list, create Jira tickets and pull requests. I am only
>> > >>>>
>> > >>>> trying to
>> > >>>>
>> > >>>> strengthen the great thing that is going on here. We can always do
>> > >>>>
>> > >>>> better.
>> > >>>>
>> > >>>> I was not trying to put down this community only to participate and
>> > >>>>
>> > >>>> make
>> > >>>>
>> > >>>> it
>> > >>>>
>> > >>>> better. I think this conversation is an indication of how great
>> > >>>>
>> > >>>> this
>> > >>>>
>> > >>>> community is.
>> > >>>>
>> > >>>> Maybe I am being sensitive about this issue and trying to
>> > >>>>
>> > >>>> strengthen
>> > >>>>
>> > >>>> the
>> > >>>>
>> > >>>> nifi community even more, after coming from a conference where it
>> > >>>>
>> > >>>> was
>> > >>>>
>> > >>>> reported there was lots of excitement at first and now the
>> > >>>>
>> > >>>> participation
>> > >>>>
>> > >>>> in
>> > >>>>
>> > >>>> the community has really died down and they are struggling. I don't
>> > >>>>
>> > >>>> want
>> > >>>>
>> > >>>> to
>> > >>>>
>> > >>>> see that happen here.
>> > >>>>
>> > >>>> Cheers,
>> > >>>>
>> > >>>> Edgardo
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>> > >>>>
>> > >>>> <javascript:;>> wrote:
>> > >>>>
>> > >>>>
>> > >>>> Edgardo,
>> > >>>>
>> > >>>> Thank you for your feedback. We hear your comments and as a
>> > >>>>
>> > >>>> committer I
>> > >>>>
>> > >>>> can
>> > >>>>
>> > >>>> share we are constantly looking to improve the PR process, having
>> > >>>>
>> > >>>> already
>> > >>>>
>> > >>>> taken many of the steps you suggest.
>> > >>>>
>> > >>>> However, it is important to notice that the number of PRs should
>> > >>>>
>> > >>>> not be
>> > >>>>
>> > >>>> seen as a metric of engagement by the development community: Most
>> > >>>>
>> > >>>> of us
>> > >>>>
>> > >>>> will submit PRs so that our work can be carefully reviewed by our
>> > >>>>
>> > >>>> peers
>> > >>>>
>> > >>>> and
>> > >>>>
>> > >>>> some of us will use JIRA patches to provide contributions.
>> > >>>>
>> > >>>> Having said that, it is true that some PRs may sit idle for a long
>> > >>>>
>> > >>>> time
>> > >>>>
>> > >>>> and
>> > >>>>
>> > >>>> we are working to improve this pipeline.
>> > >>>>
>> > >>>> It was therefore no coincidence that I  browsed most of the PRs
>> > >>>>
>> > >>>> performing
>> > >>>>
>> > >>>> a triage of items that have been superseded or diverged from the
>> > >>>>
>> > >>>> current
>> > >>>>
>> > >>>> code base.
>> > >>>>
>> > >>>> In fact, less than a month ago the dev team closed a number of
>> > >>>>
>> > >>>> stalled
>> > >>>>
>> > >>>> and
>> > >>>>
>> > >>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>> > >>>>
>> > >>>> Despite all the above, I think Joe has a point. The master
>> > >>>>
>> > >>>> contain a
>> > >>>>
>> > >>>> series
>> > >>>>
>> > >>>> of important bug fixes and suspect the community would benefit
>> > >>>>
>> > >>>> from
>> > >>>>
>> > >>>> a
>> > >>>>
>> > >>>> release sooner rather than later.
>> > >>>>
>> > >>>> Once again, thank you for your feedback and contribution. It is
>> > >>>>
>> > >>>> good to
>> > >>>>
>> > >>>> have you here.
>> > >>>>
>> > >>>> Andre
>> > >>>>
>> > >>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>> > >>>>
>> > >>>> [hidden email]
>> > >>>>
>> > >>>> <javascript:;>>
>> > >>>>
>> > >>>> wrote:
>> > >>>>
>> > >>>> Joe - You are correct I was mentioning the PRs that are
>> > >>>>
>> > >>>> currently
>> > >>>>
>> > >>>> open.
>> > >>>>
>> > >>>>
>> > >>>> Regardless of how it happens reducing the count of open PRs I
>> > >>>>
>> > >>>> believe
>> > >>>>
>> > >>>> to
>> > >>>>
>> > >>>> be
>> > >>>>
>> > >>>> extremely important. Maybe I was hoping that the release could
>> > >>>>
>> > >>>> be
>> > >>>>
>> > >>>> a
>> > >>>>
>> > >>>> forcing
>> > >>>>
>> > >>>> function to make that happen. I believe that developers are more
>> > >>>>
>> > >>>> willing
>> > >>>>
>> > >>>> to
>> > >>>>
>> > >>>> contribute when they see that their PRs will actually be able
>> > >>>>
>> > >>>> accepted
>> > >>>>
>> > >>>> and
>> > >>>>
>> > >>>> merged into the code base. Having a low number of open PRs in
>> > >>>>
>> > >>>> progress
>> > >>>>
>> > >>>> is a
>> > >>>>
>> > >>>> great indication that the main nifi developers are fully engaged
>> > >>>>
>> > >>>> with
>> > >>>>
>> > >>>> the
>> > >>>>
>> > >>>> community.
>> > >>>>
>> > >>>> There are a few PRs that don't have any comments from committers
>> > >>>>
>> > >>>> at
>> > >>>>
>> > >>>> all.
>> > >>>>
>> > >>>> I
>> > >>>>
>> > >>>> found one from August in that state. If that was my PR I don't
>> > >>>>
>> > >>>> think I
>> > >>>>
>> > >>>> would be so willing to put another one in anytime soon. I do get
>> > >>>>
>> > >>>> that
>> > >>>>
>> > >>>> sometime PRs get stalled by the originator, if so maybe a rule
>> > >>>>
>> > >>>> about
>> > >>>>
>> > >>>> closing them after a certain amount of time or being taken over
>> > >>>>
>> > >>>> by a
>> > >>>>
>> > >>>> core
>> > >>>>
>> > >>>> contributor if they think it worthwhile.
>> > >>>>
>> > >>>> I would like to shoutout to James Wing on my last PR he was
>> > >>>>
>> > >>>> quick
>> > >>>>
>> > >>>> to
>> > >>>>
>> > >>>> review, provided great comments, testing, and even some
>> > >>>>
>> > >>>> additional
>> > >>>>
>> > >>>> code.
>> > >>>>
>> > >>>> It
>> > >>>>
>> > >>>> was a great PR experience.
>> > >>>>
>> > >>>> Cheers,
>> > >>>>
>> > >>>> Edgardo
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>> > >>>>
>> > >>>> [hidden email] <javascript:;>.
>> > >>>>
>> > >>>> invalid> wrote:
>> > >>>>
>> > >>>> Joe, I think you misread. Edgardo is referring to the Pull
>> > >>>>
>> > >>>> Requests
>> > >>>>
>> > >>>> that
>> > >>>>
>> > >>>> are currently open, not the tickets assigned to the 1.1.0
>> > >>>>
>> > >>>> version.
>> > >>>>
>> > >>>>
>> > >>>> I think these goals (releasing 1.1.0 and cutting down the PR
>> > >>>>
>> > >>>> count)
>> > >>>>
>> > >>>> should
>> > >>>>
>> > >>>> be two different efforts. Doing a thorough job reviewing
>> > >>>>
>> > >>>> takes a
>> > >>>>
>> > >>>> significant amount of time from both the reviewer and
>> > >>>>
>> > >>>> contributor.
>> > >>>>
>> > >>>> In
>> > >>>>
>> > >>>> order
>> > >>>>
>> > >>>> to cut it down significantly would take much longer than a
>> > >>>>
>> > >>>> couple
>> > >>>>
>> > >>>> days.
>> > >>>>
>> > >>>>
>> > >>>> Also there has already been a lot of great new features and
>> > >>>>
>> > >>>> bug
>> > >>>>
>> > >>>> fixes
>> > >>>>
>> > >>>> contributed to the 1.X line and I don't think it's worth
>> > >>>>
>> > >>>> holding up
>> > >>>>
>> > >>>> a
>> > >>>>
>> > >>>> 1.1.0
>> > >>>>
>> > >>>> release for tickets not assigned to this fix version. As an
>> > >>>>
>> > >>>> added
>> > >>>>
>> > >>>> bonus
>> > >>>>
>> > >>>> though, I think many of the tickets tagged as 1.1.0 have PRs
>> > >>>>
>> > >>>> already
>> > >>>>
>> > >>>> open
>> > >>>>
>> > >>>> so closing those will make a large dent in the PR count.
>> > >>>>
>> > >>>>
>> > >>>> Joe
>> > >>>>
>> > >>>> - - - - - -
>> > >>>> Joseph Percivall
>> > >>>> linkedin.com/in/Percivall
>> > >>>> e: [hidden email] <javascript:;>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>> > >>>>
>> > >>>> [hidden email]
>> > >>>>
>> > >>>> <javascript:;>>
>> > >>>>
>> > >>>> wrote:
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> There are less than 30 right now.  Many of the roughly 90+
>> > >>>>
>> > >>>> JIRAs
>> > >>>>
>> > >>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>> > >>>>
>> > >>>> or
>> > >>>>
>> > >>>> just
>> > >>>>
>> > >>>> had fix versions removed.
>> > >>>>
>> > >>>> We will need to have a push over the next bunch of days to
>> > >>>>
>> > >>>> deal
>> > >>>>
>> > >>>> with
>> > >>>>
>> > >>>> reviewing/merging/moving the remaining items.
>> > >>>>
>> > >>>> Thanks
>> > >>>> Joe
>> > >>>>
>> > >>>>
>> > >>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>> > >>>>
>> > >>>> [hidden email] <javascript:;>>
>> > >>>>
>> > >>>> wrote:
>> > >>>>
>> > >>>> Joe,
>> > >>>>
>> > >>>> There are 75 PRs currently open. Why not make a push over
>> > >>>>
>> > >>>> the
>> > >>>>
>> > >>>> next
>> > >>>>
>> > >>>> bunch
>> > >>>>
>> > >>>> of
>> > >>>>
>> > >>>> days to get them closed and then cut the release after that.
>> > >>>>
>> > >>>> Cheers,
>> > >>>>
>> > >>>> Edgardo
>> > >>>>
>> > >>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>> > >>>>
>> > >>>> [hidden email]
>> > >>>>
>> > >>>> <javascript:;>>
>> > >>>>
>> > >>>> wrote:
>> > >>>>
>> > >>>>
>> > >>>> Team,
>> > >>>>
>> > >>>> There have been a ton of bugs fixed a few nice features.  I
>> > >>>>
>> > >>>> would
>> > >>>>
>> > >>>> like
>> > >>>>
>> > >>>> to move to get Apache NiFi 1.1.0 release going pretty much
>> > >>>>
>> > >>>> based
>> > >>>>
>> > >>>> on
>> > >>>>
>> > >>>> where we are now and plan to move most tickets to a new
>> > >>>>
>> > >>>> Apache
>> > >>>>
>> > >>>> NiFi
>> > >>>>
>> > >>>> 1.2.0 version.  We can try to get back on our roughly 6-8
>> > >>>>
>> > >>>> week
>> > >>>>
>> > >>>> release
>> > >>>>
>> > >>>> schedule and shoot for a mid to late Nov release for NiFi
>> > >>>>
>> > >>>> 1.2.0
>> > >>>>
>> > >>>> this
>> > >>>>
>> > >>>> way as well. Please advise if anyone has any other views on
>> > >>>>
>> > >>>> this. In
>> > >>>>
>> > >>>> the mean time I'll get the wheels in motion so you'll be
>> > >>>>
>> > >>>> seeing a
>> > >>>>
>> > >>>> lot
>> > >>>>
>> > >>>> of JIRA/issue updates to move version around.
>> > >>>>
>> > >>>> Thanks
>> > >>>> Joe
>> > >>>>
>> > >>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>> > >>>>
>> > >>>> [hidden email]
>> > >>>>
>> > >>>> <javascript:;>>
>> > >>>>
>> > >>>> wrote:
>> > >>>>
>> > >>>> Sounds good Joe. I have no issue to you doing the rm'ing
>> > >>>>
>> > >>>> for
>> > >>>>
>> > >>>> it.
>> > >>>>
>> > >>>>
>> > >>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>> > >>>>
>> > >>>> <javascript:;>> wrote:
>> > >>>>
>> > >>>>
>> > >>>> Team,
>> > >>>>
>> > >>>> There are a lot of great fixes and improvements on the
>> > >>>>
>> > >>>> master
>> > >>>>
>> > >>>> line
>> > >>>>
>> > >>>> now
>> > >>>>
>> > >>>> and we're at a good time window to start pushing for a
>> > >>>>
>> > >>>> release.
>> > >>>>
>> > >>>> There
>> > >>>>
>> > >>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>> > >>>>
>> > >>>> are
>> > >>>>
>> > >>>> open.
>> > >>>>
>> > >>>> I'm
>> > >>>>
>> > >>>> going to go through them and remove fix versions where
>> > >>>>
>> > >>>> appropriate.
>> > >>>>
>> > >>>>
>> > >>>> I'm happy to take on RM task for this release though if
>> > >>>>
>> > >>>> someone
>> > >>>>
>> > >>>> else
>> > >>>>
>> > >>>> would like to take that on please advise.
>> > >>>>
>> > >>>> Thanks
>> > >>>> Joe
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> --
>> > >>>> Cheers,
>> > >>>>
>> > >>>> Edgardo
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> --
>> > >>>> Cheers,
>> > >>>>
>> > >>>> Edgardo
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> --
>> > >>>> Cheers,
>> > >>>>
>> > >>>> Edgardo
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> --
>> > >>>> Cheers,
>> > >>>>
>> > >>>> Edgardo
>> > >>>>
>> > >>>> Sent from Gmail Mobile
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> >
>>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Joe Witt
mike - also to clarify based on what you brought up I updated the JIRA
description as follows

The following also would be true:
1) Apache NiFi 1.0.0 repositories should work just fine when applied
to an Apache NiFi 1.1.0 installation.

2) Repositories made/updated in Apache NiFi 1.1.0 onward would not
work in older Apache NiFi releases (such as 1.0.0)

On Tue, Nov 22, 2016 at 12:33 PM, Joe Witt <[hidden email]> wrote:

> Mike - yeah good catch and good question.  It does support the old
> format.  We've always been pretty good about being diligent to allow
> folks to upgrade and it honor existing state and in many cases
> configurations and it would automatically port them over.  What has
> always been far more problematic is 'rollback' where people ran on
> newer configurations but could not then go back to old framework code.
> That is what NIFI-2854 tackles at least as far as the
> content/prov/flowfile repositories go.  Now, the code and
> serialization is done in such a way that older version can simply
> ignore what never versions encoded if they don't understand it but
> they should be able to continue on.
>
> I just tried out a 1.0.0 flow with data queued up.  Upgraded to a
> latest NiFi 1.1.0-SNAPSHOT.  Moved the repos over.  And it came up
> perfectly with all the queue data ready to roll.
>
> Thanks
> Joe
>
> On Tue, Nov 22, 2016 at 12:16 PM, Michael Moser <[hidden email]> wrote:
>> After seeing "Repositories now support rollback" in the release notes and
>> reading NIFI-2854 [1], I have a question.
>>
>> Are repositories created using NiFi 1.0.0 compatible with NiFi 1.1.0
>> software?  This is the goal that the ticket seems to indicate with 1.1.0
>> onward, but it's not clear whether 1.0.0 -> 1.1.0 is included.
>>
>> Thanks,
>> -- Mike
>>
>> [1] - https://issues.apache.org/jira/browse/NIFI-2854
>>
>> On Tue, Nov 22, 2016 at 12:08 PM, James Wing <[hidden email]> wrote:
>>
>>> Going through the tickets, it seems like quite a release.  A few more
>>> things for your list:
>>>
>>> GenerateFlowFile updated to support literal/expression content and
>>> attributes
>>>
>>> AWS-related:
>>> * New processors PutCloudWatchMetric, PutKinesisStream
>>> * Updated processors PutS3Object (content type, signer options), ListS3
>>> (performance, versions)
>>> * Added support for AWS assume role credentials with proxy
>>>
>>>
>>> Thanks,
>>>
>>> James
>>>
>>> On Mon, Nov 21, 2016 at 9:30 PM, Joe Witt <[hidden email]> wrote:
>>>
>>> > matt
>>> >
>>> > i'll add a wiki page or set of instructions linked from the release
>>> notes.
>>> >
>>> > all,
>>> >
>>> > walked through the 250 or so JIRAs in the 1.1.0 release and pulled out
>>> > highlights.  The items noted are as follows.  Will likely reduce this
>>> > down further for the release notes but wanted to put this out in case
>>> > folks have things they think are really important to highlight.
>>> >
>>> > - Core Improvements:
>>> >    - Performance: Session Migration
>>> >    - Stability: Cluster Management
>>> >    - Developer: Framework supports easy user driven classloader extension
>>> >    - Expression Language: Now supports base64 and hex encoded values
>>> > and Math functions
>>> >    - Repositories now support rollback
>>> >    - Faster startup due to more efficient state restoration algorithm
>>> > - UX Improvements:
>>> >    - Visual Backpressure Indicator
>>> >    - Introduced more colors to better highlight actions and components
>>> >    - Performance: Validate non-running components
>>> >    - Provenance graph image can be exported
>>> >    - Cron Scheduling for Primary node tasks now supported
>>> > - Updated versions
>>> >    - Azure Event Hub 0.9.0
>>> >    - Spark 2.0.1
>>> >    - Hadoop 2.7.x
>>> > - New/Improved Processors
>>> >    - new Fetch/Put Elastic Search 5.0
>>> >    - new ParseCEF to parse CEF formatted logs
>>> >    - improve ExtractEmail now supports TNEF files
>>> >    - new Validate CSV
>>> >    - improved Solr processors now support SSL and Kerberos
>>> >    - new Websocket client and server processors
>>> > - New Utility
>>> >    - Zookeeper Migrator (move from one zookeeper to another)
>>> > - Security
>>> >    - Restricted Processors
>>> >    - Site-to-site now supports port forwarding
>>> >    - Improved Policy Management UX
>>> > - Migration Notes:
>>> >    - Restricted Processors
>>> >    - Twitter Processor Removed
>>> >
>>> > On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]>
>>> > wrote:
>>> > > Is there a good spot for us to put instructions on how to build the
>>> > > Twitter processor and/or the Social Media NAR in the meantime? Maybe a
>>> > > Wiki page or something simple to say "go to this directory, run this
>>> > > Maven command, drop the NAR into your deployment..." ?
>>> > >
>>> > > On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]> wrote:
>>> > >> Team,
>>> > >>
>>> > >> We appear to be very close.  Andy is working NIFI-3024 but otherwise
>>> > >> it is focus on testing.
>>> > >>
>>> > >> I'm going to prep the RC and release notes now.  Unfortunately the
>>> > >> twitter changes for json.org will need to remain.  Consensus forming
>>> > >> on the legal-discuss thread regarding a grace period has been elusive
>>> > >> and we're already prepared to make the right steps so we'll just need
>>> > >> to take that on by being empathetic to the user base.
>>> > >>
>>> > >> Thanks
>>> > >> Joe
>>> > >>
>>> > >> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]> wrote:
>>> > >>> Andy,
>>> > >>>
>>> > >>> Great to see NIFI-3050 implemented and certainly good news that NiFi
>>> > 1.1.0
>>> > >>> is set to include a number of security related improvements.
>>> > >>>
>>> > >>>
>>> > >>>
>>> > >>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <[hidden email]
>>> >
>>> > wrote:
>>> > >>>
>>> > >>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have
>>> > been
>>> > >>>> added to my plate for this release. Coordinated with Joe Witt and
>>> they
>>> > >>>> should both be included.
>>> > >>>>
>>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
>>> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
>>> > >>>>
>>> > >>>> Andy LoPresto
>>> > >>>> [hidden email]
>>> > >>>> *[hidden email] <[hidden email]>*
>>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>> > >>>>
>>> > >>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]> wrote:
>>> > >>>>
>>> > >>>> Team
>>> > >>>>
>>> > >>>> There is a thread on apache legal-discuss that might allow for a
>>> > >>>> graceperiod of continued usage of the json library.  Am going to
>>> keep
>>> > >>>> a close eye on this and if VP Legal approves we'll be able to keep
>>> the
>>> > >>>> twitter processors in which is definitely a good thing.  Will advise
>>> > >>>>
>>> > >>>> Thanks
>>> > >>>> Joe
>>> > >>>>
>>> > >>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]>
>>> > wrote:
>>> > >>>>
>>> > >>>> I've noticed an issue with the per-instance class loading capability
>>> > >>>> introduced in NIFI-2909 where the additional classpath resources can
>>> > get
>>> > >>>> incorrectly removed from the class loader.
>>> > >>>>
>>> > >>>> I was able to reproduced this with a unit test and have a fix
>>> ready. I
>>> > >>>> believe this is important and needs to go in for the 1.1 release,
>>> > going to
>>> > >>>> re-open NIFI-2909 and submit a PR shortly.
>>> > >>>>
>>> > >>>> -Bryan
>>> > >>>>
>>> > >>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <
>>> [hidden email]
>>> > >
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>> I have two items that I would like to wrap up prior to creating an
>>> RC
>>> > for
>>> > >>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process
>>> Group
>>> > port
>>> > >>>> configuration. The work is already completed and I will be reviewing
>>> > it
>>> > >>>> this today. Additionally, following recent interest on the mailing
>>> > list,
>>> > >>>> I'd like to knock out NIFI-3020. This will allow an admin to
>>> > configure a
>>> > >>>> strategy for user identity when logging in via LDAP. Specifically,
>>> it
>>> > will
>>> > >>>> support usage of the DN (the default and current implementation) as
>>> > well as
>>> > >>>> the username the user logged in as. I should be able to have a PR up
>>> > for
>>> > >>>> this work later today.
>>> > >>>>
>>> > >>>> Thanks!
>>> > >>>>
>>> > >>>> Matt
>>> > >>>>
>>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>>> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>>> > >>>>
>>> > >>>>
>>> > >>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]>
>>> wrote:
>>> > >>>>
>>> > >>>> The code is within the twitter4j library itself.  I filed a request
>>> to
>>> > >>>> twitter4jg.  The most likely case is we will need to submit a PR to
>>> > them.
>>> > >>>> However, I don't see this as something that should delay the
>>> > release.  We
>>> > >>>> can provide instructions for folks wanting to use the processor
>>> during
>>> > >>>>
>>> > >>>> the
>>> > >>>>
>>> > >>>> time we cannot make it available in a convenient manner.  I will
>>> > provide
>>> > >>>>
>>> > >>>> a
>>> > >>>>
>>> > >>>> meaningful comment about this in release notes and pointers on what
>>> > folks
>>> > >>>> can do in the meantime.
>>> > >>>>
>>> > >>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]>
>>> > wrote:
>>> > >>>>
>>> > >>>> I understand there was a discussion thread within the NiFi community
>>> > >>>>
>>> > >>>> for
>>> > >>>>
>>> > >>>> this as well and I missed responding to that at that time. It just
>>> > >>>>
>>> > >>>> seems
>>> > >>>>
>>> > >>>> to
>>> > >>>>
>>> > >>>> me like JSON processing is necessary for GetTwitter, which is
>>> > >>>>
>>> > >>>> incredibly
>>> > >>>>
>>> > >>>> useful for demonstrating NiFi’s ability to read from a high volume
>>> > >>>>
>>> > >>>> stream
>>> > >>>>
>>> > >>>> out of the box. With NIFI-3019 (Remove GetTwitter from default
>>> build),
>>> > >>>>
>>> > >>>> is
>>> > >>>>
>>> > >>>> there any related effort to substitute an acceptable replacement
>>> JSON
>>> > >>>> library to restore this functionality?
>>> > >>>>
>>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>>> > >>>>
>>> > >>>> Andy LoPresto
>>> > >>>> [hidden email]
>>> > >>>> *[hidden email] <[hidden email]>*
>>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>> > >>>>
>>> > >>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <[hidden email]>
>>> > >>>>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>>
>>> > >>>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655,
>>> and
>>> > >>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>>> > >>>> investigated some alternate TLS config options for the new version
>>> of
>>> > >>>>
>>> > >>>> the
>>> > >>>>
>>> > >>>> client library.
>>> > >>>>
>>> > >>>> Is there any alternative to excluding the GetTwitter processor?
>>> Using
>>> > >>>> Johnzon [1] or the Android re-implementation [2] discussed in the
>>> > >>>>
>>> > >>>> mailing
>>> > >>>>
>>> > >>>> list thread?
>>> > >>>>
>>> > >>>> [1] https://johnzon.apache.org/
>>> > >>>> [2] https://developer.android.com/reference/org/json/package-
>>> > >>>>
>>> > >>>> summary.html
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> Andy LoPresto
>>> > >>>> [hidden email]
>>> > >>>> *[hidden email] <[hidden email]>*
>>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>> > >>>>
>>> > >>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]> wrote:
>>> > >>>>
>>> > >>>> Team
>>> > >>>>
>>> > >>>> Very happy to see that we are down to three items remaining tagged
>>> to
>>> > >>>> 1.1.0.  Solid effort over the recent weeks to close the gap
>>> including
>>> > >>>>
>>> > >>>> work
>>> > >>>>
>>> > >>>> to get past the now category x Jason dependency we had.  The most
>>> > >>>>
>>> > >>>> notable
>>> > >>>>
>>> > >>>> impact from that is the wildly popular GetTwitter processor, the fav
>>> > >>>>
>>> > >>>> new
>>> > >>>>
>>> > >>>> nifi user and demo processor, can no longer be included in the
>>> default
>>> > >>>> build.  It is optionally available if users choose to build and use
>>> it
>>> > >>>>
>>> > >>>> but
>>> > >>>>
>>> > >>>> we won't distribute binaries that have it.
>>> > >>>>
>>> > >>>> I see some review movement on some patch available but untagged
>>> items.
>>> > >>>>
>>> > >>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri.
>>> > Anyone
>>> > >>>> have any outstanding items?
>>> > >>>>
>>> > >>>> Thanks
>>> > >>>> Joe
>>> > >>>>
>>> > >>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>>> > >>>>
>>> > >>>> Ryan
>>> > >>>>
>>> > >>>> Not officially but I think we should try to close this thing out and
>>> > >>>> start a vote in the next week or two at most.
>>> > >>>>
>>> > >>>> I'm going through the tickets again now.  There is also a new issue
>>> of
>>> > >>>> the json-p license falling out of favor in Apache legal terms and
>>> > >>>> becoming Category-X.  Am looking into that now.
>>> > >>>>
>>> > >>>> Thanks
>>> > >>>> Joe
>>> > >>>>
>>> > >>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]>
>>> > >>>>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>>
>>> > >>>> Joe - Is there a target date for 1.1?
>>> > >>>>
>>> > >>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]>
>>> > wrote:
>>> > >>>>
>>> > >>>> Team,
>>> > >>>>
>>> > >>>> Just an update on things with working toward an Apache NiFi 1.1.0
>>> > >>>> release.  There are still about 33 JIRAs there now and some are
>>> > >>>> awaiting review and are some are under active progress. Yet there is
>>> > >>>> good traction and progress. I think we should just stay vigilant
>>> with
>>> > >>>> what makes it in and keep working it down.  So let's please shoot
>>> for
>>> > >>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>>> > >>>>
>>> > >>>> Thanks
>>> > >>>> Joe
>>> > >>>>
>>> > >>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]>
>>> wrote:
>>> > >>>>
>>> > >>>> Team,
>>> > >>>>
>>> > >>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.
>>> Let's
>>> > >>>> avoiding putting more in there for now at least without a
>>> discussion.
>>> > >>>> Of the 31 JIRAs there the vast majority need review so we should be
>>> > >>>> able to close these down fairly quickly as long as we don't let the
>>> > >>>> list grow.
>>> > >>>>
>>> > >>>> Thanks
>>> > >>>> joe
>>> > >>>>
>>> > >>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <
>>> [hidden email]
>>> > >
>>> > >>>>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>> Joe,
>>> > >>>>
>>> > >>>> Appreciate the offer it isn't my PR. I was just using it as an
>>> > >>>>
>>> > >>>> example.
>>> > >>>>
>>> > >>>> All
>>> > >>>>
>>> > >>>> mine are currently closed, which I greatly appreciate.
>>> > >>>>
>>> > >>>> Cheers,
>>> > >>>>
>>> > >>>> Edgardo
>>> > >>>>
>>> > >>>> On Friday, October 14, 2016, Joe Witt <[hidden email]> wrote:
>>> > >>>>
>>> > >>>> Edgardo,
>>> > >>>>
>>> > >>>> You mentioned a PR from August. I'd be happy to help you work that
>>> > >>>> through review.
>>> > >>>>
>>> > >>>> Thanks
>>> > >>>> Joe
>>> > >>>>
>>> > >>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>>> > >>>>
>>> > >>>> [hidden email]
>>> > >>>>
>>> > >>>> <javascript:;>> wrote:
>>> > >>>>
>>> > >>>> I have agreed that at this point a release is important. My goal
>>> > >>>>
>>> > >>>> was
>>> > >>>>
>>> > >>>> try
>>> > >>>>
>>> > >>>> to
>>> > >>>>
>>> > >>>> squeeze in a much goodness as possible into the release, but the
>>> > >>>>
>>> > >>>> important
>>> > >>>>
>>> > >>>> bug fixes should come first. Getting 1.x into a state where the
>>> > >>>>
>>> > >>>> release
>>> > >>>>
>>> > >>>> notes don't say that it is geared toward developers and testers is
>>> > >>>>
>>> > >>>> really
>>> > >>>>
>>> > >>>> huge.
>>> > >>>>
>>> > >>>> I think Nifi is a great community otherwise I would participate in
>>> > >>>>
>>> > >>>> the
>>> > >>>>
>>> > >>>> mailing list, create Jira tickets and pull requests. I am only
>>> > >>>>
>>> > >>>> trying to
>>> > >>>>
>>> > >>>> strengthen the great thing that is going on here. We can always do
>>> > >>>>
>>> > >>>> better.
>>> > >>>>
>>> > >>>> I was not trying to put down this community only to participate and
>>> > >>>>
>>> > >>>> make
>>> > >>>>
>>> > >>>> it
>>> > >>>>
>>> > >>>> better. I think this conversation is an indication of how great
>>> > >>>>
>>> > >>>> this
>>> > >>>>
>>> > >>>> community is.
>>> > >>>>
>>> > >>>> Maybe I am being sensitive about this issue and trying to
>>> > >>>>
>>> > >>>> strengthen
>>> > >>>>
>>> > >>>> the
>>> > >>>>
>>> > >>>> nifi community even more, after coming from a conference where it
>>> > >>>>
>>> > >>>> was
>>> > >>>>
>>> > >>>> reported there was lots of excitement at first and now the
>>> > >>>>
>>> > >>>> participation
>>> > >>>>
>>> > >>>> in
>>> > >>>>
>>> > >>>> the community has really died down and they are struggling. I don't
>>> > >>>>
>>> > >>>> want
>>> > >>>>
>>> > >>>> to
>>> > >>>>
>>> > >>>> see that happen here.
>>> > >>>>
>>> > >>>> Cheers,
>>> > >>>>
>>> > >>>> Edgardo
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>>> > >>>>
>>> > >>>> <javascript:;>> wrote:
>>> > >>>>
>>> > >>>>
>>> > >>>> Edgardo,
>>> > >>>>
>>> > >>>> Thank you for your feedback. We hear your comments and as a
>>> > >>>>
>>> > >>>> committer I
>>> > >>>>
>>> > >>>> can
>>> > >>>>
>>> > >>>> share we are constantly looking to improve the PR process, having
>>> > >>>>
>>> > >>>> already
>>> > >>>>
>>> > >>>> taken many of the steps you suggest.
>>> > >>>>
>>> > >>>> However, it is important to notice that the number of PRs should
>>> > >>>>
>>> > >>>> not be
>>> > >>>>
>>> > >>>> seen as a metric of engagement by the development community: Most
>>> > >>>>
>>> > >>>> of us
>>> > >>>>
>>> > >>>> will submit PRs so that our work can be carefully reviewed by our
>>> > >>>>
>>> > >>>> peers
>>> > >>>>
>>> > >>>> and
>>> > >>>>
>>> > >>>> some of us will use JIRA patches to provide contributions.
>>> > >>>>
>>> > >>>> Having said that, it is true that some PRs may sit idle for a long
>>> > >>>>
>>> > >>>> time
>>> > >>>>
>>> > >>>> and
>>> > >>>>
>>> > >>>> we are working to improve this pipeline.
>>> > >>>>
>>> > >>>> It was therefore no coincidence that I  browsed most of the PRs
>>> > >>>>
>>> > >>>> performing
>>> > >>>>
>>> > >>>> a triage of items that have been superseded or diverged from the
>>> > >>>>
>>> > >>>> current
>>> > >>>>
>>> > >>>> code base.
>>> > >>>>
>>> > >>>> In fact, less than a month ago the dev team closed a number of
>>> > >>>>
>>> > >>>> stalled
>>> > >>>>
>>> > >>>> and
>>> > >>>>
>>> > >>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>>> > >>>>
>>> > >>>> Despite all the above, I think Joe has a point. The master
>>> > >>>>
>>> > >>>> contain a
>>> > >>>>
>>> > >>>> series
>>> > >>>>
>>> > >>>> of important bug fixes and suspect the community would benefit
>>> > >>>>
>>> > >>>> from
>>> > >>>>
>>> > >>>> a
>>> > >>>>
>>> > >>>> release sooner rather than later.
>>> > >>>>
>>> > >>>> Once again, thank you for your feedback and contribution. It is
>>> > >>>>
>>> > >>>> good to
>>> > >>>>
>>> > >>>> have you here.
>>> > >>>>
>>> > >>>> Andre
>>> > >>>>
>>> > >>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>>> > >>>>
>>> > >>>> [hidden email]
>>> > >>>>
>>> > >>>> <javascript:;>>
>>> > >>>>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>> Joe - You are correct I was mentioning the PRs that are
>>> > >>>>
>>> > >>>> currently
>>> > >>>>
>>> > >>>> open.
>>> > >>>>
>>> > >>>>
>>> > >>>> Regardless of how it happens reducing the count of open PRs I
>>> > >>>>
>>> > >>>> believe
>>> > >>>>
>>> > >>>> to
>>> > >>>>
>>> > >>>> be
>>> > >>>>
>>> > >>>> extremely important. Maybe I was hoping that the release could
>>> > >>>>
>>> > >>>> be
>>> > >>>>
>>> > >>>> a
>>> > >>>>
>>> > >>>> forcing
>>> > >>>>
>>> > >>>> function to make that happen. I believe that developers are more
>>> > >>>>
>>> > >>>> willing
>>> > >>>>
>>> > >>>> to
>>> > >>>>
>>> > >>>> contribute when they see that their PRs will actually be able
>>> > >>>>
>>> > >>>> accepted
>>> > >>>>
>>> > >>>> and
>>> > >>>>
>>> > >>>> merged into the code base. Having a low number of open PRs in
>>> > >>>>
>>> > >>>> progress
>>> > >>>>
>>> > >>>> is a
>>> > >>>>
>>> > >>>> great indication that the main nifi developers are fully engaged
>>> > >>>>
>>> > >>>> with
>>> > >>>>
>>> > >>>> the
>>> > >>>>
>>> > >>>> community.
>>> > >>>>
>>> > >>>> There are a few PRs that don't have any comments from committers
>>> > >>>>
>>> > >>>> at
>>> > >>>>
>>> > >>>> all.
>>> > >>>>
>>> > >>>> I
>>> > >>>>
>>> > >>>> found one from August in that state. If that was my PR I don't
>>> > >>>>
>>> > >>>> think I
>>> > >>>>
>>> > >>>> would be so willing to put another one in anytime soon. I do get
>>> > >>>>
>>> > >>>> that
>>> > >>>>
>>> > >>>> sometime PRs get stalled by the originator, if so maybe a rule
>>> > >>>>
>>> > >>>> about
>>> > >>>>
>>> > >>>> closing them after a certain amount of time or being taken over
>>> > >>>>
>>> > >>>> by a
>>> > >>>>
>>> > >>>> core
>>> > >>>>
>>> > >>>> contributor if they think it worthwhile.
>>> > >>>>
>>> > >>>> I would like to shoutout to James Wing on my last PR he was
>>> > >>>>
>>> > >>>> quick
>>> > >>>>
>>> > >>>> to
>>> > >>>>
>>> > >>>> review, provided great comments, testing, and even some
>>> > >>>>
>>> > >>>> additional
>>> > >>>>
>>> > >>>> code.
>>> > >>>>
>>> > >>>> It
>>> > >>>>
>>> > >>>> was a great PR experience.
>>> > >>>>
>>> > >>>> Cheers,
>>> > >>>>
>>> > >>>> Edgardo
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>>> > >>>>
>>> > >>>> [hidden email] <javascript:;>.
>>> > >>>>
>>> > >>>> invalid> wrote:
>>> > >>>>
>>> > >>>> Joe, I think you misread. Edgardo is referring to the Pull
>>> > >>>>
>>> > >>>> Requests
>>> > >>>>
>>> > >>>> that
>>> > >>>>
>>> > >>>> are currently open, not the tickets assigned to the 1.1.0
>>> > >>>>
>>> > >>>> version.
>>> > >>>>
>>> > >>>>
>>> > >>>> I think these goals (releasing 1.1.0 and cutting down the PR
>>> > >>>>
>>> > >>>> count)
>>> > >>>>
>>> > >>>> should
>>> > >>>>
>>> > >>>> be two different efforts. Doing a thorough job reviewing
>>> > >>>>
>>> > >>>> takes a
>>> > >>>>
>>> > >>>> significant amount of time from both the reviewer and
>>> > >>>>
>>> > >>>> contributor.
>>> > >>>>
>>> > >>>> In
>>> > >>>>
>>> > >>>> order
>>> > >>>>
>>> > >>>> to cut it down significantly would take much longer than a
>>> > >>>>
>>> > >>>> couple
>>> > >>>>
>>> > >>>> days.
>>> > >>>>
>>> > >>>>
>>> > >>>> Also there has already been a lot of great new features and
>>> > >>>>
>>> > >>>> bug
>>> > >>>>
>>> > >>>> fixes
>>> > >>>>
>>> > >>>> contributed to the 1.X line and I don't think it's worth
>>> > >>>>
>>> > >>>> holding up
>>> > >>>>
>>> > >>>> a
>>> > >>>>
>>> > >>>> 1.1.0
>>> > >>>>
>>> > >>>> release for tickets not assigned to this fix version. As an
>>> > >>>>
>>> > >>>> added
>>> > >>>>
>>> > >>>> bonus
>>> > >>>>
>>> > >>>> though, I think many of the tickets tagged as 1.1.0 have PRs
>>> > >>>>
>>> > >>>> already
>>> > >>>>
>>> > >>>> open
>>> > >>>>
>>> > >>>> so closing those will make a large dent in the PR count.
>>> > >>>>
>>> > >>>>
>>> > >>>> Joe
>>> > >>>>
>>> > >>>> - - - - - -
>>> > >>>> Joseph Percivall
>>> > >>>> linkedin.com/in/Percivall
>>> > >>>> e: [hidden email] <javascript:;>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>>> > >>>>
>>> > >>>> [hidden email]
>>> > >>>>
>>> > >>>> <javascript:;>>
>>> > >>>>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> There are less than 30 right now.  Many of the roughly 90+
>>> > >>>>
>>> > >>>> JIRAs
>>> > >>>>
>>> > >>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>>> > >>>>
>>> > >>>> or
>>> > >>>>
>>> > >>>> just
>>> > >>>>
>>> > >>>> had fix versions removed.
>>> > >>>>
>>> > >>>> We will need to have a push over the next bunch of days to
>>> > >>>>
>>> > >>>> deal
>>> > >>>>
>>> > >>>> with
>>> > >>>>
>>> > >>>> reviewing/merging/moving the remaining items.
>>> > >>>>
>>> > >>>> Thanks
>>> > >>>> Joe
>>> > >>>>
>>> > >>>>
>>> > >>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>>> > >>>>
>>> > >>>> [hidden email] <javascript:;>>
>>> > >>>>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>> Joe,
>>> > >>>>
>>> > >>>> There are 75 PRs currently open. Why not make a push over
>>> > >>>>
>>> > >>>> the
>>> > >>>>
>>> > >>>> next
>>> > >>>>
>>> > >>>> bunch
>>> > >>>>
>>> > >>>> of
>>> > >>>>
>>> > >>>> days to get them closed and then cut the release after that.
>>> > >>>>
>>> > >>>> Cheers,
>>> > >>>>
>>> > >>>> Edgardo
>>> > >>>>
>>> > >>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>>> > >>>>
>>> > >>>> [hidden email]
>>> > >>>>
>>> > >>>> <javascript:;>>
>>> > >>>>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>>
>>> > >>>> Team,
>>> > >>>>
>>> > >>>> There have been a ton of bugs fixed a few nice features.  I
>>> > >>>>
>>> > >>>> would
>>> > >>>>
>>> > >>>> like
>>> > >>>>
>>> > >>>> to move to get Apache NiFi 1.1.0 release going pretty much
>>> > >>>>
>>> > >>>> based
>>> > >>>>
>>> > >>>> on
>>> > >>>>
>>> > >>>> where we are now and plan to move most tickets to a new
>>> > >>>>
>>> > >>>> Apache
>>> > >>>>
>>> > >>>> NiFi
>>> > >>>>
>>> > >>>> 1.2.0 version.  We can try to get back on our roughly 6-8
>>> > >>>>
>>> > >>>> week
>>> > >>>>
>>> > >>>> release
>>> > >>>>
>>> > >>>> schedule and shoot for a mid to late Nov release for NiFi
>>> > >>>>
>>> > >>>> 1.2.0
>>> > >>>>
>>> > >>>> this
>>> > >>>>
>>> > >>>> way as well. Please advise if anyone has any other views on
>>> > >>>>
>>> > >>>> this. In
>>> > >>>>
>>> > >>>> the mean time I'll get the wheels in motion so you'll be
>>> > >>>>
>>> > >>>> seeing a
>>> > >>>>
>>> > >>>> lot
>>> > >>>>
>>> > >>>> of JIRA/issue updates to move version around.
>>> > >>>>
>>> > >>>> Thanks
>>> > >>>> Joe
>>> > >>>>
>>> > >>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>>> > >>>>
>>> > >>>> [hidden email]
>>> > >>>>
>>> > >>>> <javascript:;>>
>>> > >>>>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>> Sounds good Joe. I have no issue to you doing the rm'ing
>>> > >>>>
>>> > >>>> for
>>> > >>>>
>>> > >>>> it.
>>> > >>>>
>>> > >>>>
>>> > >>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>>> > >>>>
>>> > >>>> <javascript:;>> wrote:
>>> > >>>>
>>> > >>>>
>>> > >>>> Team,
>>> > >>>>
>>> > >>>> There are a lot of great fixes and improvements on the
>>> > >>>>
>>> > >>>> master
>>> > >>>>
>>> > >>>> line
>>> > >>>>
>>> > >>>> now
>>> > >>>>
>>> > >>>> and we're at a good time window to start pushing for a
>>> > >>>>
>>> > >>>> release.
>>> > >>>>
>>> > >>>> There
>>> > >>>>
>>> > >>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>>> > >>>>
>>> > >>>> are
>>> > >>>>
>>> > >>>> open.
>>> > >>>>
>>> > >>>> I'm
>>> > >>>>
>>> > >>>> going to go through them and remove fix versions where
>>> > >>>>
>>> > >>>> appropriate.
>>> > >>>>
>>> > >>>>
>>> > >>>> I'm happy to take on RM task for this release though if
>>> > >>>>
>>> > >>>> someone
>>> > >>>>
>>> > >>>> else
>>> > >>>>
>>> > >>>> would like to take that on please advise.
>>> > >>>>
>>> > >>>> Thanks
>>> > >>>> Joe
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> --
>>> > >>>> Cheers,
>>> > >>>>
>>> > >>>> Edgardo
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> --
>>> > >>>> Cheers,
>>> > >>>>
>>> > >>>> Edgardo
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> --
>>> > >>>> Cheers,
>>> > >>>>
>>> > >>>> Edgardo
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> --
>>> > >>>> Cheers,
>>> > >>>>
>>> > >>>> Edgardo
>>> > >>>>
>>> > >>>> Sent from Gmail Mobile
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> >
>>>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Michael Moser
Awesome.  Your prompt and detailed feedback is very much appreciated!


On Tue, Nov 22, 2016 at 12:47 PM, Joe Witt <[hidden email]> wrote:

> mike - also to clarify based on what you brought up I updated the JIRA
> description as follows
>
> The following also would be true:
> 1) Apache NiFi 1.0.0 repositories should work just fine when applied
> to an Apache NiFi 1.1.0 installation.
>
> 2) Repositories made/updated in Apache NiFi 1.1.0 onward would not
> work in older Apache NiFi releases (such as 1.0.0)
>
> On Tue, Nov 22, 2016 at 12:33 PM, Joe Witt <[hidden email]> wrote:
> > Mike - yeah good catch and good question.  It does support the old
> > format.  We've always been pretty good about being diligent to allow
> > folks to upgrade and it honor existing state and in many cases
> > configurations and it would automatically port them over.  What has
> > always been far more problematic is 'rollback' where people ran on
> > newer configurations but could not then go back to old framework code.
> > That is what NIFI-2854 tackles at least as far as the
> > content/prov/flowfile repositories go.  Now, the code and
> > serialization is done in such a way that older version can simply
> > ignore what never versions encoded if they don't understand it but
> > they should be able to continue on.
> >
> > I just tried out a 1.0.0 flow with data queued up.  Upgraded to a
> > latest NiFi 1.1.0-SNAPSHOT.  Moved the repos over.  And it came up
> > perfectly with all the queue data ready to roll.
> >
> > Thanks
> > Joe
> >
> > On Tue, Nov 22, 2016 at 12:16 PM, Michael Moser <[hidden email]>
> wrote:
> >> After seeing "Repositories now support rollback" in the release notes
> and
> >> reading NIFI-2854 [1], I have a question.
> >>
> >> Are repositories created using NiFi 1.0.0 compatible with NiFi 1.1.0
> >> software?  This is the goal that the ticket seems to indicate with 1.1.0
> >> onward, but it's not clear whether 1.0.0 -> 1.1.0 is included.
> >>
> >> Thanks,
> >> -- Mike
> >>
> >> [1] - https://issues.apache.org/jira/browse/NIFI-2854
> >>
> >> On Tue, Nov 22, 2016 at 12:08 PM, James Wing <[hidden email]> wrote:
> >>
> >>> Going through the tickets, it seems like quite a release.  A few more
> >>> things for your list:
> >>>
> >>> GenerateFlowFile updated to support literal/expression content and
> >>> attributes
> >>>
> >>> AWS-related:
> >>> * New processors PutCloudWatchMetric, PutKinesisStream
> >>> * Updated processors PutS3Object (content type, signer options), ListS3
> >>> (performance, versions)
> >>> * Added support for AWS assume role credentials with proxy
> >>>
> >>>
> >>> Thanks,
> >>>
> >>> James
> >>>
> >>> On Mon, Nov 21, 2016 at 9:30 PM, Joe Witt <[hidden email]> wrote:
> >>>
> >>> > matt
> >>> >
> >>> > i'll add a wiki page or set of instructions linked from the release
> >>> notes.
> >>> >
> >>> > all,
> >>> >
> >>> > walked through the 250 or so JIRAs in the 1.1.0 release and pulled
> out
> >>> > highlights.  The items noted are as follows.  Will likely reduce this
> >>> > down further for the release notes but wanted to put this out in case
> >>> > folks have things they think are really important to highlight.
> >>> >
> >>> > - Core Improvements:
> >>> >    - Performance: Session Migration
> >>> >    - Stability: Cluster Management
> >>> >    - Developer: Framework supports easy user driven classloader
> extension
> >>> >    - Expression Language: Now supports base64 and hex encoded values
> >>> > and Math functions
> >>> >    - Repositories now support rollback
> >>> >    - Faster startup due to more efficient state restoration algorithm
> >>> > - UX Improvements:
> >>> >    - Visual Backpressure Indicator
> >>> >    - Introduced more colors to better highlight actions and
> components
> >>> >    - Performance: Validate non-running components
> >>> >    - Provenance graph image can be exported
> >>> >    - Cron Scheduling for Primary node tasks now supported
> >>> > - Updated versions
> >>> >    - Azure Event Hub 0.9.0
> >>> >    - Spark 2.0.1
> >>> >    - Hadoop 2.7.x
> >>> > - New/Improved Processors
> >>> >    - new Fetch/Put Elastic Search 5.0
> >>> >    - new ParseCEF to parse CEF formatted logs
> >>> >    - improve ExtractEmail now supports TNEF files
> >>> >    - new Validate CSV
> >>> >    - improved Solr processors now support SSL and Kerberos
> >>> >    - new Websocket client and server processors
> >>> > - New Utility
> >>> >    - Zookeeper Migrator (move from one zookeeper to another)
> >>> > - Security
> >>> >    - Restricted Processors
> >>> >    - Site-to-site now supports port forwarding
> >>> >    - Improved Policy Management UX
> >>> > - Migration Notes:
> >>> >    - Restricted Processors
> >>> >    - Twitter Processor Removed
> >>> >
> >>> > On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]
> >
> >>> > wrote:
> >>> > > Is there a good spot for us to put instructions on how to build the
> >>> > > Twitter processor and/or the Social Media NAR in the meantime?
> Maybe a
> >>> > > Wiki page or something simple to say "go to this directory, run
> this
> >>> > > Maven command, drop the NAR into your deployment..." ?
> >>> > >
> >>> > > On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]>
> wrote:
> >>> > >> Team,
> >>> > >>
> >>> > >> We appear to be very close.  Andy is working NIFI-3024 but
> otherwise
> >>> > >> it is focus on testing.
> >>> > >>
> >>> > >> I'm going to prep the RC and release notes now.  Unfortunately the
> >>> > >> twitter changes for json.org will need to remain.  Consensus
> forming
> >>> > >> on the legal-discuss thread regarding a grace period has been
> elusive
> >>> > >> and we're already prepared to make the right steps so we'll just
> need
> >>> > >> to take that on by being empathetic to the user base.
> >>> > >>
> >>> > >> Thanks
> >>> > >> Joe
> >>> > >>
> >>> > >> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]>
> wrote:
> >>> > >>> Andy,
> >>> > >>>
> >>> > >>> Great to see NIFI-3050 implemented and certainly good news that
> NiFi
> >>> > 1.1.0
> >>> > >>> is set to include a number of security related improvements.
> >>> > >>>
> >>> > >>>
> >>> > >>>
> >>> > >>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <
> [hidden email]
> >>> >
> >>> > wrote:
> >>> > >>>
> >>> > >>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2]
> have
> >>> > been
> >>> > >>>> added to my plate for this release. Coordinated with Joe Witt
> and
> >>> they
> >>> > >>>> should both be included.
> >>> > >>>>
> >>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
> >>> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
> >>> > >>>>
> >>> > >>>> Andy LoPresto
> >>> > >>>> [hidden email]
> >>> > >>>> *[hidden email] <[hidden email]>*
> >>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
> EF69
> >>> > >>>>
> >>> > >>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]>
> wrote:
> >>> > >>>>
> >>> > >>>> Team
> >>> > >>>>
> >>> > >>>> There is a thread on apache legal-discuss that might allow for a
> >>> > >>>> graceperiod of continued usage of the json library.  Am going to
> >>> keep
> >>> > >>>> a close eye on this and if VP Legal approves we'll be able to
> keep
> >>> the
> >>> > >>>> twitter processors in which is definitely a good thing.  Will
> advise
> >>> > >>>>
> >>> > >>>> Thanks
> >>> > >>>> Joe
> >>> > >>>>
> >>> > >>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]
> >
> >>> > wrote:
> >>> > >>>>
> >>> > >>>> I've noticed an issue with the per-instance class loading
> capability
> >>> > >>>> introduced in NIFI-2909 where the additional classpath
> resources can
> >>> > get
> >>> > >>>> incorrectly removed from the class loader.
> >>> > >>>>
> >>> > >>>> I was able to reproduced this with a unit test and have a fix
> >>> ready. I
> >>> > >>>> believe this is important and needs to go in for the 1.1
> release,
> >>> > going to
> >>> > >>>> re-open NIFI-2909 and submit a PR shortly.
> >>> > >>>>
> >>> > >>>> -Bryan
> >>> > >>>>
> >>> > >>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <
> >>> [hidden email]
> >>> > >
> >>> > >>>> wrote:
> >>> > >>>>
> >>> > >>>> I have two items that I would like to wrap up prior to creating
> an
> >>> RC
> >>> > for
> >>> > >>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process
> >>> Group
> >>> > port
> >>> > >>>> configuration. The work is already completed and I will be
> reviewing
> >>> > it
> >>> > >>>> this today. Additionally, following recent interest on the
> mailing
> >>> > list,
> >>> > >>>> I'd like to knock out NIFI-3020. This will allow an admin to
> >>> > configure a
> >>> > >>>> strategy for user identity when logging in via LDAP.
> Specifically,
> >>> it
> >>> > will
> >>> > >>>> support usage of the DN (the default and current
> implementation) as
> >>> > well as
> >>> > >>>> the username the user logged in as. I should be able to have a
> PR up
> >>> > for
> >>> > >>>> this work later today.
> >>> > >>>>
> >>> > >>>> Thanks!
> >>> > >>>>
> >>> > >>>> Matt
> >>> > >>>>
> >>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
> >>> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]>
> >>> wrote:
> >>> > >>>>
> >>> > >>>> The code is within the twitter4j library itself.  I filed a
> request
> >>> to
> >>> > >>>> twitter4jg.  The most likely case is we will need to submit a
> PR to
> >>> > them.
> >>> > >>>> However, I don't see this as something that should delay the
> >>> > release.  We
> >>> > >>>> can provide instructions for folks wanting to use the processor
> >>> during
> >>> > >>>>
> >>> > >>>> the
> >>> > >>>>
> >>> > >>>> time we cannot make it available in a convenient manner.  I will
> >>> > provide
> >>> > >>>>
> >>> > >>>> a
> >>> > >>>>
> >>> > >>>> meaningful comment about this in release notes and pointers on
> what
> >>> > folks
> >>> > >>>> can do in the meantime.
> >>> > >>>>
> >>> > >>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]>
> >>> > wrote:
> >>> > >>>>
> >>> > >>>> I understand there was a discussion thread within the NiFi
> community
> >>> > >>>>
> >>> > >>>> for
> >>> > >>>>
> >>> > >>>> this as well and I missed responding to that at that time. It
> just
> >>> > >>>>
> >>> > >>>> seems
> >>> > >>>>
> >>> > >>>> to
> >>> > >>>>
> >>> > >>>> me like JSON processing is necessary for GetTwitter, which is
> >>> > >>>>
> >>> > >>>> incredibly
> >>> > >>>>
> >>> > >>>> useful for demonstrating NiFi’s ability to read from a high
> volume
> >>> > >>>>
> >>> > >>>> stream
> >>> > >>>>
> >>> > >>>> out of the box. With NIFI-3019 (Remove GetTwitter from default
> >>> build),
> >>> > >>>>
> >>> > >>>> is
> >>> > >>>>
> >>> > >>>> there any related effort to substitute an acceptable replacement
> >>> JSON
> >>> > >>>> library to restore this functionality?
> >>> > >>>>
> >>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
> >>> > >>>>
> >>> > >>>> Andy LoPresto
> >>> > >>>> [hidden email]
> >>> > >>>> *[hidden email] <[hidden email]>*
> >>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
> EF69
> >>> > >>>>
> >>> > >>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <
> [hidden email]>
> >>> > >>>>
> >>> > >>>> wrote:
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> I’m working with Bryan Rosander to close out NIFI-3024,
> NIFI-2655,
> >>> and
> >>> > >>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
> >>> > >>>> investigated some alternate TLS config options for the new
> version
> >>> of
> >>> > >>>>
> >>> > >>>> the
> >>> > >>>>
> >>> > >>>> client library.
> >>> > >>>>
> >>> > >>>> Is there any alternative to excluding the GetTwitter processor?
> >>> Using
> >>> > >>>> Johnzon [1] or the Android re-implementation [2] discussed in
> the
> >>> > >>>>
> >>> > >>>> mailing
> >>> > >>>>
> >>> > >>>> list thread?
> >>> > >>>>
> >>> > >>>> [1] https://johnzon.apache.org/
> >>> > >>>> [2] https://developer.android.com/reference/org/json/package-
> >>> > >>>>
> >>> > >>>> summary.html
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> Andy LoPresto
> >>> > >>>> [hidden email]
> >>> > >>>> *[hidden email] <[hidden email]>*
> >>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
> EF69
> >>> > >>>>
> >>> > >>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]>
> wrote:
> >>> > >>>>
> >>> > >>>> Team
> >>> > >>>>
> >>> > >>>> Very happy to see that we are down to three items remaining
> tagged
> >>> to
> >>> > >>>> 1.1.0.  Solid effort over the recent weeks to close the gap
> >>> including
> >>> > >>>>
> >>> > >>>> work
> >>> > >>>>
> >>> > >>>> to get past the now category x Jason dependency we had.  The
> most
> >>> > >>>>
> >>> > >>>> notable
> >>> > >>>>
> >>> > >>>> impact from that is the wildly popular GetTwitter processor,
> the fav
> >>> > >>>>
> >>> > >>>> new
> >>> > >>>>
> >>> > >>>> nifi user and demo processor, can no longer be included in the
> >>> default
> >>> > >>>> build.  It is optionally available if users choose to build and
> use
> >>> it
> >>> > >>>>
> >>> > >>>> but
> >>> > >>>>
> >>> > >>>> we won't distribute binaries that have it.
> >>> > >>>>
> >>> > >>>> I see some review movement on some patch available but untagged
> >>> items.
> >>> > >>>>
> >>> > >>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri.
> >>> > Anyone
> >>> > >>>> have any outstanding items?
> >>> > >>>>
> >>> > >>>> Thanks
> >>> > >>>> Joe
> >>> > >>>>
> >>> > >>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
> >>> > >>>>
> >>> > >>>> Ryan
> >>> > >>>>
> >>> > >>>> Not officially but I think we should try to close this thing
> out and
> >>> > >>>> start a vote in the next week or two at most.
> >>> > >>>>
> >>> > >>>> I'm going through the tickets again now.  There is also a new
> issue
> >>> of
> >>> > >>>> the json-p license falling out of favor in Apache legal terms
> and
> >>> > >>>> becoming Category-X.  Am looking into that now.
> >>> > >>>>
> >>> > >>>> Thanks
> >>> > >>>> Joe
> >>> > >>>>
> >>> > >>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]
> >
> >>> > >>>>
> >>> > >>>> wrote:
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> Joe - Is there a target date for 1.1?
> >>> > >>>>
> >>> > >>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]>
> >>> > wrote:
> >>> > >>>>
> >>> > >>>> Team,
> >>> > >>>>
> >>> > >>>> Just an update on things with working toward an Apache NiFi
> 1.1.0
> >>> > >>>> release.  There are still about 33 JIRAs there now and some are
> >>> > >>>> awaiting review and are some are under active progress. Yet
> there is
> >>> > >>>> good traction and progress. I think we should just stay vigilant
> >>> with
> >>> > >>>> what makes it in and keep working it down.  So let's please
> shoot
> >>> for
> >>> > >>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
> >>> > >>>>
> >>> > >>>> Thanks
> >>> > >>>> Joe
> >>> > >>>>
> >>> > >>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]>
> >>> wrote:
> >>> > >>>>
> >>> > >>>> Team,
> >>> > >>>>
> >>> > >>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.
> >>> Let's
> >>> > >>>> avoiding putting more in there for now at least without a
> >>> discussion.
> >>> > >>>> Of the 31 JIRAs there the vast majority need review so we
> should be
> >>> > >>>> able to close these down fairly quickly as long as we don't let
> the
> >>> > >>>> list grow.
> >>> > >>>>
> >>> > >>>> Thanks
> >>> > >>>> joe
> >>> > >>>>
> >>> > >>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <
> >>> [hidden email]
> >>> > >
> >>> > >>>>
> >>> > >>>> wrote:
> >>> > >>>>
> >>> > >>>> Joe,
> >>> > >>>>
> >>> > >>>> Appreciate the offer it isn't my PR. I was just using it as an
> >>> > >>>>
> >>> > >>>> example.
> >>> > >>>>
> >>> > >>>> All
> >>> > >>>>
> >>> > >>>> mine are currently closed, which I greatly appreciate.
> >>> > >>>>
> >>> > >>>> Cheers,
> >>> > >>>>
> >>> > >>>> Edgardo
> >>> > >>>>
> >>> > >>>> On Friday, October 14, 2016, Joe Witt <[hidden email]>
> wrote:
> >>> > >>>>
> >>> > >>>> Edgardo,
> >>> > >>>>
> >>> > >>>> You mentioned a PR from August. I'd be happy to help you work
> that
> >>> > >>>> through review.
> >>> > >>>>
> >>> > >>>> Thanks
> >>> > >>>> Joe
> >>> > >>>>
> >>> > >>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
> >>> > >>>>
> >>> > >>>> [hidden email]
> >>> > >>>>
> >>> > >>>> <javascript:;>> wrote:
> >>> > >>>>
> >>> > >>>> I have agreed that at this point a release is important. My goal
> >>> > >>>>
> >>> > >>>> was
> >>> > >>>>
> >>> > >>>> try
> >>> > >>>>
> >>> > >>>> to
> >>> > >>>>
> >>> > >>>> squeeze in a much goodness as possible into the release, but the
> >>> > >>>>
> >>> > >>>> important
> >>> > >>>>
> >>> > >>>> bug fixes should come first. Getting 1.x into a state where the
> >>> > >>>>
> >>> > >>>> release
> >>> > >>>>
> >>> > >>>> notes don't say that it is geared toward developers and testers
> is
> >>> > >>>>
> >>> > >>>> really
> >>> > >>>>
> >>> > >>>> huge.
> >>> > >>>>
> >>> > >>>> I think Nifi is a great community otherwise I would participate
> in
> >>> > >>>>
> >>> > >>>> the
> >>> > >>>>
> >>> > >>>> mailing list, create Jira tickets and pull requests. I am only
> >>> > >>>>
> >>> > >>>> trying to
> >>> > >>>>
> >>> > >>>> strengthen the great thing that is going on here. We can always
> do
> >>> > >>>>
> >>> > >>>> better.
> >>> > >>>>
> >>> > >>>> I was not trying to put down this community only to participate
> and
> >>> > >>>>
> >>> > >>>> make
> >>> > >>>>
> >>> > >>>> it
> >>> > >>>>
> >>> > >>>> better. I think this conversation is an indication of how great
> >>> > >>>>
> >>> > >>>> this
> >>> > >>>>
> >>> > >>>> community is.
> >>> > >>>>
> >>> > >>>> Maybe I am being sensitive about this issue and trying to
> >>> > >>>>
> >>> > >>>> strengthen
> >>> > >>>>
> >>> > >>>> the
> >>> > >>>>
> >>> > >>>> nifi community even more, after coming from a conference where
> it
> >>> > >>>>
> >>> > >>>> was
> >>> > >>>>
> >>> > >>>> reported there was lots of excitement at first and now the
> >>> > >>>>
> >>> > >>>> participation
> >>> > >>>>
> >>> > >>>> in
> >>> > >>>>
> >>> > >>>> the community has really died down and they are struggling. I
> don't
> >>> > >>>>
> >>> > >>>> want
> >>> > >>>>
> >>> > >>>> to
> >>> > >>>>
> >>> > >>>> see that happen here.
> >>> > >>>>
> >>> > >>>> Cheers,
> >>> > >>>>
> >>> > >>>> Edgardo
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
> >>> > >>>>
> >>> > >>>> <javascript:;>> wrote:
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> Edgardo,
> >>> > >>>>
> >>> > >>>> Thank you for your feedback. We hear your comments and as a
> >>> > >>>>
> >>> > >>>> committer I
> >>> > >>>>
> >>> > >>>> can
> >>> > >>>>
> >>> > >>>> share we are constantly looking to improve the PR process,
> having
> >>> > >>>>
> >>> > >>>> already
> >>> > >>>>
> >>> > >>>> taken many of the steps you suggest.
> >>> > >>>>
> >>> > >>>> However, it is important to notice that the number of PRs should
> >>> > >>>>
> >>> > >>>> not be
> >>> > >>>>
> >>> > >>>> seen as a metric of engagement by the development community:
> Most
> >>> > >>>>
> >>> > >>>> of us
> >>> > >>>>
> >>> > >>>> will submit PRs so that our work can be carefully reviewed by
> our
> >>> > >>>>
> >>> > >>>> peers
> >>> > >>>>
> >>> > >>>> and
> >>> > >>>>
> >>> > >>>> some of us will use JIRA patches to provide contributions.
> >>> > >>>>
> >>> > >>>> Having said that, it is true that some PRs may sit idle for a
> long
> >>> > >>>>
> >>> > >>>> time
> >>> > >>>>
> >>> > >>>> and
> >>> > >>>>
> >>> > >>>> we are working to improve this pipeline.
> >>> > >>>>
> >>> > >>>> It was therefore no coincidence that I  browsed most of the PRs
> >>> > >>>>
> >>> > >>>> performing
> >>> > >>>>
> >>> > >>>> a triage of items that have been superseded or diverged from the
> >>> > >>>>
> >>> > >>>> current
> >>> > >>>>
> >>> > >>>> code base.
> >>> > >>>>
> >>> > >>>> In fact, less than a month ago the dev team closed a number of
> >>> > >>>>
> >>> > >>>> stalled
> >>> > >>>>
> >>> > >>>> and
> >>> > >>>>
> >>> > >>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63
> c15269eea8).
> >>> > >>>>
> >>> > >>>> Despite all the above, I think Joe has a point. The master
> >>> > >>>>
> >>> > >>>> contain a
> >>> > >>>>
> >>> > >>>> series
> >>> > >>>>
> >>> > >>>> of important bug fixes and suspect the community would benefit
> >>> > >>>>
> >>> > >>>> from
> >>> > >>>>
> >>> > >>>> a
> >>> > >>>>
> >>> > >>>> release sooner rather than later.
> >>> > >>>>
> >>> > >>>> Once again, thank you for your feedback and contribution. It is
> >>> > >>>>
> >>> > >>>> good to
> >>> > >>>>
> >>> > >>>> have you here.
> >>> > >>>>
> >>> > >>>> Andre
> >>> > >>>>
> >>> > >>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
> >>> > >>>>
> >>> > >>>> [hidden email]
> >>> > >>>>
> >>> > >>>> <javascript:;>>
> >>> > >>>>
> >>> > >>>> wrote:
> >>> > >>>>
> >>> > >>>> Joe - You are correct I was mentioning the PRs that are
> >>> > >>>>
> >>> > >>>> currently
> >>> > >>>>
> >>> > >>>> open.
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> Regardless of how it happens reducing the count of open PRs I
> >>> > >>>>
> >>> > >>>> believe
> >>> > >>>>
> >>> > >>>> to
> >>> > >>>>
> >>> > >>>> be
> >>> > >>>>
> >>> > >>>> extremely important. Maybe I was hoping that the release could
> >>> > >>>>
> >>> > >>>> be
> >>> > >>>>
> >>> > >>>> a
> >>> > >>>>
> >>> > >>>> forcing
> >>> > >>>>
> >>> > >>>> function to make that happen. I believe that developers are more
> >>> > >>>>
> >>> > >>>> willing
> >>> > >>>>
> >>> > >>>> to
> >>> > >>>>
> >>> > >>>> contribute when they see that their PRs will actually be able
> >>> > >>>>
> >>> > >>>> accepted
> >>> > >>>>
> >>> > >>>> and
> >>> > >>>>
> >>> > >>>> merged into the code base. Having a low number of open PRs in
> >>> > >>>>
> >>> > >>>> progress
> >>> > >>>>
> >>> > >>>> is a
> >>> > >>>>
> >>> > >>>> great indication that the main nifi developers are fully engaged
> >>> > >>>>
> >>> > >>>> with
> >>> > >>>>
> >>> > >>>> the
> >>> > >>>>
> >>> > >>>> community.
> >>> > >>>>
> >>> > >>>> There are a few PRs that don't have any comments from committers
> >>> > >>>>
> >>> > >>>> at
> >>> > >>>>
> >>> > >>>> all.
> >>> > >>>>
> >>> > >>>> I
> >>> > >>>>
> >>> > >>>> found one from August in that state. If that was my PR I don't
> >>> > >>>>
> >>> > >>>> think I
> >>> > >>>>
> >>> > >>>> would be so willing to put another one in anytime soon. I do get
> >>> > >>>>
> >>> > >>>> that
> >>> > >>>>
> >>> > >>>> sometime PRs get stalled by the originator, if so maybe a rule
> >>> > >>>>
> >>> > >>>> about
> >>> > >>>>
> >>> > >>>> closing them after a certain amount of time or being taken over
> >>> > >>>>
> >>> > >>>> by a
> >>> > >>>>
> >>> > >>>> core
> >>> > >>>>
> >>> > >>>> contributor if they think it worthwhile.
> >>> > >>>>
> >>> > >>>> I would like to shoutout to James Wing on my last PR he was
> >>> > >>>>
> >>> > >>>> quick
> >>> > >>>>
> >>> > >>>> to
> >>> > >>>>
> >>> > >>>> review, provided great comments, testing, and even some
> >>> > >>>>
> >>> > >>>> additional
> >>> > >>>>
> >>> > >>>> code.
> >>> > >>>>
> >>> > >>>> It
> >>> > >>>>
> >>> > >>>> was a great PR experience.
> >>> > >>>>
> >>> > >>>> Cheers,
> >>> > >>>>
> >>> > >>>> Edgardo
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
> >>> > >>>>
> >>> > >>>> [hidden email] <javascript:;>.
> >>> > >>>>
> >>> > >>>> invalid> wrote:
> >>> > >>>>
> >>> > >>>> Joe, I think you misread. Edgardo is referring to the Pull
> >>> > >>>>
> >>> > >>>> Requests
> >>> > >>>>
> >>> > >>>> that
> >>> > >>>>
> >>> > >>>> are currently open, not the tickets assigned to the 1.1.0
> >>> > >>>>
> >>> > >>>> version.
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> I think these goals (releasing 1.1.0 and cutting down the PR
> >>> > >>>>
> >>> > >>>> count)
> >>> > >>>>
> >>> > >>>> should
> >>> > >>>>
> >>> > >>>> be two different efforts. Doing a thorough job reviewing
> >>> > >>>>
> >>> > >>>> takes a
> >>> > >>>>
> >>> > >>>> significant amount of time from both the reviewer and
> >>> > >>>>
> >>> > >>>> contributor.
> >>> > >>>>
> >>> > >>>> In
> >>> > >>>>
> >>> > >>>> order
> >>> > >>>>
> >>> > >>>> to cut it down significantly would take much longer than a
> >>> > >>>>
> >>> > >>>> couple
> >>> > >>>>
> >>> > >>>> days.
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> Also there has already been a lot of great new features and
> >>> > >>>>
> >>> > >>>> bug
> >>> > >>>>
> >>> > >>>> fixes
> >>> > >>>>
> >>> > >>>> contributed to the 1.X line and I don't think it's worth
> >>> > >>>>
> >>> > >>>> holding up
> >>> > >>>>
> >>> > >>>> a
> >>> > >>>>
> >>> > >>>> 1.1.0
> >>> > >>>>
> >>> > >>>> release for tickets not assigned to this fix version. As an
> >>> > >>>>
> >>> > >>>> added
> >>> > >>>>
> >>> > >>>> bonus
> >>> > >>>>
> >>> > >>>> though, I think many of the tickets tagged as 1.1.0 have PRs
> >>> > >>>>
> >>> > >>>> already
> >>> > >>>>
> >>> > >>>> open
> >>> > >>>>
> >>> > >>>> so closing those will make a large dent in the PR count.
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> Joe
> >>> > >>>>
> >>> > >>>> - - - - - -
> >>> > >>>> Joseph Percivall
> >>> > >>>> linkedin.com/in/Percivall
> >>> > >>>> e: [hidden email] <javascript:;>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
> >>> > >>>>
> >>> > >>>> [hidden email]
> >>> > >>>>
> >>> > >>>> <javascript:;>>
> >>> > >>>>
> >>> > >>>> wrote:
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> There are less than 30 right now.  Many of the roughly 90+
> >>> > >>>>
> >>> > >>>> JIRAs
> >>> > >>>>
> >>> > >>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
> >>> > >>>>
> >>> > >>>> or
> >>> > >>>>
> >>> > >>>> just
> >>> > >>>>
> >>> > >>>> had fix versions removed.
> >>> > >>>>
> >>> > >>>> We will need to have a push over the next bunch of days to
> >>> > >>>>
> >>> > >>>> deal
> >>> > >>>>
> >>> > >>>> with
> >>> > >>>>
> >>> > >>>> reviewing/merging/moving the remaining items.
> >>> > >>>>
> >>> > >>>> Thanks
> >>> > >>>> Joe
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
> >>> > >>>>
> >>> > >>>> [hidden email] <javascript:;>>
> >>> > >>>>
> >>> > >>>> wrote:
> >>> > >>>>
> >>> > >>>> Joe,
> >>> > >>>>
> >>> > >>>> There are 75 PRs currently open. Why not make a push over
> >>> > >>>>
> >>> > >>>> the
> >>> > >>>>
> >>> > >>>> next
> >>> > >>>>
> >>> > >>>> bunch
> >>> > >>>>
> >>> > >>>> of
> >>> > >>>>
> >>> > >>>> days to get them closed and then cut the release after that.
> >>> > >>>>
> >>> > >>>> Cheers,
> >>> > >>>>
> >>> > >>>> Edgardo
> >>> > >>>>
> >>> > >>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
> >>> > >>>>
> >>> > >>>> [hidden email]
> >>> > >>>>
> >>> > >>>> <javascript:;>>
> >>> > >>>>
> >>> > >>>> wrote:
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> Team,
> >>> > >>>>
> >>> > >>>> There have been a ton of bugs fixed a few nice features.  I
> >>> > >>>>
> >>> > >>>> would
> >>> > >>>>
> >>> > >>>> like
> >>> > >>>>
> >>> > >>>> to move to get Apache NiFi 1.1.0 release going pretty much
> >>> > >>>>
> >>> > >>>> based
> >>> > >>>>
> >>> > >>>> on
> >>> > >>>>
> >>> > >>>> where we are now and plan to move most tickets to a new
> >>> > >>>>
> >>> > >>>> Apache
> >>> > >>>>
> >>> > >>>> NiFi
> >>> > >>>>
> >>> > >>>> 1.2.0 version.  We can try to get back on our roughly 6-8
> >>> > >>>>
> >>> > >>>> week
> >>> > >>>>
> >>> > >>>> release
> >>> > >>>>
> >>> > >>>> schedule and shoot for a mid to late Nov release for NiFi
> >>> > >>>>
> >>> > >>>> 1.2.0
> >>> > >>>>
> >>> > >>>> this
> >>> > >>>>
> >>> > >>>> way as well. Please advise if anyone has any other views on
> >>> > >>>>
> >>> > >>>> this. In
> >>> > >>>>
> >>> > >>>> the mean time I'll get the wheels in motion so you'll be
> >>> > >>>>
> >>> > >>>> seeing a
> >>> > >>>>
> >>> > >>>> lot
> >>> > >>>>
> >>> > >>>> of JIRA/issue updates to move version around.
> >>> > >>>>
> >>> > >>>> Thanks
> >>> > >>>> Joe
> >>> > >>>>
> >>> > >>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
> >>> > >>>>
> >>> > >>>> [hidden email]
> >>> > >>>>
> >>> > >>>> <javascript:;>>
> >>> > >>>>
> >>> > >>>> wrote:
> >>> > >>>>
> >>> > >>>> Sounds good Joe. I have no issue to you doing the rm'ing
> >>> > >>>>
> >>> > >>>> for
> >>> > >>>>
> >>> > >>>> it.
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
> >>> > >>>>
> >>> > >>>> <javascript:;>> wrote:
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> Team,
> >>> > >>>>
> >>> > >>>> There are a lot of great fixes and improvements on the
> >>> > >>>>
> >>> > >>>> master
> >>> > >>>>
> >>> > >>>> line
> >>> > >>>>
> >>> > >>>> now
> >>> > >>>>
> >>> > >>>> and we're at a good time window to start pushing for a
> >>> > >>>>
> >>> > >>>> release.
> >>> > >>>>
> >>> > >>>> There
> >>> > >>>>
> >>> > >>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
> >>> > >>>>
> >>> > >>>> are
> >>> > >>>>
> >>> > >>>> open.
> >>> > >>>>
> >>> > >>>> I'm
> >>> > >>>>
> >>> > >>>> going to go through them and remove fix versions where
> >>> > >>>>
> >>> > >>>> appropriate.
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> I'm happy to take on RM task for this release though if
> >>> > >>>>
> >>> > >>>> someone
> >>> > >>>>
> >>> > >>>> else
> >>> > >>>>
> >>> > >>>> would like to take that on please advise.
> >>> > >>>>
> >>> > >>>> Thanks
> >>> > >>>> Joe
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> --
> >>> > >>>> Cheers,
> >>> > >>>>
> >>> > >>>> Edgardo
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> --
> >>> > >>>> Cheers,
> >>> > >>>>
> >>> > >>>> Edgardo
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> --
> >>> > >>>> Cheers,
> >>> > >>>>
> >>> > >>>> Edgardo
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>> --
> >>> > >>>> Cheers,
> >>> > >>>>
> >>> > >>>> Edgardo
> >>> > >>>>
> >>> > >>>> Sent from Gmail Mobile
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> > >>>>
> >>> >
> >>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Joe Witt
Some good news - we can put our twitter processor back in the game.

Legal VP at Apache just sent out the decision.

Will immediately restore that to action and put in the follow-on
ticket to ensure the twitter4j library moves away from the old Json
lib (there is a PR to replace it).  We can only keep it this way until
April so unless twitter4j resolves their source dependency we'll be
back in this position.

thanks
joe

On Tue, Nov 22, 2016 at 1:38 PM, Michael Moser <[hidden email]> wrote:

> Awesome.  Your prompt and detailed feedback is very much appreciated!
>
>
> On Tue, Nov 22, 2016 at 12:47 PM, Joe Witt <[hidden email]> wrote:
>
>> mike - also to clarify based on what you brought up I updated the JIRA
>> description as follows
>>
>> The following also would be true:
>> 1) Apache NiFi 1.0.0 repositories should work just fine when applied
>> to an Apache NiFi 1.1.0 installation.
>>
>> 2) Repositories made/updated in Apache NiFi 1.1.0 onward would not
>> work in older Apache NiFi releases (such as 1.0.0)
>>
>> On Tue, Nov 22, 2016 at 12:33 PM, Joe Witt <[hidden email]> wrote:
>> > Mike - yeah good catch and good question.  It does support the old
>> > format.  We've always been pretty good about being diligent to allow
>> > folks to upgrade and it honor existing state and in many cases
>> > configurations and it would automatically port them over.  What has
>> > always been far more problematic is 'rollback' where people ran on
>> > newer configurations but could not then go back to old framework code.
>> > That is what NIFI-2854 tackles at least as far as the
>> > content/prov/flowfile repositories go.  Now, the code and
>> > serialization is done in such a way that older version can simply
>> > ignore what never versions encoded if they don't understand it but
>> > they should be able to continue on.
>> >
>> > I just tried out a 1.0.0 flow with data queued up.  Upgraded to a
>> > latest NiFi 1.1.0-SNAPSHOT.  Moved the repos over.  And it came up
>> > perfectly with all the queue data ready to roll.
>> >
>> > Thanks
>> > Joe
>> >
>> > On Tue, Nov 22, 2016 at 12:16 PM, Michael Moser <[hidden email]>
>> wrote:
>> >> After seeing "Repositories now support rollback" in the release notes
>> and
>> >> reading NIFI-2854 [1], I have a question.
>> >>
>> >> Are repositories created using NiFi 1.0.0 compatible with NiFi 1.1.0
>> >> software?  This is the goal that the ticket seems to indicate with 1.1.0
>> >> onward, but it's not clear whether 1.0.0 -> 1.1.0 is included.
>> >>
>> >> Thanks,
>> >> -- Mike
>> >>
>> >> [1] - https://issues.apache.org/jira/browse/NIFI-2854
>> >>
>> >> On Tue, Nov 22, 2016 at 12:08 PM, James Wing <[hidden email]> wrote:
>> >>
>> >>> Going through the tickets, it seems like quite a release.  A few more
>> >>> things for your list:
>> >>>
>> >>> GenerateFlowFile updated to support literal/expression content and
>> >>> attributes
>> >>>
>> >>> AWS-related:
>> >>> * New processors PutCloudWatchMetric, PutKinesisStream
>> >>> * Updated processors PutS3Object (content type, signer options), ListS3
>> >>> (performance, versions)
>> >>> * Added support for AWS assume role credentials with proxy
>> >>>
>> >>>
>> >>> Thanks,
>> >>>
>> >>> James
>> >>>
>> >>> On Mon, Nov 21, 2016 at 9:30 PM, Joe Witt <[hidden email]> wrote:
>> >>>
>> >>> > matt
>> >>> >
>> >>> > i'll add a wiki page or set of instructions linked from the release
>> >>> notes.
>> >>> >
>> >>> > all,
>> >>> >
>> >>> > walked through the 250 or so JIRAs in the 1.1.0 release and pulled
>> out
>> >>> > highlights.  The items noted are as follows.  Will likely reduce this
>> >>> > down further for the release notes but wanted to put this out in case
>> >>> > folks have things they think are really important to highlight.
>> >>> >
>> >>> > - Core Improvements:
>> >>> >    - Performance: Session Migration
>> >>> >    - Stability: Cluster Management
>> >>> >    - Developer: Framework supports easy user driven classloader
>> extension
>> >>> >    - Expression Language: Now supports base64 and hex encoded values
>> >>> > and Math functions
>> >>> >    - Repositories now support rollback
>> >>> >    - Faster startup due to more efficient state restoration algorithm
>> >>> > - UX Improvements:
>> >>> >    - Visual Backpressure Indicator
>> >>> >    - Introduced more colors to better highlight actions and
>> components
>> >>> >    - Performance: Validate non-running components
>> >>> >    - Provenance graph image can be exported
>> >>> >    - Cron Scheduling for Primary node tasks now supported
>> >>> > - Updated versions
>> >>> >    - Azure Event Hub 0.9.0
>> >>> >    - Spark 2.0.1
>> >>> >    - Hadoop 2.7.x
>> >>> > - New/Improved Processors
>> >>> >    - new Fetch/Put Elastic Search 5.0
>> >>> >    - new ParseCEF to parse CEF formatted logs
>> >>> >    - improve ExtractEmail now supports TNEF files
>> >>> >    - new Validate CSV
>> >>> >    - improved Solr processors now support SSL and Kerberos
>> >>> >    - new Websocket client and server processors
>> >>> > - New Utility
>> >>> >    - Zookeeper Migrator (move from one zookeeper to another)
>> >>> > - Security
>> >>> >    - Restricted Processors
>> >>> >    - Site-to-site now supports port forwarding
>> >>> >    - Improved Policy Management UX
>> >>> > - Migration Notes:
>> >>> >    - Restricted Processors
>> >>> >    - Twitter Processor Removed
>> >>> >
>> >>> > On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]
>> >
>> >>> > wrote:
>> >>> > > Is there a good spot for us to put instructions on how to build the
>> >>> > > Twitter processor and/or the Social Media NAR in the meantime?
>> Maybe a
>> >>> > > Wiki page or something simple to say "go to this directory, run
>> this
>> >>> > > Maven command, drop the NAR into your deployment..." ?
>> >>> > >
>> >>> > > On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]>
>> wrote:
>> >>> > >> Team,
>> >>> > >>
>> >>> > >> We appear to be very close.  Andy is working NIFI-3024 but
>> otherwise
>> >>> > >> it is focus on testing.
>> >>> > >>
>> >>> > >> I'm going to prep the RC and release notes now.  Unfortunately the
>> >>> > >> twitter changes for json.org will need to remain.  Consensus
>> forming
>> >>> > >> on the legal-discuss thread regarding a grace period has been
>> elusive
>> >>> > >> and we're already prepared to make the right steps so we'll just
>> need
>> >>> > >> to take that on by being empathetic to the user base.
>> >>> > >>
>> >>> > >> Thanks
>> >>> > >> Joe
>> >>> > >>
>> >>> > >> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]>
>> wrote:
>> >>> > >>> Andy,
>> >>> > >>>
>> >>> > >>> Great to see NIFI-3050 implemented and certainly good news that
>> NiFi
>> >>> > 1.1.0
>> >>> > >>> is set to include a number of security related improvements.
>> >>> > >>>
>> >>> > >>>
>> >>> > >>>
>> >>> > >>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <
>> [hidden email]
>> >>> >
>> >>> > wrote:
>> >>> > >>>
>> >>> > >>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2]
>> have
>> >>> > been
>> >>> > >>>> added to my plate for this release. Coordinated with Joe Witt
>> and
>> >>> they
>> >>> > >>>> should both be included.
>> >>> > >>>>
>> >>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
>> >>> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
>> >>> > >>>>
>> >>> > >>>> Andy LoPresto
>> >>> > >>>> [hidden email]
>> >>> > >>>> *[hidden email] <[hidden email]>*
>> >>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
>> EF69
>> >>> > >>>>
>> >>> > >>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]>
>> wrote:
>> >>> > >>>>
>> >>> > >>>> Team
>> >>> > >>>>
>> >>> > >>>> There is a thread on apache legal-discuss that might allow for a
>> >>> > >>>> graceperiod of continued usage of the json library.  Am going to
>> >>> keep
>> >>> > >>>> a close eye on this and if VP Legal approves we'll be able to
>> keep
>> >>> the
>> >>> > >>>> twitter processors in which is definitely a good thing.  Will
>> advise
>> >>> > >>>>
>> >>> > >>>> Thanks
>> >>> > >>>> Joe
>> >>> > >>>>
>> >>> > >>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]
>> >
>> >>> > wrote:
>> >>> > >>>>
>> >>> > >>>> I've noticed an issue with the per-instance class loading
>> capability
>> >>> > >>>> introduced in NIFI-2909 where the additional classpath
>> resources can
>> >>> > get
>> >>> > >>>> incorrectly removed from the class loader.
>> >>> > >>>>
>> >>> > >>>> I was able to reproduced this with a unit test and have a fix
>> >>> ready. I
>> >>> > >>>> believe this is important and needs to go in for the 1.1
>> release,
>> >>> > going to
>> >>> > >>>> re-open NIFI-2909 and submit a PR shortly.
>> >>> > >>>>
>> >>> > >>>> -Bryan
>> >>> > >>>>
>> >>> > >>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <
>> >>> [hidden email]
>> >>> > >
>> >>> > >>>> wrote:
>> >>> > >>>>
>> >>> > >>>> I have two items that I would like to wrap up prior to creating
>> an
>> >>> RC
>> >>> > for
>> >>> > >>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process
>> >>> Group
>> >>> > port
>> >>> > >>>> configuration. The work is already completed and I will be
>> reviewing
>> >>> > it
>> >>> > >>>> this today. Additionally, following recent interest on the
>> mailing
>> >>> > list,
>> >>> > >>>> I'd like to knock out NIFI-3020. This will allow an admin to
>> >>> > configure a
>> >>> > >>>> strategy for user identity when logging in via LDAP.
>> Specifically,
>> >>> it
>> >>> > will
>> >>> > >>>> support usage of the DN (the default and current
>> implementation) as
>> >>> > well as
>> >>> > >>>> the username the user logged in as. I should be able to have a
>> PR up
>> >>> > for
>> >>> > >>>> this work later today.
>> >>> > >>>>
>> >>> > >>>> Thanks!
>> >>> > >>>>
>> >>> > >>>> Matt
>> >>> > >>>>
>> >>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>> >>> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]>
>> >>> wrote:
>> >>> > >>>>
>> >>> > >>>> The code is within the twitter4j library itself.  I filed a
>> request
>> >>> to
>> >>> > >>>> twitter4jg.  The most likely case is we will need to submit a
>> PR to
>> >>> > them.
>> >>> > >>>> However, I don't see this as something that should delay the
>> >>> > release.  We
>> >>> > >>>> can provide instructions for folks wanting to use the processor
>> >>> during
>> >>> > >>>>
>> >>> > >>>> the
>> >>> > >>>>
>> >>> > >>>> time we cannot make it available in a convenient manner.  I will
>> >>> > provide
>> >>> > >>>>
>> >>> > >>>> a
>> >>> > >>>>
>> >>> > >>>> meaningful comment about this in release notes and pointers on
>> what
>> >>> > folks
>> >>> > >>>> can do in the meantime.
>> >>> > >>>>
>> >>> > >>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]>
>> >>> > wrote:
>> >>> > >>>>
>> >>> > >>>> I understand there was a discussion thread within the NiFi
>> community
>> >>> > >>>>
>> >>> > >>>> for
>> >>> > >>>>
>> >>> > >>>> this as well and I missed responding to that at that time. It
>> just
>> >>> > >>>>
>> >>> > >>>> seems
>> >>> > >>>>
>> >>> > >>>> to
>> >>> > >>>>
>> >>> > >>>> me like JSON processing is necessary for GetTwitter, which is
>> >>> > >>>>
>> >>> > >>>> incredibly
>> >>> > >>>>
>> >>> > >>>> useful for demonstrating NiFi’s ability to read from a high
>> volume
>> >>> > >>>>
>> >>> > >>>> stream
>> >>> > >>>>
>> >>> > >>>> out of the box. With NIFI-3019 (Remove GetTwitter from default
>> >>> build),
>> >>> > >>>>
>> >>> > >>>> is
>> >>> > >>>>
>> >>> > >>>> there any related effort to substitute an acceptable replacement
>> >>> JSON
>> >>> > >>>> library to restore this functionality?
>> >>> > >>>>
>> >>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>> >>> > >>>>
>> >>> > >>>> Andy LoPresto
>> >>> > >>>> [hidden email]
>> >>> > >>>> *[hidden email] <[hidden email]>*
>> >>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
>> EF69
>> >>> > >>>>
>> >>> > >>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <
>> [hidden email]>
>> >>> > >>>>
>> >>> > >>>> wrote:
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> I’m working with Bryan Rosander to close out NIFI-3024,
>> NIFI-2655,
>> >>> and
>> >>> > >>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>> >>> > >>>> investigated some alternate TLS config options for the new
>> version
>> >>> of
>> >>> > >>>>
>> >>> > >>>> the
>> >>> > >>>>
>> >>> > >>>> client library.
>> >>> > >>>>
>> >>> > >>>> Is there any alternative to excluding the GetTwitter processor?
>> >>> Using
>> >>> > >>>> Johnzon [1] or the Android re-implementation [2] discussed in
>> the
>> >>> > >>>>
>> >>> > >>>> mailing
>> >>> > >>>>
>> >>> > >>>> list thread?
>> >>> > >>>>
>> >>> > >>>> [1] https://johnzon.apache.org/
>> >>> > >>>> [2] https://developer.android.com/reference/org/json/package-
>> >>> > >>>>
>> >>> > >>>> summary.html
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> Andy LoPresto
>> >>> > >>>> [hidden email]
>> >>> > >>>> *[hidden email] <[hidden email]>*
>> >>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
>> EF69
>> >>> > >>>>
>> >>> > >>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]>
>> wrote:
>> >>> > >>>>
>> >>> > >>>> Team
>> >>> > >>>>
>> >>> > >>>> Very happy to see that we are down to three items remaining
>> tagged
>> >>> to
>> >>> > >>>> 1.1.0.  Solid effort over the recent weeks to close the gap
>> >>> including
>> >>> > >>>>
>> >>> > >>>> work
>> >>> > >>>>
>> >>> > >>>> to get past the now category x Jason dependency we had.  The
>> most
>> >>> > >>>>
>> >>> > >>>> notable
>> >>> > >>>>
>> >>> > >>>> impact from that is the wildly popular GetTwitter processor,
>> the fav
>> >>> > >>>>
>> >>> > >>>> new
>> >>> > >>>>
>> >>> > >>>> nifi user and demo processor, can no longer be included in the
>> >>> default
>> >>> > >>>> build.  It is optionally available if users choose to build and
>> use
>> >>> it
>> >>> > >>>>
>> >>> > >>>> but
>> >>> > >>>>
>> >>> > >>>> we won't distribute binaries that have it.
>> >>> > >>>>
>> >>> > >>>> I see some review movement on some patch available but untagged
>> >>> items.
>> >>> > >>>>
>> >>> > >>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri.
>> >>> > Anyone
>> >>> > >>>> have any outstanding items?
>> >>> > >>>>
>> >>> > >>>> Thanks
>> >>> > >>>> Joe
>> >>> > >>>>
>> >>> > >>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>> >>> > >>>>
>> >>> > >>>> Ryan
>> >>> > >>>>
>> >>> > >>>> Not officially but I think we should try to close this thing
>> out and
>> >>> > >>>> start a vote in the next week or two at most.
>> >>> > >>>>
>> >>> > >>>> I'm going through the tickets again now.  There is also a new
>> issue
>> >>> of
>> >>> > >>>> the json-p license falling out of favor in Apache legal terms
>> and
>> >>> > >>>> becoming Category-X.  Am looking into that now.
>> >>> > >>>>
>> >>> > >>>> Thanks
>> >>> > >>>> Joe
>> >>> > >>>>
>> >>> > >>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]
>> >
>> >>> > >>>>
>> >>> > >>>> wrote:
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> Joe - Is there a target date for 1.1?
>> >>> > >>>>
>> >>> > >>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]>
>> >>> > wrote:
>> >>> > >>>>
>> >>> > >>>> Team,
>> >>> > >>>>
>> >>> > >>>> Just an update on things with working toward an Apache NiFi
>> 1.1.0
>> >>> > >>>> release.  There are still about 33 JIRAs there now and some are
>> >>> > >>>> awaiting review and are some are under active progress. Yet
>> there is
>> >>> > >>>> good traction and progress. I think we should just stay vigilant
>> >>> with
>> >>> > >>>> what makes it in and keep working it down.  So let's please
>> shoot
>> >>> for
>> >>> > >>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>> >>> > >>>>
>> >>> > >>>> Thanks
>> >>> > >>>> Joe
>> >>> > >>>>
>> >>> > >>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]>
>> >>> wrote:
>> >>> > >>>>
>> >>> > >>>> Team,
>> >>> > >>>>
>> >>> > >>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.
>> >>> Let's
>> >>> > >>>> avoiding putting more in there for now at least without a
>> >>> discussion.
>> >>> > >>>> Of the 31 JIRAs there the vast majority need review so we
>> should be
>> >>> > >>>> able to close these down fairly quickly as long as we don't let
>> the
>> >>> > >>>> list grow.
>> >>> > >>>>
>> >>> > >>>> Thanks
>> >>> > >>>> joe
>> >>> > >>>>
>> >>> > >>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <
>> >>> [hidden email]
>> >>> > >
>> >>> > >>>>
>> >>> > >>>> wrote:
>> >>> > >>>>
>> >>> > >>>> Joe,
>> >>> > >>>>
>> >>> > >>>> Appreciate the offer it isn't my PR. I was just using it as an
>> >>> > >>>>
>> >>> > >>>> example.
>> >>> > >>>>
>> >>> > >>>> All
>> >>> > >>>>
>> >>> > >>>> mine are currently closed, which I greatly appreciate.
>> >>> > >>>>
>> >>> > >>>> Cheers,
>> >>> > >>>>
>> >>> > >>>> Edgardo
>> >>> > >>>>
>> >>> > >>>> On Friday, October 14, 2016, Joe Witt <[hidden email]>
>> wrote:
>> >>> > >>>>
>> >>> > >>>> Edgardo,
>> >>> > >>>>
>> >>> > >>>> You mentioned a PR from August. I'd be happy to help you work
>> that
>> >>> > >>>> through review.
>> >>> > >>>>
>> >>> > >>>> Thanks
>> >>> > >>>> Joe
>> >>> > >>>>
>> >>> > >>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>> >>> > >>>>
>> >>> > >>>> [hidden email]
>> >>> > >>>>
>> >>> > >>>> <javascript:;>> wrote:
>> >>> > >>>>
>> >>> > >>>> I have agreed that at this point a release is important. My goal
>> >>> > >>>>
>> >>> > >>>> was
>> >>> > >>>>
>> >>> > >>>> try
>> >>> > >>>>
>> >>> > >>>> to
>> >>> > >>>>
>> >>> > >>>> squeeze in a much goodness as possible into the release, but the
>> >>> > >>>>
>> >>> > >>>> important
>> >>> > >>>>
>> >>> > >>>> bug fixes should come first. Getting 1.x into a state where the
>> >>> > >>>>
>> >>> > >>>> release
>> >>> > >>>>
>> >>> > >>>> notes don't say that it is geared toward developers and testers
>> is
>> >>> > >>>>
>> >>> > >>>> really
>> >>> > >>>>
>> >>> > >>>> huge.
>> >>> > >>>>
>> >>> > >>>> I think Nifi is a great community otherwise I would participate
>> in
>> >>> > >>>>
>> >>> > >>>> the
>> >>> > >>>>
>> >>> > >>>> mailing list, create Jira tickets and pull requests. I am only
>> >>> > >>>>
>> >>> > >>>> trying to
>> >>> > >>>>
>> >>> > >>>> strengthen the great thing that is going on here. We can always
>> do
>> >>> > >>>>
>> >>> > >>>> better.
>> >>> > >>>>
>> >>> > >>>> I was not trying to put down this community only to participate
>> and
>> >>> > >>>>
>> >>> > >>>> make
>> >>> > >>>>
>> >>> > >>>> it
>> >>> > >>>>
>> >>> > >>>> better. I think this conversation is an indication of how great
>> >>> > >>>>
>> >>> > >>>> this
>> >>> > >>>>
>> >>> > >>>> community is.
>> >>> > >>>>
>> >>> > >>>> Maybe I am being sensitive about this issue and trying to
>> >>> > >>>>
>> >>> > >>>> strengthen
>> >>> > >>>>
>> >>> > >>>> the
>> >>> > >>>>
>> >>> > >>>> nifi community even more, after coming from a conference where
>> it
>> >>> > >>>>
>> >>> > >>>> was
>> >>> > >>>>
>> >>> > >>>> reported there was lots of excitement at first and now the
>> >>> > >>>>
>> >>> > >>>> participation
>> >>> > >>>>
>> >>> > >>>> in
>> >>> > >>>>
>> >>> > >>>> the community has really died down and they are struggling. I
>> don't
>> >>> > >>>>
>> >>> > >>>> want
>> >>> > >>>>
>> >>> > >>>> to
>> >>> > >>>>
>> >>> > >>>> see that happen here.
>> >>> > >>>>
>> >>> > >>>> Cheers,
>> >>> > >>>>
>> >>> > >>>> Edgardo
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>> >>> > >>>>
>> >>> > >>>> <javascript:;>> wrote:
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> Edgardo,
>> >>> > >>>>
>> >>> > >>>> Thank you for your feedback. We hear your comments and as a
>> >>> > >>>>
>> >>> > >>>> committer I
>> >>> > >>>>
>> >>> > >>>> can
>> >>> > >>>>
>> >>> > >>>> share we are constantly looking to improve the PR process,
>> having
>> >>> > >>>>
>> >>> > >>>> already
>> >>> > >>>>
>> >>> > >>>> taken many of the steps you suggest.
>> >>> > >>>>
>> >>> > >>>> However, it is important to notice that the number of PRs should
>> >>> > >>>>
>> >>> > >>>> not be
>> >>> > >>>>
>> >>> > >>>> seen as a metric of engagement by the development community:
>> Most
>> >>> > >>>>
>> >>> > >>>> of us
>> >>> > >>>>
>> >>> > >>>> will submit PRs so that our work can be carefully reviewed by
>> our
>> >>> > >>>>
>> >>> > >>>> peers
>> >>> > >>>>
>> >>> > >>>> and
>> >>> > >>>>
>> >>> > >>>> some of us will use JIRA patches to provide contributions.
>> >>> > >>>>
>> >>> > >>>> Having said that, it is true that some PRs may sit idle for a
>> long
>> >>> > >>>>
>> >>> > >>>> time
>> >>> > >>>>
>> >>> > >>>> and
>> >>> > >>>>
>> >>> > >>>> we are working to improve this pipeline.
>> >>> > >>>>
>> >>> > >>>> It was therefore no coincidence that I  browsed most of the PRs
>> >>> > >>>>
>> >>> > >>>> performing
>> >>> > >>>>
>> >>> > >>>> a triage of items that have been superseded or diverged from the
>> >>> > >>>>
>> >>> > >>>> current
>> >>> > >>>>
>> >>> > >>>> code base.
>> >>> > >>>>
>> >>> > >>>> In fact, less than a month ago the dev team closed a number of
>> >>> > >>>>
>> >>> > >>>> stalled
>> >>> > >>>>
>> >>> > >>>> and
>> >>> > >>>>
>> >>> > >>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63
>> c15269eea8).
>> >>> > >>>>
>> >>> > >>>> Despite all the above, I think Joe has a point. The master
>> >>> > >>>>
>> >>> > >>>> contain a
>> >>> > >>>>
>> >>> > >>>> series
>> >>> > >>>>
>> >>> > >>>> of important bug fixes and suspect the community would benefit
>> >>> > >>>>
>> >>> > >>>> from
>> >>> > >>>>
>> >>> > >>>> a
>> >>> > >>>>
>> >>> > >>>> release sooner rather than later.
>> >>> > >>>>
>> >>> > >>>> Once again, thank you for your feedback and contribution. It is
>> >>> > >>>>
>> >>> > >>>> good to
>> >>> > >>>>
>> >>> > >>>> have you here.
>> >>> > >>>>
>> >>> > >>>> Andre
>> >>> > >>>>
>> >>> > >>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>> >>> > >>>>
>> >>> > >>>> [hidden email]
>> >>> > >>>>
>> >>> > >>>> <javascript:;>>
>> >>> > >>>>
>> >>> > >>>> wrote:
>> >>> > >>>>
>> >>> > >>>> Joe - You are correct I was mentioning the PRs that are
>> >>> > >>>>
>> >>> > >>>> currently
>> >>> > >>>>
>> >>> > >>>> open.
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> Regardless of how it happens reducing the count of open PRs I
>> >>> > >>>>
>> >>> > >>>> believe
>> >>> > >>>>
>> >>> > >>>> to
>> >>> > >>>>
>> >>> > >>>> be
>> >>> > >>>>
>> >>> > >>>> extremely important. Maybe I was hoping that the release could
>> >>> > >>>>
>> >>> > >>>> be
>> >>> > >>>>
>> >>> > >>>> a
>> >>> > >>>>
>> >>> > >>>> forcing
>> >>> > >>>>
>> >>> > >>>> function to make that happen. I believe that developers are more
>> >>> > >>>>
>> >>> > >>>> willing
>> >>> > >>>>
>> >>> > >>>> to
>> >>> > >>>>
>> >>> > >>>> contribute when they see that their PRs will actually be able
>> >>> > >>>>
>> >>> > >>>> accepted
>> >>> > >>>>
>> >>> > >>>> and
>> >>> > >>>>
>> >>> > >>>> merged into the code base. Having a low number of open PRs in
>> >>> > >>>>
>> >>> > >>>> progress
>> >>> > >>>>
>> >>> > >>>> is a
>> >>> > >>>>
>> >>> > >>>> great indication that the main nifi developers are fully engaged
>> >>> > >>>>
>> >>> > >>>> with
>> >>> > >>>>
>> >>> > >>>> the
>> >>> > >>>>
>> >>> > >>>> community.
>> >>> > >>>>
>> >>> > >>>> There are a few PRs that don't have any comments from committers
>> >>> > >>>>
>> >>> > >>>> at
>> >>> > >>>>
>> >>> > >>>> all.
>> >>> > >>>>
>> >>> > >>>> I
>> >>> > >>>>
>> >>> > >>>> found one from August in that state. If that was my PR I don't
>> >>> > >>>>
>> >>> > >>>> think I
>> >>> > >>>>
>> >>> > >>>> would be so willing to put another one in anytime soon. I do get
>> >>> > >>>>
>> >>> > >>>> that
>> >>> > >>>>
>> >>> > >>>> sometime PRs get stalled by the originator, if so maybe a rule
>> >>> > >>>>
>> >>> > >>>> about
>> >>> > >>>>
>> >>> > >>>> closing them after a certain amount of time or being taken over
>> >>> > >>>>
>> >>> > >>>> by a
>> >>> > >>>>
>> >>> > >>>> core
>> >>> > >>>>
>> >>> > >>>> contributor if they think it worthwhile.
>> >>> > >>>>
>> >>> > >>>> I would like to shoutout to James Wing on my last PR he was
>> >>> > >>>>
>> >>> > >>>> quick
>> >>> > >>>>
>> >>> > >>>> to
>> >>> > >>>>
>> >>> > >>>> review, provided great comments, testing, and even some
>> >>> > >>>>
>> >>> > >>>> additional
>> >>> > >>>>
>> >>> > >>>> code.
>> >>> > >>>>
>> >>> > >>>> It
>> >>> > >>>>
>> >>> > >>>> was a great PR experience.
>> >>> > >>>>
>> >>> > >>>> Cheers,
>> >>> > >>>>
>> >>> > >>>> Edgardo
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>> >>> > >>>>
>> >>> > >>>> [hidden email] <javascript:;>.
>> >>> > >>>>
>> >>> > >>>> invalid> wrote:
>> >>> > >>>>
>> >>> > >>>> Joe, I think you misread. Edgardo is referring to the Pull
>> >>> > >>>>
>> >>> > >>>> Requests
>> >>> > >>>>
>> >>> > >>>> that
>> >>> > >>>>
>> >>> > >>>> are currently open, not the tickets assigned to the 1.1.0
>> >>> > >>>>
>> >>> > >>>> version.
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> I think these goals (releasing 1.1.0 and cutting down the PR
>> >>> > >>>>
>> >>> > >>>> count)
>> >>> > >>>>
>> >>> > >>>> should
>> >>> > >>>>
>> >>> > >>>> be two different efforts. Doing a thorough job reviewing
>> >>> > >>>>
>> >>> > >>>> takes a
>> >>> > >>>>
>> >>> > >>>> significant amount of time from both the reviewer and
>> >>> > >>>>
>> >>> > >>>> contributor.
>> >>> > >>>>
>> >>> > >>>> In
>> >>> > >>>>
>> >>> > >>>> order
>> >>> > >>>>
>> >>> > >>>> to cut it down significantly would take much longer than a
>> >>> > >>>>
>> >>> > >>>> couple
>> >>> > >>>>
>> >>> > >>>> days.
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> Also there has already been a lot of great new features and
>> >>> > >>>>
>> >>> > >>>> bug
>> >>> > >>>>
>> >>> > >>>> fixes
>> >>> > >>>>
>> >>> > >>>> contributed to the 1.X line and I don't think it's worth
>> >>> > >>>>
>> >>> > >>>> holding up
>> >>> > >>>>
>> >>> > >>>> a
>> >>> > >>>>
>> >>> > >>>> 1.1.0
>> >>> > >>>>
>> >>> > >>>> release for tickets not assigned to this fix version. As an
>> >>> > >>>>
>> >>> > >>>> added
>> >>> > >>>>
>> >>> > >>>> bonus
>> >>> > >>>>
>> >>> > >>>> though, I think many of the tickets tagged as 1.1.0 have PRs
>> >>> > >>>>
>> >>> > >>>> already
>> >>> > >>>>
>> >>> > >>>> open
>> >>> > >>>>
>> >>> > >>>> so closing those will make a large dent in the PR count.
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> Joe
>> >>> > >>>>
>> >>> > >>>> - - - - - -
>> >>> > >>>> Joseph Percivall
>> >>> > >>>> linkedin.com/in/Percivall
>> >>> > >>>> e: [hidden email] <javascript:;>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>> >>> > >>>>
>> >>> > >>>> [hidden email]
>> >>> > >>>>
>> >>> > >>>> <javascript:;>>
>> >>> > >>>>
>> >>> > >>>> wrote:
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> There are less than 30 right now.  Many of the roughly 90+
>> >>> > >>>>
>> >>> > >>>> JIRAs
>> >>> > >>>>
>> >>> > >>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>> >>> > >>>>
>> >>> > >>>> or
>> >>> > >>>>
>> >>> > >>>> just
>> >>> > >>>>
>> >>> > >>>> had fix versions removed.
>> >>> > >>>>
>> >>> > >>>> We will need to have a push over the next bunch of days to
>> >>> > >>>>
>> >>> > >>>> deal
>> >>> > >>>>
>> >>> > >>>> with
>> >>> > >>>>
>> >>> > >>>> reviewing/merging/moving the remaining items.
>> >>> > >>>>
>> >>> > >>>> Thanks
>> >>> > >>>> Joe
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>> >>> > >>>>
>> >>> > >>>> [hidden email] <javascript:;>>
>> >>> > >>>>
>> >>> > >>>> wrote:
>> >>> > >>>>
>> >>> > >>>> Joe,
>> >>> > >>>>
>> >>> > >>>> There are 75 PRs currently open. Why not make a push over
>> >>> > >>>>
>> >>> > >>>> the
>> >>> > >>>>
>> >>> > >>>> next
>> >>> > >>>>
>> >>> > >>>> bunch
>> >>> > >>>>
>> >>> > >>>> of
>> >>> > >>>>
>> >>> > >>>> days to get them closed and then cut the release after that.
>> >>> > >>>>
>> >>> > >>>> Cheers,
>> >>> > >>>>
>> >>> > >>>> Edgardo
>> >>> > >>>>
>> >>> > >>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>> >>> > >>>>
>> >>> > >>>> [hidden email]
>> >>> > >>>>
>> >>> > >>>> <javascript:;>>
>> >>> > >>>>
>> >>> > >>>> wrote:
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> Team,
>> >>> > >>>>
>> >>> > >>>> There have been a ton of bugs fixed a few nice features.  I
>> >>> > >>>>
>> >>> > >>>> would
>> >>> > >>>>
>> >>> > >>>> like
>> >>> > >>>>
>> >>> > >>>> to move to get Apache NiFi 1.1.0 release going pretty much
>> >>> > >>>>
>> >>> > >>>> based
>> >>> > >>>>
>> >>> > >>>> on
>> >>> > >>>>
>> >>> > >>>> where we are now and plan to move most tickets to a new
>> >>> > >>>>
>> >>> > >>>> Apache
>> >>> > >>>>
>> >>> > >>>> NiFi
>> >>> > >>>>
>> >>> > >>>> 1.2.0 version.  We can try to get back on our roughly 6-8
>> >>> > >>>>
>> >>> > >>>> week
>> >>> > >>>>
>> >>> > >>>> release
>> >>> > >>>>
>> >>> > >>>> schedule and shoot for a mid to late Nov release for NiFi
>> >>> > >>>>
>> >>> > >>>> 1.2.0
>> >>> > >>>>
>> >>> > >>>> this
>> >>> > >>>>
>> >>> > >>>> way as well. Please advise if anyone has any other views on
>> >>> > >>>>
>> >>> > >>>> this. In
>> >>> > >>>>
>> >>> > >>>> the mean time I'll get the wheels in motion so you'll be
>> >>> > >>>>
>> >>> > >>>> seeing a
>> >>> > >>>>
>> >>> > >>>> lot
>> >>> > >>>>
>> >>> > >>>> of JIRA/issue updates to move version around.
>> >>> > >>>>
>> >>> > >>>> Thanks
>> >>> > >>>> Joe
>> >>> > >>>>
>> >>> > >>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>> >>> > >>>>
>> >>> > >>>> [hidden email]
>> >>> > >>>>
>> >>> > >>>> <javascript:;>>
>> >>> > >>>>
>> >>> > >>>> wrote:
>> >>> > >>>>
>> >>> > >>>> Sounds good Joe. I have no issue to you doing the rm'ing
>> >>> > >>>>
>> >>> > >>>> for
>> >>> > >>>>
>> >>> > >>>> it.
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>> >>> > >>>>
>> >>> > >>>> <javascript:;>> wrote:
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> Team,
>> >>> > >>>>
>> >>> > >>>> There are a lot of great fixes and improvements on the
>> >>> > >>>>
>> >>> > >>>> master
>> >>> > >>>>
>> >>> > >>>> line
>> >>> > >>>>
>> >>> > >>>> now
>> >>> > >>>>
>> >>> > >>>> and we're at a good time window to start pushing for a
>> >>> > >>>>
>> >>> > >>>> release.
>> >>> > >>>>
>> >>> > >>>> There
>> >>> > >>>>
>> >>> > >>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>> >>> > >>>>
>> >>> > >>>> are
>> >>> > >>>>
>> >>> > >>>> open.
>> >>> > >>>>
>> >>> > >>>> I'm
>> >>> > >>>>
>> >>> > >>>> going to go through them and remove fix versions where
>> >>> > >>>>
>> >>> > >>>> appropriate.
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> I'm happy to take on RM task for this release though if
>> >>> > >>>>
>> >>> > >>>> someone
>> >>> > >>>>
>> >>> > >>>> else
>> >>> > >>>>
>> >>> > >>>> would like to take that on please advise.
>> >>> > >>>>
>> >>> > >>>> Thanks
>> >>> > >>>> Joe
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> --
>> >>> > >>>> Cheers,
>> >>> > >>>>
>> >>> > >>>> Edgardo
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> --
>> >>> > >>>> Cheers,
>> >>> > >>>>
>> >>> > >>>> Edgardo
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> --
>> >>> > >>>> Cheers,
>> >>> > >>>>
>> >>> > >>>> Edgardo
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>> --
>> >>> > >>>> Cheers,
>> >>> > >>>>
>> >>> > >>>> Edgardo
>> >>> > >>>>
>> >>> > >>>> Sent from Gmail Mobile
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> > >>>>
>> >>> >
>> >>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] NiFi 1.1.0 release

Joe Witt
release notes here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.1.0

migration notes here:
https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance

Will kick out an RC today or in next day or two hopefully.

Thanks
Joe

On Wed, Nov 23, 2016 at 9:13 AM, Joe Witt <[hidden email]> wrote:

> Some good news - we can put our twitter processor back in the game.
>
> Legal VP at Apache just sent out the decision.
>
> Will immediately restore that to action and put in the follow-on
> ticket to ensure the twitter4j library moves away from the old Json
> lib (there is a PR to replace it).  We can only keep it this way until
> April so unless twitter4j resolves their source dependency we'll be
> back in this position.
>
> thanks
> joe
>
> On Tue, Nov 22, 2016 at 1:38 PM, Michael Moser <[hidden email]> wrote:
>> Awesome.  Your prompt and detailed feedback is very much appreciated!
>>
>>
>> On Tue, Nov 22, 2016 at 12:47 PM, Joe Witt <[hidden email]> wrote:
>>
>>> mike - also to clarify based on what you brought up I updated the JIRA
>>> description as follows
>>>
>>> The following also would be true:
>>> 1) Apache NiFi 1.0.0 repositories should work just fine when applied
>>> to an Apache NiFi 1.1.0 installation.
>>>
>>> 2) Repositories made/updated in Apache NiFi 1.1.0 onward would not
>>> work in older Apache NiFi releases (such as 1.0.0)
>>>
>>> On Tue, Nov 22, 2016 at 12:33 PM, Joe Witt <[hidden email]> wrote:
>>> > Mike - yeah good catch and good question.  It does support the old
>>> > format.  We've always been pretty good about being diligent to allow
>>> > folks to upgrade and it honor existing state and in many cases
>>> > configurations and it would automatically port them over.  What has
>>> > always been far more problematic is 'rollback' where people ran on
>>> > newer configurations but could not then go back to old framework code.
>>> > That is what NIFI-2854 tackles at least as far as the
>>> > content/prov/flowfile repositories go.  Now, the code and
>>> > serialization is done in such a way that older version can simply
>>> > ignore what never versions encoded if they don't understand it but
>>> > they should be able to continue on.
>>> >
>>> > I just tried out a 1.0.0 flow with data queued up.  Upgraded to a
>>> > latest NiFi 1.1.0-SNAPSHOT.  Moved the repos over.  And it came up
>>> > perfectly with all the queue data ready to roll.
>>> >
>>> > Thanks
>>> > Joe
>>> >
>>> > On Tue, Nov 22, 2016 at 12:16 PM, Michael Moser <[hidden email]>
>>> wrote:
>>> >> After seeing "Repositories now support rollback" in the release notes
>>> and
>>> >> reading NIFI-2854 [1], I have a question.
>>> >>
>>> >> Are repositories created using NiFi 1.0.0 compatible with NiFi 1.1.0
>>> >> software?  This is the goal that the ticket seems to indicate with 1.1.0
>>> >> onward, but it's not clear whether 1.0.0 -> 1.1.0 is included.
>>> >>
>>> >> Thanks,
>>> >> -- Mike
>>> >>
>>> >> [1] - https://issues.apache.org/jira/browse/NIFI-2854
>>> >>
>>> >> On Tue, Nov 22, 2016 at 12:08 PM, James Wing <[hidden email]> wrote:
>>> >>
>>> >>> Going through the tickets, it seems like quite a release.  A few more
>>> >>> things for your list:
>>> >>>
>>> >>> GenerateFlowFile updated to support literal/expression content and
>>> >>> attributes
>>> >>>
>>> >>> AWS-related:
>>> >>> * New processors PutCloudWatchMetric, PutKinesisStream
>>> >>> * Updated processors PutS3Object (content type, signer options), ListS3
>>> >>> (performance, versions)
>>> >>> * Added support for AWS assume role credentials with proxy
>>> >>>
>>> >>>
>>> >>> Thanks,
>>> >>>
>>> >>> James
>>> >>>
>>> >>> On Mon, Nov 21, 2016 at 9:30 PM, Joe Witt <[hidden email]> wrote:
>>> >>>
>>> >>> > matt
>>> >>> >
>>> >>> > i'll add a wiki page or set of instructions linked from the release
>>> >>> notes.
>>> >>> >
>>> >>> > all,
>>> >>> >
>>> >>> > walked through the 250 or so JIRAs in the 1.1.0 release and pulled
>>> out
>>> >>> > highlights.  The items noted are as follows.  Will likely reduce this
>>> >>> > down further for the release notes but wanted to put this out in case
>>> >>> > folks have things they think are really important to highlight.
>>> >>> >
>>> >>> > - Core Improvements:
>>> >>> >    - Performance: Session Migration
>>> >>> >    - Stability: Cluster Management
>>> >>> >    - Developer: Framework supports easy user driven classloader
>>> extension
>>> >>> >    - Expression Language: Now supports base64 and hex encoded values
>>> >>> > and Math functions
>>> >>> >    - Repositories now support rollback
>>> >>> >    - Faster startup due to more efficient state restoration algorithm
>>> >>> > - UX Improvements:
>>> >>> >    - Visual Backpressure Indicator
>>> >>> >    - Introduced more colors to better highlight actions and
>>> components
>>> >>> >    - Performance: Validate non-running components
>>> >>> >    - Provenance graph image can be exported
>>> >>> >    - Cron Scheduling for Primary node tasks now supported
>>> >>> > - Updated versions
>>> >>> >    - Azure Event Hub 0.9.0
>>> >>> >    - Spark 2.0.1
>>> >>> >    - Hadoop 2.7.x
>>> >>> > - New/Improved Processors
>>> >>> >    - new Fetch/Put Elastic Search 5.0
>>> >>> >    - new ParseCEF to parse CEF formatted logs
>>> >>> >    - improve ExtractEmail now supports TNEF files
>>> >>> >    - new Validate CSV
>>> >>> >    - improved Solr processors now support SSL and Kerberos
>>> >>> >    - new Websocket client and server processors
>>> >>> > - New Utility
>>> >>> >    - Zookeeper Migrator (move from one zookeeper to another)
>>> >>> > - Security
>>> >>> >    - Restricted Processors
>>> >>> >    - Site-to-site now supports port forwarding
>>> >>> >    - Improved Policy Management UX
>>> >>> > - Migration Notes:
>>> >>> >    - Restricted Processors
>>> >>> >    - Twitter Processor Removed
>>> >>> >
>>> >>> > On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <[hidden email]
>>> >
>>> >>> > wrote:
>>> >>> > > Is there a good spot for us to put instructions on how to build the
>>> >>> > > Twitter processor and/or the Social Media NAR in the meantime?
>>> Maybe a
>>> >>> > > Wiki page or something simple to say "go to this directory, run
>>> this
>>> >>> > > Maven command, drop the NAR into your deployment..." ?
>>> >>> > >
>>> >>> > > On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <[hidden email]>
>>> wrote:
>>> >>> > >> Team,
>>> >>> > >>
>>> >>> > >> We appear to be very close.  Andy is working NIFI-3024 but
>>> otherwise
>>> >>> > >> it is focus on testing.
>>> >>> > >>
>>> >>> > >> I'm going to prep the RC and release notes now.  Unfortunately the
>>> >>> > >> twitter changes for json.org will need to remain.  Consensus
>>> forming
>>> >>> > >> on the legal-discuss thread regarding a grace period has been
>>> elusive
>>> >>> > >> and we're already prepared to make the right steps so we'll just
>>> need
>>> >>> > >> to take that on by being empathetic to the user base.
>>> >>> > >>
>>> >>> > >> Thanks
>>> >>> > >> Joe
>>> >>> > >>
>>> >>> > >> On Thu, Nov 17, 2016 at 7:37 AM, Andre <[hidden email]>
>>> wrote:
>>> >>> > >>> Andy,
>>> >>> > >>>
>>> >>> > >>> Great to see NIFI-3050 implemented and certainly good news that
>>> NiFi
>>> >>> > 1.1.0
>>> >>> > >>> is set to include a number of security related improvements.
>>> >>> > >>>
>>> >>> > >>>
>>> >>> > >>>
>>> >>> > >>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <
>>> [hidden email]
>>> >>> >
>>> >>> > wrote:
>>> >>> > >>>
>>> >>> > >>>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2]
>>> have
>>> >>> > been
>>> >>> > >>>> added to my plate for this release. Coordinated with Joe Witt
>>> and
>>> >>> they
>>> >>> > >>>> should both be included.
>>> >>> > >>>>
>>> >>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3050
>>> >>> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3051
>>> >>> > >>>>
>>> >>> > >>>> Andy LoPresto
>>> >>> > >>>> [hidden email]
>>> >>> > >>>> *[hidden email] <[hidden email]>*
>>> >>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
>>> EF69
>>> >>> > >>>>
>>> >>> > >>>> On Nov 16, 2016, at 12:08 PM, Joe Witt <[hidden email]>
>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> Team
>>> >>> > >>>>
>>> >>> > >>>> There is a thread on apache legal-discuss that might allow for a
>>> >>> > >>>> graceperiod of continued usage of the json library.  Am going to
>>> >>> keep
>>> >>> > >>>> a close eye on this and if VP Legal approves we'll be able to
>>> keep
>>> >>> the
>>> >>> > >>>> twitter processors in which is definitely a good thing.  Will
>>> advise
>>> >>> > >>>>
>>> >>> > >>>> Thanks
>>> >>> > >>>> Joe
>>> >>> > >>>>
>>> >>> > >>>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <[hidden email]
>>> >
>>> >>> > wrote:
>>> >>> > >>>>
>>> >>> > >>>> I've noticed an issue with the per-instance class loading
>>> capability
>>> >>> > >>>> introduced in NIFI-2909 where the additional classpath
>>> resources can
>>> >>> > get
>>> >>> > >>>> incorrectly removed from the class loader.
>>> >>> > >>>>
>>> >>> > >>>> I was able to reproduced this with a unit test and have a fix
>>> >>> ready. I
>>> >>> > >>>> believe this is important and needs to go in for the 1.1
>>> release,
>>> >>> > going to
>>> >>> > >>>> re-open NIFI-2909 and submit a PR shortly.
>>> >>> > >>>>
>>> >>> > >>>> -Bryan
>>> >>> > >>>>
>>> >>> > >>>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <
>>> >>> [hidden email]
>>> >>> > >
>>> >>> > >>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> I have two items that I would like to wrap up prior to creating
>>> an
>>> >>> RC
>>> >>> > for
>>> >>> > >>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process
>>> >>> Group
>>> >>> > port
>>> >>> > >>>> configuration. The work is already completed and I will be
>>> reviewing
>>> >>> > it
>>> >>> > >>>> this today. Additionally, following recent interest on the
>>> mailing
>>> >>> > list,
>>> >>> > >>>> I'd like to knock out NIFI-3020. This will allow an admin to
>>> >>> > configure a
>>> >>> > >>>> strategy for user identity when logging in via LDAP.
>>> Specifically,
>>> >>> it
>>> >>> > will
>>> >>> > >>>> support usage of the DN (the default and current
>>> implementation) as
>>> >>> > well as
>>> >>> > >>>> the username the user logged in as. I should be able to have a
>>> PR up
>>> >>> > for
>>> >>> > >>>> this work later today.
>>> >>> > >>>>
>>> >>> > >>>> Thanks!
>>> >>> > >>>>
>>> >>> > >>>> Matt
>>> >>> > >>>>
>>> >>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>>> >>> > >>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <[hidden email]>
>>> >>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> The code is within the twitter4j library itself.  I filed a
>>> request
>>> >>> to
>>> >>> > >>>> twitter4jg.  The most likely case is we will need to submit a
>>> PR to
>>> >>> > them.
>>> >>> > >>>> However, I don't see this as something that should delay the
>>> >>> > release.  We
>>> >>> > >>>> can provide instructions for folks wanting to use the processor
>>> >>> during
>>> >>> > >>>>
>>> >>> > >>>> the
>>> >>> > >>>>
>>> >>> > >>>> time we cannot make it available in a convenient manner.  I will
>>> >>> > provide
>>> >>> > >>>>
>>> >>> > >>>> a
>>> >>> > >>>>
>>> >>> > >>>> meaningful comment about this in release notes and pointers on
>>> what
>>> >>> > folks
>>> >>> > >>>> can do in the meantime.
>>> >>> > >>>>
>>> >>> > >>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <[hidden email]>
>>> >>> > wrote:
>>> >>> > >>>>
>>> >>> > >>>> I understand there was a discussion thread within the NiFi
>>> community
>>> >>> > >>>>
>>> >>> > >>>> for
>>> >>> > >>>>
>>> >>> > >>>> this as well and I missed responding to that at that time. It
>>> just
>>> >>> > >>>>
>>> >>> > >>>> seems
>>> >>> > >>>>
>>> >>> > >>>> to
>>> >>> > >>>>
>>> >>> > >>>> me like JSON processing is necessary for GetTwitter, which is
>>> >>> > >>>>
>>> >>> > >>>> incredibly
>>> >>> > >>>>
>>> >>> > >>>> useful for demonstrating NiFi’s ability to read from a high
>>> volume
>>> >>> > >>>>
>>> >>> > >>>> stream
>>> >>> > >>>>
>>> >>> > >>>> out of the box. With NIFI-3019 (Remove GetTwitter from default
>>> >>> build),
>>> >>> > >>>>
>>> >>> > >>>> is
>>> >>> > >>>>
>>> >>> > >>>> there any related effort to substitute an acceptable replacement
>>> >>> JSON
>>> >>> > >>>> library to restore this functionality?
>>> >>> > >>>>
>>> >>> > >>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>>> >>> > >>>>
>>> >>> > >>>> Andy LoPresto
>>> >>> > >>>> [hidden email]
>>> >>> > >>>> *[hidden email] <[hidden email]>*
>>> >>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
>>> EF69
>>> >>> > >>>>
>>> >>> > >>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <
>>> [hidden email]>
>>> >>> > >>>>
>>> >>> > >>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> I’m working with Bryan Rosander to close out NIFI-3024,
>>> NIFI-2655,
>>> >>> and
>>> >>> > >>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>>> >>> > >>>> investigated some alternate TLS config options for the new
>>> version
>>> >>> of
>>> >>> > >>>>
>>> >>> > >>>> the
>>> >>> > >>>>
>>> >>> > >>>> client library.
>>> >>> > >>>>
>>> >>> > >>>> Is there any alternative to excluding the GetTwitter processor?
>>> >>> Using
>>> >>> > >>>> Johnzon [1] or the Android re-implementation [2] discussed in
>>> the
>>> >>> > >>>>
>>> >>> > >>>> mailing
>>> >>> > >>>>
>>> >>> > >>>> list thread?
>>> >>> > >>>>
>>> >>> > >>>> [1] https://johnzon.apache.org/
>>> >>> > >>>> [2] https://developer.android.com/reference/org/json/package-
>>> >>> > >>>>
>>> >>> > >>>> summary.html
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> Andy LoPresto
>>> >>> > >>>> [hidden email]
>>> >>> > >>>> *[hidden email] <[hidden email]>*
>>> >>> > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
>>> EF69
>>> >>> > >>>>
>>> >>> > >>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <[hidden email]>
>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> Team
>>> >>> > >>>>
>>> >>> > >>>> Very happy to see that we are down to three items remaining
>>> tagged
>>> >>> to
>>> >>> > >>>> 1.1.0.  Solid effort over the recent weeks to close the gap
>>> >>> including
>>> >>> > >>>>
>>> >>> > >>>> work
>>> >>> > >>>>
>>> >>> > >>>> to get past the now category x Jason dependency we had.  The
>>> most
>>> >>> > >>>>
>>> >>> > >>>> notable
>>> >>> > >>>>
>>> >>> > >>>> impact from that is the wildly popular GetTwitter processor,
>>> the fav
>>> >>> > >>>>
>>> >>> > >>>> new
>>> >>> > >>>>
>>> >>> > >>>> nifi user and demo processor, can no longer be included in the
>>> >>> default
>>> >>> > >>>> build.  It is optionally available if users choose to build and
>>> use
>>> >>> it
>>> >>> > >>>>
>>> >>> > >>>> but
>>> >>> > >>>>
>>> >>> > >>>> we won't distribute binaries that have it.
>>> >>> > >>>>
>>> >>> > >>>> I see some review movement on some patch available but untagged
>>> >>> items.
>>> >>> > >>>>
>>> >>> > >>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri.
>>> >>> > Anyone
>>> >>> > >>>> have any outstanding items?
>>> >>> > >>>>
>>> >>> > >>>> Thanks
>>> >>> > >>>> Joe
>>> >>> > >>>>
>>> >>> > >>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <[hidden email]> wrote:
>>> >>> > >>>>
>>> >>> > >>>> Ryan
>>> >>> > >>>>
>>> >>> > >>>> Not officially but I think we should try to close this thing
>>> out and
>>> >>> > >>>> start a vote in the next week or two at most.
>>> >>> > >>>>
>>> >>> > >>>> I'm going through the tickets again now.  There is also a new
>>> issue
>>> >>> of
>>> >>> > >>>> the json-p license falling out of favor in Apache legal terms
>>> and
>>> >>> > >>>> becoming Category-X.  Am looking into that now.
>>> >>> > >>>>
>>> >>> > >>>> Thanks
>>> >>> > >>>> Joe
>>> >>> > >>>>
>>> >>> > >>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <[hidden email]
>>> >
>>> >>> > >>>>
>>> >>> > >>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> Joe - Is there a target date for 1.1?
>>> >>> > >>>>
>>> >>> > >>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <[hidden email]>
>>> >>> > wrote:
>>> >>> > >>>>
>>> >>> > >>>> Team,
>>> >>> > >>>>
>>> >>> > >>>> Just an update on things with working toward an Apache NiFi
>>> 1.1.0
>>> >>> > >>>> release.  There are still about 33 JIRAs there now and some are
>>> >>> > >>>> awaiting review and are some are under active progress. Yet
>>> there is
>>> >>> > >>>> good traction and progress. I think we should just stay vigilant
>>> >>> with
>>> >>> > >>>> what makes it in and keep working it down.  So let's please
>>> shoot
>>> >>> for
>>> >>> > >>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>>> >>> > >>>>
>>> >>> > >>>> Thanks
>>> >>> > >>>> Joe
>>> >>> > >>>>
>>> >>> > >>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <[hidden email]>
>>> >>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> Team,
>>> >>> > >>>>
>>> >>> > >>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.
>>> >>> Let's
>>> >>> > >>>> avoiding putting more in there for now at least without a
>>> >>> discussion.
>>> >>> > >>>> Of the 31 JIRAs there the vast majority need review so we
>>> should be
>>> >>> > >>>> able to close these down fairly quickly as long as we don't let
>>> the
>>> >>> > >>>> list grow.
>>> >>> > >>>>
>>> >>> > >>>> Thanks
>>> >>> > >>>> joe
>>> >>> > >>>>
>>> >>> > >>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <
>>> >>> [hidden email]
>>> >>> > >
>>> >>> > >>>>
>>> >>> > >>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> Joe,
>>> >>> > >>>>
>>> >>> > >>>> Appreciate the offer it isn't my PR. I was just using it as an
>>> >>> > >>>>
>>> >>> > >>>> example.
>>> >>> > >>>>
>>> >>> > >>>> All
>>> >>> > >>>>
>>> >>> > >>>> mine are currently closed, which I greatly appreciate.
>>> >>> > >>>>
>>> >>> > >>>> Cheers,
>>> >>> > >>>>
>>> >>> > >>>> Edgardo
>>> >>> > >>>>
>>> >>> > >>>> On Friday, October 14, 2016, Joe Witt <[hidden email]>
>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> Edgardo,
>>> >>> > >>>>
>>> >>> > >>>> You mentioned a PR from August. I'd be happy to help you work
>>> that
>>> >>> > >>>> through review.
>>> >>> > >>>>
>>> >>> > >>>> Thanks
>>> >>> > >>>> Joe
>>> >>> > >>>>
>>> >>> > >>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>>> >>> > >>>>
>>> >>> > >>>> [hidden email]
>>> >>> > >>>>
>>> >>> > >>>> <javascript:;>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> I have agreed that at this point a release is important. My goal
>>> >>> > >>>>
>>> >>> > >>>> was
>>> >>> > >>>>
>>> >>> > >>>> try
>>> >>> > >>>>
>>> >>> > >>>> to
>>> >>> > >>>>
>>> >>> > >>>> squeeze in a much goodness as possible into the release, but the
>>> >>> > >>>>
>>> >>> > >>>> important
>>> >>> > >>>>
>>> >>> > >>>> bug fixes should come first. Getting 1.x into a state where the
>>> >>> > >>>>
>>> >>> > >>>> release
>>> >>> > >>>>
>>> >>> > >>>> notes don't say that it is geared toward developers and testers
>>> is
>>> >>> > >>>>
>>> >>> > >>>> really
>>> >>> > >>>>
>>> >>> > >>>> huge.
>>> >>> > >>>>
>>> >>> > >>>> I think Nifi is a great community otherwise I would participate
>>> in
>>> >>> > >>>>
>>> >>> > >>>> the
>>> >>> > >>>>
>>> >>> > >>>> mailing list, create Jira tickets and pull requests. I am only
>>> >>> > >>>>
>>> >>> > >>>> trying to
>>> >>> > >>>>
>>> >>> > >>>> strengthen the great thing that is going on here. We can always
>>> do
>>> >>> > >>>>
>>> >>> > >>>> better.
>>> >>> > >>>>
>>> >>> > >>>> I was not trying to put down this community only to participate
>>> and
>>> >>> > >>>>
>>> >>> > >>>> make
>>> >>> > >>>>
>>> >>> > >>>> it
>>> >>> > >>>>
>>> >>> > >>>> better. I think this conversation is an indication of how great
>>> >>> > >>>>
>>> >>> > >>>> this
>>> >>> > >>>>
>>> >>> > >>>> community is.
>>> >>> > >>>>
>>> >>> > >>>> Maybe I am being sensitive about this issue and trying to
>>> >>> > >>>>
>>> >>> > >>>> strengthen
>>> >>> > >>>>
>>> >>> > >>>> the
>>> >>> > >>>>
>>> >>> > >>>> nifi community even more, after coming from a conference where
>>> it
>>> >>> > >>>>
>>> >>> > >>>> was
>>> >>> > >>>>
>>> >>> > >>>> reported there was lots of excitement at first and now the
>>> >>> > >>>>
>>> >>> > >>>> participation
>>> >>> > >>>>
>>> >>> > >>>> in
>>> >>> > >>>>
>>> >>> > >>>> the community has really died down and they are struggling. I
>>> don't
>>> >>> > >>>>
>>> >>> > >>>> want
>>> >>> > >>>>
>>> >>> > >>>> to
>>> >>> > >>>>
>>> >>> > >>>> see that happen here.
>>> >>> > >>>>
>>> >>> > >>>> Cheers,
>>> >>> > >>>>
>>> >>> > >>>> Edgardo
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <[hidden email]
>>> >>> > >>>>
>>> >>> > >>>> <javascript:;>> wrote:
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> Edgardo,
>>> >>> > >>>>
>>> >>> > >>>> Thank you for your feedback. We hear your comments and as a
>>> >>> > >>>>
>>> >>> > >>>> committer I
>>> >>> > >>>>
>>> >>> > >>>> can
>>> >>> > >>>>
>>> >>> > >>>> share we are constantly looking to improve the PR process,
>>> having
>>> >>> > >>>>
>>> >>> > >>>> already
>>> >>> > >>>>
>>> >>> > >>>> taken many of the steps you suggest.
>>> >>> > >>>>
>>> >>> > >>>> However, it is important to notice that the number of PRs should
>>> >>> > >>>>
>>> >>> > >>>> not be
>>> >>> > >>>>
>>> >>> > >>>> seen as a metric of engagement by the development community:
>>> Most
>>> >>> > >>>>
>>> >>> > >>>> of us
>>> >>> > >>>>
>>> >>> > >>>> will submit PRs so that our work can be carefully reviewed by
>>> our
>>> >>> > >>>>
>>> >>> > >>>> peers
>>> >>> > >>>>
>>> >>> > >>>> and
>>> >>> > >>>>
>>> >>> > >>>> some of us will use JIRA patches to provide contributions.
>>> >>> > >>>>
>>> >>> > >>>> Having said that, it is true that some PRs may sit idle for a
>>> long
>>> >>> > >>>>
>>> >>> > >>>> time
>>> >>> > >>>>
>>> >>> > >>>> and
>>> >>> > >>>>
>>> >>> > >>>> we are working to improve this pipeline.
>>> >>> > >>>>
>>> >>> > >>>> It was therefore no coincidence that I  browsed most of the PRs
>>> >>> > >>>>
>>> >>> > >>>> performing
>>> >>> > >>>>
>>> >>> > >>>> a triage of items that have been superseded or diverged from the
>>> >>> > >>>>
>>> >>> > >>>> current
>>> >>> > >>>>
>>> >>> > >>>> code base.
>>> >>> > >>>>
>>> >>> > >>>> In fact, less than a month ago the dev team closed a number of
>>> >>> > >>>>
>>> >>> > >>>> stalled
>>> >>> > >>>>
>>> >>> > >>>> and
>>> >>> > >>>>
>>> >>> > >>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63
>>> c15269eea8).
>>> >>> > >>>>
>>> >>> > >>>> Despite all the above, I think Joe has a point. The master
>>> >>> > >>>>
>>> >>> > >>>> contain a
>>> >>> > >>>>
>>> >>> > >>>> series
>>> >>> > >>>>
>>> >>> > >>>> of important bug fixes and suspect the community would benefit
>>> >>> > >>>>
>>> >>> > >>>> from
>>> >>> > >>>>
>>> >>> > >>>> a
>>> >>> > >>>>
>>> >>> > >>>> release sooner rather than later.
>>> >>> > >>>>
>>> >>> > >>>> Once again, thank you for your feedback and contribution. It is
>>> >>> > >>>>
>>> >>> > >>>> good to
>>> >>> > >>>>
>>> >>> > >>>> have you here.
>>> >>> > >>>>
>>> >>> > >>>> Andre
>>> >>> > >>>>
>>> >>> > >>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>>> >>> > >>>>
>>> >>> > >>>> [hidden email]
>>> >>> > >>>>
>>> >>> > >>>> <javascript:;>>
>>> >>> > >>>>
>>> >>> > >>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> Joe - You are correct I was mentioning the PRs that are
>>> >>> > >>>>
>>> >>> > >>>> currently
>>> >>> > >>>>
>>> >>> > >>>> open.
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> Regardless of how it happens reducing the count of open PRs I
>>> >>> > >>>>
>>> >>> > >>>> believe
>>> >>> > >>>>
>>> >>> > >>>> to
>>> >>> > >>>>
>>> >>> > >>>> be
>>> >>> > >>>>
>>> >>> > >>>> extremely important. Maybe I was hoping that the release could
>>> >>> > >>>>
>>> >>> > >>>> be
>>> >>> > >>>>
>>> >>> > >>>> a
>>> >>> > >>>>
>>> >>> > >>>> forcing
>>> >>> > >>>>
>>> >>> > >>>> function to make that happen. I believe that developers are more
>>> >>> > >>>>
>>> >>> > >>>> willing
>>> >>> > >>>>
>>> >>> > >>>> to
>>> >>> > >>>>
>>> >>> > >>>> contribute when they see that their PRs will actually be able
>>> >>> > >>>>
>>> >>> > >>>> accepted
>>> >>> > >>>>
>>> >>> > >>>> and
>>> >>> > >>>>
>>> >>> > >>>> merged into the code base. Having a low number of open PRs in
>>> >>> > >>>>
>>> >>> > >>>> progress
>>> >>> > >>>>
>>> >>> > >>>> is a
>>> >>> > >>>>
>>> >>> > >>>> great indication that the main nifi developers are fully engaged
>>> >>> > >>>>
>>> >>> > >>>> with
>>> >>> > >>>>
>>> >>> > >>>> the
>>> >>> > >>>>
>>> >>> > >>>> community.
>>> >>> > >>>>
>>> >>> > >>>> There are a few PRs that don't have any comments from committers
>>> >>> > >>>>
>>> >>> > >>>> at
>>> >>> > >>>>
>>> >>> > >>>> all.
>>> >>> > >>>>
>>> >>> > >>>> I
>>> >>> > >>>>
>>> >>> > >>>> found one from August in that state. If that was my PR I don't
>>> >>> > >>>>
>>> >>> > >>>> think I
>>> >>> > >>>>
>>> >>> > >>>> would be so willing to put another one in anytime soon. I do get
>>> >>> > >>>>
>>> >>> > >>>> that
>>> >>> > >>>>
>>> >>> > >>>> sometime PRs get stalled by the originator, if so maybe a rule
>>> >>> > >>>>
>>> >>> > >>>> about
>>> >>> > >>>>
>>> >>> > >>>> closing them after a certain amount of time or being taken over
>>> >>> > >>>>
>>> >>> > >>>> by a
>>> >>> > >>>>
>>> >>> > >>>> core
>>> >>> > >>>>
>>> >>> > >>>> contributor if they think it worthwhile.
>>> >>> > >>>>
>>> >>> > >>>> I would like to shoutout to James Wing on my last PR he was
>>> >>> > >>>>
>>> >>> > >>>> quick
>>> >>> > >>>>
>>> >>> > >>>> to
>>> >>> > >>>>
>>> >>> > >>>> review, provided great comments, testing, and even some
>>> >>> > >>>>
>>> >>> > >>>> additional
>>> >>> > >>>>
>>> >>> > >>>> code.
>>> >>> > >>>>
>>> >>> > >>>> It
>>> >>> > >>>>
>>> >>> > >>>> was a great PR experience.
>>> >>> > >>>>
>>> >>> > >>>> Cheers,
>>> >>> > >>>>
>>> >>> > >>>> Edgardo
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>>> >>> > >>>>
>>> >>> > >>>> [hidden email] <javascript:;>.
>>> >>> > >>>>
>>> >>> > >>>> invalid> wrote:
>>> >>> > >>>>
>>> >>> > >>>> Joe, I think you misread. Edgardo is referring to the Pull
>>> >>> > >>>>
>>> >>> > >>>> Requests
>>> >>> > >>>>
>>> >>> > >>>> that
>>> >>> > >>>>
>>> >>> > >>>> are currently open, not the tickets assigned to the 1.1.0
>>> >>> > >>>>
>>> >>> > >>>> version.
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> I think these goals (releasing 1.1.0 and cutting down the PR
>>> >>> > >>>>
>>> >>> > >>>> count)
>>> >>> > >>>>
>>> >>> > >>>> should
>>> >>> > >>>>
>>> >>> > >>>> be two different efforts. Doing a thorough job reviewing
>>> >>> > >>>>
>>> >>> > >>>> takes a
>>> >>> > >>>>
>>> >>> > >>>> significant amount of time from both the reviewer and
>>> >>> > >>>>
>>> >>> > >>>> contributor.
>>> >>> > >>>>
>>> >>> > >>>> In
>>> >>> > >>>>
>>> >>> > >>>> order
>>> >>> > >>>>
>>> >>> > >>>> to cut it down significantly would take much longer than a
>>> >>> > >>>>
>>> >>> > >>>> couple
>>> >>> > >>>>
>>> >>> > >>>> days.
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> Also there has already been a lot of great new features and
>>> >>> > >>>>
>>> >>> > >>>> bug
>>> >>> > >>>>
>>> >>> > >>>> fixes
>>> >>> > >>>>
>>> >>> > >>>> contributed to the 1.X line and I don't think it's worth
>>> >>> > >>>>
>>> >>> > >>>> holding up
>>> >>> > >>>>
>>> >>> > >>>> a
>>> >>> > >>>>
>>> >>> > >>>> 1.1.0
>>> >>> > >>>>
>>> >>> > >>>> release for tickets not assigned to this fix version. As an
>>> >>> > >>>>
>>> >>> > >>>> added
>>> >>> > >>>>
>>> >>> > >>>> bonus
>>> >>> > >>>>
>>> >>> > >>>> though, I think many of the tickets tagged as 1.1.0 have PRs
>>> >>> > >>>>
>>> >>> > >>>> already
>>> >>> > >>>>
>>> >>> > >>>> open
>>> >>> > >>>>
>>> >>> > >>>> so closing those will make a large dent in the PR count.
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> Joe
>>> >>> > >>>>
>>> >>> > >>>> - - - - - -
>>> >>> > >>>> Joseph Percivall
>>> >>> > >>>> linkedin.com/in/Percivall
>>> >>> > >>>> e: [hidden email] <javascript:;>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>>> >>> > >>>>
>>> >>> > >>>> [hidden email]
>>> >>> > >>>>
>>> >>> > >>>> <javascript:;>>
>>> >>> > >>>>
>>> >>> > >>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> There are less than 30 right now.  Many of the roughly 90+
>>> >>> > >>>>
>>> >>> > >>>> JIRAs
>>> >>> > >>>>
>>> >>> > >>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>>> >>> > >>>>
>>> >>> > >>>> or
>>> >>> > >>>>
>>> >>> > >>>> just
>>> >>> > >>>>
>>> >>> > >>>> had fix versions removed.
>>> >>> > >>>>
>>> >>> > >>>> We will need to have a push over the next bunch of days to
>>> >>> > >>>>
>>> >>> > >>>> deal
>>> >>> > >>>>
>>> >>> > >>>> with
>>> >>> > >>>>
>>> >>> > >>>> reviewing/merging/moving the remaining items.
>>> >>> > >>>>
>>> >>> > >>>> Thanks
>>> >>> > >>>> Joe
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>>> >>> > >>>>
>>> >>> > >>>> [hidden email] <javascript:;>>
>>> >>> > >>>>
>>> >>> > >>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> Joe,
>>> >>> > >>>>
>>> >>> > >>>> There are 75 PRs currently open. Why not make a push over
>>> >>> > >>>>
>>> >>> > >>>> the
>>> >>> > >>>>
>>> >>> > >>>> next
>>> >>> > >>>>
>>> >>> > >>>> bunch
>>> >>> > >>>>
>>> >>> > >>>> of
>>> >>> > >>>>
>>> >>> > >>>> days to get them closed and then cut the release after that.
>>> >>> > >>>>
>>> >>> > >>>> Cheers,
>>> >>> > >>>>
>>> >>> > >>>> Edgardo
>>> >>> > >>>>
>>> >>> > >>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>>> >>> > >>>>
>>> >>> > >>>> [hidden email]
>>> >>> > >>>>
>>> >>> > >>>> <javascript:;>>
>>> >>> > >>>>
>>> >>> > >>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> Team,
>>> >>> > >>>>
>>> >>> > >>>> There have been a ton of bugs fixed a few nice features.  I
>>> >>> > >>>>
>>> >>> > >>>> would
>>> >>> > >>>>
>>> >>> > >>>> like
>>> >>> > >>>>
>>> >>> > >>>> to move to get Apache NiFi 1.1.0 release going pretty much
>>> >>> > >>>>
>>> >>> > >>>> based
>>> >>> > >>>>
>>> >>> > >>>> on
>>> >>> > >>>>
>>> >>> > >>>> where we are now and plan to move most tickets to a new
>>> >>> > >>>>
>>> >>> > >>>> Apache
>>> >>> > >>>>
>>> >>> > >>>> NiFi
>>> >>> > >>>>
>>> >>> > >>>> 1.2.0 version.  We can try to get back on our roughly 6-8
>>> >>> > >>>>
>>> >>> > >>>> week
>>> >>> > >>>>
>>> >>> > >>>> release
>>> >>> > >>>>
>>> >>> > >>>> schedule and shoot for a mid to late Nov release for NiFi
>>> >>> > >>>>
>>> >>> > >>>> 1.2.0
>>> >>> > >>>>
>>> >>> > >>>> this
>>> >>> > >>>>
>>> >>> > >>>> way as well. Please advise if anyone has any other views on
>>> >>> > >>>>
>>> >>> > >>>> this. In
>>> >>> > >>>>
>>> >>> > >>>> the mean time I'll get the wheels in motion so you'll be
>>> >>> > >>>>
>>> >>> > >>>> seeing a
>>> >>> > >>>>
>>> >>> > >>>> lot
>>> >>> > >>>>
>>> >>> > >>>> of JIRA/issue updates to move version around.
>>> >>> > >>>>
>>> >>> > >>>> Thanks
>>> >>> > >>>> Joe
>>> >>> > >>>>
>>> >>> > >>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>>> >>> > >>>>
>>> >>> > >>>> [hidden email]
>>> >>> > >>>>
>>> >>> > >>>> <javascript:;>>
>>> >>> > >>>>
>>> >>> > >>>> wrote:
>>> >>> > >>>>
>>> >>> > >>>> Sounds good Joe. I have no issue to you doing the rm'ing
>>> >>> > >>>>
>>> >>> > >>>> for
>>> >>> > >>>>
>>> >>> > >>>> it.
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <[hidden email]
>>> >>> > >>>>
>>> >>> > >>>> <javascript:;>> wrote:
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> Team,
>>> >>> > >>>>
>>> >>> > >>>> There are a lot of great fixes and improvements on the
>>> >>> > >>>>
>>> >>> > >>>> master
>>> >>> > >>>>
>>> >>> > >>>> line
>>> >>> > >>>>
>>> >>> > >>>> now
>>> >>> > >>>>
>>> >>> > >>>> and we're at a good time window to start pushing for a
>>> >>> > >>>>
>>> >>> > >>>> release.
>>> >>> > >>>>
>>> >>> > >>>> There
>>> >>> > >>>>
>>> >>> > >>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>>> >>> > >>>>
>>> >>> > >>>> are
>>> >>> > >>>>
>>> >>> > >>>> open.
>>> >>> > >>>>
>>> >>> > >>>> I'm
>>> >>> > >>>>
>>> >>> > >>>> going to go through them and remove fix versions where
>>> >>> > >>>>
>>> >>> > >>>> appropriate.
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> I'm happy to take on RM task for this release though if
>>> >>> > >>>>
>>> >>> > >>>> someone
>>> >>> > >>>>
>>> >>> > >>>> else
>>> >>> > >>>>
>>> >>> > >>>> would like to take that on please advise.
>>> >>> > >>>>
>>> >>> > >>>> Thanks
>>> >>> > >>>> Joe
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> --
>>> >>> > >>>> Cheers,
>>> >>> > >>>>
>>> >>> > >>>> Edgardo
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> --
>>> >>> > >>>> Cheers,
>>> >>> > >>>>
>>> >>> > >>>> Edgardo
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> --
>>> >>> > >>>> Cheers,
>>> >>> > >>>>
>>> >>> > >>>> Edgardo
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>> --
>>> >>> > >>>> Cheers,
>>> >>> > >>>>
>>> >>> > >>>> Edgardo
>>> >>> > >>>>
>>> >>> > >>>> Sent from Gmail Mobile
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> > >>>>
>>> >>> >
>>> >>>
>>>

123