[DISCUSS] First Release of NiFi Registry

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSS] First Release of NiFi Registry

Bryan Bende
Hey folks,

There has been a lot of great work done on the NiFi Registry [1] and I
think we are probably very close to an initial release focused on
storing "versioned flows".

Since NiFi will have a dependency on client code provided by the
registry, the first release of the registry would need to occur before
the first release of NiFi that integrates with it. The work on the
NiFi side is being done as part of NIFI-4436, which can be followed
along on PR 2219 [2].

Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I would
propose the first release should be 0.1.0.

Let me know if anyone has any thoughts or comments.  I'm happy to act
as RM if no one else is interested in doing so, and we can start the
process of going through JIRA to see what is left.

Thanks,

Bryan

[1] https://nifi.apache.org/registry.html
[2] https://github.com/apache/nifi/pull/2219
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First Release of NiFi Registry

Joe Witt
Bryan - very exciting and awesome.  Having experimented with the
registry on the JIRAs/PRs you mention I must say this is going to be a
huge step forward for NiFi!

Since we'll also be doing a NiFi release soon (1.5.0?) I am happy to
volunteer to RM that as well if needed.

Thanks

On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende <[hidden email]> wrote:

> Hey folks,
>
> There has been a lot of great work done on the NiFi Registry [1] and I
> think we are probably very close to an initial release focused on
> storing "versioned flows".
>
> Since NiFi will have a dependency on client code provided by the
> registry, the first release of the registry would need to occur before
> the first release of NiFi that integrates with it. The work on the
> NiFi side is being done as part of NIFI-4436, which can be followed
> along on PR 2219 [2].
>
> Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I would
> propose the first release should be 0.1.0.
>
> Let me know if anyone has any thoughts or comments.  I'm happy to act
> as RM if no one else is interested in doing so, and we can start the
> process of going through JIRA to see what is left.
>
> Thanks,
>
> Bryan
>
> [1] https://nifi.apache.org/registry.html
> [2] https://github.com/apache/nifi/pull/2219
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First Release of NiFi Registry

Kevin Doran
Thanks for kicking off this discussion thread, Bryan.

I support prepping a release of NiFi Registry and making it version 0.1.0 as you propose.

Thanks!
Kevin

On 12/7/17, 10:45, "Joe Witt" <[hidden email]> wrote:

    Bryan - very exciting and awesome.  Having experimented with the
    registry on the JIRAs/PRs you mention I must say this is going to be a
    huge step forward for NiFi!
   
    Since we'll also be doing a NiFi release soon (1.5.0?) I am happy to
    volunteer to RM that as well if needed.
   
    Thanks
   
    On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende <[hidden email]> wrote:
    > Hey folks,
    >
    > There has been a lot of great work done on the NiFi Registry [1] and I
    > think we are probably very close to an initial release focused on
    > storing "versioned flows".
    >
    > Since NiFi will have a dependency on client code provided by the
    > registry, the first release of the registry would need to occur before
    > the first release of NiFi that integrates with it. The work on the
    > NiFi side is being done as part of NIFI-4436, which can be followed
    > along on PR 2219 [2].
    >
    > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I would
    > propose the first release should be 0.1.0.
    >
    > Let me know if anyone has any thoughts or comments.  I'm happy to act
    > as RM if no one else is interested in doing so, and we can start the
    > process of going through JIRA to see what is left.
    >
    > Thanks,
    >
    > Bryan
    >
    > [1] https://nifi.apache.org/registry.html
    > [2] https://github.com/apache/nifi/pull/2219
   


Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First Release of NiFi Registry

Russell Bateman-2
Our down-stream users are excited at the prospect of using this registry
capability for their flows. So, we're eager to see it integrated into
the earliest NiFi version you can choose (1.5.0?).

Russ

On 12/07/2017 08:49 AM, Kevin Doran wrote:

> Thanks for kicking off this discussion thread, Bryan.
>
> I support prepping a release of NiFi Registry and making it version 0.1.0 as you propose.
>
> Thanks!
> Kevin
>
> On 12/7/17, 10:45, "Joe Witt" <[hidden email]> wrote:
>
>      Bryan - very exciting and awesome.  Having experimented with the
>      registry on the JIRAs/PRs you mention I must say this is going to be a
>      huge step forward for NiFi!
>      
>      Since we'll also be doing a NiFi release soon (1.5.0?) I am happy to
>      volunteer to RM that as well if needed.
>      
>      Thanks
>      
>      On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende <[hidden email]> wrote:
>      > Hey folks,
>      >
>      > There has been a lot of great work done on the NiFi Registry [1] and I
>      > think we are probably very close to an initial release focused on
>      > storing "versioned flows".
>      >
>      > Since NiFi will have a dependency on client code provided by the
>      > registry, the first release of the registry would need to occur before
>      > the first release of NiFi that integrates with it. The work on the
>      > NiFi side is being done as part of NIFI-4436, which can be followed
>      > along on PR 2219 [2].
>      >
>      > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I would
>      > propose the first release should be 0.1.0.
>      >
>      > Let me know if anyone has any thoughts or comments.  I'm happy to act
>      > as RM if no one else is interested in doing so, and we can start the
>      > process of going through JIRA to see what is left.
>      >
>      > Thanks,
>      >
>      > Bryan
>      >
>      > [1] https://nifi.apache.org/registry.html
>      > [2] https://github.com/apache/nifi/pull/2219
>      
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First Release of NiFi Registry

Jeff
Bryan,

+1 to getting an initial release of NiFi Registry out to the community.
Definitely a huge step in the evolution of NiFi!

On Thu, Dec 7, 2017 at 11:29 AM Russell Bateman <[hidden email]>
wrote:

> Our down-stream users are excited at the prospect of using this registry
> capability for their flows. So, we're eager to see it integrated into
> the earliest NiFi version you can choose (1.5.0?).
>
> Russ
>
> On 12/07/2017 08:49 AM, Kevin Doran wrote:
> > Thanks for kicking off this discussion thread, Bryan.
> >
> > I support prepping a release of NiFi Registry and making it version
> 0.1.0 as you propose.
> >
> > Thanks!
> > Kevin
> >
> > On 12/7/17, 10:45, "Joe Witt" <[hidden email]> wrote:
> >
> >      Bryan - very exciting and awesome.  Having experimented with the
> >      registry on the JIRAs/PRs you mention I must say this is going to
> be a
> >      huge step forward for NiFi!
> >
> >      Since we'll also be doing a NiFi release soon (1.5.0?) I am happy to
> >      volunteer to RM that as well if needed.
> >
> >      Thanks
> >
> >      On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende <[hidden email]>
> wrote:
> >      > Hey folks,
> >      >
> >      > There has been a lot of great work done on the NiFi Registry [1]
> and I
> >      > think we are probably very close to an initial release focused on
> >      > storing "versioned flows".
> >      >
> >      > Since NiFi will have a dependency on client code provided by the
> >      > registry, the first release of the registry would need to occur
> before
> >      > the first release of NiFi that integrates with it. The work on the
> >      > NiFi side is being done as part of NIFI-4436, which can be
> followed
> >      > along on PR 2219 [2].
> >      >
> >      > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I
> would
> >      > propose the first release should be 0.1.0.
> >      >
> >      > Let me know if anyone has any thoughts or comments.  I'm happy to
> act
> >      > as RM if no one else is interested in doing so, and we can start
> the
> >      > process of going through JIRA to see what is left.
> >      >
> >      > Thanks,
> >      >
> >      > Bryan
> >      >
> >      > [1] https://nifi.apache.org/registry.html
> >      > [2] https://github.com/apache/nifi/pull/2219
> >
> >
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First Release of NiFi Registry

Pierre Villard
Strong +1!!

Really impressed by all the work you guys did on the registry stuff. Very
impatient to use it in official releases!

Le 7 déc. 2017 18:52, "Jeff" <[hidden email]> a écrit :

Bryan,

+1 to getting an initial release of NiFi Registry out to the community.
Definitely a huge step in the evolution of NiFi!

On Thu, Dec 7, 2017 at 11:29 AM Russell Bateman <[hidden email]>
wrote:

> Our down-stream users are excited at the prospect of using this registry
> capability for their flows. So, we're eager to see it integrated into
> the earliest NiFi version you can choose (1.5.0?).
>
> Russ
>
> On 12/07/2017 08:49 AM, Kevin Doran wrote:
> > Thanks for kicking off this discussion thread, Bryan.
> >
> > I support prepping a release of NiFi Registry and making it version
> 0.1.0 as you propose.
> >
> > Thanks!
> > Kevin
> >
> > On 12/7/17, 10:45, "Joe Witt" <[hidden email]> wrote:
> >
> >      Bryan - very exciting and awesome.  Having experimented with the
> >      registry on the JIRAs/PRs you mention I must say this is going to
> be a
> >      huge step forward for NiFi!
> >
> >      Since we'll also be doing a NiFi release soon (1.5.0?) I am happy
to

> >      volunteer to RM that as well if needed.
> >
> >      Thanks
> >
> >      On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende <[hidden email]>
> wrote:
> >      > Hey folks,
> >      >
> >      > There has been a lot of great work done on the NiFi Registry [1]
> and I
> >      > think we are probably very close to an initial release focused on
> >      > storing "versioned flows".
> >      >
> >      > Since NiFi will have a dependency on client code provided by the
> >      > registry, the first release of the registry would need to occur
> before
> >      > the first release of NiFi that integrates with it. The work on
the

> >      > NiFi side is being done as part of NIFI-4436, which can be
> followed
> >      > along on PR 2219 [2].
> >      >
> >      > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I
> would
> >      > propose the first release should be 0.1.0.
> >      >
> >      > Let me know if anyone has any thoughts or comments.  I'm happy to
> act
> >      > as RM if no one else is interested in doing so, and we can start
> the
> >      > process of going through JIRA to see what is left.
> >      >
> >      > Thanks,
> >      >
> >      > Bryan
> >      >
> >      > [1] https://nifi.apache.org/registry.html
> >      > [2] https://github.com/apache/nifi/pull/2219
> >
> >
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First Release of NiFi Registry

Mike Thomsen
Is there a good description/detail page somewhere going over the registry?

On Thu, Dec 7, 2017 at 2:06 PM, Pierre Villard <[hidden email]>
wrote:

> Strong +1!!
>
> Really impressed by all the work you guys did on the registry stuff. Very
> impatient to use it in official releases!
>
> Le 7 déc. 2017 18:52, "Jeff" <[hidden email]> a écrit :
>
> Bryan,
>
> +1 to getting an initial release of NiFi Registry out to the community.
> Definitely a huge step in the evolution of NiFi!
>
> On Thu, Dec 7, 2017 at 11:29 AM Russell Bateman <[hidden email]>
> wrote:
>
> > Our down-stream users are excited at the prospect of using this registry
> > capability for their flows. So, we're eager to see it integrated into
> > the earliest NiFi version you can choose (1.5.0?).
> >
> > Russ
> >
> > On 12/07/2017 08:49 AM, Kevin Doran wrote:
> > > Thanks for kicking off this discussion thread, Bryan.
> > >
> > > I support prepping a release of NiFi Registry and making it version
> > 0.1.0 as you propose.
> > >
> > > Thanks!
> > > Kevin
> > >
> > > On 12/7/17, 10:45, "Joe Witt" <[hidden email]> wrote:
> > >
> > >      Bryan - very exciting and awesome.  Having experimented with the
> > >      registry on the JIRAs/PRs you mention I must say this is going to
> > be a
> > >      huge step forward for NiFi!
> > >
> > >      Since we'll also be doing a NiFi release soon (1.5.0?) I am happy
> to
> > >      volunteer to RM that as well if needed.
> > >
> > >      Thanks
> > >
> > >      On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende <[hidden email]>
> > wrote:
> > >      > Hey folks,
> > >      >
> > >      > There has been a lot of great work done on the NiFi Registry [1]
> > and I
> > >      > think we are probably very close to an initial release focused
> on
> > >      > storing "versioned flows".
> > >      >
> > >      > Since NiFi will have a dependency on client code provided by the
> > >      > registry, the first release of the registry would need to occur
> > before
> > >      > the first release of NiFi that integrates with it. The work on
> the
> > >      > NiFi side is being done as part of NIFI-4436, which can be
> > followed
> > >      > along on PR 2219 [2].
> > >      >
> > >      > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I
> > would
> > >      > propose the first release should be 0.1.0.
> > >      >
> > >      > Let me know if anyone has any thoughts or comments.  I'm happy
> to
> > act
> > >      > as RM if no one else is interested in doing so, and we can start
> > the
> > >      > process of going through JIRA to see what is left.
> > >      >
> > >      > Thanks,
> > >      >
> > >      > Bryan
> > >      >
> > >      > [1] https://nifi.apache.org/registry.html
> > >      > [2] https://github.com/apache/nifi/pull/2219
> > >
> > >
> > >
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First Release of NiFi Registry

Bryan Bende
Mike,

You brought up a good point... documentation is one of the things that
still needs to be done.

There is some information that might be helpful though...

I would suggest reading this Wiki page for the feature proposal of
"Configuration Management of Flows" [1].

There is also a JIRA from a few months ago with initial mock ups for
the registry UI [2].

As part of the RC I can provide some instructions on how it can be
tested with NiFi using PR 2219.

Thanks,

Bryan

[1] https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows
[2] https://issues.apache.org/jira/browse/NIFIREG-3


On Thu, Dec 7, 2017 at 8:21 PM, Mike Thomsen <[hidden email]> wrote:

> Is there a good description/detail page somewhere going over the registry?
>
> On Thu, Dec 7, 2017 at 2:06 PM, Pierre Villard <[hidden email]>
> wrote:
>
>> Strong +1!!
>>
>> Really impressed by all the work you guys did on the registry stuff. Very
>> impatient to use it in official releases!
>>
>> Le 7 déc. 2017 18:52, "Jeff" <[hidden email]> a écrit :
>>
>> Bryan,
>>
>> +1 to getting an initial release of NiFi Registry out to the community.
>> Definitely a huge step in the evolution of NiFi!
>>
>> On Thu, Dec 7, 2017 at 11:29 AM Russell Bateman <[hidden email]>
>> wrote:
>>
>> > Our down-stream users are excited at the prospect of using this registry
>> > capability for their flows. So, we're eager to see it integrated into
>> > the earliest NiFi version you can choose (1.5.0?).
>> >
>> > Russ
>> >
>> > On 12/07/2017 08:49 AM, Kevin Doran wrote:
>> > > Thanks for kicking off this discussion thread, Bryan.
>> > >
>> > > I support prepping a release of NiFi Registry and making it version
>> > 0.1.0 as you propose.
>> > >
>> > > Thanks!
>> > > Kevin
>> > >
>> > > On 12/7/17, 10:45, "Joe Witt" <[hidden email]> wrote:
>> > >
>> > >      Bryan - very exciting and awesome.  Having experimented with the
>> > >      registry on the JIRAs/PRs you mention I must say this is going to
>> > be a
>> > >      huge step forward for NiFi!
>> > >
>> > >      Since we'll also be doing a NiFi release soon (1.5.0?) I am happy
>> to
>> > >      volunteer to RM that as well if needed.
>> > >
>> > >      Thanks
>> > >
>> > >      On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende <[hidden email]>
>> > wrote:
>> > >      > Hey folks,
>> > >      >
>> > >      > There has been a lot of great work done on the NiFi Registry [1]
>> > and I
>> > >      > think we are probably very close to an initial release focused
>> on
>> > >      > storing "versioned flows".
>> > >      >
>> > >      > Since NiFi will have a dependency on client code provided by the
>> > >      > registry, the first release of the registry would need to occur
>> > before
>> > >      > the first release of NiFi that integrates with it. The work on
>> the
>> > >      > NiFi side is being done as part of NIFI-4436, which can be
>> > followed
>> > >      > along on PR 2219 [2].
>> > >      >
>> > >      > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I
>> > would
>> > >      > propose the first release should be 0.1.0.
>> > >      >
>> > >      > Let me know if anyone has any thoughts or comments.  I'm happy
>> to
>> > act
>> > >      > as RM if no one else is interested in doing so, and we can start
>> > the
>> > >      > process of going through JIRA to see what is left.
>> > >      >
>> > >      > Thanks,
>> > >      >
>> > >      > Bryan
>> > >      >
>> > >      > [1] https://nifi.apache.org/registry.html
>> > >      > [2] https://github.com/apache/nifi/pull/2219
>> > >
>> > >
>> > >
>> >
>> >
>>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First Release of NiFi Registry

Bryan Bende
Just wanted to give an update on this...

Great progress has been made in the last two weeks in terms of getting
ready for an RC. Still a few outstanding items, but I think we could
have those wrapped up soon and kick out an RC some time next week.
Depending when everything is ready we can adjust the voting period if
needed to account for holidays and make sure there is adequate time
for review.

In the mean time, I encourage anyone who is interested to give it a
try. Here is some info about how to get started...

1) Get the code for the registry

The Apache repo is here:

https://git-wip-us.apache.org/repos/asf/nifi-registry.git

The github repo is here if you prefer to fork that:

https://github.com/apache/nifi-registry

2) Build the registry code

cd nifi-registry
mvn clean install

3) Start the registry

cd nifi-registry-assembly/target/nifi-registry-0.0.1-SNAPSHOT-bin/nifi-registry-0.0.1-SNAPSHOT/
./bin/nifi-registry.sh start

4) Create a bucket in the registry

- Go to the registry UI at http://localhost:18080/nifi-registry
- Click the tool icon in the top right corner
- Click New Bucket from the bucket table
- Enter a name and click create

5) Get the NiFi PR which adds the support for integrating with the registry

https://github.com/apache/nifi/pull/2219

Build that PR like normal.

NOTE: That you must have already built nifi-registry with "mvn clean
install" in order to build this PR because it depends on snapshot JARs
being in your local Maven repo.

6) Tell NiFi about your local registry instance

- Go the controller settings for NiFi from the top-right menu
- Select the Registry Clients tab
- Add a new Registry Client giving it a name and the url of
http://localhost:18080

7) Create a process group and place it under version control

- Right click on the PG and select the Version menu
- Select Start Version Control
- Choose the registry instance and bucket you want to use
- Enter a name, description, and comment

8) Go back to the registry and refresh the main page and you should
see the versioned flow you just saved

9) Import a new PG from a versioned flow

- Drag on a new PG like normal
- Instead of entering a name, click the Import link
- Now choose the flow you saved before

You should have a second identical PG now.

From there you can try making changes to one of them, view local
changes, revert changes, save a version 2, upgrade the other one to
version 2, etc.

Hope that helps.

-Bryan

On Fri, Dec 8, 2017 at 9:19 AM, Bryan Bende <[hidden email]> wrote:

> Mike,
>
> You brought up a good point... documentation is one of the things that
> still needs to be done.
>
> There is some information that might be helpful though...
>
> I would suggest reading this Wiki page for the feature proposal of
> "Configuration Management of Flows" [1].
>
> There is also a JIRA from a few months ago with initial mock ups for
> the registry UI [2].
>
> As part of the RC I can provide some instructions on how it can be
> tested with NiFi using PR 2219.
>
> Thanks,
>
> Bryan
>
> [1] https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows
> [2] https://issues.apache.org/jira/browse/NIFIREG-3
>
>
> On Thu, Dec 7, 2017 at 8:21 PM, Mike Thomsen <[hidden email]> wrote:
>> Is there a good description/detail page somewhere going over the registry?
>>
>> On Thu, Dec 7, 2017 at 2:06 PM, Pierre Villard <[hidden email]>
>> wrote:
>>
>>> Strong +1!!
>>>
>>> Really impressed by all the work you guys did on the registry stuff. Very
>>> impatient to use it in official releases!
>>>
>>> Le 7 déc. 2017 18:52, "Jeff" <[hidden email]> a écrit :
>>>
>>> Bryan,
>>>
>>> +1 to getting an initial release of NiFi Registry out to the community.
>>> Definitely a huge step in the evolution of NiFi!
>>>
>>> On Thu, Dec 7, 2017 at 11:29 AM Russell Bateman <[hidden email]>
>>> wrote:
>>>
>>> > Our down-stream users are excited at the prospect of using this registry
>>> > capability for their flows. So, we're eager to see it integrated into
>>> > the earliest NiFi version you can choose (1.5.0?).
>>> >
>>> > Russ
>>> >
>>> > On 12/07/2017 08:49 AM, Kevin Doran wrote:
>>> > > Thanks for kicking off this discussion thread, Bryan.
>>> > >
>>> > > I support prepping a release of NiFi Registry and making it version
>>> > 0.1.0 as you propose.
>>> > >
>>> > > Thanks!
>>> > > Kevin
>>> > >
>>> > > On 12/7/17, 10:45, "Joe Witt" <[hidden email]> wrote:
>>> > >
>>> > >      Bryan - very exciting and awesome.  Having experimented with the
>>> > >      registry on the JIRAs/PRs you mention I must say this is going to
>>> > be a
>>> > >      huge step forward for NiFi!
>>> > >
>>> > >      Since we'll also be doing a NiFi release soon (1.5.0?) I am happy
>>> to
>>> > >      volunteer to RM that as well if needed.
>>> > >
>>> > >      Thanks
>>> > >
>>> > >      On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende <[hidden email]>
>>> > wrote:
>>> > >      > Hey folks,
>>> > >      >
>>> > >      > There has been a lot of great work done on the NiFi Registry [1]
>>> > and I
>>> > >      > think we are probably very close to an initial release focused
>>> on
>>> > >      > storing "versioned flows".
>>> > >      >
>>> > >      > Since NiFi will have a dependency on client code provided by the
>>> > >      > registry, the first release of the registry would need to occur
>>> > before
>>> > >      > the first release of NiFi that integrates with it. The work on
>>> the
>>> > >      > NiFi side is being done as part of NIFI-4436, which can be
>>> > followed
>>> > >      > along on PR 2219 [2].
>>> > >      >
>>> > >      > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I
>>> > would
>>> > >      > propose the first release should be 0.1.0.
>>> > >      >
>>> > >      > Let me know if anyone has any thoughts or comments.  I'm happy
>>> to
>>> > act
>>> > >      > as RM if no one else is interested in doing so, and we can start
>>> > the
>>> > >      > process of going through JIRA to see what is left.
>>> > >      >
>>> > >      > Thanks,
>>> > >      >
>>> > >      > Bryan
>>> > >      >
>>> > >      > [1] https://nifi.apache.org/registry.html
>>> > >      > [2] https://github.com/apache/nifi/pull/2219
>>> > >
>>> > >
>>> > >
>>> >
>>> >
>>>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First Release of NiFi Registry

Pierre Villard
Hey guys,

Not sure that's the best place to give my feedbacks after running some
tests, let me know if I should open a new thread.

(I believe Joe P. already made some similar comments, but just in case...)

- in an unsecure environment, it's probably better to disable the "Add new
policy" button (NIFIREG-78)
- I've seen some logs that could be set to debug? “Access tokens are only
issued over HTTPS. Returning Conflict response.”, “Registry is not
configured to internally manage users, groups, or policies. Please contact
your system administrator.. Returning Conflict response.“
- general comment for NiFi UI: add tooltips on the icons of the upper
status bar? We've quite a few new icons coming with the Registry and I
guess it could help people not very familiar with it yet.
- is it possible to do a diff between two versions in the Registry UI?
- when adding a variable to a versioned PG, it does not show changes to
commit. Is it expected? (it does not to me)
- how to set a previous version as the new current one? does not seem
possible unless you stop version control and start again?
- very very minor comment, in the Registry UI, in the actions list, I'd set
"Delete" instead of "delete".

Another observation:

I have PG A containing PG B, both versioned. And I have two instances of PG
A in my NiFi UI PG A1 and PG A2.
- I deleted PG B tracking in NiFi Registry. I now have 404 errors on the
PGAx because PG B is not found in registry. All good. Then I disconnect PG
B in PG A1. PG A1 is shown as OK / up-to-date with nothing to commit.
- If I try to import a new instance of PG A, it’s not working because “The
Flow Registry with ID 893e20cc-0160-1000-8ab8-e0507c36aa94 reports that no
Flow exists with Bucket f66d8eb1-b893-41ad-974b-565bc33c8104, Flow
8d2df468-e8e4-4138-aaef-c7eadb71c2c4, Version 4”
- In PG A2, if I delete PG B, then it shows local change but I cannot
revert local changes: "Failed to retrieve flow with Flow Registry in order
to calculate local differences due to Error retrieving flow snapshot:
Versioned flow does not exist with identifier
08e85785-cb41-4cae-a516-6b4d3506960e"
- In the end I have to delete PG B, commit changes to get everything back
to normal. I’m wondering if disconnecting PG B shouldn’t be considered as a
local change to be committed? Because, I could be in a situation where I
don’t want to delete PG B, I just want to stop version control on it, no?

I'll run some more tests in secured environments.

Pierre




2017-12-21 18:50 GMT+01:00 Bryan Bende <[hidden email]>:

> Just wanted to give an update on this...
>
> Great progress has been made in the last two weeks in terms of getting
> ready for an RC. Still a few outstanding items, but I think we could
> have those wrapped up soon and kick out an RC some time next week.
> Depending when everything is ready we can adjust the voting period if
> needed to account for holidays and make sure there is adequate time
> for review.
>
> In the mean time, I encourage anyone who is interested to give it a
> try. Here is some info about how to get started...
>
> 1) Get the code for the registry
>
> The Apache repo is here:
>
> https://git-wip-us.apache.org/repos/asf/nifi-registry.git
>
> The github repo is here if you prefer to fork that:
>
> https://github.com/apache/nifi-registry
>
> 2) Build the registry code
>
> cd nifi-registry
> mvn clean install
>
> 3) Start the registry
>
> cd nifi-registry-assembly/target/nifi-registry-0.0.1-SNAPSHOT-
> bin/nifi-registry-0.0.1-SNAPSHOT/
> ./bin/nifi-registry.sh start
>
> 4) Create a bucket in the registry
>
> - Go to the registry UI at http://localhost:18080/nifi-registry
> - Click the tool icon in the top right corner
> - Click New Bucket from the bucket table
> - Enter a name and click create
>
> 5) Get the NiFi PR which adds the support for integrating with the registry
>
> https://github.com/apache/nifi/pull/2219
>
> Build that PR like normal.
>
> NOTE: That you must have already built nifi-registry with "mvn clean
> install" in order to build this PR because it depends on snapshot JARs
> being in your local Maven repo.
>
> 6) Tell NiFi about your local registry instance
>
> - Go the controller settings for NiFi from the top-right menu
> - Select the Registry Clients tab
> - Add a new Registry Client giving it a name and the url of
> http://localhost:18080
>
> 7) Create a process group and place it under version control
>
> - Right click on the PG and select the Version menu
> - Select Start Version Control
> - Choose the registry instance and bucket you want to use
> - Enter a name, description, and comment
>
> 8) Go back to the registry and refresh the main page and you should
> see the versioned flow you just saved
>
> 9) Import a new PG from a versioned flow
>
> - Drag on a new PG like normal
> - Instead of entering a name, click the Import link
> - Now choose the flow you saved before
>
> You should have a second identical PG now.
>
> From there you can try making changes to one of them, view local
> changes, revert changes, save a version 2, upgrade the other one to
> version 2, etc.
>
> Hope that helps.
>
> -Bryan
>
> On Fri, Dec 8, 2017 at 9:19 AM, Bryan Bende <[hidden email]> wrote:
> > Mike,
> >
> > You brought up a good point... documentation is one of the things that
> > still needs to be done.
> >
> > There is some information that might be helpful though...
> >
> > I would suggest reading this Wiki page for the feature proposal of
> > "Configuration Management of Flows" [1].
> >
> > There is also a JIRA from a few months ago with initial mock ups for
> > the registry UI [2].
> >
> > As part of the RC I can provide some instructions on how it can be
> > tested with NiFi using PR 2219.
> >
> > Thanks,
> >
> > Bryan
> >
> > [1] https://cwiki.apache.org/confluence/display/NIFI/
> Configuration+Management+of+Flows
> > [2] https://issues.apache.org/jira/browse/NIFIREG-3
> >
> >
> > On Thu, Dec 7, 2017 at 8:21 PM, Mike Thomsen <[hidden email]>
> wrote:
> >> Is there a good description/detail page somewhere going over the
> registry?
> >>
> >> On Thu, Dec 7, 2017 at 2:06 PM, Pierre Villard <
> [hidden email]>
> >> wrote:
> >>
> >>> Strong +1!!
> >>>
> >>> Really impressed by all the work you guys did on the registry stuff.
> Very
> >>> impatient to use it in official releases!
> >>>
> >>> Le 7 déc. 2017 18:52, "Jeff" <[hidden email]> a écrit :
> >>>
> >>> Bryan,
> >>>
> >>> +1 to getting an initial release of NiFi Registry out to the community.
> >>> Definitely a huge step in the evolution of NiFi!
> >>>
> >>> On Thu, Dec 7, 2017 at 11:29 AM Russell Bateman <[hidden email]
> >
> >>> wrote:
> >>>
> >>> > Our down-stream users are excited at the prospect of using this
> registry
> >>> > capability for their flows. So, we're eager to see it integrated into
> >>> > the earliest NiFi version you can choose (1.5.0?).
> >>> >
> >>> > Russ
> >>> >
> >>> > On 12/07/2017 08:49 AM, Kevin Doran wrote:
> >>> > > Thanks for kicking off this discussion thread, Bryan.
> >>> > >
> >>> > > I support prepping a release of NiFi Registry and making it version
> >>> > 0.1.0 as you propose.
> >>> > >
> >>> > > Thanks!
> >>> > > Kevin
> >>> > >
> >>> > > On 12/7/17, 10:45, "Joe Witt" <[hidden email]> wrote:
> >>> > >
> >>> > >      Bryan - very exciting and awesome.  Having experimented with
> the
> >>> > >      registry on the JIRAs/PRs you mention I must say this is
> going to
> >>> > be a
> >>> > >      huge step forward for NiFi!
> >>> > >
> >>> > >      Since we'll also be doing a NiFi release soon (1.5.0?) I am
> happy
> >>> to
> >>> > >      volunteer to RM that as well if needed.
> >>> > >
> >>> > >      Thanks
> >>> > >
> >>> > >      On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende <
> [hidden email]>
> >>> > wrote:
> >>> > >      > Hey folks,
> >>> > >      >
> >>> > >      > There has been a lot of great work done on the NiFi
> Registry [1]
> >>> > and I
> >>> > >      > think we are probably very close to an initial release
> focused
> >>> on
> >>> > >      > storing "versioned flows".
> >>> > >      >
> >>> > >      > Since NiFi will have a dependency on client code provided
> by the
> >>> > >      > registry, the first release of the registry would need to
> occur
> >>> > before
> >>> > >      > the first release of NiFi that integrates with it. The work
> on
> >>> the
> >>> > >      > NiFi side is being done as part of NIFI-4436, which can be
> >>> > followed
> >>> > >      > along on PR 2219 [2].
> >>> > >      >
> >>> > >      > Currently nifi-registry master is set to 0.0.1-SNAPSHOT,
> but I
> >>> > would
> >>> > >      > propose the first release should be 0.1.0.
> >>> > >      >
> >>> > >      > Let me know if anyone has any thoughts or comments.  I'm
> happy
> >>> to
> >>> > act
> >>> > >      > as RM if no one else is interested in doing so, and we can
> start
> >>> > the
> >>> > >      > process of going through JIRA to see what is left.
> >>> > >      >
> >>> > >      > Thanks,
> >>> > >      >
> >>> > >      > Bryan
> >>> > >      >
> >>> > >      > [1] https://nifi.apache.org/registry.html
> >>> > >      > [2] https://github.com/apache/nifi/pull/2219
> >>> > >
> >>> > >
> >>> > >
> >>> >
> >>> >
> >>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First Release of NiFi Registry

Bryan Bende
Looks like the work for the initial docs has wrapped up and we are in
a good place to kick out an RC.

I'll start pulling things together and should be able to get something
out tomorrow.

Pierre,

I created a JIRA [1] to capture an improvement for the scenario you
described with the nested process groups.

As you said, I think most of the other stuff has been captured in
JIRA's created by Joe P.

Thanks,

Bryan

[1] https://issues.apache.org/jira/browse/NIFIREG-86

On Mon, Dec 25, 2017 at 5:40 AM, Pierre Villard
<[hidden email]> wrote:

> Hey guys,
>
> Not sure that's the best place to give my feedbacks after running some
> tests, let me know if I should open a new thread.
>
> (I believe Joe P. already made some similar comments, but just in case...)
>
> - in an unsecure environment, it's probably better to disable the "Add new
> policy" button (NIFIREG-78)
> - I've seen some logs that could be set to debug? “Access tokens are only
> issued over HTTPS. Returning Conflict response.”, “Registry is not
> configured to internally manage users, groups, or policies. Please contact
> your system administrator.. Returning Conflict response.“
> - general comment for NiFi UI: add tooltips on the icons of the upper
> status bar? We've quite a few new icons coming with the Registry and I
> guess it could help people not very familiar with it yet.
> - is it possible to do a diff between two versions in the Registry UI?
> - when adding a variable to a versioned PG, it does not show changes to
> commit. Is it expected? (it does not to me)
> - how to set a previous version as the new current one? does not seem
> possible unless you stop version control and start again?
> - very very minor comment, in the Registry UI, in the actions list, I'd set
> "Delete" instead of "delete".
>
> Another observation:
>
> I have PG A containing PG B, both versioned. And I have two instances of PG
> A in my NiFi UI PG A1 and PG A2.
> - I deleted PG B tracking in NiFi Registry. I now have 404 errors on the
> PGAx because PG B is not found in registry. All good. Then I disconnect PG
> B in PG A1. PG A1 is shown as OK / up-to-date with nothing to commit.
> - If I try to import a new instance of PG A, it’s not working because “The
> Flow Registry with ID 893e20cc-0160-1000-8ab8-e0507c36aa94 reports that no
> Flow exists with Bucket f66d8eb1-b893-41ad-974b-565bc33c8104, Flow
> 8d2df468-e8e4-4138-aaef-c7eadb71c2c4, Version 4”
> - In PG A2, if I delete PG B, then it shows local change but I cannot
> revert local changes: "Failed to retrieve flow with Flow Registry in order
> to calculate local differences due to Error retrieving flow snapshot:
> Versioned flow does not exist with identifier
> 08e85785-cb41-4cae-a516-6b4d3506960e"
> - In the end I have to delete PG B, commit changes to get everything back
> to normal. I’m wondering if disconnecting PG B shouldn’t be considered as a
> local change to be committed? Because, I could be in a situation where I
> don’t want to delete PG B, I just want to stop version control on it, no?
>
> I'll run some more tests in secured environments.
>
> Pierre
>
>
>
>
> 2017-12-21 18:50 GMT+01:00 Bryan Bende <[hidden email]>:
>
>> Just wanted to give an update on this...
>>
>> Great progress has been made in the last two weeks in terms of getting
>> ready for an RC. Still a few outstanding items, but I think we could
>> have those wrapped up soon and kick out an RC some time next week.
>> Depending when everything is ready we can adjust the voting period if
>> needed to account for holidays and make sure there is adequate time
>> for review.
>>
>> In the mean time, I encourage anyone who is interested to give it a
>> try. Here is some info about how to get started...
>>
>> 1) Get the code for the registry
>>
>> The Apache repo is here:
>>
>> https://git-wip-us.apache.org/repos/asf/nifi-registry.git
>>
>> The github repo is here if you prefer to fork that:
>>
>> https://github.com/apache/nifi-registry
>>
>> 2) Build the registry code
>>
>> cd nifi-registry
>> mvn clean install
>>
>> 3) Start the registry
>>
>> cd nifi-registry-assembly/target/nifi-registry-0.0.1-SNAPSHOT-
>> bin/nifi-registry-0.0.1-SNAPSHOT/
>> ./bin/nifi-registry.sh start
>>
>> 4) Create a bucket in the registry
>>
>> - Go to the registry UI at http://localhost:18080/nifi-registry
>> - Click the tool icon in the top right corner
>> - Click New Bucket from the bucket table
>> - Enter a name and click create
>>
>> 5) Get the NiFi PR which adds the support for integrating with the registry
>>
>> https://github.com/apache/nifi/pull/2219
>>
>> Build that PR like normal.
>>
>> NOTE: That you must have already built nifi-registry with "mvn clean
>> install" in order to build this PR because it depends on snapshot JARs
>> being in your local Maven repo.
>>
>> 6) Tell NiFi about your local registry instance
>>
>> - Go the controller settings for NiFi from the top-right menu
>> - Select the Registry Clients tab
>> - Add a new Registry Client giving it a name and the url of
>> http://localhost:18080
>>
>> 7) Create a process group and place it under version control
>>
>> - Right click on the PG and select the Version menu
>> - Select Start Version Control
>> - Choose the registry instance and bucket you want to use
>> - Enter a name, description, and comment
>>
>> 8) Go back to the registry and refresh the main page and you should
>> see the versioned flow you just saved
>>
>> 9) Import a new PG from a versioned flow
>>
>> - Drag on a new PG like normal
>> - Instead of entering a name, click the Import link
>> - Now choose the flow you saved before
>>
>> You should have a second identical PG now.
>>
>> From there you can try making changes to one of them, view local
>> changes, revert changes, save a version 2, upgrade the other one to
>> version 2, etc.
>>
>> Hope that helps.
>>
>> -Bryan
>>
>> On Fri, Dec 8, 2017 at 9:19 AM, Bryan Bende <[hidden email]> wrote:
>> > Mike,
>> >
>> > You brought up a good point... documentation is one of the things that
>> > still needs to be done.
>> >
>> > There is some information that might be helpful though...
>> >
>> > I would suggest reading this Wiki page for the feature proposal of
>> > "Configuration Management of Flows" [1].
>> >
>> > There is also a JIRA from a few months ago with initial mock ups for
>> > the registry UI [2].
>> >
>> > As part of the RC I can provide some instructions on how it can be
>> > tested with NiFi using PR 2219.
>> >
>> > Thanks,
>> >
>> > Bryan
>> >
>> > [1] https://cwiki.apache.org/confluence/display/NIFI/
>> Configuration+Management+of+Flows
>> > [2] https://issues.apache.org/jira/browse/NIFIREG-3
>> >
>> >
>> > On Thu, Dec 7, 2017 at 8:21 PM, Mike Thomsen <[hidden email]>
>> wrote:
>> >> Is there a good description/detail page somewhere going over the
>> registry?
>> >>
>> >> On Thu, Dec 7, 2017 at 2:06 PM, Pierre Villard <
>> [hidden email]>
>> >> wrote:
>> >>
>> >>> Strong +1!!
>> >>>
>> >>> Really impressed by all the work you guys did on the registry stuff.
>> Very
>> >>> impatient to use it in official releases!
>> >>>
>> >>> Le 7 déc. 2017 18:52, "Jeff" <[hidden email]> a écrit :
>> >>>
>> >>> Bryan,
>> >>>
>> >>> +1 to getting an initial release of NiFi Registry out to the community.
>> >>> Definitely a huge step in the evolution of NiFi!
>> >>>
>> >>> On Thu, Dec 7, 2017 at 11:29 AM Russell Bateman <[hidden email]
>> >
>> >>> wrote:
>> >>>
>> >>> > Our down-stream users are excited at the prospect of using this
>> registry
>> >>> > capability for their flows. So, we're eager to see it integrated into
>> >>> > the earliest NiFi version you can choose (1.5.0?).
>> >>> >
>> >>> > Russ
>> >>> >
>> >>> > On 12/07/2017 08:49 AM, Kevin Doran wrote:
>> >>> > > Thanks for kicking off this discussion thread, Bryan.
>> >>> > >
>> >>> > > I support prepping a release of NiFi Registry and making it version
>> >>> > 0.1.0 as you propose.
>> >>> > >
>> >>> > > Thanks!
>> >>> > > Kevin
>> >>> > >
>> >>> > > On 12/7/17, 10:45, "Joe Witt" <[hidden email]> wrote:
>> >>> > >
>> >>> > >      Bryan - very exciting and awesome.  Having experimented with
>> the
>> >>> > >      registry on the JIRAs/PRs you mention I must say this is
>> going to
>> >>> > be a
>> >>> > >      huge step forward for NiFi!
>> >>> > >
>> >>> > >      Since we'll also be doing a NiFi release soon (1.5.0?) I am
>> happy
>> >>> to
>> >>> > >      volunteer to RM that as well if needed.
>> >>> > >
>> >>> > >      Thanks
>> >>> > >
>> >>> > >      On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende <
>> [hidden email]>
>> >>> > wrote:
>> >>> > >      > Hey folks,
>> >>> > >      >
>> >>> > >      > There has been a lot of great work done on the NiFi
>> Registry [1]
>> >>> > and I
>> >>> > >      > think we are probably very close to an initial release
>> focused
>> >>> on
>> >>> > >      > storing "versioned flows".
>> >>> > >      >
>> >>> > >      > Since NiFi will have a dependency on client code provided
>> by the
>> >>> > >      > registry, the first release of the registry would need to
>> occur
>> >>> > before
>> >>> > >      > the first release of NiFi that integrates with it. The work
>> on
>> >>> the
>> >>> > >      > NiFi side is being done as part of NIFI-4436, which can be
>> >>> > followed
>> >>> > >      > along on PR 2219 [2].
>> >>> > >      >
>> >>> > >      > Currently nifi-registry master is set to 0.0.1-SNAPSHOT,
>> but I
>> >>> > would
>> >>> > >      > propose the first release should be 0.1.0.
>> >>> > >      >
>> >>> > >      > Let me know if anyone has any thoughts or comments.  I'm
>> happy
>> >>> to
>> >>> > act
>> >>> > >      > as RM if no one else is interested in doing so, and we can
>> start
>> >>> > the
>> >>> > >      > process of going through JIRA to see what is left.
>> >>> > >      >
>> >>> > >      > Thanks,
>> >>> > >      >
>> >>> > >      > Bryan
>> >>> > >      >
>> >>> > >      > [1] https://nifi.apache.org/registry.html
>> >>> > >      > [2] https://github.com/apache/nifi/pull/2219
>> >>> > >
>> >>> > >
>> >>> > >
>> >>> >
>> >>> >
>> >>>
>>