CM of nar via toolkit cli and regisry

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

CM of nar via toolkit cli and regisry

Aaron Rich
Hi,

I'm trying to determine the best way to CM custom nar files for sharing
between team. We are using nifi-registry for the flows and I saw there was
a new capability via the toolkit cli for "upload-bundle".

I'm trying to use that but have ran into a few issues:
1) I first wasn't getting the META-INF/docs/ in the .nar. I had built the
initial project from mvn archetype:generate with version 1.9.2. I added:
<build>
<plugins>
<plugin>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-nar-maven-plugin</artifactId>
<version>1.3.1</version>
<extensions>true</extensions>
</plugin>
</plugins>
</build>

To the base pom.xml. It has the parent of:
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-nar-bundles</artifactId>
<version>1.9.2</version>
</parent>

That got the docs in the jar:
     0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
    72 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/extension-manifest.xml
     0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/additional-details/

2)I tried to use cli then with command:
./bin/cli.sh registry upload-bundle --baseUrl http://localhost:18080/
--bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar -ebf
.custom-processor-nar-1.0-SNAPSHOT.nar

I get back "ERROR: Error executing command 'upload-bundle' : Error creating
extension bundle version: An unexpected error has occurred. Please check
the logs for additional details."

In Registry log, error is:
Exception: Unable to obtain extension info for bundle due to: Unable to
parse extension manifest due to: Unexpected end of input block; expected an
identifier
 at [row,col {unknown-source}]: [1,72]
 at [Source:
(org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
line: 1, column: 73]. Returning Internal Server Error response.
org.apache.nifi.registry.bundle.extract.BundleException: Unable to obtain
extension info for bundle due to: Unable to parse extension manifest due
to: Unexpected end of input
block; expected an identifier

Questions:
1) Should I need to set the 1.3.1 plugin do get the docs in nar? I thought
that would be automatic from parent version being 1.9.2
2) Is there an example for how to use the toolkit cli to upload-bundle?
3) Is there a better way to CM .nar? Archiva? Artifactory?

Thanks.

-Aaron
Reply | Threaded
Open this post in threaded view
|

Re: CM of nar via toolkit cli and regisry

Aaron Rich
I forgot to include that when generating the .nar, I do get the warnings of:
[WARNING] Could not generate extensions' documentation
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension
Documentation
...
Caused by: java.lang.NoSuchMethodException:
org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)

I believe this is tied to not getting the right nfi-api version? But the
parent is set to 1.9.2 so not sure how that is happening.

Thanks again.

-Aaron

On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <[hidden email]> wrote:

> Hi,
>
> I'm trying to determine the best way to CM custom nar files for sharing
> between team. We are using nifi-registry for the flows and I saw there was
> a new capability via the toolkit cli for "upload-bundle".
>
> I'm trying to use that but have ran into a few issues:
> 1) I first wasn't getting the META-INF/docs/ in the .nar. I had built the
> initial project from mvn archetype:generate with version 1.9.2. I added:
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.nifi</groupId>
> <artifactId>nifi-nar-maven-plugin</artifactId>
> <version>1.3.1</version>
> <extensions>true</extensions>
> </plugin>
> </plugins>
> </build>
>
> To the base pom.xml. It has the parent of:
> <parent>
> <groupId>org.apache.nifi</groupId>
> <artifactId>nifi-nar-bundles</artifactId>
> <version>1.9.2</version>
> </parent>
>
> That got the docs in the jar:
>      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
>     72 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/extension-manifest.xml
>      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/additional-details/
>
> 2)I tried to use cli then with command:
> ./bin/cli.sh registry upload-bundle --baseUrl http://localhost:18080/
> --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar -ebf
> .custom-processor-nar-1.0-SNAPSHOT.nar
>
> I get back "ERROR: Error executing command 'upload-bundle' : Error
> creating extension bundle version: An unexpected error has occurred. Please
> check the logs for additional details."
>
> In Registry log, error is:
> Exception: Unable to obtain extension info for bundle due to: Unable to
> parse extension manifest due to: Unexpected end of input block; expected an
> identifier
>  at [row,col {unknown-source}]: [1,72]
>  at [Source:
> (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
> line: 1, column: 73]. Returning Internal Server Error response.
> org.apache.nifi.registry.bundle.extract.BundleException: Unable to obtain
> extension info for bundle due to: Unable to parse extension manifest due
> to: Unexpected end of input
> block; expected an identifier
>
> Questions:
> 1) Should I need to set the 1.3.1 plugin do get the docs in nar? I thought
> that would be automatic from parent version being 1.9.2
> 2) Is there an example for how to use the toolkit cli to upload-bundle?
> 3) Is there a better way to CM .nar? Archiva? Artifactory?
>
> Thanks.
>
> -Aaron
>
>
Reply | Threaded
Open this post in threaded view
|

Re: CM of nar via toolkit cli and regisry

Bryan Bende
Hello,

The issue is that in order to correctly generate the extension manifest
with the new NAR plugin, it requires changes from nifi-api that are not
released yet.

You should be able to build NiFi on the master branch, really just the
nifi-api module, doing a mvn clean install.

Then in your NAR you’ll need to force it to use 1.10.0-SNAPSHOT of
nifi-api. Currently you are getting 1.9.2 because of the parent of
nifi-nar-bundles.

Once we get 1.10.0 released then this won’t be an issue anymore.

Thanks,

Bryan

On Sat, Jul 20, 2019 at 11:36 PM Aaron Rich <[hidden email]> wrote:

> I forgot to include that when generating the .nar, I do get the warnings
> of:
> [WARNING] Could not generate extensions' documentation
> org.apache.maven.plugin.MojoExecutionException: Failed to create Extension
> Documentation
> ...
> Caused by: java.lang.NoSuchMethodException:
>
> org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)
>
> I believe this is tied to not getting the right nfi-api version? But the
> parent is set to 1.9.2 so not sure how that is happening.
>
> Thanks again.
>
> -Aaron
>
> On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <[hidden email]> wrote:
>
> > Hi,
> >
> > I'm trying to determine the best way to CM custom nar files for sharing
> > between team. We are using nifi-registry for the flows and I saw there
> was
> > a new capability via the toolkit cli for "upload-bundle".
> >
> > I'm trying to use that but have ran into a few issues:
> > 1) I first wasn't getting the META-INF/docs/ in the .nar. I had built the
> > initial project from mvn archetype:generate with version 1.9.2. I added:
> > <build>
> > <plugins>
> > <plugin>
> > <groupId>org.apache.nifi</groupId>
> > <artifactId>nifi-nar-maven-plugin</artifactId>
> > <version>1.3.1</version>
> > <extensions>true</extensions>
> > </plugin>
> > </plugins>
> > </build>
> >
> > To the base pom.xml. It has the parent of:
> > <parent>
> > <groupId>org.apache.nifi</groupId>
> > <artifactId>nifi-nar-bundles</artifactId>
> > <version>1.9.2</version>
> > </parent>
> >
> > That got the docs in the jar:
> >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
> >     72 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/extension-manifest.xml
> >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/additional-details/
> >
> > 2)I tried to use cli then with command:
> > ./bin/cli.sh registry upload-bundle --baseUrl http://localhost:18080/
> > --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar -ebf
> > .custom-processor-nar-1.0-SNAPSHOT.nar
> >
> > I get back "ERROR: Error executing command 'upload-bundle' : Error
> > creating extension bundle version: An unexpected error has occurred.
> Please
> > check the logs for additional details."
> >
> > In Registry log, error is:
> > Exception: Unable to obtain extension info for bundle due to: Unable to
> > parse extension manifest due to: Unexpected end of input block; expected
> an
> > identifier
> >  at [row,col {unknown-source}]: [1,72]
> >  at [Source:
> >
> (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
> > line: 1, column: 73]. Returning Internal Server Error response.
> > org.apache.nifi.registry.bundle.extract.BundleException: Unable to obtain
> > extension info for bundle due to: Unable to parse extension manifest due
> > to: Unexpected end of input
> > block; expected an identifier
> >
> > Questions:
> > 1) Should I need to set the 1.3.1 plugin do get the docs in nar? I
> thought
> > that would be automatic from parent version being 1.9.2
> > 2) Is there an example for how to use the toolkit cli to upload-bundle?
> > 3) Is there a better way to CM .nar? Archiva? Artifactory?
> >
> > Thanks.
> >
> > -Aaron
> >
> >
>
--
Sent from Gmail Mobile
Reply | Threaded
Open this post in threaded view
|

Re: CM of nar via toolkit cli and regisry

Aaron Rich
Great.

Thank you.

Is the best way to force my nar to use the 1.10 nifi-api via adding it as
dependency is main pom file?

Assuming I still need the build plugin also?

Thanks again.

-Aaron

On Sun, Jul 21, 2019 at 9:30 AM Bryan Bende <[hidden email]> wrote:

> Hello,
>
> The issue is that in order to correctly generate the extension manifest
> with the new NAR plugin, it requires changes from nifi-api that are not
> released yet.
>
> You should be able to build NiFi on the master branch, really just the
> nifi-api module, doing a mvn clean install.
>
> Then in your NAR you’ll need to force it to use 1.10.0-SNAPSHOT of
> nifi-api. Currently you are getting 1.9.2 because of the parent of
> nifi-nar-bundles.
>
> Once we get 1.10.0 released then this won’t be an issue anymore.
>
> Thanks,
>
> Bryan
>
> On Sat, Jul 20, 2019 at 11:36 PM Aaron Rich <[hidden email]> wrote:
>
> > I forgot to include that when generating the .nar, I do get the warnings
> > of:
> > [WARNING] Could not generate extensions' documentation
> > org.apache.maven.plugin.MojoExecutionException: Failed to create
> Extension
> > Documentation
> > ...
> > Caused by: java.lang.NoSuchMethodException:
> >
> >
> org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)
> >
> > I believe this is tied to not getting the right nfi-api version? But the
> > parent is set to 1.9.2 so not sure how that is happening.
> >
> > Thanks again.
> >
> > -Aaron
> >
> > On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <[hidden email]> wrote:
> >
> > > Hi,
> > >
> > > I'm trying to determine the best way to CM custom nar files for sharing
> > > between team. We are using nifi-registry for the flows and I saw there
> > was
> > > a new capability via the toolkit cli for "upload-bundle".
> > >
> > > I'm trying to use that but have ran into a few issues:
> > > 1) I first wasn't getting the META-INF/docs/ in the .nar. I had built
> the
> > > initial project from mvn archetype:generate with version 1.9.2. I
> added:
> > > <build>
> > > <plugins>
> > > <plugin>
> > > <groupId>org.apache.nifi</groupId>
> > > <artifactId>nifi-nar-maven-plugin</artifactId>
> > > <version>1.3.1</version>
> > > <extensions>true</extensions>
> > > </plugin>
> > > </plugins>
> > > </build>
> > >
> > > To the base pom.xml. It has the parent of:
> > > <parent>
> > > <groupId>org.apache.nifi</groupId>
> > > <artifactId>nifi-nar-bundles</artifactId>
> > > <version>1.9.2</version>
> > > </parent>
> > >
> > > That got the docs in the jar:
> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
> > >     72 Sat Jul 20 20:53:44 MDT 2019
> META-INF/docs/extension-manifest.xml
> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/additional-details/
> > >
> > > 2)I tried to use cli then with command:
> > > ./bin/cli.sh registry upload-bundle --baseUrl http://localhost:18080/
> > > --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar -ebf
> > > .custom-processor-nar-1.0-SNAPSHOT.nar
> > >
> > > I get back "ERROR: Error executing command 'upload-bundle' : Error
> > > creating extension bundle version: An unexpected error has occurred.
> > Please
> > > check the logs for additional details."
> > >
> > > In Registry log, error is:
> > > Exception: Unable to obtain extension info for bundle due to: Unable to
> > > parse extension manifest due to: Unexpected end of input block;
> expected
> > an
> > > identifier
> > >  at [row,col {unknown-source}]: [1,72]
> > >  at [Source:
> > >
> >
> (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
> > > line: 1, column: 73]. Returning Internal Server Error response.
> > > org.apache.nifi.registry.bundle.extract.BundleException: Unable to
> obtain
> > > extension info for bundle due to: Unable to parse extension manifest
> due
> > > to: Unexpected end of input
> > > block; expected an identifier
> > >
> > > Questions:
> > > 1) Should I need to set the 1.3.1 plugin do get the docs in nar? I
> > thought
> > > that would be automatic from parent version being 1.9.2
> > > 2) Is there an example for how to use the toolkit cli to upload-bundle?
> > > 3) Is there a better way to CM .nar? Archiva? Artifactory?
> > >
> > > Thanks.
> > >
> > > -Aaron
> > >
> > >
> >
> --
> Sent from Gmail Mobile
>
Reply | Threaded
Open this post in threaded view
|

Re: CM of nar via toolkit cli and regisry

Aaron Rich
Also,

When is 1.10.0 planned for release?

Thanks.

-Aaron


On Sun, Jul 21, 2019 at 9:29 PM Aaron Rich <[hidden email]> wrote:

> Great.
>
> Thank you.
>
> Is the best way to force my nar to use the 1.10 nifi-api via adding it as
> dependency is main pom file?
>
> Assuming I still need the build plugin also?
>
> Thanks again.
>
> -Aaron
>
> On Sun, Jul 21, 2019 at 9:30 AM Bryan Bende <[hidden email]> wrote:
>
>> Hello,
>>
>> The issue is that in order to correctly generate the extension manifest
>> with the new NAR plugin, it requires changes from nifi-api that are not
>> released yet.
>>
>> You should be able to build NiFi on the master branch, really just the
>> nifi-api module, doing a mvn clean install.
>>
>> Then in your NAR you’ll need to force it to use 1.10.0-SNAPSHOT of
>> nifi-api. Currently you are getting 1.9.2 because of the parent of
>> nifi-nar-bundles.
>>
>> Once we get 1.10.0 released then this won’t be an issue anymore.
>>
>> Thanks,
>>
>> Bryan
>>
>> On Sat, Jul 20, 2019 at 11:36 PM Aaron Rich <[hidden email]> wrote:
>>
>> > I forgot to include that when generating the .nar, I do get the warnings
>> > of:
>> > [WARNING] Could not generate extensions' documentation
>> > org.apache.maven.plugin.MojoExecutionException: Failed to create
>> Extension
>> > Documentation
>> > ...
>> > Caused by: java.lang.NoSuchMethodException:
>> >
>> >
>> org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)
>> >
>> > I believe this is tied to not getting the right nfi-api version? But the
>> > parent is set to 1.9.2 so not sure how that is happening.
>> >
>> > Thanks again.
>> >
>> > -Aaron
>> >
>> > On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <[hidden email]>
>> wrote:
>> >
>> > > Hi,
>> > >
>> > > I'm trying to determine the best way to CM custom nar files for
>> sharing
>> > > between team. We are using nifi-registry for the flows and I saw there
>> > was
>> > > a new capability via the toolkit cli for "upload-bundle".
>> > >
>> > > I'm trying to use that but have ran into a few issues:
>> > > 1) I first wasn't getting the META-INF/docs/ in the .nar. I had built
>> the
>> > > initial project from mvn archetype:generate with version 1.9.2. I
>> added:
>> > > <build>
>> > > <plugins>
>> > > <plugin>
>> > > <groupId>org.apache.nifi</groupId>
>> > > <artifactId>nifi-nar-maven-plugin</artifactId>
>> > > <version>1.3.1</version>
>> > > <extensions>true</extensions>
>> > > </plugin>
>> > > </plugins>
>> > > </build>
>> > >
>> > > To the base pom.xml. It has the parent of:
>> > > <parent>
>> > > <groupId>org.apache.nifi</groupId>
>> > > <artifactId>nifi-nar-bundles</artifactId>
>> > > <version>1.9.2</version>
>> > > </parent>
>> > >
>> > > That got the docs in the jar:
>> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
>> > >     72 Sat Jul 20 20:53:44 MDT 2019
>> META-INF/docs/extension-manifest.xml
>> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/additional-details/
>> > >
>> > > 2)I tried to use cli then with command:
>> > > ./bin/cli.sh registry upload-bundle --baseUrl http://localhost:18080/
>> > > --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar -ebf
>> > > .custom-processor-nar-1.0-SNAPSHOT.nar
>> > >
>> > > I get back "ERROR: Error executing command 'upload-bundle' : Error
>> > > creating extension bundle version: An unexpected error has occurred.
>> > Please
>> > > check the logs for additional details."
>> > >
>> > > In Registry log, error is:
>> > > Exception: Unable to obtain extension info for bundle due to: Unable
>> to
>> > > parse extension manifest due to: Unexpected end of input block;
>> expected
>> > an
>> > > identifier
>> > >  at [row,col {unknown-source}]: [1,72]
>> > >  at [Source:
>> > >
>> >
>> (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
>> > > line: 1, column: 73]. Returning Internal Server Error response.
>> > > org.apache.nifi.registry.bundle.extract.BundleException: Unable to
>> obtain
>> > > extension info for bundle due to: Unable to parse extension manifest
>> due
>> > > to: Unexpected end of input
>> > > block; expected an identifier
>> > >
>> > > Questions:
>> > > 1) Should I need to set the 1.3.1 plugin do get the docs in nar? I
>> > thought
>> > > that would be automatic from parent version being 1.9.2
>> > > 2) Is there an example for how to use the toolkit cli to
>> upload-bundle?
>> > > 3) Is there a better way to CM .nar? Archiva? Artifactory?
>> > >
>> > > Thanks.
>> > >
>> > > -Aaron
>> > >
>> > >
>> >
>> --
>> Sent from Gmail Mobile
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: CM of nar via toolkit cli and regisry

Aaron Rich
I updated  pom  for  a bundle with just a processor and was able to get nar
generated and loaded into registry via cli.

When I tried a bundle with a custom service though, I get failures in the
test:
java.lang.AbstractMethodError:
org.apache.nifi.util.MockControllerServiceInitializationContext.getNodeTypeProvider()Lorg/apache/nifi/controller/NodeTypeProvider;
        at
com.ngc.swordfish.nifi.multipart.TestStandardMyService.testService(TestStandardMyService.java:36)

Do I need to updated nifi-mock to 1.10 also?

Thanks.

-Aaron

On Sun, Jul 21, 2019 at 9:59 PM Aaron Rich <[hidden email]> wrote:

> Also,
>
> When is 1.10.0 planned for release?
>
> Thanks.
>
> -Aaron
>
>
> On Sun, Jul 21, 2019 at 9:29 PM Aaron Rich <[hidden email]> wrote:
>
>> Great.
>>
>> Thank you.
>>
>> Is the best way to force my nar to use the 1.10 nifi-api via adding it as
>> dependency is main pom file?
>>
>> Assuming I still need the build plugin also?
>>
>> Thanks again.
>>
>> -Aaron
>>
>> On Sun, Jul 21, 2019 at 9:30 AM Bryan Bende <[hidden email]> wrote:
>>
>>> Hello,
>>>
>>> The issue is that in order to correctly generate the extension manifest
>>> with the new NAR plugin, it requires changes from nifi-api that are not
>>> released yet.
>>>
>>> You should be able to build NiFi on the master branch, really just the
>>> nifi-api module, doing a mvn clean install.
>>>
>>> Then in your NAR you’ll need to force it to use 1.10.0-SNAPSHOT of
>>> nifi-api. Currently you are getting 1.9.2 because of the parent of
>>> nifi-nar-bundles.
>>>
>>> Once we get 1.10.0 released then this won’t be an issue anymore.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>> On Sat, Jul 20, 2019 at 11:36 PM Aaron Rich <[hidden email]>
>>> wrote:
>>>
>>> > I forgot to include that when generating the .nar, I do get the
>>> warnings
>>> > of:
>>> > [WARNING] Could not generate extensions' documentation
>>> > org.apache.maven.plugin.MojoExecutionException: Failed to create
>>> Extension
>>> > Documentation
>>> > ...
>>> > Caused by: java.lang.NoSuchMethodException:
>>> >
>>> >
>>> org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)
>>> >
>>> > I believe this is tied to not getting the right nfi-api version? But
>>> the
>>> > parent is set to 1.9.2 so not sure how that is happening.
>>> >
>>> > Thanks again.
>>> >
>>> > -Aaron
>>> >
>>> > On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <[hidden email]>
>>> wrote:
>>> >
>>> > > Hi,
>>> > >
>>> > > I'm trying to determine the best way to CM custom nar files for
>>> sharing
>>> > > between team. We are using nifi-registry for the flows and I saw
>>> there
>>> > was
>>> > > a new capability via the toolkit cli for "upload-bundle".
>>> > >
>>> > > I'm trying to use that but have ran into a few issues:
>>> > > 1) I first wasn't getting the META-INF/docs/ in the .nar. I had
>>> built the
>>> > > initial project from mvn archetype:generate with version 1.9.2. I
>>> added:
>>> > > <build>
>>> > > <plugins>
>>> > > <plugin>
>>> > > <groupId>org.apache.nifi</groupId>
>>> > > <artifactId>nifi-nar-maven-plugin</artifactId>
>>> > > <version>1.3.1</version>
>>> > > <extensions>true</extensions>
>>> > > </plugin>
>>> > > </plugins>
>>> > > </build>
>>> > >
>>> > > To the base pom.xml. It has the parent of:
>>> > > <parent>
>>> > > <groupId>org.apache.nifi</groupId>
>>> > > <artifactId>nifi-nar-bundles</artifactId>
>>> > > <version>1.9.2</version>
>>> > > </parent>
>>> > >
>>> > > That got the docs in the jar:
>>> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
>>> > >     72 Sat Jul 20 20:53:44 MDT 2019
>>> META-INF/docs/extension-manifest.xml
>>> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/additional-details/
>>> > >
>>> > > 2)I tried to use cli then with command:
>>> > > ./bin/cli.sh registry upload-bundle --baseUrl
>>> http://localhost:18080/
>>> > > --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar -ebf
>>> > > .custom-processor-nar-1.0-SNAPSHOT.nar
>>> > >
>>> > > I get back "ERROR: Error executing command 'upload-bundle' : Error
>>> > > creating extension bundle version: An unexpected error has occurred.
>>> > Please
>>> > > check the logs for additional details."
>>> > >
>>> > > In Registry log, error is:
>>> > > Exception: Unable to obtain extension info for bundle due to: Unable
>>> to
>>> > > parse extension manifest due to: Unexpected end of input block;
>>> expected
>>> > an
>>> > > identifier
>>> > >  at [row,col {unknown-source}]: [1,72]
>>> > >  at [Source:
>>> > >
>>> >
>>> (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
>>> > > line: 1, column: 73]. Returning Internal Server Error response.
>>> > > org.apache.nifi.registry.bundle.extract.BundleException: Unable to
>>> obtain
>>> > > extension info for bundle due to: Unable to parse extension manifest
>>> due
>>> > > to: Unexpected end of input
>>> > > block; expected an identifier
>>> > >
>>> > > Questions:
>>> > > 1) Should I need to set the 1.3.1 plugin do get the docs in nar? I
>>> > thought
>>> > > that would be automatic from parent version being 1.9.2
>>> > > 2) Is there an example for how to use the toolkit cli to
>>> upload-bundle?
>>> > > 3) Is there a better way to CM .nar? Archiva? Artifactory?
>>> > >
>>> > > Thanks.
>>> > >
>>> > > -Aaron
>>> > >
>>> > >
>>> >
>>> --
>>> Sent from Gmail Mobile
>>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: CM of nar via toolkit cli and regisry

Jeff
Aaron,

New versions of NiFi are generally released after a member of the community
proposes a release and someone volunteers to perform the duties of the
release manager.  There is no predefined schedule or projected release
dates.  I think there are a few major feature JIRAs on which work is
currently being done, and once those are resolved/finished, at that point
someone may make proposal to create a release candidate for 1.10.0.

Regarding the error to which you are referring, does the
TestStandardMyService class make use of nifi-mock classes?  Is
TestStandardMyService within the "src/main" folder of your module?  If so,
you may want to move that class into "src/test", as I don't believe the
nifi-maven-nar plugin bundles test code when it creates the NAR.  From the
stacktrace, I see that TestStandardMyService makes a call to a
org.apache.nifi.util.MockControllerServiceIntitializationContext method,
which comes from nifi-mock.  If TestStandardMyService is included with the
rest of your "production" code, you might want to refactor your module so
that it is part of the "test" code.  That should remove the dependency on
nifi-mock.

On Mon, Jul 22, 2019 at 12:24 AM Aaron Rich <[hidden email]> wrote:

> I updated  pom  for  a bundle with just a processor and was able to get nar
> generated and loaded into registry via cli.
>
> When I tried a bundle with a custom service though, I get failures in the
> test:
> java.lang.AbstractMethodError:
>
> org.apache.nifi.util.MockControllerServiceInitializationContext.getNodeTypeProvider()Lorg/apache/nifi/controller/NodeTypeProvider;
>         at
>
> com.ngc.swordfish.nifi.multipart.TestStandardMyService.testService(TestStandardMyService.java:36)
>
> Do I need to updated nifi-mock to 1.10 also?
>
> Thanks.
>
> -Aaron
>
> On Sun, Jul 21, 2019 at 9:59 PM Aaron Rich <[hidden email]> wrote:
>
> > Also,
> >
> > When is 1.10.0 planned for release?
> >
> > Thanks.
> >
> > -Aaron
> >
> >
> > On Sun, Jul 21, 2019 at 9:29 PM Aaron Rich <[hidden email]> wrote:
> >
> >> Great.
> >>
> >> Thank you.
> >>
> >> Is the best way to force my nar to use the 1.10 nifi-api via adding it
> as
> >> dependency is main pom file?
> >>
> >> Assuming I still need the build plugin also?
> >>
> >> Thanks again.
> >>
> >> -Aaron
> >>
> >> On Sun, Jul 21, 2019 at 9:30 AM Bryan Bende <[hidden email]> wrote:
> >>
> >>> Hello,
> >>>
> >>> The issue is that in order to correctly generate the extension manifest
> >>> with the new NAR plugin, it requires changes from nifi-api that are not
> >>> released yet.
> >>>
> >>> You should be able to build NiFi on the master branch, really just the
> >>> nifi-api module, doing a mvn clean install.
> >>>
> >>> Then in your NAR you’ll need to force it to use 1.10.0-SNAPSHOT of
> >>> nifi-api. Currently you are getting 1.9.2 because of the parent of
> >>> nifi-nar-bundles.
> >>>
> >>> Once we get 1.10.0 released then this won’t be an issue anymore.
> >>>
> >>> Thanks,
> >>>
> >>> Bryan
> >>>
> >>> On Sat, Jul 20, 2019 at 11:36 PM Aaron Rich <[hidden email]>
> >>> wrote:
> >>>
> >>> > I forgot to include that when generating the .nar, I do get the
> >>> warnings
> >>> > of:
> >>> > [WARNING] Could not generate extensions' documentation
> >>> > org.apache.maven.plugin.MojoExecutionException: Failed to create
> >>> Extension
> >>> > Documentation
> >>> > ...
> >>> > Caused by: java.lang.NoSuchMethodException:
> >>> >
> >>> >
> >>>
> org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)
> >>> >
> >>> > I believe this is tied to not getting the right nfi-api version? But
> >>> the
> >>> > parent is set to 1.9.2 so not sure how that is happening.
> >>> >
> >>> > Thanks again.
> >>> >
> >>> > -Aaron
> >>> >
> >>> > On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <[hidden email]>
> >>> wrote:
> >>> >
> >>> > > Hi,
> >>> > >
> >>> > > I'm trying to determine the best way to CM custom nar files for
> >>> sharing
> >>> > > between team. We are using nifi-registry for the flows and I saw
> >>> there
> >>> > was
> >>> > > a new capability via the toolkit cli for "upload-bundle".
> >>> > >
> >>> > > I'm trying to use that but have ran into a few issues:
> >>> > > 1) I first wasn't getting the META-INF/docs/ in the .nar. I had
> >>> built the
> >>> > > initial project from mvn archetype:generate with version 1.9.2. I
> >>> added:
> >>> > > <build>
> >>> > > <plugins>
> >>> > > <plugin>
> >>> > > <groupId>org.apache.nifi</groupId>
> >>> > > <artifactId>nifi-nar-maven-plugin</artifactId>
> >>> > > <version>1.3.1</version>
> >>> > > <extensions>true</extensions>
> >>> > > </plugin>
> >>> > > </plugins>
> >>> > > </build>
> >>> > >
> >>> > > To the base pom.xml. It has the parent of:
> >>> > > <parent>
> >>> > > <groupId>org.apache.nifi</groupId>
> >>> > > <artifactId>nifi-nar-bundles</artifactId>
> >>> > > <version>1.9.2</version>
> >>> > > </parent>
> >>> > >
> >>> > > That got the docs in the jar:
> >>> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
> >>> > >     72 Sat Jul 20 20:53:44 MDT 2019
> >>> META-INF/docs/extension-manifest.xml
> >>> > >      0 Sat Jul 20 20:53:44 MDT 2019
> META-INF/docs/additional-details/
> >>> > >
> >>> > > 2)I tried to use cli then with command:
> >>> > > ./bin/cli.sh registry upload-bundle --baseUrl
> >>> http://localhost:18080/
> >>> > > --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar -ebf
> >>> > > .custom-processor-nar-1.0-SNAPSHOT.nar
> >>> > >
> >>> > > I get back "ERROR: Error executing command 'upload-bundle' : Error
> >>> > > creating extension bundle version: An unexpected error has
> occurred.
> >>> > Please
> >>> > > check the logs for additional details."
> >>> > >
> >>> > > In Registry log, error is:
> >>> > > Exception: Unable to obtain extension info for bundle due to:
> Unable
> >>> to
> >>> > > parse extension manifest due to: Unexpected end of input block;
> >>> expected
> >>> > an
> >>> > > identifier
> >>> > >  at [row,col {unknown-source}]: [1,72]
> >>> > >  at [Source:
> >>> > >
> >>> >
> >>>
> (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
> >>> > > line: 1, column: 73]. Returning Internal Server Error response.
> >>> > > org.apache.nifi.registry.bundle.extract.BundleException: Unable to
> >>> obtain
> >>> > > extension info for bundle due to: Unable to parse extension
> manifest
> >>> due
> >>> > > to: Unexpected end of input
> >>> > > block; expected an identifier
> >>> > >
> >>> > > Questions:
> >>> > > 1) Should I need to set the 1.3.1 plugin do get the docs in nar? I
> >>> > thought
> >>> > > that would be automatic from parent version being 1.9.2
> >>> > > 2) Is there an example for how to use the toolkit cli to
> >>> upload-bundle?
> >>> > > 3) Is there a better way to CM .nar? Archiva? Artifactory?
> >>> > >
> >>> > > Thanks.
> >>> > >
> >>> > > -Aaron
> >>> > >
> >>> > >
> >>> >
> >>> --
> >>> Sent from Gmail Mobile
> >>>
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: CM of nar via toolkit cli and regisry

Aaron Rich
Hi Jeff,
Thanks for the information on the release process.

For the error I'm getting, I started a new custom service with:
mvn archetype:generate -DarchetypeGroupId=org.apache.nifi
-DarchetypeArtifactId=nifi-service-bundle-archetype
-DarchetypeVersion=1.9.2 -DnifiVersion=1.9.2

This provides a shell test which I haven't updated.

I updated the testservice-api-nar and the  testservice-nar poms like I did
with the processor to include:
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-api</artifactId>
<version>1.10.0-SNAPSHOT</version>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-nar-maven-plugin</artifactId>
<version>1.3.1</version>
<extensions>true</extensions>
</plugin>
</plugins>
</build>

On the testService-api-nar I still get the warning of:
[INFO] Generating documentation for NiFi extensions in the NAR...
[INFO] Found NAR dependency of
com.test.nifi:nifi-testService-api-nar:nar:1.0-SNAPSHOT:compile
[INFO] Found NAR dependency of
org.apache.nifi:nifi-standard-services-api-nar:nar:1.9.2:compile
[INFO] Found NAR dependency of
org.apache.nifi:nifi-jetty-bundle:nar:1.9.2:compile
[INFO] Found a dependency on version 1.9.2 of NiFi API
[WARNING] Could not generate extensions' documentation
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension
Documentation

When I was trying to trace back the 1.9.2 NiFi API dependency, I changed
the testService to have nifi-api version 1.10. This is when the compile
errors showed up.

So at this point my question would be, for a custom service based on 1.9.2,
which pom.xml files need to be changed, and with what changes, to have the
service.nar file have the documentation to be able to be uploaded to nifi
registry via cli? Seems like there is more then just moving nifi-api to
version 1.10.

Thank you again for the help.

-Aaron



On Mon, Jul 22, 2019 at 12:43 AM Jeff <[hidden email]> wrote:

> Aaron,
>
> New versions of NiFi are generally released after a member of the community
> proposes a release and someone volunteers to perform the duties of the
> release manager.  There is no predefined schedule or projected release
> dates.  I think there are a few major feature JIRAs on which work is
> currently being done, and once those are resolved/finished, at that point
> someone may make proposal to create a release candidate for 1.10.0.
>
> Regarding the error to which you are referring, does the
> TestStandardMyService class make use of nifi-mock classes?  Is
> TestStandardMyService within the "src/main" folder of your module?  If so,
> you may want to move that class into "src/test", as I don't believe the
> nifi-maven-nar plugin bundles test code when it creates the NAR.  From the
> stacktrace, I see that TestStandardMyService makes a call to a
> org.apache.nifi.util.MockControllerServiceIntitializationContext method,
> which comes from nifi-mock.  If TestStandardMyService is included with the
> rest of your "production" code, you might want to refactor your module so
> that it is part of the "test" code.  That should remove the dependency on
> nifi-mock.
>
> On Mon, Jul 22, 2019 at 12:24 AM Aaron Rich <[hidden email]> wrote:
>
> > I updated  pom  for  a bundle with just a processor and was able to get
> nar
> > generated and loaded into registry via cli.
> >
> > When I tried a bundle with a custom service though, I get failures in the
> > test:
> > java.lang.AbstractMethodError:
> >
> >
> org.apache.nifi.util.MockControllerServiceInitializationContext.getNodeTypeProvider()Lorg/apache/nifi/controller/NodeTypeProvider;
> >         at
> >
> >
> com.ngc.swordfish.nifi.multipart.TestStandardMyService.testService(TestStandardMyService.java:36)
> >
> > Do I need to updated nifi-mock to 1.10 also?
> >
> > Thanks.
> >
> > -Aaron
> >
> > On Sun, Jul 21, 2019 at 9:59 PM Aaron Rich <[hidden email]> wrote:
> >
> > > Also,
> > >
> > > When is 1.10.0 planned for release?
> > >
> > > Thanks.
> > >
> > > -Aaron
> > >
> > >
> > > On Sun, Jul 21, 2019 at 9:29 PM Aaron Rich <[hidden email]>
> wrote:
> > >
> > >> Great.
> > >>
> > >> Thank you.
> > >>
> > >> Is the best way to force my nar to use the 1.10 nifi-api via adding it
> > as
> > >> dependency is main pom file?
> > >>
> > >> Assuming I still need the build plugin also?
> > >>
> > >> Thanks again.
> > >>
> > >> -Aaron
> > >>
> > >> On Sun, Jul 21, 2019 at 9:30 AM Bryan Bende <[hidden email]> wrote:
> > >>
> > >>> Hello,
> > >>>
> > >>> The issue is that in order to correctly generate the extension
> manifest
> > >>> with the new NAR plugin, it requires changes from nifi-api that are
> not
> > >>> released yet.
> > >>>
> > >>> You should be able to build NiFi on the master branch, really just
> the
> > >>> nifi-api module, doing a mvn clean install.
> > >>>
> > >>> Then in your NAR you’ll need to force it to use 1.10.0-SNAPSHOT of
> > >>> nifi-api. Currently you are getting 1.9.2 because of the parent of
> > >>> nifi-nar-bundles.
> > >>>
> > >>> Once we get 1.10.0 released then this won’t be an issue anymore.
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Bryan
> > >>>
> > >>> On Sat, Jul 20, 2019 at 11:36 PM Aaron Rich <[hidden email]>
> > >>> wrote:
> > >>>
> > >>> > I forgot to include that when generating the .nar, I do get the
> > >>> warnings
> > >>> > of:
> > >>> > [WARNING] Could not generate extensions' documentation
> > >>> > org.apache.maven.plugin.MojoExecutionException: Failed to create
> > >>> Extension
> > >>> > Documentation
> > >>> > ...
> > >>> > Caused by: java.lang.NoSuchMethodException:
> > >>> >
> > >>> >
> > >>>
> >
> org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)
> > >>> >
> > >>> > I believe this is tied to not getting the right nfi-api version?
> But
> > >>> the
> > >>> > parent is set to 1.9.2 so not sure how that is happening.
> > >>> >
> > >>> > Thanks again.
> > >>> >
> > >>> > -Aaron
> > >>> >
> > >>> > On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <[hidden email]>
> > >>> wrote:
> > >>> >
> > >>> > > Hi,
> > >>> > >
> > >>> > > I'm trying to determine the best way to CM custom nar files for
> > >>> sharing
> > >>> > > between team. We are using nifi-registry for the flows and I saw
> > >>> there
> > >>> > was
> > >>> > > a new capability via the toolkit cli for "upload-bundle".
> > >>> > >
> > >>> > > I'm trying to use that but have ran into a few issues:
> > >>> > > 1) I first wasn't getting the META-INF/docs/ in the .nar. I had
> > >>> built the
> > >>> > > initial project from mvn archetype:generate with version 1.9.2. I
> > >>> added:
> > >>> > > <build>
> > >>> > > <plugins>
> > >>> > > <plugin>
> > >>> > > <groupId>org.apache.nifi</groupId>
> > >>> > > <artifactId>nifi-nar-maven-plugin</artifactId>
> > >>> > > <version>1.3.1</version>
> > >>> > > <extensions>true</extensions>
> > >>> > > </plugin>
> > >>> > > </plugins>
> > >>> > > </build>
> > >>> > >
> > >>> > > To the base pom.xml. It has the parent of:
> > >>> > > <parent>
> > >>> > > <groupId>org.apache.nifi</groupId>
> > >>> > > <artifactId>nifi-nar-bundles</artifactId>
> > >>> > > <version>1.9.2</version>
> > >>> > > </parent>
> > >>> > >
> > >>> > > That got the docs in the jar:
> > >>> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
> > >>> > >     72 Sat Jul 20 20:53:44 MDT 2019
> > >>> META-INF/docs/extension-manifest.xml
> > >>> > >      0 Sat Jul 20 20:53:44 MDT 2019
> > META-INF/docs/additional-details/
> > >>> > >
> > >>> > > 2)I tried to use cli then with command:
> > >>> > > ./bin/cli.sh registry upload-bundle --baseUrl
> > >>> http://localhost:18080/
> > >>> > > --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar
> -ebf
> > >>> > > .custom-processor-nar-1.0-SNAPSHOT.nar
> > >>> > >
> > >>> > > I get back "ERROR: Error executing command 'upload-bundle' :
> Error
> > >>> > > creating extension bundle version: An unexpected error has
> > occurred.
> > >>> > Please
> > >>> > > check the logs for additional details."
> > >>> > >
> > >>> > > In Registry log, error is:
> > >>> > > Exception: Unable to obtain extension info for bundle due to:
> > Unable
> > >>> to
> > >>> > > parse extension manifest due to: Unexpected end of input block;
> > >>> expected
> > >>> > an
> > >>> > > identifier
> > >>> > >  at [row,col {unknown-source}]: [1,72]
> > >>> > >  at [Source:
> > >>> > >
> > >>> >
> > >>>
> >
> (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
> > >>> > > line: 1, column: 73]. Returning Internal Server Error response.
> > >>> > > org.apache.nifi.registry.bundle.extract.BundleException: Unable
> to
> > >>> obtain
> > >>> > > extension info for bundle due to: Unable to parse extension
> > manifest
> > >>> due
> > >>> > > to: Unexpected end of input
> > >>> > > block; expected an identifier
> > >>> > >
> > >>> > > Questions:
> > >>> > > 1) Should I need to set the 1.3.1 plugin do get the docs in nar?
> I
> > >>> > thought
> > >>> > > that would be automatic from parent version being 1.9.2
> > >>> > > 2) Is there an example for how to use the toolkit cli to
> > >>> upload-bundle?
> > >>> > > 3) Is there a better way to CM .nar? Archiva? Artifactory?
> > >>> > >
> > >>> > > Thanks.
> > >>> > >
> > >>> > > -Aaron
> > >>> > >
> > >>> > >
> > >>> >
> > >>> --
> > >>> Sent from Gmail Mobile
> > >>>
> > >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: CM of nar via toolkit cli and regisry

Bryan Bende
I think you are correct that you would need to also bump nifi-mock to
1.10.0-SNAPSHPOT.

An easy way to make sure everything gets set to 1.10.0-SNAPSHOT is to
add a <dependencyManagement> block to the top-level pom of your
services project (the parent pom of the service, service api, and
NARs) and in the dep management declare the 1.10.0-SNAPSHOT for
nifi-api and nifi-mock.

As a side note, there is obviously a slight risk here building against
1.10.0 API and deploying to 1.9.2 since you could end up calling
something in your code that only exists in 1.10.0, but since it sounds
like you originally implemented against 1.9.2 then you are probably
fine. Just wanted to point this out.

On Mon, Jul 22, 2019 at 2:04 PM Aaron Rich <[hidden email]> wrote:

>
> Hi Jeff,
> Thanks for the information on the release process.
>
> For the error I'm getting, I started a new custom service with:
> mvn archetype:generate -DarchetypeGroupId=org.apache.nifi
> -DarchetypeArtifactId=nifi-service-bundle-archetype
> -DarchetypeVersion=1.9.2 -DnifiVersion=1.9.2
>
> This provides a shell test which I haven't updated.
>
> I updated the testservice-api-nar and the  testservice-nar poms like I did
> with the processor to include:
> <dependency>
> <groupId>org.apache.nifi</groupId>
> <artifactId>nifi-api</artifactId>
> <version>1.10.0-SNAPSHOT</version>
> </dependency>
> </dependencies>
>
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.nifi</groupId>
> <artifactId>nifi-nar-maven-plugin</artifactId>
> <version>1.3.1</version>
> <extensions>true</extensions>
> </plugin>
> </plugins>
> </build>
>
> On the testService-api-nar I still get the warning of:
> [INFO] Generating documentation for NiFi extensions in the NAR...
> [INFO] Found NAR dependency of
> com.test.nifi:nifi-testService-api-nar:nar:1.0-SNAPSHOT:compile
> [INFO] Found NAR dependency of
> org.apache.nifi:nifi-standard-services-api-nar:nar:1.9.2:compile
> [INFO] Found NAR dependency of
> org.apache.nifi:nifi-jetty-bundle:nar:1.9.2:compile
> [INFO] Found a dependency on version 1.9.2 of NiFi API
> [WARNING] Could not generate extensions' documentation
> org.apache.maven.plugin.MojoExecutionException: Failed to create Extension
> Documentation
>
> When I was trying to trace back the 1.9.2 NiFi API dependency, I changed
> the testService to have nifi-api version 1.10. This is when the compile
> errors showed up.
>
> So at this point my question would be, for a custom service based on 1.9.2,
> which pom.xml files need to be changed, and with what changes, to have the
> service.nar file have the documentation to be able to be uploaded to nifi
> registry via cli? Seems like there is more then just moving nifi-api to
> version 1.10.
>
> Thank you again for the help.
>
> -Aaron
>
>
>
> On Mon, Jul 22, 2019 at 12:43 AM Jeff <[hidden email]> wrote:
>
> > Aaron,
> >
> > New versions of NiFi are generally released after a member of the community
> > proposes a release and someone volunteers to perform the duties of the
> > release manager.  There is no predefined schedule or projected release
> > dates.  I think there are a few major feature JIRAs on which work is
> > currently being done, and once those are resolved/finished, at that point
> > someone may make proposal to create a release candidate for 1.10.0.
> >
> > Regarding the error to which you are referring, does the
> > TestStandardMyService class make use of nifi-mock classes?  Is
> > TestStandardMyService within the "src/main" folder of your module?  If so,
> > you may want to move that class into "src/test", as I don't believe the
> > nifi-maven-nar plugin bundles test code when it creates the NAR.  From the
> > stacktrace, I see that TestStandardMyService makes a call to a
> > org.apache.nifi.util.MockControllerServiceIntitializationContext method,
> > which comes from nifi-mock.  If TestStandardMyService is included with the
> > rest of your "production" code, you might want to refactor your module so
> > that it is part of the "test" code.  That should remove the dependency on
> > nifi-mock.
> >
> > On Mon, Jul 22, 2019 at 12:24 AM Aaron Rich <[hidden email]> wrote:
> >
> > > I updated  pom  for  a bundle with just a processor and was able to get
> > nar
> > > generated and loaded into registry via cli.
> > >
> > > When I tried a bundle with a custom service though, I get failures in the
> > > test:
> > > java.lang.AbstractMethodError:
> > >
> > >
> > org.apache.nifi.util.MockControllerServiceInitializationContext.getNodeTypeProvider()Lorg/apache/nifi/controller/NodeTypeProvider;
> > >         at
> > >
> > >
> > com.ngc.swordfish.nifi.multipart.TestStandardMyService.testService(TestStandardMyService.java:36)
> > >
> > > Do I need to updated nifi-mock to 1.10 also?
> > >
> > > Thanks.
> > >
> > > -Aaron
> > >
> > > On Sun, Jul 21, 2019 at 9:59 PM Aaron Rich <[hidden email]> wrote:
> > >
> > > > Also,
> > > >
> > > > When is 1.10.0 planned for release?
> > > >
> > > > Thanks.
> > > >
> > > > -Aaron
> > > >
> > > >
> > > > On Sun, Jul 21, 2019 at 9:29 PM Aaron Rich <[hidden email]>
> > wrote:
> > > >
> > > >> Great.
> > > >>
> > > >> Thank you.
> > > >>
> > > >> Is the best way to force my nar to use the 1.10 nifi-api via adding it
> > > as
> > > >> dependency is main pom file?
> > > >>
> > > >> Assuming I still need the build plugin also?
> > > >>
> > > >> Thanks again.
> > > >>
> > > >> -Aaron
> > > >>
> > > >> On Sun, Jul 21, 2019 at 9:30 AM Bryan Bende <[hidden email]> wrote:
> > > >>
> > > >>> Hello,
> > > >>>
> > > >>> The issue is that in order to correctly generate the extension
> > manifest
> > > >>> with the new NAR plugin, it requires changes from nifi-api that are
> > not
> > > >>> released yet.
> > > >>>
> > > >>> You should be able to build NiFi on the master branch, really just
> > the
> > > >>> nifi-api module, doing a mvn clean install.
> > > >>>
> > > >>> Then in your NAR you’ll need to force it to use 1.10.0-SNAPSHOT of
> > > >>> nifi-api. Currently you are getting 1.9.2 because of the parent of
> > > >>> nifi-nar-bundles.
> > > >>>
> > > >>> Once we get 1.10.0 released then this won’t be an issue anymore.
> > > >>>
> > > >>> Thanks,
> > > >>>
> > > >>> Bryan
> > > >>>
> > > >>> On Sat, Jul 20, 2019 at 11:36 PM Aaron Rich <[hidden email]>
> > > >>> wrote:
> > > >>>
> > > >>> > I forgot to include that when generating the .nar, I do get the
> > > >>> warnings
> > > >>> > of:
> > > >>> > [WARNING] Could not generate extensions' documentation
> > > >>> > org.apache.maven.plugin.MojoExecutionException: Failed to create
> > > >>> Extension
> > > >>> > Documentation
> > > >>> > ...
> > > >>> > Caused by: java.lang.NoSuchMethodException:
> > > >>> >
> > > >>> >
> > > >>>
> > >
> > org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)
> > > >>> >
> > > >>> > I believe this is tied to not getting the right nfi-api version?
> > But
> > > >>> the
> > > >>> > parent is set to 1.9.2 so not sure how that is happening.
> > > >>> >
> > > >>> > Thanks again.
> > > >>> >
> > > >>> > -Aaron
> > > >>> >
> > > >>> > On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <[hidden email]>
> > > >>> wrote:
> > > >>> >
> > > >>> > > Hi,
> > > >>> > >
> > > >>> > > I'm trying to determine the best way to CM custom nar files for
> > > >>> sharing
> > > >>> > > between team. We are using nifi-registry for the flows and I saw
> > > >>> there
> > > >>> > was
> > > >>> > > a new capability via the toolkit cli for "upload-bundle".
> > > >>> > >
> > > >>> > > I'm trying to use that but have ran into a few issues:
> > > >>> > > 1) I first wasn't getting the META-INF/docs/ in the .nar. I had
> > > >>> built the
> > > >>> > > initial project from mvn archetype:generate with version 1.9.2. I
> > > >>> added:
> > > >>> > > <build>
> > > >>> > > <plugins>
> > > >>> > > <plugin>
> > > >>> > > <groupId>org.apache.nifi</groupId>
> > > >>> > > <artifactId>nifi-nar-maven-plugin</artifactId>
> > > >>> > > <version>1.3.1</version>
> > > >>> > > <extensions>true</extensions>
> > > >>> > > </plugin>
> > > >>> > > </plugins>
> > > >>> > > </build>
> > > >>> > >
> > > >>> > > To the base pom.xml. It has the parent of:
> > > >>> > > <parent>
> > > >>> > > <groupId>org.apache.nifi</groupId>
> > > >>> > > <artifactId>nifi-nar-bundles</artifactId>
> > > >>> > > <version>1.9.2</version>
> > > >>> > > </parent>
> > > >>> > >
> > > >>> > > That got the docs in the jar:
> > > >>> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
> > > >>> > >     72 Sat Jul 20 20:53:44 MDT 2019
> > > >>> META-INF/docs/extension-manifest.xml
> > > >>> > >      0 Sat Jul 20 20:53:44 MDT 2019
> > > META-INF/docs/additional-details/
> > > >>> > >
> > > >>> > > 2)I tried to use cli then with command:
> > > >>> > > ./bin/cli.sh registry upload-bundle --baseUrl
> > > >>> http://localhost:18080/
> > > >>> > > --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar
> > -ebf
> > > >>> > > .custom-processor-nar-1.0-SNAPSHOT.nar
> > > >>> > >
> > > >>> > > I get back "ERROR: Error executing command 'upload-bundle' :
> > Error
> > > >>> > > creating extension bundle version: An unexpected error has
> > > occurred.
> > > >>> > Please
> > > >>> > > check the logs for additional details."
> > > >>> > >
> > > >>> > > In Registry log, error is:
> > > >>> > > Exception: Unable to obtain extension info for bundle due to:
> > > Unable
> > > >>> to
> > > >>> > > parse extension manifest due to: Unexpected end of input block;
> > > >>> expected
> > > >>> > an
> > > >>> > > identifier
> > > >>> > >  at [row,col {unknown-source}]: [1,72]
> > > >>> > >  at [Source:
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> > (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
> > > >>> > > line: 1, column: 73]. Returning Internal Server Error response.
> > > >>> > > org.apache.nifi.registry.bundle.extract.BundleException: Unable
> > to
> > > >>> obtain
> > > >>> > > extension info for bundle due to: Unable to parse extension
> > > manifest
> > > >>> due
> > > >>> > > to: Unexpected end of input
> > > >>> > > block; expected an identifier
> > > >>> > >
> > > >>> > > Questions:
> > > >>> > > 1) Should I need to set the 1.3.1 plugin do get the docs in nar?
> > I
> > > >>> > thought
> > > >>> > > that would be automatic from parent version being 1.9.2
> > > >>> > > 2) Is there an example for how to use the toolkit cli to
> > > >>> upload-bundle?
> > > >>> > > 3) Is there a better way to CM .nar? Archiva? Artifactory?
> > > >>> > >
> > > >>> > > Thanks.
> > > >>> > >
> > > >>> > > -Aaron
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>> --
> > > >>> Sent from Gmail Mobile
> > > >>>
> > > >>
> > >
> >
Reply | Threaded
Open this post in threaded view
|

Re: CM of nar via toolkit cli and regisry

Bryan Bende
One more thing, make sure the <scope> on the dependencies stays the
same, meaning nifi-api should still be provided. You don't want to end
up bundling the 1.10.0 nifi-api into your NARs.

On Mon, Jul 22, 2019 at 2:21 PM Bryan Bende <[hidden email]> wrote:

>
> I think you are correct that you would need to also bump nifi-mock to
> 1.10.0-SNAPSHPOT.
>
> An easy way to make sure everything gets set to 1.10.0-SNAPSHOT is to
> add a <dependencyManagement> block to the top-level pom of your
> services project (the parent pom of the service, service api, and
> NARs) and in the dep management declare the 1.10.0-SNAPSHOT for
> nifi-api and nifi-mock.
>
> As a side note, there is obviously a slight risk here building against
> 1.10.0 API and deploying to 1.9.2 since you could end up calling
> something in your code that only exists in 1.10.0, but since it sounds
> like you originally implemented against 1.9.2 then you are probably
> fine. Just wanted to point this out.
>
> On Mon, Jul 22, 2019 at 2:04 PM Aaron Rich <[hidden email]> wrote:
> >
> > Hi Jeff,
> > Thanks for the information on the release process.
> >
> > For the error I'm getting, I started a new custom service with:
> > mvn archetype:generate -DarchetypeGroupId=org.apache.nifi
> > -DarchetypeArtifactId=nifi-service-bundle-archetype
> > -DarchetypeVersion=1.9.2 -DnifiVersion=1.9.2
> >
> > This provides a shell test which I haven't updated.
> >
> > I updated the testservice-api-nar and the  testservice-nar poms like I did
> > with the processor to include:
> > <dependency>
> > <groupId>org.apache.nifi</groupId>
> > <artifactId>nifi-api</artifactId>
> > <version>1.10.0-SNAPSHOT</version>
> > </dependency>
> > </dependencies>
> >
> > <build>
> > <plugins>
> > <plugin>
> > <groupId>org.apache.nifi</groupId>
> > <artifactId>nifi-nar-maven-plugin</artifactId>
> > <version>1.3.1</version>
> > <extensions>true</extensions>
> > </plugin>
> > </plugins>
> > </build>
> >
> > On the testService-api-nar I still get the warning of:
> > [INFO] Generating documentation for NiFi extensions in the NAR...
> > [INFO] Found NAR dependency of
> > com.test.nifi:nifi-testService-api-nar:nar:1.0-SNAPSHOT:compile
> > [INFO] Found NAR dependency of
> > org.apache.nifi:nifi-standard-services-api-nar:nar:1.9.2:compile
> > [INFO] Found NAR dependency of
> > org.apache.nifi:nifi-jetty-bundle:nar:1.9.2:compile
> > [INFO] Found a dependency on version 1.9.2 of NiFi API
> > [WARNING] Could not generate extensions' documentation
> > org.apache.maven.plugin.MojoExecutionException: Failed to create Extension
> > Documentation
> >
> > When I was trying to trace back the 1.9.2 NiFi API dependency, I changed
> > the testService to have nifi-api version 1.10. This is when the compile
> > errors showed up.
> >
> > So at this point my question would be, for a custom service based on 1.9.2,
> > which pom.xml files need to be changed, and with what changes, to have the
> > service.nar file have the documentation to be able to be uploaded to nifi
> > registry via cli? Seems like there is more then just moving nifi-api to
> > version 1.10.
> >
> > Thank you again for the help.
> >
> > -Aaron
> >
> >
> >
> > On Mon, Jul 22, 2019 at 12:43 AM Jeff <[hidden email]> wrote:
> >
> > > Aaron,
> > >
> > > New versions of NiFi are generally released after a member of the community
> > > proposes a release and someone volunteers to perform the duties of the
> > > release manager.  There is no predefined schedule or projected release
> > > dates.  I think there are a few major feature JIRAs on which work is
> > > currently being done, and once those are resolved/finished, at that point
> > > someone may make proposal to create a release candidate for 1.10.0.
> > >
> > > Regarding the error to which you are referring, does the
> > > TestStandardMyService class make use of nifi-mock classes?  Is
> > > TestStandardMyService within the "src/main" folder of your module?  If so,
> > > you may want to move that class into "src/test", as I don't believe the
> > > nifi-maven-nar plugin bundles test code when it creates the NAR.  From the
> > > stacktrace, I see that TestStandardMyService makes a call to a
> > > org.apache.nifi.util.MockControllerServiceIntitializationContext method,
> > > which comes from nifi-mock.  If TestStandardMyService is included with the
> > > rest of your "production" code, you might want to refactor your module so
> > > that it is part of the "test" code.  That should remove the dependency on
> > > nifi-mock.
> > >
> > > On Mon, Jul 22, 2019 at 12:24 AM Aaron Rich <[hidden email]> wrote:
> > >
> > > > I updated  pom  for  a bundle with just a processor and was able to get
> > > nar
> > > > generated and loaded into registry via cli.
> > > >
> > > > When I tried a bundle with a custom service though, I get failures in the
> > > > test:
> > > > java.lang.AbstractMethodError:
> > > >
> > > >
> > > org.apache.nifi.util.MockControllerServiceInitializationContext.getNodeTypeProvider()Lorg/apache/nifi/controller/NodeTypeProvider;
> > > >         at
> > > >
> > > >
> > > com.ngc.swordfish.nifi.multipart.TestStandardMyService.testService(TestStandardMyService.java:36)
> > > >
> > > > Do I need to updated nifi-mock to 1.10 also?
> > > >
> > > > Thanks.
> > > >
> > > > -Aaron
> > > >
> > > > On Sun, Jul 21, 2019 at 9:59 PM Aaron Rich <[hidden email]> wrote:
> > > >
> > > > > Also,
> > > > >
> > > > > When is 1.10.0 planned for release?
> > > > >
> > > > > Thanks.
> > > > >
> > > > > -Aaron
> > > > >
> > > > >
> > > > > On Sun, Jul 21, 2019 at 9:29 PM Aaron Rich <[hidden email]>
> > > wrote:
> > > > >
> > > > >> Great.
> > > > >>
> > > > >> Thank you.
> > > > >>
> > > > >> Is the best way to force my nar to use the 1.10 nifi-api via adding it
> > > > as
> > > > >> dependency is main pom file?
> > > > >>
> > > > >> Assuming I still need the build plugin also?
> > > > >>
> > > > >> Thanks again.
> > > > >>
> > > > >> -Aaron
> > > > >>
> > > > >> On Sun, Jul 21, 2019 at 9:30 AM Bryan Bende <[hidden email]> wrote:
> > > > >>
> > > > >>> Hello,
> > > > >>>
> > > > >>> The issue is that in order to correctly generate the extension
> > > manifest
> > > > >>> with the new NAR plugin, it requires changes from nifi-api that are
> > > not
> > > > >>> released yet.
> > > > >>>
> > > > >>> You should be able to build NiFi on the master branch, really just
> > > the
> > > > >>> nifi-api module, doing a mvn clean install.
> > > > >>>
> > > > >>> Then in your NAR you’ll need to force it to use 1.10.0-SNAPSHOT of
> > > > >>> nifi-api. Currently you are getting 1.9.2 because of the parent of
> > > > >>> nifi-nar-bundles.
> > > > >>>
> > > > >>> Once we get 1.10.0 released then this won’t be an issue anymore.
> > > > >>>
> > > > >>> Thanks,
> > > > >>>
> > > > >>> Bryan
> > > > >>>
> > > > >>> On Sat, Jul 20, 2019 at 11:36 PM Aaron Rich <[hidden email]>
> > > > >>> wrote:
> > > > >>>
> > > > >>> > I forgot to include that when generating the .nar, I do get the
> > > > >>> warnings
> > > > >>> > of:
> > > > >>> > [WARNING] Could not generate extensions' documentation
> > > > >>> > org.apache.maven.plugin.MojoExecutionException: Failed to create
> > > > >>> Extension
> > > > >>> > Documentation
> > > > >>> > ...
> > > > >>> > Caused by: java.lang.NoSuchMethodException:
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > > org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)
> > > > >>> >
> > > > >>> > I believe this is tied to not getting the right nfi-api version?
> > > But
> > > > >>> the
> > > > >>> > parent is set to 1.9.2 so not sure how that is happening.
> > > > >>> >
> > > > >>> > Thanks again.
> > > > >>> >
> > > > >>> > -Aaron
> > > > >>> >
> > > > >>> > On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <[hidden email]>
> > > > >>> wrote:
> > > > >>> >
> > > > >>> > > Hi,
> > > > >>> > >
> > > > >>> > > I'm trying to determine the best way to CM custom nar files for
> > > > >>> sharing
> > > > >>> > > between team. We are using nifi-registry for the flows and I saw
> > > > >>> there
> > > > >>> > was
> > > > >>> > > a new capability via the toolkit cli for "upload-bundle".
> > > > >>> > >
> > > > >>> > > I'm trying to use that but have ran into a few issues:
> > > > >>> > > 1) I first wasn't getting the META-INF/docs/ in the .nar. I had
> > > > >>> built the
> > > > >>> > > initial project from mvn archetype:generate with version 1.9.2. I
> > > > >>> added:
> > > > >>> > > <build>
> > > > >>> > > <plugins>
> > > > >>> > > <plugin>
> > > > >>> > > <groupId>org.apache.nifi</groupId>
> > > > >>> > > <artifactId>nifi-nar-maven-plugin</artifactId>
> > > > >>> > > <version>1.3.1</version>
> > > > >>> > > <extensions>true</extensions>
> > > > >>> > > </plugin>
> > > > >>> > > </plugins>
> > > > >>> > > </build>
> > > > >>> > >
> > > > >>> > > To the base pom.xml. It has the parent of:
> > > > >>> > > <parent>
> > > > >>> > > <groupId>org.apache.nifi</groupId>
> > > > >>> > > <artifactId>nifi-nar-bundles</artifactId>
> > > > >>> > > <version>1.9.2</version>
> > > > >>> > > </parent>
> > > > >>> > >
> > > > >>> > > That got the docs in the jar:
> > > > >>> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
> > > > >>> > >     72 Sat Jul 20 20:53:44 MDT 2019
> > > > >>> META-INF/docs/extension-manifest.xml
> > > > >>> > >      0 Sat Jul 20 20:53:44 MDT 2019
> > > > META-INF/docs/additional-details/
> > > > >>> > >
> > > > >>> > > 2)I tried to use cli then with command:
> > > > >>> > > ./bin/cli.sh registry upload-bundle --baseUrl
> > > > >>> http://localhost:18080/
> > > > >>> > > --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar
> > > -ebf
> > > > >>> > > .custom-processor-nar-1.0-SNAPSHOT.nar
> > > > >>> > >
> > > > >>> > > I get back "ERROR: Error executing command 'upload-bundle' :
> > > Error
> > > > >>> > > creating extension bundle version: An unexpected error has
> > > > occurred.
> > > > >>> > Please
> > > > >>> > > check the logs for additional details."
> > > > >>> > >
> > > > >>> > > In Registry log, error is:
> > > > >>> > > Exception: Unable to obtain extension info for bundle due to:
> > > > Unable
> > > > >>> to
> > > > >>> > > parse extension manifest due to: Unexpected end of input block;
> > > > >>> expected
> > > > >>> > an
> > > > >>> > > identifier
> > > > >>> > >  at [row,col {unknown-source}]: [1,72]
> > > > >>> > >  at [Source:
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > > (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
> > > > >>> > > line: 1, column: 73]. Returning Internal Server Error response.
> > > > >>> > > org.apache.nifi.registry.bundle.extract.BundleException: Unable
> > > to
> > > > >>> obtain
> > > > >>> > > extension info for bundle due to: Unable to parse extension
> > > > manifest
> > > > >>> due
> > > > >>> > > to: Unexpected end of input
> > > > >>> > > block; expected an identifier
> > > > >>> > >
> > > > >>> > > Questions:
> > > > >>> > > 1) Should I need to set the 1.3.1 plugin do get the docs in nar?
> > > I
> > > > >>> > thought
> > > > >>> > > that would be automatic from parent version being 1.9.2
> > > > >>> > > 2) Is there an example for how to use the toolkit cli to
> > > > >>> upload-bundle?
> > > > >>> > > 3) Is there a better way to CM .nar? Archiva? Artifactory?
> > > > >>> > >
> > > > >>> > > Thanks.
> > > > >>> > >
> > > > >>> > > -Aaron
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>> --
> > > > >>> Sent from Gmail Mobile
> > > > >>>
> > > > >>
> > > >
> > >
Reply | Threaded
Open this post in threaded view
|

Re: CM of nar via toolkit cli and regisry

Jeff
Aaron,

Sorry, late night and I read your previous email too quickly.  Bryan is
correct, and I just tested it.  In the root pom, I added:

    <dependencyManagement>

>         <dependencies>
>             <dependency>
>                 <groupId>org.apache.nifi</groupId>
>                 <artifactId>nifi-api</artifactId>
>                 <version>1.10.0-SNAPSHOT</version>
>             </dependency>
>             <dependency>
>                 <groupId>org.apache.nifi</groupId>
>                 <artifactId>nifi-mock</artifactId>
>                 <version>1.10.0-SNAPSHOT</version>
>             </dependency>
>         </dependencies>
>     </dependencyManagement>
>

I also removed the version tag for nifi-mock (so that it inherits the
version specified in dependencyManagement) from the nifi-module (not
nifi-module-api, nifi-module-api-nar, or nifi-module-nar), and the build
succeeded.

On Mon, Jul 22, 2019 at 2:24 PM Bryan Bende <[hidden email]> wrote:

> One more thing, make sure the <scope> on the dependencies stays the
> same, meaning nifi-api should still be provided. You don't want to end
> up bundling the 1.10.0 nifi-api into your NARs.
>
> On Mon, Jul 22, 2019 at 2:21 PM Bryan Bende <[hidden email]> wrote:
> >
> > I think you are correct that you would need to also bump nifi-mock to
> > 1.10.0-SNAPSHPOT.
> >
> > An easy way to make sure everything gets set to 1.10.0-SNAPSHOT is to
> > add a <dependencyManagement> block to the top-level pom of your
> > services project (the parent pom of the service, service api, and
> > NARs) and in the dep management declare the 1.10.0-SNAPSHOT for
> > nifi-api and nifi-mock.
> >
> > As a side note, there is obviously a slight risk here building against
> > 1.10.0 API and deploying to 1.9.2 since you could end up calling
> > something in your code that only exists in 1.10.0, but since it sounds
> > like you originally implemented against 1.9.2 then you are probably
> > fine. Just wanted to point this out.
> >
> > On Mon, Jul 22, 2019 at 2:04 PM Aaron Rich <[hidden email]> wrote:
> > >
> > > Hi Jeff,
> > > Thanks for the information on the release process.
> > >
> > > For the error I'm getting, I started a new custom service with:
> > > mvn archetype:generate -DarchetypeGroupId=org.apache.nifi
> > > -DarchetypeArtifactId=nifi-service-bundle-archetype
> > > -DarchetypeVersion=1.9.2 -DnifiVersion=1.9.2
> > >
> > > This provides a shell test which I haven't updated.
> > >
> > > I updated the testservice-api-nar and the  testservice-nar poms like I
> did
> > > with the processor to include:
> > > <dependency>
> > > <groupId>org.apache.nifi</groupId>
> > > <artifactId>nifi-api</artifactId>
> > > <version>1.10.0-SNAPSHOT</version>
> > > </dependency>
> > > </dependencies>
> > >
> > > <build>
> > > <plugins>
> > > <plugin>
> > > <groupId>org.apache.nifi</groupId>
> > > <artifactId>nifi-nar-maven-plugin</artifactId>
> > > <version>1.3.1</version>
> > > <extensions>true</extensions>
> > > </plugin>
> > > </plugins>
> > > </build>
> > >
> > > On the testService-api-nar I still get the warning of:
> > > [INFO] Generating documentation for NiFi extensions in the NAR...
> > > [INFO] Found NAR dependency of
> > > com.test.nifi:nifi-testService-api-nar:nar:1.0-SNAPSHOT:compile
> > > [INFO] Found NAR dependency of
> > > org.apache.nifi:nifi-standard-services-api-nar:nar:1.9.2:compile
> > > [INFO] Found NAR dependency of
> > > org.apache.nifi:nifi-jetty-bundle:nar:1.9.2:compile
> > > [INFO] Found a dependency on version 1.9.2 of NiFi API
> > > [WARNING] Could not generate extensions' documentation
> > > org.apache.maven.plugin.MojoExecutionException: Failed to create
> Extension
> > > Documentation
> > >
> > > When I was trying to trace back the 1.9.2 NiFi API dependency, I
> changed
> > > the testService to have nifi-api version 1.10. This is when the compile
> > > errors showed up.
> > >
> > > So at this point my question would be, for a custom service based on
> 1.9.2,
> > > which pom.xml files need to be changed, and with what changes, to have
> the
> > > service.nar file have the documentation to be able to be uploaded to
> nifi
> > > registry via cli? Seems like there is more then just moving nifi-api to
> > > version 1.10.
> > >
> > > Thank you again for the help.
> > >
> > > -Aaron
> > >
> > >
> > >
> > > On Mon, Jul 22, 2019 at 12:43 AM Jeff <[hidden email]> wrote:
> > >
> > > > Aaron,
> > > >
> > > > New versions of NiFi are generally released after a member of the
> community
> > > > proposes a release and someone volunteers to perform the duties of
> the
> > > > release manager.  There is no predefined schedule or projected
> release
> > > > dates.  I think there are a few major feature JIRAs on which work is
> > > > currently being done, and once those are resolved/finished, at that
> point
> > > > someone may make proposal to create a release candidate for 1.10.0.
> > > >
> > > > Regarding the error to which you are referring, does the
> > > > TestStandardMyService class make use of nifi-mock classes?  Is
> > > > TestStandardMyService within the "src/main" folder of your module?
> If so,
> > > > you may want to move that class into "src/test", as I don't believe
> the
> > > > nifi-maven-nar plugin bundles test code when it creates the NAR.
> From the
> > > > stacktrace, I see that TestStandardMyService makes a call to a
> > > > org.apache.nifi.util.MockControllerServiceIntitializationContext
> method,
> > > > which comes from nifi-mock.  If TestStandardMyService is included
> with the
> > > > rest of your "production" code, you might want to refactor your
> module so
> > > > that it is part of the "test" code.  That should remove the
> dependency on
> > > > nifi-mock.
> > > >
> > > > On Mon, Jul 22, 2019 at 12:24 AM Aaron Rich <[hidden email]>
> wrote:
> > > >
> > > > > I updated  pom  for  a bundle with just a processor and was able
> to get
> > > > nar
> > > > > generated and loaded into registry via cli.
> > > > >
> > > > > When I tried a bundle with a custom service though, I get failures
> in the
> > > > > test:
> > > > > java.lang.AbstractMethodError:
> > > > >
> > > > >
> > > >
> org.apache.nifi.util.MockControllerServiceInitializationContext.getNodeTypeProvider()Lorg/apache/nifi/controller/NodeTypeProvider;
> > > > >         at
> > > > >
> > > > >
> > > >
> com.ngc.swordfish.nifi.multipart.TestStandardMyService.testService(TestStandardMyService.java:36)
> > > > >
> > > > > Do I need to updated nifi-mock to 1.10 also?
> > > > >
> > > > > Thanks.
> > > > >
> > > > > -Aaron
> > > > >
> > > > > On Sun, Jul 21, 2019 at 9:59 PM Aaron Rich <[hidden email]>
> wrote:
> > > > >
> > > > > > Also,
> > > > > >
> > > > > > When is 1.10.0 planned for release?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > -Aaron
> > > > > >
> > > > > >
> > > > > > On Sun, Jul 21, 2019 at 9:29 PM Aaron Rich <[hidden email]
> >
> > > > wrote:
> > > > > >
> > > > > >> Great.
> > > > > >>
> > > > > >> Thank you.
> > > > > >>
> > > > > >> Is the best way to force my nar to use the 1.10 nifi-api via
> adding it
> > > > > as
> > > > > >> dependency is main pom file?
> > > > > >>
> > > > > >> Assuming I still need the build plugin also?
> > > > > >>
> > > > > >> Thanks again.
> > > > > >>
> > > > > >> -Aaron
> > > > > >>
> > > > > >> On Sun, Jul 21, 2019 at 9:30 AM Bryan Bende <[hidden email]>
> wrote:
> > > > > >>
> > > > > >>> Hello,
> > > > > >>>
> > > > > >>> The issue is that in order to correctly generate the extension
> > > > manifest
> > > > > >>> with the new NAR plugin, it requires changes from nifi-api
> that are
> > > > not
> > > > > >>> released yet.
> > > > > >>>
> > > > > >>> You should be able to build NiFi on the master branch, really
> just
> > > > the
> > > > > >>> nifi-api module, doing a mvn clean install.
> > > > > >>>
> > > > > >>> Then in your NAR you’ll need to force it to use
> 1.10.0-SNAPSHOT of
> > > > > >>> nifi-api. Currently you are getting 1.9.2 because of the
> parent of
> > > > > >>> nifi-nar-bundles.
> > > > > >>>
> > > > > >>> Once we get 1.10.0 released then this won’t be an issue
> anymore.
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>>
> > > > > >>> Bryan
> > > > > >>>
> > > > > >>> On Sat, Jul 20, 2019 at 11:36 PM Aaron Rich <
> [hidden email]>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>> > I forgot to include that when generating the .nar, I do get
> the
> > > > > >>> warnings
> > > > > >>> > of:
> > > > > >>> > [WARNING] Could not generate extensions' documentation
> > > > > >>> > org.apache.maven.plugin.MojoExecutionException: Failed to
> create
> > > > > >>> Extension
> > > > > >>> > Documentation
> > > > > >>> > ...
> > > > > >>> > Caused by: java.lang.NoSuchMethodException:
> > > > > >>> >
> > > > > >>> >
> > > > > >>>
> > > > >
> > > >
> org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)
> > > > > >>> >
> > > > > >>> > I believe this is tied to not getting the right nfi-api
> version?
> > > > But
> > > > > >>> the
> > > > > >>> > parent is set to 1.9.2 so not sure how that is happening.
> > > > > >>> >
> > > > > >>> > Thanks again.
> > > > > >>> >
> > > > > >>> > -Aaron
> > > > > >>> >
> > > > > >>> > On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <
> [hidden email]>
> > > > > >>> wrote:
> > > > > >>> >
> > > > > >>> > > Hi,
> > > > > >>> > >
> > > > > >>> > > I'm trying to determine the best way to CM custom nar
> files for
> > > > > >>> sharing
> > > > > >>> > > between team. We are using nifi-registry for the flows and
> I saw
> > > > > >>> there
> > > > > >>> > was
> > > > > >>> > > a new capability via the toolkit cli for "upload-bundle".
> > > > > >>> > >
> > > > > >>> > > I'm trying to use that but have ran into a few issues:
> > > > > >>> > > 1) I first wasn't getting the META-INF/docs/ in the .nar.
> I had
> > > > > >>> built the
> > > > > >>> > > initial project from mvn archetype:generate with version
> 1.9.2. I
> > > > > >>> added:
> > > > > >>> > > <build>
> > > > > >>> > > <plugins>
> > > > > >>> > > <plugin>
> > > > > >>> > > <groupId>org.apache.nifi</groupId>
> > > > > >>> > > <artifactId>nifi-nar-maven-plugin</artifactId>
> > > > > >>> > > <version>1.3.1</version>
> > > > > >>> > > <extensions>true</extensions>
> > > > > >>> > > </plugin>
> > > > > >>> > > </plugins>
> > > > > >>> > > </build>
> > > > > >>> > >
> > > > > >>> > > To the base pom.xml. It has the parent of:
> > > > > >>> > > <parent>
> > > > > >>> > > <groupId>org.apache.nifi</groupId>
> > > > > >>> > > <artifactId>nifi-nar-bundles</artifactId>
> > > > > >>> > > <version>1.9.2</version>
> > > > > >>> > > </parent>
> > > > > >>> > >
> > > > > >>> > > That got the docs in the jar:
> > > > > >>> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
> > > > > >>> > >     72 Sat Jul 20 20:53:44 MDT 2019
> > > > > >>> META-INF/docs/extension-manifest.xml
> > > > > >>> > >      0 Sat Jul 20 20:53:44 MDT 2019
> > > > > META-INF/docs/additional-details/
> > > > > >>> > >
> > > > > >>> > > 2)I tried to use cli then with command:
> > > > > >>> > > ./bin/cli.sh registry upload-bundle --baseUrl
> > > > > >>> http://localhost:18080/
> > > > > >>> > > --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt
> nifi-nar
> > > > -ebf
> > > > > >>> > > .custom-processor-nar-1.0-SNAPSHOT.nar
> > > > > >>> > >
> > > > > >>> > > I get back "ERROR: Error executing command 'upload-bundle'
> :
> > > > Error
> > > > > >>> > > creating extension bundle version: An unexpected error has
> > > > > occurred.
> > > > > >>> > Please
> > > > > >>> > > check the logs for additional details."
> > > > > >>> > >
> > > > > >>> > > In Registry log, error is:
> > > > > >>> > > Exception: Unable to obtain extension info for bundle due
> to:
> > > > > Unable
> > > > > >>> to
> > > > > >>> > > parse extension manifest due to: Unexpected end of input
> block;
> > > > > >>> expected
> > > > > >>> > an
> > > > > >>> > > identifier
> > > > > >>> > >  at [row,col {unknown-source}]: [1,72]
> > > > > >>> > >  at [Source:
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > >
> > > >
> (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
> > > > > >>> > > line: 1, column: 73]. Returning Internal Server Error
> response.
> > > > > >>> > > org.apache.nifi.registry.bundle.extract.BundleException:
> Unable
> > > > to
> > > > > >>> obtain
> > > > > >>> > > extension info for bundle due to: Unable to parse extension
> > > > > manifest
> > > > > >>> due
> > > > > >>> > > to: Unexpected end of input
> > > > > >>> > > block; expected an identifier
> > > > > >>> > >
> > > > > >>> > > Questions:
> > > > > >>> > > 1) Should I need to set the 1.3.1 plugin do get the docs
> in nar?
> > > > I
> > > > > >>> > thought
> > > > > >>> > > that would be automatic from parent version being 1.9.2
> > > > > >>> > > 2) Is there an example for how to use the toolkit cli to
> > > > > >>> upload-bundle?
> > > > > >>> > > 3) Is there a better way to CM .nar? Archiva? Artifactory?
> > > > > >>> > >
> > > > > >>> > > Thanks.
> > > > > >>> > >
> > > > > >>> > > -Aaron
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>> --
> > > > > >>> Sent from Gmail Mobile
> > > > > >>>
> > > > > >>
> > > > >
> > > >
>