Classloader issues

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Classloader issues

Phil H
Hi guys,

I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.

I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.

Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).

Pulling my hair out here - would love some help!

Cheers,
Phil
Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Joe Witt
Can you share the code by chance for a review?  Otherwise, you'll want
to add a debugger at runtime and examine the context classloader as it
goes through to see where it goes wonky.

What version are you on?

Thanks

On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:

> Hi guys,
>
> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>
> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>
> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>
> Pulling my hair out here - would love some help!
>
> Cheers,
> Phil
Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Matt Burgess-2
In reply to this post by Phil H
Phil,

Any chance you could share your NAR and processor POMs? Your processor
POM should have a dependency on nifi-utils. What are the other classes
that aren't found? I'm curious if they are in the same JAR/NAR or not.
Do you have a parent NAR set (in your bundle's NAR POM)? Also once
your NAR is built, does it appear to have the correct contents? You
can check with "jar tvf /path/to/my/nifi-bundle.nar"

Regards,
Matt


On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:

> Hi guys,
>
> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>
> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>
> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>
> Pulling my hair out here - would love some help!
>
> Cheers,
> Phil
Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Phil H
In reply to this post by Joe Witt
I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:

    public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
            .Builder().name("ID_PATH")
            .displayName("JSON ID Path")
            .description("The path to the JSON attribute that represents the unique ID for the object")
            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
            .required(true)
            .build();


2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
        at java.util.ServiceLoader.fail(ServiceLoader.java:232)
        at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
        at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
        at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
        at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
        at org.apache.nifi.NiFi.<init>(NiFi.java:160)
        at org.apache.nifi.NiFi.main(NiFi.java:267)
Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
        at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.Class.newInstance(Class.java:442)
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
        ... 7 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 14 common frames omitted


> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>
> Can you share the code by chance for a review?  Otherwise, you'll want
> to add a debugger at runtime and examine the context classloader as it
> goes through to see where it goes wonky.
>
> What version are you on?
>
> Thanks
>
> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>> Hi guys,
>>
>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>
>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>
>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>
>> Pulling my hair out here - would love some help!
>>
>> Cheers,
>> Phil

Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Phil H
Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)

> On 7 Nov 2017, at 4:17 am, Phil H <[hidden email]> wrote:
>
> I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:
>
>    public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>            .Builder().name("ID_PATH")
>            .displayName("JSON ID Path")
>            .description("The path to the JSON attribute that represents the unique ID for the object")
>            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>            .required(true)
>            .build();
>
>
> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
> at org.apache.nifi.NiFi.main(NiFi.java:267)
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at java.lang.Class.newInstance(Class.java:442)
> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
> ... 7 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 14 common frames omitted
>
>
>> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>>
>> Can you share the code by chance for a review?  Otherwise, you'll want
>> to add a debugger at runtime and examine the context classloader as it
>> goes through to see where it goes wonky.
>>
>> What version are you on?
>>
>> Thanks
>>
>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>>> Hi guys,
>>>
>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>
>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>
>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>
>>> Pulling my hair out here - would love some help!
>>>
>>> Cheers,
>>> Phil
>

Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Bryan Bende
It is most likely an issue with the Maven configuration in one of your modules.

Can you share your project, or the pom files for the processors, NAR,
and bundle?

Thanks,

Bryan


On Mon, Nov 6, 2017 at 12:20 PM, Phil H <[hidden email]> wrote:

> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>
>> On 7 Nov 2017, at 4:17 am, Phil H <[hidden email]> wrote:
>>
>> I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>
>>    public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>            .Builder().name("ID_PATH")
>>            .displayName("JSON ID Path")
>>            .description("The path to the JSON attribute that represents the unique ID for the object")
>>            .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>            .required(true)
>>            .build();
>>
>>
>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>       at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>       at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>       at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>       at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>       at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>       at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>       at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>       at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>       at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>       at org.apache.nifi.NiFi.main(NiFi.java:267)
>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>       at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>       at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>       at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>       at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>       at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>       at java.lang.Class.newInstance(Class.java:442)
>>       at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>       ... 7 common frames omitted
>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>       at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>       at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>       at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>       at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>       ... 14 common frames omitted
>>
>>
>>> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>>>
>>> Can you share the code by chance for a review?  Otherwise, you'll want
>>> to add a debugger at runtime and examine the context classloader as it
>>> goes through to see where it goes wonky.
>>>
>>> What version are you on?
>>>
>>> Thanks
>>>
>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>>>> Hi guys,
>>>>
>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>
>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>
>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>
>>>> Pulling my hair out here - would love some help!
>>>>
>>>> Cheers,
>>>> Phil
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Phil H
./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-nar-bundles</artifactId>
        <version>1.3.0</version>
    </parent>

    <repositories>
        <repository>
            <id>project.local</id>
            <name>projects</name>
            <url>file:${project.basedir}/repo</url>
        </repository>
    </repositories>

    <groupId>com.jidmu</groupId>
    <artifactId>JSONCondenser</artifactId>
    <version>0.1</version>
    <packaging>pom</packaging>

    <modules>
        <module>nifi-JSONCondenser-processors</module>
        <module>nifi-JSONCondenser-nar</module>
    </modules>

    <dependencies>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>JSON</artifactId>
            <version>1.0</version>
        </dependency>
    </dependencies>

</project>



./nifi-JSONCondenser-processors/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>com.jidmu</groupId>
        <artifactId>JSONCondenser</artifactId>
        <version>0.1</version>
    </parent>

    <artifactId>nifi-JSONCondenser-processors</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-utils</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-mock</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

./nifi-JSONCondenser-nar/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>com.jidmu</groupId>
        <artifactId>JSONCondenser</artifactId>
        <version>0.1</version>
    </parent>

    <artifactId>nifi-JSONCondenser-nar</artifactId>
    <version>0.1</version>
    <packaging>nar</packaging>
    <properties>
        <maven.javadoc.skip>true</maven.javadoc.skip>
        <source.skip>true</source.skip>
    </properties>

    <dependencies>
        <dependency>
            <groupId>com.jidmu</groupId>
            <artifactId>nifi-JSONCondenser-processors</artifactId>
            <version>0.1</version>
        </dependency>
    </dependencies>

</project>



> On 7 Nov 2017, at 4:39 am, Bryan Bende <[hidden email]> wrote:
>
> It is most likely an issue with the Maven configuration in one of your modules.
>
> Can you share your project, or the pom files for the processors, NAR,
> and bundle?
>
> Thanks,
>
> Bryan
>
>
> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <[hidden email]> wrote:
>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>
>>> On 7 Nov 2017, at 4:17 am, Phil H <[hidden email]> wrote:
>>>
>>> I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>
>>>   public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>           .Builder().name("ID_PATH")
>>>           .displayName("JSON ID Path")
>>>           .description("The path to the JSON attribute that represents the unique ID for the object")
>>>           .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>           .required(true)
>>>           .build();
>>>
>>>
>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>      at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>      at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>      at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>      at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>      at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>      at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>      at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>      at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>      at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>      at org.apache.nifi.NiFi.main(NiFi.java:267)
>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>      at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>      at java.lang.Class.newInstance(Class.java:442)
>>>      at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>      ... 7 common frames omitted
>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>      at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>      ... 14 common frames omitted
>>>
>>>
>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>>>>
>>>> Can you share the code by chance for a review?  Otherwise, you'll want
>>>> to add a debugger at runtime and examine the context classloader as it
>>>> goes through to see where it goes wonky.
>>>>
>>>> What version are you on?
>>>>
>>>> Thanks
>>>>
>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>>>>> Hi guys,
>>>>>
>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>
>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>
>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>
>>>>> Pulling my hair out here - would love some help!
>>>>>
>>>>> Cheers,
>>>>> Phil
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Bryan Bende
Thanks for the poms.

Can you provide the output of listing
NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
?

and also the output of listing JARs that are in NiFi's lib directory?
ls -l NIFI_HOME/lib/ | grep jar

Want to verify that nifi-utils JAR is actually in your NAR and also
that no other unexpected JARs are in your lib directory.

Thanks,

Bryan

On Mon, Nov 6, 2017 at 12:48 PM, Phil H <[hidden email]> wrote:

> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>
> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>     <modelVersion>4.0.0</modelVersion>
>
>     <parent>
>         <groupId>org.apache.nifi</groupId>
>         <artifactId>nifi-nar-bundles</artifactId>
>         <version>1.3.0</version>
>     </parent>
>
>     <repositories>
>         <repository>
>             <id>project.local</id>
>             <name>projects</name>
>             <url>file:${project.basedir}/repo</url>
>         </repository>
>     </repositories>
>
>     <groupId>com.jidmu</groupId>
>     <artifactId>JSONCondenser</artifactId>
>     <version>0.1</version>
>     <packaging>pom</packaging>
>
>     <modules>
>         <module>nifi-JSONCondenser-processors</module>
>         <module>nifi-JSONCondenser-nar</module>
>     </modules>
>
>     <dependencies>
>         <dependency>
>             <groupId>org.json</groupId>
>             <artifactId>JSON</artifactId>
>             <version>1.0</version>
>         </dependency>
>     </dependencies>
>
> </project>
>
>
>
> ./nifi-JSONCondenser-processors/pom.xml
> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>     <modelVersion>4.0.0</modelVersion>
>
>     <parent>
>         <groupId>com.jidmu</groupId>
>         <artifactId>JSONCondenser</artifactId>
>         <version>0.1</version>
>     </parent>
>
>     <artifactId>nifi-JSONCondenser-processors</artifactId>
>     <packaging>jar</packaging>
>
>     <dependencies>
>         <dependency>
>             <groupId>org.apache.nifi</groupId>
>             <artifactId>nifi-api</artifactId>
>         </dependency>
>         <dependency>
>             <groupId>org.apache.nifi</groupId>
>             <artifactId>nifi-utils</artifactId>
>         </dependency>
>         <dependency>
>             <groupId>org.apache.nifi</groupId>
>             <artifactId>nifi-mock</artifactId>
>             <scope>test</scope>
>         </dependency>
>         <dependency>
>             <groupId>org.slf4j</groupId>
>             <artifactId>slf4j-simple</artifactId>
>             <scope>test</scope>
>         </dependency>
>         <dependency>
>             <groupId>junit</groupId>
>             <artifactId>junit</artifactId>
>             <scope>test</scope>
>         </dependency>
>     </dependencies>
> </project>
>
> ./nifi-JSONCondenser-nar/pom.xml
> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>     <modelVersion>4.0.0</modelVersion>
>
>     <parent>
>         <groupId>com.jidmu</groupId>
>         <artifactId>JSONCondenser</artifactId>
>         <version>0.1</version>
>     </parent>
>
>     <artifactId>nifi-JSONCondenser-nar</artifactId>
>     <version>0.1</version>
>     <packaging>nar</packaging>
>     <properties>
>         <maven.javadoc.skip>true</maven.javadoc.skip>
>         <source.skip>true</source.skip>
>     </properties>
>
>     <dependencies>
>         <dependency>
>             <groupId>com.jidmu</groupId>
>             <artifactId>nifi-JSONCondenser-processors</artifactId>
>             <version>0.1</version>
>         </dependency>
>     </dependencies>
>
> </project>
>
>
>
>> On 7 Nov 2017, at 4:39 am, Bryan Bende <[hidden email]> wrote:
>>
>> It is most likely an issue with the Maven configuration in one of your modules.
>>
>> Can you share your project, or the pom files for the processors, NAR,
>> and bundle?
>>
>> Thanks,
>>
>> Bryan
>>
>>
>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <[hidden email]> wrote:
>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>
>>>> On 7 Nov 2017, at 4:17 am, Phil H <[hidden email]> wrote:
>>>>
>>>> I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>
>>>>   public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>           .Builder().name("ID_PATH")
>>>>           .displayName("JSON ID Path")
>>>>           .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>           .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>           .required(true)
>>>>           .build();
>>>>
>>>>
>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>      at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>      at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>      at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>      at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>      at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>      at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>      at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>      at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>      at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>      at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>      at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>      at java.lang.Class.newInstance(Class.java:442)
>>>>      at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>      ... 7 common frames omitted
>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>      at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>      ... 14 common frames omitted
>>>>
>>>>
>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>>>>>
>>>>> Can you share the code by chance for a review?  Otherwise, you'll want
>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>> goes through to see where it goes wonky.
>>>>>
>>>>> What version are you on?
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>>>>>> Hi guys,
>>>>>>
>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>
>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>
>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>
>>>>>> Pulling my hair out here - would love some help!
>>>>>>
>>>>>> Cheers,
>>>>>> Phil
>>>>
>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Phil H
Thanks Bryan,

This new NAR does not appear in the extensions directory (my other working ones do).

As for your second question

[phil@localhost JSONCondenser]$ ls  ~/nifi-1.3.0/lib/ | grep jar
javax.servlet-api-3.1.0.jar
jcl-over-slf4j-1.7.25.jar
jetty-schemas-3.1.jar
jul-to-slf4j-1.7.25.jar
log4j-over-slf4j-1.7.25.jar
logback-classic-1.2.3.jar
logback-core-1.2.3.jar
nifi-api-1.3.0.jar
nifi-framework-api-1.3.0.jar
nifi-JSONCondenser-processors-0.1.jar
nifi-nar-utils-1.3.0.jar
nifi-properties-1.3.0.jar
nifi-runtime-1.3.0.jar
slf4j-api-1.7.25.jar


> On 7 Nov 2017, at 5:13 am, Bryan Bende <[hidden email]> wrote:
>
> Thanks for the poms.
>
> Can you provide the output of listing
> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
> ?
>
> and also the output of listing JARs that are in NiFi's lib directory?
> ls -l NIFI_HOME/lib/ | grep jar
>
> Want to verify that nifi-utils JAR is actually in your NAR and also
> that no other unexpected JARs are in your lib directory.
>
> Thanks,
>
> Bryan
>
> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <[hidden email]> wrote:
>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>
>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>    <modelVersion>4.0.0</modelVersion>
>>
>>    <parent>
>>        <groupId>org.apache.nifi</groupId>
>>        <artifactId>nifi-nar-bundles</artifactId>
>>        <version>1.3.0</version>
>>    </parent>
>>
>>    <repositories>
>>        <repository>
>>            <id>project.local</id>
>>            <name>projects</name>
>>            <url>file:${project.basedir}/repo</url>
>>        </repository>
>>    </repositories>
>>
>>    <groupId>com.jidmu</groupId>
>>    <artifactId>JSONCondenser</artifactId>
>>    <version>0.1</version>
>>    <packaging>pom</packaging>
>>
>>    <modules>
>>        <module>nifi-JSONCondenser-processors</module>
>>        <module>nifi-JSONCondenser-nar</module>
>>    </modules>
>>
>>    <dependencies>
>>        <dependency>
>>            <groupId>org.json</groupId>
>>            <artifactId>JSON</artifactId>
>>            <version>1.0</version>
>>        </dependency>
>>    </dependencies>
>>
>> </project>
>>
>>
>>
>> ./nifi-JSONCondenser-processors/pom.xml
>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>    <modelVersion>4.0.0</modelVersion>
>>
>>    <parent>
>>        <groupId>com.jidmu</groupId>
>>        <artifactId>JSONCondenser</artifactId>
>>        <version>0.1</version>
>>    </parent>
>>
>>    <artifactId>nifi-JSONCondenser-processors</artifactId>
>>    <packaging>jar</packaging>
>>
>>    <dependencies>
>>        <dependency>
>>            <groupId>org.apache.nifi</groupId>
>>            <artifactId>nifi-api</artifactId>
>>        </dependency>
>>        <dependency>
>>            <groupId>org.apache.nifi</groupId>
>>            <artifactId>nifi-utils</artifactId>
>>        </dependency>
>>        <dependency>
>>            <groupId>org.apache.nifi</groupId>
>>            <artifactId>nifi-mock</artifactId>
>>            <scope>test</scope>
>>        </dependency>
>>        <dependency>
>>            <groupId>org.slf4j</groupId>
>>            <artifactId>slf4j-simple</artifactId>
>>            <scope>test</scope>
>>        </dependency>
>>        <dependency>
>>            <groupId>junit</groupId>
>>            <artifactId>junit</artifactId>
>>            <scope>test</scope>
>>        </dependency>
>>    </dependencies>
>> </project>
>>
>> ./nifi-JSONCondenser-nar/pom.xml
>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>    <modelVersion>4.0.0</modelVersion>
>>
>>    <parent>
>>        <groupId>com.jidmu</groupId>
>>        <artifactId>JSONCondenser</artifactId>
>>        <version>0.1</version>
>>    </parent>
>>
>>    <artifactId>nifi-JSONCondenser-nar</artifactId>
>>    <version>0.1</version>
>>    <packaging>nar</packaging>
>>    <properties>
>>        <maven.javadoc.skip>true</maven.javadoc.skip>
>>        <source.skip>true</source.skip>
>>    </properties>
>>
>>    <dependencies>
>>        <dependency>
>>            <groupId>com.jidmu</groupId>
>>            <artifactId>nifi-JSONCondenser-processors</artifactId>
>>            <version>0.1</version>
>>        </dependency>
>>    </dependencies>
>>
>> </project>
>>
>>
>>
>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <[hidden email]> wrote:
>>>
>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>
>>> Can you share your project, or the pom files for the processors, NAR,
>>> and bundle?
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>>
>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <[hidden email]> wrote:
>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>
>>>>> On 7 Nov 2017, at 4:17 am, Phil H <[hidden email]> wrote:
>>>>>
>>>>> I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>
>>>>>  public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>          .Builder().name("ID_PATH")
>>>>>          .displayName("JSON ID Path")
>>>>>          .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>          .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>          .required(true)
>>>>>          .build();
>>>>>
>>>>>
>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>     at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>     at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>     at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>     at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>     at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>     at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>     at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>     at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>     at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>     at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>     at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>     at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>     at java.lang.Class.newInstance(Class.java:442)
>>>>>     at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>     ... 7 common frames omitted
>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>     ... 14 common frames omitted
>>>>>
>>>>>
>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>>>>>>
>>>>>> Can you share the code by chance for a review?  Otherwise, you'll want
>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>> goes through to see where it goes wonky.
>>>>>>
>>>>>> What version are you on?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>>>>>>> Hi guys,
>>>>>>>
>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>
>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>
>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>
>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Phil
>>>>>
>>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Phil H
Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file



> On 7 Nov 2017, at 5:17 am, Phil H <[hidden email]> wrote:
>
> Thanks Bryan,
>
> This new NAR does not appear in the extensions directory (my other working ones do).
>
> As for your second question
>
> [phil@localhost JSONCondenser]$ ls  ~/nifi-1.3.0/lib/ | grep jar
> javax.servlet-api-3.1.0.jar
> jcl-over-slf4j-1.7.25.jar
> jetty-schemas-3.1.jar
> jul-to-slf4j-1.7.25.jar
> log4j-over-slf4j-1.7.25.jar
> logback-classic-1.2.3.jar
> logback-core-1.2.3.jar
> nifi-api-1.3.0.jar
> nifi-framework-api-1.3.0.jar
> nifi-JSONCondenser-processors-0.1.jar
> nifi-nar-utils-1.3.0.jar
> nifi-properties-1.3.0.jar
> nifi-runtime-1.3.0.jar
> slf4j-api-1.7.25.jar
>
>
>> On 7 Nov 2017, at 5:13 am, Bryan Bende <[hidden email]> wrote:
>>
>> Thanks for the poms.
>>
>> Can you provide the output of listing
>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>> ?
>>
>> and also the output of listing JARs that are in NiFi's lib directory?
>> ls -l NIFI_HOME/lib/ | grep jar
>>
>> Want to verify that nifi-utils JAR is actually in your NAR and also
>> that no other unexpected JARs are in your lib directory.
>>
>> Thanks,
>>
>> Bryan
>>
>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <[hidden email]> wrote:
>>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>>
>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>   <modelVersion>4.0.0</modelVersion>
>>>
>>>   <parent>
>>>       <groupId>org.apache.nifi</groupId>
>>>       <artifactId>nifi-nar-bundles</artifactId>
>>>       <version>1.3.0</version>
>>>   </parent>
>>>
>>>   <repositories>
>>>       <repository>
>>>           <id>project.local</id>
>>>           <name>projects</name>
>>>           <url>file:${project.basedir}/repo</url>
>>>       </repository>
>>>   </repositories>
>>>
>>>   <groupId>com.jidmu</groupId>
>>>   <artifactId>JSONCondenser</artifactId>
>>>   <version>0.1</version>
>>>   <packaging>pom</packaging>
>>>
>>>   <modules>
>>>       <module>nifi-JSONCondenser-processors</module>
>>>       <module>nifi-JSONCondenser-nar</module>
>>>   </modules>
>>>
>>>   <dependencies>
>>>       <dependency>
>>>           <groupId>org.json</groupId>
>>>           <artifactId>JSON</artifactId>
>>>           <version>1.0</version>
>>>       </dependency>
>>>   </dependencies>
>>>
>>> </project>
>>>
>>>
>>>
>>> ./nifi-JSONCondenser-processors/pom.xml
>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>   <modelVersion>4.0.0</modelVersion>
>>>
>>>   <parent>
>>>       <groupId>com.jidmu</groupId>
>>>       <artifactId>JSONCondenser</artifactId>
>>>       <version>0.1</version>
>>>   </parent>
>>>
>>>   <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>   <packaging>jar</packaging>
>>>
>>>   <dependencies>
>>>       <dependency>
>>>           <groupId>org.apache.nifi</groupId>
>>>           <artifactId>nifi-api</artifactId>
>>>       </dependency>
>>>       <dependency>
>>>           <groupId>org.apache.nifi</groupId>
>>>           <artifactId>nifi-utils</artifactId>
>>>       </dependency>
>>>       <dependency>
>>>           <groupId>org.apache.nifi</groupId>
>>>           <artifactId>nifi-mock</artifactId>
>>>           <scope>test</scope>
>>>       </dependency>
>>>       <dependency>
>>>           <groupId>org.slf4j</groupId>
>>>           <artifactId>slf4j-simple</artifactId>
>>>           <scope>test</scope>
>>>       </dependency>
>>>       <dependency>
>>>           <groupId>junit</groupId>
>>>           <artifactId>junit</artifactId>
>>>           <scope>test</scope>
>>>       </dependency>
>>>   </dependencies>
>>> </project>
>>>
>>> ./nifi-JSONCondenser-nar/pom.xml
>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>   <modelVersion>4.0.0</modelVersion>
>>>
>>>   <parent>
>>>       <groupId>com.jidmu</groupId>
>>>       <artifactId>JSONCondenser</artifactId>
>>>       <version>0.1</version>
>>>   </parent>
>>>
>>>   <artifactId>nifi-JSONCondenser-nar</artifactId>
>>>   <version>0.1</version>
>>>   <packaging>nar</packaging>
>>>   <properties>
>>>       <maven.javadoc.skip>true</maven.javadoc.skip>
>>>       <source.skip>true</source.skip>
>>>   </properties>
>>>
>>>   <dependencies>
>>>       <dependency>
>>>           <groupId>com.jidmu</groupId>
>>>           <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>           <version>0.1</version>
>>>       </dependency>
>>>   </dependencies>
>>>
>>> </project>
>>>
>>>
>>>
>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <[hidden email]> wrote:
>>>>
>>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>>
>>>> Can you share your project, or the pom files for the processors, NAR,
>>>> and bundle?
>>>>
>>>> Thanks,
>>>>
>>>> Bryan
>>>>
>>>>
>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <[hidden email]> wrote:
>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>>
>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <[hidden email]> wrote:
>>>>>>
>>>>>> I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>>
>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>>         .Builder().name("ID_PATH")
>>>>>>         .displayName("JSON ID Path")
>>>>>>         .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>>         .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>>         .required(true)
>>>>>>         .build();
>>>>>>
>>>>>>
>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>    at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>>    at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>>    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>>    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>>    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>>    at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>>    at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>>    at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>>    at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>>    at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>>    at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>>    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>>    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>>    at java.lang.Class.newInstance(Class.java:442)
>>>>>>    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>>    ... 7 common frames omitted
>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>>    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>    ... 14 common frames omitted
>>>>>>
>>>>>>
>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>>>>>>>
>>>>>>> Can you share the code by chance for a review?  Otherwise, you'll want
>>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>>> goes through to see where it goes wonky.
>>>>>>>
>>>>>>> What version are you on?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>>>>>>>> Hi guys,
>>>>>>>>
>>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>
>>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>>
>>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>>
>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Phil
>>>>>>
>>>>>
>>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Bryan Bende
Correct :)

Technically we do support deploying extensions in JARs, but if you did
that you would only be able to utilize JARs that are directly in lib,
and since nifi-utils is not one of those, it explains why your
processor coming from the JAR cannot find StandardValidators.

Long story short, I think removing the JAR and copying in your NAR should work.

-Bryan


On Mon, Nov 6, 2017 at 1:20 PM, Phil H <[hidden email]> wrote:

> Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file
>
>
>
>> On 7 Nov 2017, at 5:17 am, Phil H <[hidden email]> wrote:
>>
>> Thanks Bryan,
>>
>> This new NAR does not appear in the extensions directory (my other working ones do).
>>
>> As for your second question
>>
>> [phil@localhost JSONCondenser]$ ls  ~/nifi-1.3.0/lib/ | grep jar
>> javax.servlet-api-3.1.0.jar
>> jcl-over-slf4j-1.7.25.jar
>> jetty-schemas-3.1.jar
>> jul-to-slf4j-1.7.25.jar
>> log4j-over-slf4j-1.7.25.jar
>> logback-classic-1.2.3.jar
>> logback-core-1.2.3.jar
>> nifi-api-1.3.0.jar
>> nifi-framework-api-1.3.0.jar
>> nifi-JSONCondenser-processors-0.1.jar
>> nifi-nar-utils-1.3.0.jar
>> nifi-properties-1.3.0.jar
>> nifi-runtime-1.3.0.jar
>> slf4j-api-1.7.25.jar
>>
>>
>>> On 7 Nov 2017, at 5:13 am, Bryan Bende <[hidden email]> wrote:
>>>
>>> Thanks for the poms.
>>>
>>> Can you provide the output of listing
>>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>>> ?
>>>
>>> and also the output of listing JARs that are in NiFi's lib directory?
>>> ls -l NIFI_HOME/lib/ | grep jar
>>>
>>> Want to verify that nifi-utils JAR is actually in your NAR and also
>>> that no other unexpected JARs are in your lib directory.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <[hidden email]> wrote:
>>>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>>>
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>   <modelVersion>4.0.0</modelVersion>
>>>>
>>>>   <parent>
>>>>       <groupId>org.apache.nifi</groupId>
>>>>       <artifactId>nifi-nar-bundles</artifactId>
>>>>       <version>1.3.0</version>
>>>>   </parent>
>>>>
>>>>   <repositories>
>>>>       <repository>
>>>>           <id>project.local</id>
>>>>           <name>projects</name>
>>>>           <url>file:${project.basedir}/repo</url>
>>>>       </repository>
>>>>   </repositories>
>>>>
>>>>   <groupId>com.jidmu</groupId>
>>>>   <artifactId>JSONCondenser</artifactId>
>>>>   <version>0.1</version>
>>>>   <packaging>pom</packaging>
>>>>
>>>>   <modules>
>>>>       <module>nifi-JSONCondenser-processors</module>
>>>>       <module>nifi-JSONCondenser-nar</module>
>>>>   </modules>
>>>>
>>>>   <dependencies>
>>>>       <dependency>
>>>>           <groupId>org.json</groupId>
>>>>           <artifactId>JSON</artifactId>
>>>>           <version>1.0</version>
>>>>       </dependency>
>>>>   </dependencies>
>>>>
>>>> </project>
>>>>
>>>>
>>>>
>>>> ./nifi-JSONCondenser-processors/pom.xml
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>   <modelVersion>4.0.0</modelVersion>
>>>>
>>>>   <parent>
>>>>       <groupId>com.jidmu</groupId>
>>>>       <artifactId>JSONCondenser</artifactId>
>>>>       <version>0.1</version>
>>>>   </parent>
>>>>
>>>>   <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>   <packaging>jar</packaging>
>>>>
>>>>   <dependencies>
>>>>       <dependency>
>>>>           <groupId>org.apache.nifi</groupId>
>>>>           <artifactId>nifi-api</artifactId>
>>>>       </dependency>
>>>>       <dependency>
>>>>           <groupId>org.apache.nifi</groupId>
>>>>           <artifactId>nifi-utils</artifactId>
>>>>       </dependency>
>>>>       <dependency>
>>>>           <groupId>org.apache.nifi</groupId>
>>>>           <artifactId>nifi-mock</artifactId>
>>>>           <scope>test</scope>
>>>>       </dependency>
>>>>       <dependency>
>>>>           <groupId>org.slf4j</groupId>
>>>>           <artifactId>slf4j-simple</artifactId>
>>>>           <scope>test</scope>
>>>>       </dependency>
>>>>       <dependency>
>>>>           <groupId>junit</groupId>
>>>>           <artifactId>junit</artifactId>
>>>>           <scope>test</scope>
>>>>       </dependency>
>>>>   </dependencies>
>>>> </project>
>>>>
>>>> ./nifi-JSONCondenser-nar/pom.xml
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>   <modelVersion>4.0.0</modelVersion>
>>>>
>>>>   <parent>
>>>>       <groupId>com.jidmu</groupId>
>>>>       <artifactId>JSONCondenser</artifactId>
>>>>       <version>0.1</version>
>>>>   </parent>
>>>>
>>>>   <artifactId>nifi-JSONCondenser-nar</artifactId>
>>>>   <version>0.1</version>
>>>>   <packaging>nar</packaging>
>>>>   <properties>
>>>>       <maven.javadoc.skip>true</maven.javadoc.skip>
>>>>       <source.skip>true</source.skip>
>>>>   </properties>
>>>>
>>>>   <dependencies>
>>>>       <dependency>
>>>>           <groupId>com.jidmu</groupId>
>>>>           <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>           <version>0.1</version>
>>>>       </dependency>
>>>>   </dependencies>
>>>>
>>>> </project>
>>>>
>>>>
>>>>
>>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <[hidden email]> wrote:
>>>>>
>>>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>>>
>>>>> Can you share your project, or the pom files for the processors, NAR,
>>>>> and bundle?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Bryan
>>>>>
>>>>>
>>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <[hidden email]> wrote:
>>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>>>
>>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <[hidden email]> wrote:
>>>>>>>
>>>>>>> I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>>>
>>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>>>         .Builder().name("ID_PATH")
>>>>>>>         .displayName("JSON ID Path")
>>>>>>>         .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>>>         .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>>>         .required(true)
>>>>>>>         .build();
>>>>>>>
>>>>>>>
>>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>    at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>>>    at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>>>    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>>>    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>>>    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>>>    at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>>>    at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>>>    at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>>>    at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>>>    at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>>>    at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>>>    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>>    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>>>    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>>    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>>>    at java.lang.Class.newInstance(Class.java:442)
>>>>>>>    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>>>    ... 7 common frames omitted
>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>>>    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>>    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>>    ... 14 common frames omitted
>>>>>>>
>>>>>>>
>>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>>>>>>>>
>>>>>>>> Can you share the code by chance for a review?  Otherwise, you'll want
>>>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>>>> goes through to see where it goes wonky.
>>>>>>>>
>>>>>>>> What version are you on?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>>>>>>>>> Hi guys,
>>>>>>>>>
>>>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>>
>>>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>>>
>>>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>>>
>>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Phil
>>>>>>>
>>>>>>
>>>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Joe Witt
In reply to this post by Phil H
Phil - you are the first person to ever make that mistake.

Nope, not true :)  No worries and thanks for following up.  I believe
once we have the extension registry in play this sort of thing will be
much easier/simpler.

Thanks

On Mon, Nov 6, 2017 at 1:20 PM, Phil H <[hidden email]> wrote:

> Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file
>
>
>
>> On 7 Nov 2017, at 5:17 am, Phil H <[hidden email]> wrote:
>>
>> Thanks Bryan,
>>
>> This new NAR does not appear in the extensions directory (my other working ones do).
>>
>> As for your second question
>>
>> [phil@localhost JSONCondenser]$ ls  ~/nifi-1.3.0/lib/ | grep jar
>> javax.servlet-api-3.1.0.jar
>> jcl-over-slf4j-1.7.25.jar
>> jetty-schemas-3.1.jar
>> jul-to-slf4j-1.7.25.jar
>> log4j-over-slf4j-1.7.25.jar
>> logback-classic-1.2.3.jar
>> logback-core-1.2.3.jar
>> nifi-api-1.3.0.jar
>> nifi-framework-api-1.3.0.jar
>> nifi-JSONCondenser-processors-0.1.jar
>> nifi-nar-utils-1.3.0.jar
>> nifi-properties-1.3.0.jar
>> nifi-runtime-1.3.0.jar
>> slf4j-api-1.7.25.jar
>>
>>
>>> On 7 Nov 2017, at 5:13 am, Bryan Bende <[hidden email]> wrote:
>>>
>>> Thanks for the poms.
>>>
>>> Can you provide the output of listing
>>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>>> ?
>>>
>>> and also the output of listing JARs that are in NiFi's lib directory?
>>> ls -l NIFI_HOME/lib/ | grep jar
>>>
>>> Want to verify that nifi-utils JAR is actually in your NAR and also
>>> that no other unexpected JARs are in your lib directory.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <[hidden email]> wrote:
>>>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>>>
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>   <modelVersion>4.0.0</modelVersion>
>>>>
>>>>   <parent>
>>>>       <groupId>org.apache.nifi</groupId>
>>>>       <artifactId>nifi-nar-bundles</artifactId>
>>>>       <version>1.3.0</version>
>>>>   </parent>
>>>>
>>>>   <repositories>
>>>>       <repository>
>>>>           <id>project.local</id>
>>>>           <name>projects</name>
>>>>           <url>file:${project.basedir}/repo</url>
>>>>       </repository>
>>>>   </repositories>
>>>>
>>>>   <groupId>com.jidmu</groupId>
>>>>   <artifactId>JSONCondenser</artifactId>
>>>>   <version>0.1</version>
>>>>   <packaging>pom</packaging>
>>>>
>>>>   <modules>
>>>>       <module>nifi-JSONCondenser-processors</module>
>>>>       <module>nifi-JSONCondenser-nar</module>
>>>>   </modules>
>>>>
>>>>   <dependencies>
>>>>       <dependency>
>>>>           <groupId>org.json</groupId>
>>>>           <artifactId>JSON</artifactId>
>>>>           <version>1.0</version>
>>>>       </dependency>
>>>>   </dependencies>
>>>>
>>>> </project>
>>>>
>>>>
>>>>
>>>> ./nifi-JSONCondenser-processors/pom.xml
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>   <modelVersion>4.0.0</modelVersion>
>>>>
>>>>   <parent>
>>>>       <groupId>com.jidmu</groupId>
>>>>       <artifactId>JSONCondenser</artifactId>
>>>>       <version>0.1</version>
>>>>   </parent>
>>>>
>>>>   <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>   <packaging>jar</packaging>
>>>>
>>>>   <dependencies>
>>>>       <dependency>
>>>>           <groupId>org.apache.nifi</groupId>
>>>>           <artifactId>nifi-api</artifactId>
>>>>       </dependency>
>>>>       <dependency>
>>>>           <groupId>org.apache.nifi</groupId>
>>>>           <artifactId>nifi-utils</artifactId>
>>>>       </dependency>
>>>>       <dependency>
>>>>           <groupId>org.apache.nifi</groupId>
>>>>           <artifactId>nifi-mock</artifactId>
>>>>           <scope>test</scope>
>>>>       </dependency>
>>>>       <dependency>
>>>>           <groupId>org.slf4j</groupId>
>>>>           <artifactId>slf4j-simple</artifactId>
>>>>           <scope>test</scope>
>>>>       </dependency>
>>>>       <dependency>
>>>>           <groupId>junit</groupId>
>>>>           <artifactId>junit</artifactId>
>>>>           <scope>test</scope>
>>>>       </dependency>
>>>>   </dependencies>
>>>> </project>
>>>>
>>>> ./nifi-JSONCondenser-nar/pom.xml
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>   <modelVersion>4.0.0</modelVersion>
>>>>
>>>>   <parent>
>>>>       <groupId>com.jidmu</groupId>
>>>>       <artifactId>JSONCondenser</artifactId>
>>>>       <version>0.1</version>
>>>>   </parent>
>>>>
>>>>   <artifactId>nifi-JSONCondenser-nar</artifactId>
>>>>   <version>0.1</version>
>>>>   <packaging>nar</packaging>
>>>>   <properties>
>>>>       <maven.javadoc.skip>true</maven.javadoc.skip>
>>>>       <source.skip>true</source.skip>
>>>>   </properties>
>>>>
>>>>   <dependencies>
>>>>       <dependency>
>>>>           <groupId>com.jidmu</groupId>
>>>>           <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>           <version>0.1</version>
>>>>       </dependency>
>>>>   </dependencies>
>>>>
>>>> </project>
>>>>
>>>>
>>>>
>>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <[hidden email]> wrote:
>>>>>
>>>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>>>
>>>>> Can you share your project, or the pom files for the processors, NAR,
>>>>> and bundle?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Bryan
>>>>>
>>>>>
>>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <[hidden email]> wrote:
>>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>>>
>>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <[hidden email]> wrote:
>>>>>>>
>>>>>>> I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>>>
>>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>>>         .Builder().name("ID_PATH")
>>>>>>>         .displayName("JSON ID Path")
>>>>>>>         .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>>>         .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>>>         .required(true)
>>>>>>>         .build();
>>>>>>>
>>>>>>>
>>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>    at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>>>    at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>>>    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>>>    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>>>    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>>>    at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>>>    at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>>>    at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>>>    at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>>>    at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>>>    at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>>>    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>>    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>>>    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>>    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>>>    at java.lang.Class.newInstance(Class.java:442)
>>>>>>>    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>>>    ... 7 common frames omitted
>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>>>    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>>    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>>    ... 14 common frames omitted
>>>>>>>
>>>>>>>
>>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>>>>>>>>
>>>>>>>> Can you share the code by chance for a review?  Otherwise, you'll want
>>>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>>>> goes through to see where it goes wonky.
>>>>>>>>
>>>>>>>> What version are you on?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>>>>>>>>> Hi guys,
>>>>>>>>>
>>>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>>
>>>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>>>
>>>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>>>
>>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Phil
>>>>>>>
>>>>>>
>>>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Phil H
Yep, replacing the JAR with the NAR (and then re-connecting it through the UI) worked.  As a nice postscript to the story, the previously untested filtering algorithm i had written worked as expected first time too :-)

Thanks so much for your help, team!

> On 7 Nov 2017, at 5:22 am, Joe Witt <[hidden email]> wrote:
>
> Phil - you are the first person to ever make that mistake.
>
> Nope, not true :)  No worries and thanks for following up.  I believe
> once we have the extension registry in play this sort of thing will be
> much easier/simpler.
>
> Thanks
>
> On Mon, Nov 6, 2017 at 1:20 PM, Phil H <[hidden email]> wrote:
>> Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file
>>
>>
>>
>>> On 7 Nov 2017, at 5:17 am, Phil H <[hidden email]> wrote:
>>>
>>> Thanks Bryan,
>>>
>>> This new NAR does not appear in the extensions directory (my other working ones do).
>>>
>>> As for your second question
>>>
>>> [phil@localhost JSONCondenser]$ ls  ~/nifi-1.3.0/lib/ | grep jar
>>> javax.servlet-api-3.1.0.jar
>>> jcl-over-slf4j-1.7.25.jar
>>> jetty-schemas-3.1.jar
>>> jul-to-slf4j-1.7.25.jar
>>> log4j-over-slf4j-1.7.25.jar
>>> logback-classic-1.2.3.jar
>>> logback-core-1.2.3.jar
>>> nifi-api-1.3.0.jar
>>> nifi-framework-api-1.3.0.jar
>>> nifi-JSONCondenser-processors-0.1.jar
>>> nifi-nar-utils-1.3.0.jar
>>> nifi-properties-1.3.0.jar
>>> nifi-runtime-1.3.0.jar
>>> slf4j-api-1.7.25.jar
>>>
>>>
>>>> On 7 Nov 2017, at 5:13 am, Bryan Bende <[hidden email]> wrote:
>>>>
>>>> Thanks for the poms.
>>>>
>>>> Can you provide the output of listing
>>>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>>>> ?
>>>>
>>>> and also the output of listing JARs that are in NiFi's lib directory?
>>>> ls -l NIFI_HOME/lib/ | grep jar
>>>>
>>>> Want to verify that nifi-utils JAR is actually in your NAR and also
>>>> that no other unexpected JARs are in your lib directory.
>>>>
>>>> Thanks,
>>>>
>>>> Bryan
>>>>
>>>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <[hidden email]> wrote:
>>>>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>>>>
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>  <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>>  <parent>
>>>>>      <groupId>org.apache.nifi</groupId>
>>>>>      <artifactId>nifi-nar-bundles</artifactId>
>>>>>      <version>1.3.0</version>
>>>>>  </parent>
>>>>>
>>>>>  <repositories>
>>>>>      <repository>
>>>>>          <id>project.local</id>
>>>>>          <name>projects</name>
>>>>>          <url>file:${project.basedir}/repo</url>
>>>>>      </repository>
>>>>>  </repositories>
>>>>>
>>>>>  <groupId>com.jidmu</groupId>
>>>>>  <artifactId>JSONCondenser</artifactId>
>>>>>  <version>0.1</version>
>>>>>  <packaging>pom</packaging>
>>>>>
>>>>>  <modules>
>>>>>      <module>nifi-JSONCondenser-processors</module>
>>>>>      <module>nifi-JSONCondenser-nar</module>
>>>>>  </modules>
>>>>>
>>>>>  <dependencies>
>>>>>      <dependency>
>>>>>          <groupId>org.json</groupId>
>>>>>          <artifactId>JSON</artifactId>
>>>>>          <version>1.0</version>
>>>>>      </dependency>
>>>>>  </dependencies>
>>>>>
>>>>> </project>
>>>>>
>>>>>
>>>>>
>>>>> ./nifi-JSONCondenser-processors/pom.xml
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>  <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>>  <parent>
>>>>>      <groupId>com.jidmu</groupId>
>>>>>      <artifactId>JSONCondenser</artifactId>
>>>>>      <version>0.1</version>
>>>>>  </parent>
>>>>>
>>>>>  <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>>  <packaging>jar</packaging>
>>>>>
>>>>>  <dependencies>
>>>>>      <dependency>
>>>>>          <groupId>org.apache.nifi</groupId>
>>>>>          <artifactId>nifi-api</artifactId>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>org.apache.nifi</groupId>
>>>>>          <artifactId>nifi-utils</artifactId>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>org.apache.nifi</groupId>
>>>>>          <artifactId>nifi-mock</artifactId>
>>>>>          <scope>test</scope>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>org.slf4j</groupId>
>>>>>          <artifactId>slf4j-simple</artifactId>
>>>>>          <scope>test</scope>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>junit</groupId>
>>>>>          <artifactId>junit</artifactId>
>>>>>          <scope>test</scope>
>>>>>      </dependency>
>>>>>  </dependencies>
>>>>> </project>
>>>>>
>>>>> ./nifi-JSONCondenser-nar/pom.xml
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>  <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>>  <parent>
>>>>>      <groupId>com.jidmu</groupId>
>>>>>      <artifactId>JSONCondenser</artifactId>
>>>>>      <version>0.1</version>
>>>>>  </parent>
>>>>>
>>>>>  <artifactId>nifi-JSONCondenser-nar</artifactId>
>>>>>  <version>0.1</version>
>>>>>  <packaging>nar</packaging>
>>>>>  <properties>
>>>>>      <maven.javadoc.skip>true</maven.javadoc.skip>
>>>>>      <source.skip>true</source.skip>
>>>>>  </properties>
>>>>>
>>>>>  <dependencies>
>>>>>      <dependency>
>>>>>          <groupId>com.jidmu</groupId>
>>>>>          <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>>          <version>0.1</version>
>>>>>      </dependency>
>>>>>  </dependencies>
>>>>>
>>>>> </project>
>>>>>
>>>>>
>>>>>
>>>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <[hidden email]> wrote:
>>>>>>
>>>>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>>>>
>>>>>> Can you share your project, or the pom files for the processors, NAR,
>>>>>> and bundle?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Bryan
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <[hidden email]> wrote:
>>>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>>>>
>>>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <[hidden email]> wrote:
>>>>>>>>
>>>>>>>> I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>>>>
>>>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>>>>        .Builder().name("ID_PATH")
>>>>>>>>        .displayName("JSON ID Path")
>>>>>>>>        .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>>>>        .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>>>>        .required(true)
>>>>>>>>        .build();
>>>>>>>>
>>>>>>>>
>>>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>>   at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>>>>   at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>>>>   at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>>>>   at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>>>>   at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>>>>   at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>>>>   at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>>>>   at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>>>>   at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>>>>   at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>>>>   at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>>>>   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>>>   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>>>>   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>>>   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>>>>   at java.lang.Class.newInstance(Class.java:442)
>>>>>>>>   at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>>>>   ... 7 common frames omitted
>>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>>>>   at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>>>   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>>>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>>>>   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>>>   ... 14 common frames omitted
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>>>>>>>>>
>>>>>>>>> Can you share the code by chance for a review?  Otherwise, you'll want
>>>>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>>>>> goes through to see where it goes wonky.
>>>>>>>>>
>>>>>>>>> What version are you on?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>>>>>>>>>> Hi guys,
>>>>>>>>>>
>>>>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>>>
>>>>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>>>>
>>>>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>>>>
>>>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Phil
>>>>>>>>
>>>>>>>
>>>>>
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Classloader issues

Phil H
In reply to this post by Joe Witt
While we're talking about this, is there a simple way to have maven put the NAR file into my NiFi lib directory automatically as part of the "mvn install" script?

Cheers,
Phil

> On 7 Nov 2017, at 05:22, Joe Witt <[hidden email]> wrote:
>
> Phil - you are the first person to ever make that mistake.
>
> Nope, not true :)  No worries and thanks for following up.  I believe
> once we have the extension registry in play this sort of thing will be
> much easier/simpler.
>
> Thanks
>
>> On Mon, Nov 6, 2017 at 1:20 PM, Phil H <[hidden email]> wrote:
>> Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file
>>
>>
>>
>>> On 7 Nov 2017, at 5:17 am, Phil H <[hidden email]> wrote:
>>>
>>> Thanks Bryan,
>>>
>>> This new NAR does not appear in the extensions directory (my other working ones do).
>>>
>>> As for your second question
>>>
>>> [phil@localhost JSONCondenser]$ ls  ~/nifi-1.3.0/lib/ | grep jar
>>> javax.servlet-api-3.1.0.jar
>>> jcl-over-slf4j-1.7.25.jar
>>> jetty-schemas-3.1.jar
>>> jul-to-slf4j-1.7.25.jar
>>> log4j-over-slf4j-1.7.25.jar
>>> logback-classic-1.2.3.jar
>>> logback-core-1.2.3.jar
>>> nifi-api-1.3.0.jar
>>> nifi-framework-api-1.3.0.jar
>>> nifi-JSONCondenser-processors-0.1.jar
>>> nifi-nar-utils-1.3.0.jar
>>> nifi-properties-1.3.0.jar
>>> nifi-runtime-1.3.0.jar
>>> slf4j-api-1.7.25.jar
>>>
>>>
>>>> On 7 Nov 2017, at 5:13 am, Bryan Bende <[hidden email]> wrote:
>>>>
>>>> Thanks for the poms.
>>>>
>>>> Can you provide the output of listing
>>>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>>>> ?
>>>>
>>>> and also the output of listing JARs that are in NiFi's lib directory?
>>>> ls -l NIFI_HOME/lib/ | grep jar
>>>>
>>>> Want to verify that nifi-utils JAR is actually in your NAR and also
>>>> that no other unexpected JARs are in your lib directory.
>>>>
>>>> Thanks,
>>>>
>>>> Bryan
>>>>
>>>>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <[hidden email]> wrote:
>>>>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>>>>
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>  <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>>  <parent>
>>>>>      <groupId>org.apache.nifi</groupId>
>>>>>      <artifactId>nifi-nar-bundles</artifactId>
>>>>>      <version>1.3.0</version>
>>>>>  </parent>
>>>>>
>>>>>  <repositories>
>>>>>      <repository>
>>>>>          <id>project.local</id>
>>>>>          <name>projects</name>
>>>>>          <url>file:${project.basedir}/repo</url>
>>>>>      </repository>
>>>>>  </repositories>
>>>>>
>>>>>  <groupId>com.jidmu</groupId>
>>>>>  <artifactId>JSONCondenser</artifactId>
>>>>>  <version>0.1</version>
>>>>>  <packaging>pom</packaging>
>>>>>
>>>>>  <modules>
>>>>>      <module>nifi-JSONCondenser-processors</module>
>>>>>      <module>nifi-JSONCondenser-nar</module>
>>>>>  </modules>
>>>>>
>>>>>  <dependencies>
>>>>>      <dependency>
>>>>>          <groupId>org.json</groupId>
>>>>>          <artifactId>JSON</artifactId>
>>>>>          <version>1.0</version>
>>>>>      </dependency>
>>>>>  </dependencies>
>>>>>
>>>>> </project>
>>>>>
>>>>>
>>>>>
>>>>> ./nifi-JSONCondenser-processors/pom.xml
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>  <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>>  <parent>
>>>>>      <groupId>com.jidmu</groupId>
>>>>>      <artifactId>JSONCondenser</artifactId>
>>>>>      <version>0.1</version>
>>>>>  </parent>
>>>>>
>>>>>  <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>>  <packaging>jar</packaging>
>>>>>
>>>>>  <dependencies>
>>>>>      <dependency>
>>>>>          <groupId>org.apache.nifi</groupId>
>>>>>          <artifactId>nifi-api</artifactId>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>org.apache.nifi</groupId>
>>>>>          <artifactId>nifi-utils</artifactId>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>org.apache.nifi</groupId>
>>>>>          <artifactId>nifi-mock</artifactId>
>>>>>          <scope>test</scope>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>org.slf4j</groupId>
>>>>>          <artifactId>slf4j-simple</artifactId>
>>>>>          <scope>test</scope>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>junit</groupId>
>>>>>          <artifactId>junit</artifactId>
>>>>>          <scope>test</scope>
>>>>>      </dependency>
>>>>>  </dependencies>
>>>>> </project>
>>>>>
>>>>> ./nifi-JSONCondenser-nar/pom.xml
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>  <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>>  <parent>
>>>>>      <groupId>com.jidmu</groupId>
>>>>>      <artifactId>JSONCondenser</artifactId>
>>>>>      <version>0.1</version>
>>>>>  </parent>
>>>>>
>>>>>  <artifactId>nifi-JSONCondenser-nar</artifactId>
>>>>>  <version>0.1</version>
>>>>>  <packaging>nar</packaging>
>>>>>  <properties>
>>>>>      <maven.javadoc.skip>true</maven.javadoc.skip>
>>>>>      <source.skip>true</source.skip>
>>>>>  </properties>
>>>>>
>>>>>  <dependencies>
>>>>>      <dependency>
>>>>>          <groupId>com.jidmu</groupId>
>>>>>          <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>>          <version>0.1</version>
>>>>>      </dependency>
>>>>>  </dependencies>
>>>>>
>>>>> </project>
>>>>>
>>>>>
>>>>>
>>>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <[hidden email]> wrote:
>>>>>>
>>>>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>>>>
>>>>>> Can you share your project, or the pom files for the processors, NAR,
>>>>>> and bundle?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Bryan
>>>>>>
>>>>>>
>>>>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <[hidden email]> wrote:
>>>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>>>>
>>>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <[hidden email]> wrote:
>>>>>>>>
>>>>>>>> I added the StandardValidators reference back in and the original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>>>>
>>>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>>>>        .Builder().name("ID_PATH")
>>>>>>>>        .displayName("JSON ID Path")
>>>>>>>>        .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>>>>        .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>>>>        .required(true)
>>>>>>>>        .build();
>>>>>>>>
>>>>>>>>
>>>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>>   at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>>>>   at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>>>>   at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>>>>   at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>>>>   at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>>>>   at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>>>>   at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>>>>   at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>>>>   at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>>>>   at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>>>>   at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>>>>   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>>>   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>>>>   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>>>   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>>>>   at java.lang.Class.newInstance(Class.java:442)
>>>>>>>>   at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>>>>   ... 7 common frames omitted
>>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>>>>   at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>>>   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>>>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>>>>   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>>>   ... 14 common frames omitted
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <[hidden email]> wrote:
>>>>>>>>>
>>>>>>>>> Can you share the code by chance for a review?  Otherwise, you'll want
>>>>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>>>>> goes through to see where it goes wonky.
>>>>>>>>>
>>>>>>>>> What version are you on?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <[hidden email]> wrote:
>>>>>>>>>> Hi guys,
>>>>>>>>>>
>>>>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>>>
>>>>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>>>>
>>>>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>>>>
>>>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Phil
>>>>>>>>
>>>>>>>
>>>>>
>>>
>>