Pheonix client jar required for HBase conn

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Pheonix client jar required for HBase conn

Martini, Adam
Hello all,

We were struggling with our HBase client connection in the NiFi 1.7.1 release. We are not using Phoenix and our client was erroring with a missing class exception:

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.util.ClassSize
                at org.apache.hadoop.hbase.ipc.IPCUtil.<init>(IPCUtil.java:72)
                at org.apache.hadoop.hbase.ipc.AbstractRpcClient.<init>(AbstractRpcClient.java:91)
                at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1113)
                at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1139)
                ... 33 common frames omitted

Adding a Phoenix client (which we do not need) helps resolve this class dependency, but fails if NiFi is running on Java 10:

Caused by: java.lang.RuntimeException: Unexpected version format: 10.0.1
    at org.apache.hadoop.hbase.util.ClassSize.<clinit>(ClassSize.java:119)
    ... 37 common frames omitted

Starting NiFi with Java 9 fixes this error, but complicates our devops as Java 9 is no longer supported and cannot be installed via package management.

Please fix this dependency issue so that a Phoenix client is not required to resolve the org.apache.hadoop.hbase.util.ClassSize class. Also, please migrate to Java 10 as soon as possible.

Let me know if I can help with this process.

Thanks,

Adam Martini
Senior Software Dev, Nike Digital

Reply | Threaded
Open this post in threaded view
|

Re: Pheonix client jar required for HBase conn

Bryan Bende
Adam,

The ClassSize class comes from hbase-common [1] so I'm not sure how
that would related to the Phoenix client JAR. What version of HBase
was this against?

The only case I know of that needs the Phoenix client jar is when
Phoenix has been installed which then modifies the HBase config files
to specify some class names that come from phoenix client JAR.

As far as Java 10, there has been an on-going effort to become
compatible with Java 9+, but it is a non-trivial effort and not
complete yet [2].

Thanks,

Bryan

[1] https://github.com/apache/hbase/blob/rel/1.1.2/hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java
[2] https://issues.apache.org/jira/browse/NIFI-5174


On Thu, Aug 23, 2018 at 1:47 PM, Martini, Adam <[hidden email]> wrote:

> Hello all,
>
> We were struggling with our HBase client connection in the NiFi 1.7.1 release. We are not using Phoenix and our client was erroring with a missing class exception:
>
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.util.ClassSize
>                 at org.apache.hadoop.hbase.ipc.IPCUtil.<init>(IPCUtil.java:72)
>                 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.<init>(AbstractRpcClient.java:91)
>                 at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1113)
>                 at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1139)
>                 ... 33 common frames omitted
>
> Adding a Phoenix client (which we do not need) helps resolve this class dependency, but fails if NiFi is running on Java 10:
>
> Caused by: java.lang.RuntimeException: Unexpected version format: 10.0.1
>     at org.apache.hadoop.hbase.util.ClassSize.<clinit>(ClassSize.java:119)
>     ... 37 common frames omitted
>
> Starting NiFi with Java 9 fixes this error, but complicates our devops as Java 9 is no longer supported and cannot be installed via package management.
>
> Please fix this dependency issue so that a Phoenix client is not required to resolve the org.apache.hadoop.hbase.util.ClassSize class. Also, please migrate to Java 10 as soon as possible.
>
> Let me know if I can help with this process.
>
> Thanks,
>
> Adam Martini
> Senior Software Dev, Nike Digital
>
Reply | Threaded
Open this post in threaded view
|

Re: <External>Re: Pheonix client jar required for HBase conn

Martini, Adam
Bryan,

We are running HBase 1.2.  I am confused as well about the missing ClassSize class.  Does seem completely unrelated to Pheonix and I do not understand why adding the client would help resolve the class.

Would you advise reverting to Java 8 until Java 10 is fully supported?

Thanks,

Adam

On 8/23/18, 11:27 AM, "Bryan Bende" <[hidden email]> wrote:

    Adam,
   
    The ClassSize class comes from hbase-common [1] so I'm not sure how
    that would related to the Phoenix client JAR. What version of HBase
    was this against?
   
    The only case I know of that needs the Phoenix client jar is when
    Phoenix has been installed which then modifies the HBase config files
    to specify some class names that come from phoenix client JAR.
   
    As far as Java 10, there has been an on-going effort to become
    compatible with Java 9+, but it is a non-trivial effort and not
    complete yet [2].
   
    Thanks,
   
    Bryan
   
    [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_hbase_blob_rel_1.1.2_hbase-2Dcommon_src_main_java_org_apache_hadoop_hbase_util_ClassSize.java&d=DwIBaQ&c=7DfhQjPWzR3PmWBQVpi-kw&r=Sb-vyl3zZFvHwGIFN-QEmQCWZUYgyH4UBZPmcWFaTC8&m=bnetA9Y56Fx_Tiq-PCdevSuGr34t8ixQz5sDQMViOmc&s=XVjke9vTfRooHYCqZCNp1AS5kUbiGaJxHHF6OyNcnfQ&e=
    [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_NIFI-2D5174&d=DwIBaQ&c=7DfhQjPWzR3PmWBQVpi-kw&r=Sb-vyl3zZFvHwGIFN-QEmQCWZUYgyH4UBZPmcWFaTC8&m=bnetA9Y56Fx_Tiq-PCdevSuGr34t8ixQz5sDQMViOmc&s=dHD8gEEW65OaoGA1DC-FE3i7EJRLatqydkeFpaHB1IA&e=
   
   
    On Thu, Aug 23, 2018 at 1:47 PM, Martini, Adam <[hidden email]> wrote:
    > Hello all,
    >
    > We were struggling with our HBase client connection in the NiFi 1.7.1 release. We are not using Phoenix and our client was erroring with a missing class exception:
    >
    > Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.util.ClassSize
    >                 at org.apache.hadoop.hbase.ipc.IPCUtil.<init>(IPCUtil.java:72)
    >                 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.<init>(AbstractRpcClient.java:91)
    >                 at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1113)
    >                 at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1139)
    >                 ... 33 common frames omitted
    >
    > Adding a Phoenix client (which we do not need) helps resolve this class dependency, but fails if NiFi is running on Java 10:
    >
    > Caused by: java.lang.RuntimeException: Unexpected version format: 10.0.1
    >     at org.apache.hadoop.hbase.util.ClassSize.<clinit>(ClassSize.java:119)
    >     ... 37 common frames omitted
    >
    > Starting NiFi with Java 9 fixes this error, but complicates our devops as Java 9 is no longer supported and cannot be installed via package management.
    >
    > Please fix this dependency issue so that a Phoenix client is not required to resolve the org.apache.hadoop.hbase.util.ClassSize class. Also, please migrate to Java 10 as soon as possible.
    >
    > Let me know if I can help with this process.
    >
    > Thanks,
    >
    > Adam Martini
    > Senior Software Dev, Nike Digital
    >
   

Reply | Threaded
Open this post in threaded view
|

Re: <External>Re: Pheonix client jar required for HBase conn

Bryan Bende
Adam,

Yes for now Java 8 is what is fully supported.

Hopefully the remaining issues can be resolved in the not too distant
future to become compatible with Java 9+.

I know in the HBase case it requires changing to a newer version of
the client because the 1.1.2 client has the code that caused the issue
you ran into with "Unexpected version format: 10.0.1".

Thanks,

Bryan

On Thu, Aug 23, 2018 at 2:54 PM, Martini, Adam <[hidden email]> wrote:

> Bryan,
>
> We are running HBase 1.2.  I am confused as well about the missing ClassSize class.  Does seem completely unrelated to Pheonix and I do not understand why adding the client would help resolve the class.
>
> Would you advise reverting to Java 8 until Java 10 is fully supported?
>
> Thanks,
>
> Adam
>
> On 8/23/18, 11:27 AM, "Bryan Bende" <[hidden email]> wrote:
>
>     Adam,
>
>     The ClassSize class comes from hbase-common [1] so I'm not sure how
>     that would related to the Phoenix client JAR. What version of HBase
>     was this against?
>
>     The only case I know of that needs the Phoenix client jar is when
>     Phoenix has been installed which then modifies the HBase config files
>     to specify some class names that come from phoenix client JAR.
>
>     As far as Java 10, there has been an on-going effort to become
>     compatible with Java 9+, but it is a non-trivial effort and not
>     complete yet [2].
>
>     Thanks,
>
>     Bryan
>
>     [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_hbase_blob_rel_1.1.2_hbase-2Dcommon_src_main_java_org_apache_hadoop_hbase_util_ClassSize.java&d=DwIBaQ&c=7DfhQjPWzR3PmWBQVpi-kw&r=Sb-vyl3zZFvHwGIFN-QEmQCWZUYgyH4UBZPmcWFaTC8&m=bnetA9Y56Fx_Tiq-PCdevSuGr34t8ixQz5sDQMViOmc&s=XVjke9vTfRooHYCqZCNp1AS5kUbiGaJxHHF6OyNcnfQ&e=
>     [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_NIFI-2D5174&d=DwIBaQ&c=7DfhQjPWzR3PmWBQVpi-kw&r=Sb-vyl3zZFvHwGIFN-QEmQCWZUYgyH4UBZPmcWFaTC8&m=bnetA9Y56Fx_Tiq-PCdevSuGr34t8ixQz5sDQMViOmc&s=dHD8gEEW65OaoGA1DC-FE3i7EJRLatqydkeFpaHB1IA&e=
>
>
>     On Thu, Aug 23, 2018 at 1:47 PM, Martini, Adam <[hidden email]> wrote:
>     > Hello all,
>     >
>     > We were struggling with our HBase client connection in the NiFi 1.7.1 release. We are not using Phoenix and our client was erroring with a missing class exception:
>     >
>     > Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.util.ClassSize
>     >                 at org.apache.hadoop.hbase.ipc.IPCUtil.<init>(IPCUtil.java:72)
>     >                 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.<init>(AbstractRpcClient.java:91)
>     >                 at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1113)
>     >                 at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1139)
>     >                 ... 33 common frames omitted
>     >
>     > Adding a Phoenix client (which we do not need) helps resolve this class dependency, but fails if NiFi is running on Java 10:
>     >
>     > Caused by: java.lang.RuntimeException: Unexpected version format: 10.0.1
>     >     at org.apache.hadoop.hbase.util.ClassSize.<clinit>(ClassSize.java:119)
>     >     ... 37 common frames omitted
>     >
>     > Starting NiFi with Java 9 fixes this error, but complicates our devops as Java 9 is no longer supported and cannot be installed via package management.
>     >
>     > Please fix this dependency issue so that a Phoenix client is not required to resolve the org.apache.hadoop.hbase.util.ClassSize class. Also, please migrate to Java 10 as soon as possible.
>     >
>     > Let me know if I can help with this process.
>     >
>     > Thanks,
>     >
>     > Adam Martini
>     > Senior Software Dev, Nike Digital
>     >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: <External>Re: Pheonix client jar required for HBase conn

Martini, Adam
Bryan,

Yes, an HBase client upgrade makes sense for the Java 10 upgrade path. However, the NoClassDefFoundError is more mysterious and does concern me.

Thanks,
Adam

On 8/23/18, 12:12 PM, "Bryan Bende" <[hidden email]> wrote:

    Adam,
   
    Yes for now Java 8 is what is fully supported.
   
    Hopefully the remaining issues can be resolved in the not too distant
    future to become compatible with Java 9+.
   
    I know in the HBase case it requires changing to a newer version of
    the client because the 1.1.2 client has the code that caused the issue
    you ran into with "Unexpected version format: 10.0.1".
   
    Thanks,
   
    Bryan
   
    On Thu, Aug 23, 2018 at 2:54 PM, Martini, Adam <[hidden email]> wrote:
    > Bryan,
    >
    > We are running HBase 1.2.  I am confused as well about the missing ClassSize class.  Does seem completely unrelated to Pheonix and I do not understand why adding the client would help resolve the class.
    >
    > Would you advise reverting to Java 8 until Java 10 is fully supported?
    >
    > Thanks,
    >
    > Adam
    >
    > On 8/23/18, 11:27 AM, "Bryan Bende" <[hidden email]> wrote:
    >
    >     Adam,
    >
    >     The ClassSize class comes from hbase-common [1] so I'm not sure how
    >     that would related to the Phoenix client JAR. What version of HBase
    >     was this against?
    >
    >     The only case I know of that needs the Phoenix client jar is when
    >     Phoenix has been installed which then modifies the HBase config files
    >     to specify some class names that come from phoenix client JAR.
    >
    >     As far as Java 10, there has been an on-going effort to become
    >     compatible with Java 9+, but it is a non-trivial effort and not
    >     complete yet [2].
    >
    >     Thanks,
    >
    >     Bryan
    >
    >     [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_hbase_blob_rel_1.1.2_hbase-2Dcommon_src_main_java_org_apache_hadoop_hbase_util_ClassSize.java&d=DwIBaQ&c=7DfhQjPWzR3PmWBQVpi-kw&r=Sb-vyl3zZFvHwGIFN-QEmQCWZUYgyH4UBZPmcWFaTC8&m=bnetA9Y56Fx_Tiq-PCdevSuGr34t8ixQz5sDQMViOmc&s=XVjke9vTfRooHYCqZCNp1AS5kUbiGaJxHHF6OyNcnfQ&e=
    >     [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_NIFI-2D5174&d=DwIBaQ&c=7DfhQjPWzR3PmWBQVpi-kw&r=Sb-vyl3zZFvHwGIFN-QEmQCWZUYgyH4UBZPmcWFaTC8&m=bnetA9Y56Fx_Tiq-PCdevSuGr34t8ixQz5sDQMViOmc&s=dHD8gEEW65OaoGA1DC-FE3i7EJRLatqydkeFpaHB1IA&e=
    >
    >
    >     On Thu, Aug 23, 2018 at 1:47 PM, Martini, Adam <[hidden email]> wrote:
    >     > Hello all,
    >     >
    >     > We were struggling with our HBase client connection in the NiFi 1.7.1 release. We are not using Phoenix and our client was erroring with a missing class exception:
    >     >
    >     > Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.util.ClassSize
    >     >                 at org.apache.hadoop.hbase.ipc.IPCUtil.<init>(IPCUtil.java:72)
    >     >                 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.<init>(AbstractRpcClient.java:91)
    >     >                 at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1113)
    >     >                 at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1139)
    >     >                 ... 33 common frames omitted
    >     >
    >     > Adding a Phoenix client (which we do not need) helps resolve this class dependency, but fails if NiFi is running on Java 10:
    >     >
    >     > Caused by: java.lang.RuntimeException: Unexpected version format: 10.0.1
    >     >     at org.apache.hadoop.hbase.util.ClassSize.<clinit>(ClassSize.java:119)
    >     >     ... 37 common frames omitted
    >     >
    >     > Starting NiFi with Java 9 fixes this error, but complicates our devops as Java 9 is no longer supported and cannot be installed via package management.
    >     >
    >     > Please fix this dependency issue so that a Phoenix client is not required to resolve the org.apache.hadoop.hbase.util.ClassSize class. Also, please migrate to Java 10 as soon as possible.
    >     >
    >     > Let me know if I can help with this process.
    >     >
    >     > Thanks,
    >     >
    >     > Adam Martini
    >     > Senior Software Dev, Nike Digital
    >     >
    >
    >
   

Reply | Threaded
Open this post in threaded view
|

Re: <External>Re: Pheonix client jar required for HBase conn

Bryan Bende
Adam,

One thought... the phoenix client jar is a shaded jar which I think
bundles hbase-common so maybe the end result was two versions of
ClassSize being on the classpath and it not being clear which one was
being used.

Also, a NoClassDefFoundError can be thrown if some static code in the
class produces an error, which seems oddly related to the static code
that produced "Unexpected version format: 10.0.1".

Long story short I would be curious to see if everything worked on
Java 8 without phoenix client, or if you'd end up back with the
NoClassDefError. If you still got the error then I'd be stumped.

-Bryan


On Thu, Aug 23, 2018 at 3:16 PM, Martini, Adam <[hidden email]> wrote:

> Bryan,
>
> Yes, an HBase client upgrade makes sense for the Java 10 upgrade path. However, the NoClassDefFoundError is more mysterious and does concern me.
>
> Thanks,
> Adam
>
> On 8/23/18, 12:12 PM, "Bryan Bende" <[hidden email]> wrote:
>
>     Adam,
>
>     Yes for now Java 8 is what is fully supported.
>
>     Hopefully the remaining issues can be resolved in the not too distant
>     future to become compatible with Java 9+.
>
>     I know in the HBase case it requires changing to a newer version of
>     the client because the 1.1.2 client has the code that caused the issue
>     you ran into with "Unexpected version format: 10.0.1".
>
>     Thanks,
>
>     Bryan
>
>     On Thu, Aug 23, 2018 at 2:54 PM, Martini, Adam <[hidden email]> wrote:
>     > Bryan,
>     >
>     > We are running HBase 1.2.  I am confused as well about the missing ClassSize class.  Does seem completely unrelated to Pheonix and I do not understand why adding the client would help resolve the class.
>     >
>     > Would you advise reverting to Java 8 until Java 10 is fully supported?
>     >
>     > Thanks,
>     >
>     > Adam
>     >
>     > On 8/23/18, 11:27 AM, "Bryan Bende" <[hidden email]> wrote:
>     >
>     >     Adam,
>     >
>     >     The ClassSize class comes from hbase-common [1] so I'm not sure how
>     >     that would related to the Phoenix client JAR. What version of HBase
>     >     was this against?
>     >
>     >     The only case I know of that needs the Phoenix client jar is when
>     >     Phoenix has been installed which then modifies the HBase config files
>     >     to specify some class names that come from phoenix client JAR.
>     >
>     >     As far as Java 10, there has been an on-going effort to become
>     >     compatible with Java 9+, but it is a non-trivial effort and not
>     >     complete yet [2].
>     >
>     >     Thanks,
>     >
>     >     Bryan
>     >
>     >     [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_hbase_blob_rel_1.1.2_hbase-2Dcommon_src_main_java_org_apache_hadoop_hbase_util_ClassSize.java&d=DwIBaQ&c=7DfhQjPWzR3PmWBQVpi-kw&r=Sb-vyl3zZFvHwGIFN-QEmQCWZUYgyH4UBZPmcWFaTC8&m=bnetA9Y56Fx_Tiq-PCdevSuGr34t8ixQz5sDQMViOmc&s=XVjke9vTfRooHYCqZCNp1AS5kUbiGaJxHHF6OyNcnfQ&e=
>     >     [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_NIFI-2D5174&d=DwIBaQ&c=7DfhQjPWzR3PmWBQVpi-kw&r=Sb-vyl3zZFvHwGIFN-QEmQCWZUYgyH4UBZPmcWFaTC8&m=bnetA9Y56Fx_Tiq-PCdevSuGr34t8ixQz5sDQMViOmc&s=dHD8gEEW65OaoGA1DC-FE3i7EJRLatqydkeFpaHB1IA&e=
>     >
>     >
>     >     On Thu, Aug 23, 2018 at 1:47 PM, Martini, Adam <[hidden email]> wrote:
>     >     > Hello all,
>     >     >
>     >     > We were struggling with our HBase client connection in the NiFi 1.7.1 release. We are not using Phoenix and our client was erroring with a missing class exception:
>     >     >
>     >     > Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.util.ClassSize
>     >     >                 at org.apache.hadoop.hbase.ipc.IPCUtil.<init>(IPCUtil.java:72)
>     >     >                 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.<init>(AbstractRpcClient.java:91)
>     >     >                 at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1113)
>     >     >                 at org.apache.hadoop.hbase.ipc.RpcClientImpl.<init>(RpcClientImpl.java:1139)
>     >     >                 ... 33 common frames omitted
>     >     >
>     >     > Adding a Phoenix client (which we do not need) helps resolve this class dependency, but fails if NiFi is running on Java 10:
>     >     >
>     >     > Caused by: java.lang.RuntimeException: Unexpected version format: 10.0.1
>     >     >     at org.apache.hadoop.hbase.util.ClassSize.<clinit>(ClassSize.java:119)
>     >     >     ... 37 common frames omitted
>     >     >
>     >     > Starting NiFi with Java 9 fixes this error, but complicates our devops as Java 9 is no longer supported and cannot be installed via package management.
>     >     >
>     >     > Please fix this dependency issue so that a Phoenix client is not required to resolve the org.apache.hadoop.hbase.util.ClassSize class. Also, please migrate to Java 10 as soon as possible.
>     >     >
>     >     > Let me know if I can help with this process.
>     >     >
>     >     > Thanks,
>     >     >
>     >     > Adam Martini
>     >     > Senior Software Dev, Nike Digital
>     >     >
>     >
>     >
>
>