NiFi 1.4: PublishKafkaRecord_0_10: failed to process due to java.lang.NullPointerException

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

NiFi 1.4: PublishKafkaRecord_0_10: failed to process due to java.lang.NullPointerException

mayank rathi
Hello All,

I am moving data to Kafka using NiFi's PublishKafkaRecord processor. I am
using ConfluentSchemaRegistry Controller service and getting below error:

2017-08-11 20:54:25,937 ERROR [Timer-Driven Process Thread-4]
o.a.n.p.k.pubsub.PublishKafkaRecord_0_10
PublishKafkaRecord_0_10[id=b3c03961-015d-1000-0946-79ccbe2ffbbd]
PublishKafkaRecord_0_10[id=b3c03961-015d-1000-0946-79ccbe2ffbbd] failed to
process due to java.lang.NullPointerException; rolling back session: {}
java.lang.NullPointerException: null

I do not see any error on Kafka side.

How can I debug and resolve this issue?

Thanks!!

--
NOTICE: This email message is for the sole use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: NiFi 1.4: PublishKafkaRecord_0_10: failed to process due to java.lang.NullPointerException

mayank rathi
Attached are the logs after setting processors in Debug mode. Here is the flow:

ExecuteSQL --> SplitAvro --> PublishKafkaRecord_0_10

Thanks!!

On Fri, Aug 11, 2017 at 9:01 PM, mayank rathi <[hidden email]> wrote:
Hello All,

I am moving data to Kafka using NiFi's PublishKafkaRecord processor. I am using ConfluentSchemaRegistry Controller service and getting below error:

2017-08-11 20:54:25,937 ERROR [Timer-Driven Process Thread-4] o.a.n.p.k.pubsub.PublishKafkaRecord_0_10 PublishKafkaRecord_0_10[id=b3c03961-015d-1000-0946-79ccbe2ffbbd] PublishKafkaRecord_0_10[id=b3c03961-015d-1000-0946-79ccbe2ffbbd] failed to process due to java.lang.NullPointerException; rolling back session: {}
java.lang.NullPointerException: null

I do not see any error on Kafka side.

How can I debug and resolve this issue?

Thanks!!

--
NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.



--
NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: NiFi 1.4: PublishKafkaRecord_0_10: failed to process due to java.lang.NullPointerException

Joe Witt
Hello

Can you please file a jira and attach the logs to the jira.

Thanks
Joe

On Aug 11, 2017 6:23 PM, "mayank rathi" <[hidden email]> wrote:

> Attached are the logs after setting processors in Debug mode. Here is the
> flow:
>
> ExecuteSQL --> SplitAvro --> PublishKafkaRecord_0_10
>
> Thanks!!
>
> On Fri, Aug 11, 2017 at 9:01 PM, mayank rathi <[hidden email]>
> wrote:
>
>> Hello All,
>>
>> I am moving data to Kafka using NiFi's PublishKafkaRecord processor. I am
>> using ConfluentSchemaRegistry Controller service and getting below error:
>>
>> 2017-08-11 20:54:25,937 ERROR [Timer-Driven Process Thread-4]
>> o.a.n.p.k.pubsub.PublishKafkaRecord_0_10 PublishKafkaRecord_0_10[id=b3c
>> 03961-015d-1000-0946-79ccbe2ffbbd] PublishKafkaRecord_0_10[id=b3c
>> 03961-015d-1000-0946-79ccbe2ffbbd] failed to process due to
>> java.lang.NullPointerException; rolling back session: {}
>> java.lang.NullPointerException: null
>>
>> I do not see any error on Kafka side.
>>
>> How can I debug and resolve this issue?
>>
>> Thanks!!
>>
>> --
>> NOTICE: This email message is for the sole use of the intended
>> recipient(s) and may contain confidential and privileged information. Any
>> unauthorized review, use, disclosure or distribution is prohibited. If you
>> are not the intended recipient, please contact the sender by reply email
>> and destroy all copies of the original message.
>>
>
>
>
> --
> NOTICE: This email message is for the sole use of the intended
> recipient(s) and may contain confidential and privileged information. Any
> unauthorized review, use, disclosure or distribution is prohibited. If you
> are not the intended recipient, please contact the sender by reply email
> and destroy all copies of the original message.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: NiFi 1.4: PublishKafkaRecord_0_10: failed to process due to java.lang.NullPointerException

mayank rathi
Hello Joe,

JIRA logged.

https://issues.apache.org/jira/browse/NIFI-4290

Thanks!!

On Fri, Aug 11, 2017 at 9:48 PM, Joe Witt <[hidden email]> wrote:

> Hello
>
> Can you please file a jira and attach the logs to the jira.
>
> Thanks
> Joe
>
> On Aug 11, 2017 6:23 PM, "mayank rathi" <[hidden email]> wrote:
>
> > Attached are the logs after setting processors in Debug mode. Here is the
> > flow:
> >
> > ExecuteSQL --> SplitAvro --> PublishKafkaRecord_0_10
> >
> > Thanks!!
> >
> > On Fri, Aug 11, 2017 at 9:01 PM, mayank rathi <[hidden email]>
> > wrote:
> >
> >> Hello All,
> >>
> >> I am moving data to Kafka using NiFi's PublishKafkaRecord processor. I
> am
> >> using ConfluentSchemaRegistry Controller service and getting below
> error:
> >>
> >> 2017-08-11 20:54:25,937 ERROR [Timer-Driven Process Thread-4]
> >> o.a.n.p.k.pubsub.PublishKafkaRecord_0_10 PublishKafkaRecord_0_10[id=b3c
> >> 03961-015d-1000-0946-79ccbe2ffbbd] PublishKafkaRecord_0_10[id=b3c
> >> 03961-015d-1000-0946-79ccbe2ffbbd] failed to process due to
> >> java.lang.NullPointerException; rolling back session: {}
> >> java.lang.NullPointerException: null
> >>
> >> I do not see any error on Kafka side.
> >>
> >> How can I debug and resolve this issue?
> >>
> >> Thanks!!
> >>
> >> --
> >> NOTICE: This email message is for the sole use of the intended
> >> recipient(s) and may contain confidential and privileged information.
> Any
> >> unauthorized review, use, disclosure or distribution is prohibited. If
> you
> >> are not the intended recipient, please contact the sender by reply email
> >> and destroy all copies of the original message.
> >>
> >
> >
> >
> > --
> > NOTICE: This email message is for the sole use of the intended
> > recipient(s) and may contain confidential and privileged information. Any
> > unauthorized review, use, disclosure or distribution is prohibited. If
> you
> > are not the intended recipient, please contact the sender by reply email
> > and destroy all copies of the original message.
> >
>



--
NOTICE: This email message is for the sole use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: NiFi 1.4: PublishKafkaRecord_0_10: failed to process due to java.lang.NullPointerException

Joe Witt
Thanks for reporting the issue.  It appears the NPE can occur if zero
records end up getting published and it fails to lookup the schema
because the message tracker is not yet initialized.

So a couple things:
1) Verify the settings being used to lookup the schema.  You can show
the record reader, writer, and schema registry settings being used for
that publisher.
2) You do not need to SplitAvro before sending it via publish kafka
record.  Avoiding doing the split beforehand can result in vastly
superior performance.

Thanks
Joe

On Fri, Aug 11, 2017 at 8:03 PM, mayank rathi <[hidden email]> wrote:

> Hello Joe,
>
> JIRA logged.
>
> https://issues.apache.org/jira/browse/NIFI-4290
>
> Thanks!!
>
> On Fri, Aug 11, 2017 at 9:48 PM, Joe Witt <[hidden email]> wrote:
>
>> Hello
>>
>> Can you please file a jira and attach the logs to the jira.
>>
>> Thanks
>> Joe
>>
>> On Aug 11, 2017 6:23 PM, "mayank rathi" <[hidden email]> wrote:
>>
>> > Attached are the logs after setting processors in Debug mode. Here is the
>> > flow:
>> >
>> > ExecuteSQL --> SplitAvro --> PublishKafkaRecord_0_10
>> >
>> > Thanks!!
>> >
>> > On Fri, Aug 11, 2017 at 9:01 PM, mayank rathi <[hidden email]>
>> > wrote:
>> >
>> >> Hello All,
>> >>
>> >> I am moving data to Kafka using NiFi's PublishKafkaRecord processor. I
>> am
>> >> using ConfluentSchemaRegistry Controller service and getting below
>> error:
>> >>
>> >> 2017-08-11 20:54:25,937 ERROR [Timer-Driven Process Thread-4]
>> >> o.a.n.p.k.pubsub.PublishKafkaRecord_0_10 PublishKafkaRecord_0_10[id=b3c
>> >> 03961-015d-1000-0946-79ccbe2ffbbd] PublishKafkaRecord_0_10[id=b3c
>> >> 03961-015d-1000-0946-79ccbe2ffbbd] failed to process due to
>> >> java.lang.NullPointerException; rolling back session: {}
>> >> java.lang.NullPointerException: null
>> >>
>> >> I do not see any error on Kafka side.
>> >>
>> >> How can I debug and resolve this issue?
>> >>
>> >> Thanks!!
>> >>
>> >> --
>> >> NOTICE: This email message is for the sole use of the intended
>> >> recipient(s) and may contain confidential and privileged information.
>> Any
>> >> unauthorized review, use, disclosure or distribution is prohibited. If
>> you
>> >> are not the intended recipient, please contact the sender by reply email
>> >> and destroy all copies of the original message.
>> >>
>> >
>> >
>> >
>> > --
>> > NOTICE: This email message is for the sole use of the intended
>> > recipient(s) and may contain confidential and privileged information. Any
>> > unauthorized review, use, disclosure or distribution is prohibited. If
>> you
>> > are not the intended recipient, please contact the sender by reply email
>> > and destroy all copies of the original message.
>> >
>>
>
>
>
> --
> NOTICE: This email message is for the sole use of the intended recipient(s)
> and may contain confidential and privileged information. Any unauthorized
> review, use, disclosure or distribution is prohibited. If you are not the
> intended recipient, please contact the sender by reply email and destroy
> all copies of the original message.
Loading...