KITESDK fails creation of schema

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

KITESDK fails creation of schema

pradeepbill
This post was updated on .
if my JSON has   "Blah_bc_sc-status":"407" , I can not create a valid schema using kiteSDK , fails saying

[parumalla@step6 ~]$ ./kite-dataset json-schema --class BCSchemaClass  hdfs:/user/parumalla/test_bc.txt  --output hdfs:/user/parumalla/nifi/schemas/bc.avsc

Unknown error: Illegal character in: Blah_bc_sc-status

If I replace the '-' with '_' , it works, I probably will try modifying the schema by hand and see what happens.

Thanks
Pradeep
Reply | Threaded
Open this post in threaded view
|

Re: KITESDK fails creation of schema

Andrew Psaltis
Hi Pradeep,
That error is due to "-" being an illegal character for an Avro name, per
the spec here[1]. Unfortunately the error message is not as descriptive as
it could be.

Thanks,
Andrew

[1] https://avro.apache.org/docs/1.7.7/spec.html#Names

On Wed, Jun 22, 2016 at 12:02 PM, pradeepbill <[hidden email]>
wrote:

> if my JSON has   "Blah_bc_sc-status":"407" , I can not create a valid
> schema
> using kiteSDK , fails saying
>
> [parumalla@sxmn5 ~]$ ./kite-dataset json-schema --class BCSchemaClass
> hdfs:/user/parumalla/test_bc.txt  --output
> hdfs:/user/parumalla/nifi/schemas/bc.avsc
>
> Unknown error: Illegal character in: Blah_bc_sc-status
>
> If I replace the '-' with '_' , it works, I probably will try modifying the
> schema by hand and see what happens.
>
> Thanks
> Pradeep
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/KITESDK-fails-creation-of-schema-tp12105.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>



--
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
Reply | Threaded
Open this post in threaded view
|

Re: KITESDK fails creation of schema

pradeepbill
awesome, thanks Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: KITESDK fails creation of schema

pradeepbill
In reply to this post by Andrew Psaltis
I probably will replace incoming json text "Blah_bc_sc-status" to "Blah_bc_sc_status" and then operate on it,

THanks
Pradeep
Reply | Threaded
Open this post in threaded view
|

Re: KITESDK fails creation of schema

Andrew Psaltis
That makes sense. You may want to take into consideration the following
lifted from the spec:


The name portion of a fullname, record field names, and enum symbols must:


   - start with [A-Za-z_]
      - subsequently contain only [A-Za-z0-9_]


Thus, you may want to replace any other characters with perhaps a "_". It
wold be nice if the kiteSDK or another Avro processor provided an option to
substitute illegal characters with a valid on.

Hope that helps.

Andrew

On Wed, Jun 22, 2016 at 12:21 PM, pradeepbill <[hidden email]>
wrote:

> I probably will replace incoming json text "Blah_bc_sc-status" to
> "Blah_bc_sc_status" and then operate on it,
>
> THanks
> Pradeep
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/KITESDK-fails-creation-of-schema-tp12105p12113.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>



--
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
Reply | Threaded
Open this post in threaded view
|

Re: KITESDK fails creation of schema

pradeepbill
thanks man.