How to capture more than 40 groups in Extract Text

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How to capture more than 40 groups in Extract Text

shweta
Hi All,

I have requirement wherein I need to convert a csv file to JSON. The input csv file has 135 attributes.
I referred to nifi example template csv-to-json.xml which uses a combination of replaceText and ExtractText processor.
But I think ExtractText has limitation of capturing not more that 40 groups. Is there a way around to handle this scenario.

Regards,
Shweta
Reply | Threaded
Open this post in threaded view
|

Re: How to capture more than 40 groups in Extract Text

Bryan Bende
Hi Shweta,

You may want to consider a custom processor at this point.
The csv-to-json example works ok for smaller csv files, but admittedly is
not a great solution when there are a lot of columns.
There has been interest from the community in the past on having a
ConvertCsvToJson processor, but no one has taken on the task yet [1].

-Bryan

[1] https://issues.apache.org/jira/browse/NIFI-1398


On Tue, Feb 2, 2016 at 11:40 PM, shweta <[hidden email]> wrote:

> Hi All,
>
> I have requirement wherein I need to convert a csv file to JSON. The input
> csv file has 135 attributes.
> I referred to nifi example template csv-to-json.xml which uses a
> combination
> of replaceText and ExtractText processor.
> But I think ExtractText has limitation of capturing not more that 40
> groups.
> Is there a way around to handle this scenario.
>
> Regards,
> Shweta
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/How-to-capture-more-than-40-groups-in-Extract-Text-tp7115.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: How to capture more than 40 groups in Extract Text

Matt Burgess
Schweta,

The ExecuteScript processor (coming in NiFi 0.5.0) will allow you to do this in code without having to build a whole processor bundle. I have a template where I do something similar, although I only grab two columns not 135 :)  However it seems very possible and likely more efficient than the regex approach.




On 2/3/16, 9:30 AM, "Bryan Bende" <[hidden email]> wrote:

>Hi Shweta,
>
>You may want to consider a custom processor at this point.
>The csv-to-json example works ok for smaller csv files, but admittedly is
>not a great solution when there are a lot of columns.
>There has been interest from the community in the past on having a
>ConvertCsvToJson processor, but no one has taken on the task yet [1].
>
>-Bryan
>
>[1] https://issues.apache.org/jira/browse/NIFI-1398
>
>
>On Tue, Feb 2, 2016 at 11:40 PM, shweta <[hidden email]> wrote:
>
>> Hi All,
>>
>> I have requirement wherein I need to convert a csv file to JSON. The input
>> csv file has 135 attributes.
>> I referred to nifi example template csv-to-json.xml which uses a
>> combination
>> of replaceText and ExtractText processor.
>> But I think ExtractText has limitation of capturing not more that 40
>> groups.
>> Is there a way around to handle this scenario.
>>
>> Regards,
>> Shweta
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-nifi-developer-list.39713.n7.nabble.com/How-to-capture-more-than-40-groups-in-Extract-Text-tp7115.html
>> Sent from the Apache NiFi Developer List mailing list archive at
>> Nabble.com.
>>