AttributesToJSON Multi-Valued Fields

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

AttributesToJSON Multi-Valued Fields

dale.chang13
I don't know if this use-case is too specific to be a feature for a future release, but I would like to see the AttributesToJSON processor support multi-valued fields.

In my use-case, I am storing JSON documents into Solr, and there are two ways to store multi-valued fields: using an array and repeating keys in the JSON document. The result is a JSON array stored in that field in Solr, regardless if you submit repeated keys or not.

Would we like to see this implemented, and how should we go about doing so? I was thinking that if a FlowFile attribute is a comma-separated list, we could continue to use the Jackson ObjectMapper() and convert it to an array.
Reply | Threaded
Open this post in threaded view
|

Re: AttributesToJSON Multi-Valued Fields

Bryan Bende
Supporting multi-valued fields on AttributesToJson would be a nice
feature...

One approach might be to provide a strategy property that lets the user
choose the handling.

For example, if there was an attribute called attribute1 with a value of
a,b,c...

One user may want the json to end up as:
{ "attribute1" : "a,b,c" }

Another user may want it to be:
{ "attribute1" : [ "a", "b", "c" ] }

So maybe a property like "Multi-Value Handling" with options of "Array" or
"Single Value" ? (could even have a third option of "Repeating Field")

The downside is that you have to check the value of every single property
to see if it meets the pattern of a comma-separated list, although this
would only be done if you selected "Array" handling.

Another approach might be to have a naming convention where any attribute
who's name ends with "_multivalue" (or something better) would have the
value automatically split on commas and turned into an array.

-Bryan


On Tue, May 17, 2016 at 3:00 PM, dale.chang13 <[hidden email]>
wrote:

> I don't know if this use-case is too specific to be a feature for a future
> release, but I would like to see the AttributesToJSON processor support
> multi-valued fields.
>
> In my use-case, I am storing JSON documents into Solr, and there are two
> ways to store multi-valued fields: using an array and repeating keys in the
> JSON document. The result is a JSON array stored in that field in Solr,
> regardless if you submit repeated keys or not.
>
> *Would we like to see this implemented, and how should we go about doing
> so?* I was thinking that if a FlowFile attribute is a comma-separated list,
> we could continue to use the Jackson ObjectMapper() and convert it to an
> array.
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/AttributesToJSON-Multi-Valued-Fields-tp10452.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>