Checking for existence of mandatory attributes

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Checking for existence of mandatory attributes

Mike Thomsen
Do we have any processors that would be particularly good at checking a
flowfile for the existence of certain mandatory attributes and then routing
flowfiles missing them to a failure relationship?

I have a processor sketched out in my head for this already, but don't want
to reinvent the wheel.

Thanks,

Mike
Reply | Threaded
Open this post in threaded view
|

Re: Checking for existence of mandatory attributes

Bryan Bende
RouteOnAttribute?

On Thu, Dec 20, 2018 at 11:45 AM Mike Thomsen <[hidden email]> wrote:

>
> Do we have any processors that would be particularly good at checking a
> flowfile for the existence of certain mandatory attributes and then routing
> flowfiles missing them to a failure relationship?
>
> I have a processor sketched out in my head for this already, but don't want
> to reinvent the wheel.
>
> Thanks,
>
> Mike
Reply | Threaded
Open this post in threaded view
|

Re: Checking for existence of mandatory attributes

Mark Payne
In reply to this post by Mike Thomsen
Hey Mike,

I would recommend RouteOnAttribute for that.

Thanks
-Mark

Sent from my iPhone

> On Dec 20, 2018, at 11:45 AM, Mike Thomsen <[hidden email]> wrote:
>
> Do we have any processors that would be particularly good at checking a
> flowfile for the existence of certain mandatory attributes and then routing
> flowfiles missing them to a failure relationship?
>
> I have a processor sketched out in my head for this already, but don't want
> to reinvent the wheel.
>
> Thanks,
>
> Mike
Reply | Threaded
Open this post in threaded view
|

Re: Checking for existence of mandatory attributes

Mike Thomsen
This is probably enough:
https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#allattributes

I missed that because I was thinking "how do I do a bunch of 'and must
exist' statements."

Thanks,

Mike

On Thu, Dec 20, 2018 at 12:01 PM Mark Payne <[hidden email]> wrote:

> Hey Mike,
>
> I would recommend RouteOnAttribute for that.
>
> Thanks
> -Mark
>
> Sent from my iPhone
>
> > On Dec 20, 2018, at 11:45 AM, Mike Thomsen <[hidden email]>
> wrote:
> >
> > Do we have any processors that would be particularly good at checking a
> > flowfile for the existence of certain mandatory attributes and then
> routing
> > flowfiles missing them to a failure relationship?
> >
> > I have a processor sketched out in my head for this already, but don't
> want
> > to reinvent the wheel.
> >
> > Thanks,
> >
> > Mike
>
Reply | Threaded
Open this post in threaded view
|

Re: Checking for existence of mandatory attributes

Mike Thomsen
I might still hack out a prototype and send it as a PR because it would be
like a really beefed up version of RouteOnAttribute that is built around
policy enforcement with attributes.

On Thu, Dec 20, 2018 at 12:18 PM Mike Thomsen <[hidden email]>
wrote:

> This is probably enough:
> https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#allattributes
>
> I missed that because I was thinking "how do I do a bunch of 'and must
> exist' statements."
>
> Thanks,
>
> Mike
>
> On Thu, Dec 20, 2018 at 12:01 PM Mark Payne <[hidden email]> wrote:
>
>> Hey Mike,
>>
>> I would recommend RouteOnAttribute for that.
>>
>> Thanks
>> -Mark
>>
>> Sent from my iPhone
>>
>> > On Dec 20, 2018, at 11:45 AM, Mike Thomsen <[hidden email]>
>> wrote:
>> >
>> > Do we have any processors that would be particularly good at checking a
>> > flowfile for the existence of certain mandatory attributes and then
>> routing
>> > flowfiles missing them to a failure relationship?
>> >
>> > I have a processor sketched out in my head for this already, but don't
>> want
>> > to reinvent the wheel.
>> >
>> > Thanks,
>> >
>> > Mike
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Checking for existence of mandatory attributes

Mark Payne
Mike,

I do think that RouteOnAttribute is the correct play here, given the details at hand.
Other options, should that be too cumbersome to configure may be UpdateAttribute using the
Advanced configuration, which works sort of like a mini rules engine. Or, to be honest,
this sounds like it could be a great use of ExecuteScript.

If you want to create a new Processor, then by all means do so, but I think we'd need a better
description of the use case for it and description of where the other approaches fall short and how this new
Processor would fill in the gap in order for it to be warranted as a standard processor.

Thanks
-Mark

> On Dec 20, 2018, at 12:19 PM, Mike Thomsen <[hidden email]> wrote:
>
> I might still hack out a prototype and send it as a PR because it would be
> like a really beefed up version of RouteOnAttribute that is built around
> policy enforcement with attributes.
>
> On Thu, Dec 20, 2018 at 12:18 PM Mike Thomsen <[hidden email]>
> wrote:
>
>> This is probably enough:
>> https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#allattributes
>>
>> I missed that because I was thinking "how do I do a bunch of 'and must
>> exist' statements."
>>
>> Thanks,
>>
>> Mike
>>
>> On Thu, Dec 20, 2018 at 12:01 PM Mark Payne <[hidden email]> wrote:
>>
>>> Hey Mike,
>>>
>>> I would recommend RouteOnAttribute for that.
>>>
>>> Thanks
>>> -Mark
>>>
>>> Sent from my iPhone
>>>
>>>> On Dec 20, 2018, at 11:45 AM, Mike Thomsen <[hidden email]>
>>> wrote:
>>>>
>>>> Do we have any processors that would be particularly good at checking a
>>>> flowfile for the existence of certain mandatory attributes and then
>>> routing
>>>> flowfiles missing them to a failure relationship?
>>>>
>>>> I have a processor sketched out in my head for this already, but don't
>>> want
>>>> to reinvent the wheel.
>>>>
>>>> Thanks,
>>>>
>>>> Mike
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Checking for existence of mandatory attributes

Mike Thomsen
I ended up getting a little creative by chaining a few Puts with multiple
RouteOnAttributes. Worked quite well so far.

On Thu, Dec 20, 2018 at 1:59 PM Mark Payne <[hidden email]> wrote:

> Mike,
>
> I do think that RouteOnAttribute is the correct play here, given the
> details at hand.
> Other options, should that be too cumbersome to configure may be
> UpdateAttribute using the
> Advanced configuration, which works sort of like a mini rules engine. Or,
> to be honest,
> this sounds like it could be a great use of ExecuteScript.
>
> If you want to create a new Processor, then by all means do so, but I
> think we'd need a better
> description of the use case for it and description of where the other
> approaches fall short and how this new
> Processor would fill in the gap in order for it to be warranted as a
> standard processor.
>
> Thanks
> -Mark
>
> > On Dec 20, 2018, at 12:19 PM, Mike Thomsen <[hidden email]>
> wrote:
> >
> > I might still hack out a prototype and send it as a PR because it would
> be
> > like a really beefed up version of RouteOnAttribute that is built around
> > policy enforcement with attributes.
> >
> > On Thu, Dec 20, 2018 at 12:18 PM Mike Thomsen <[hidden email]>
> > wrote:
> >
> >> This is probably enough:
> >>
> https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#allattributes
> >>
> >> I missed that because I was thinking "how do I do a bunch of 'and must
> >> exist' statements."
> >>
> >> Thanks,
> >>
> >> Mike
> >>
> >> On Thu, Dec 20, 2018 at 12:01 PM Mark Payne <[hidden email]>
> wrote:
> >>
> >>> Hey Mike,
> >>>
> >>> I would recommend RouteOnAttribute for that.
> >>>
> >>> Thanks
> >>> -Mark
> >>>
> >>> Sent from my iPhone
> >>>
> >>>> On Dec 20, 2018, at 11:45 AM, Mike Thomsen <[hidden email]>
> >>> wrote:
> >>>>
> >>>> Do we have any processors that would be particularly good at checking
> a
> >>>> flowfile for the existence of certain mandatory attributes and then
> >>> routing
> >>>> flowfiles missing them to a failure relationship?
> >>>>
> >>>> I have a processor sketched out in my head for this already, but don't
> >>> want
> >>>> to reinvent the wheel.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Mike
> >>>
> >>
>
>