Route Based on Result

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Route Based on Result

Toivo Adams
Data must be sent to channel (PutEmail, PutFile, PutSFTP,etc)
In case of channel failure we should retry to send again 3 times before giving up.

How to achieve this?

One possibility is to create custom router RouterRetry which will examine FlowFile certain attribute FailedCount to check how many times this FlowFile sending is failed.
RouterRetry has 2 relationships REL_DOWORK, REL_FAILURE

It seems another processor IncFailedCount is needed. Its only task is to set FailedCount attribute or to increment it.

Let's connect channel to  RouterRetry  REL_DOWORK and channel REL_FAILURE to  IncFailedCount.
And  IncFailedCount relationship back to RouterRetry.

Now  RouterRetry can decide whetever to send  FlowFile to  REL_DOWORK
or to REL_FAILURE when all retries failed.

Is this reasonable?
Any other (simpler) way?

Thanks
toivo
Reply | Threaded
Open this post in threaded view
|

Re: Route Based on Result

Joe Witt
Toivo

This is definitely something you can do.  So let's use your example.
You have a PutFile processor and if a given object routes to failure
three times you want to be able to detect that and do something else.
Here is how:

- Before PutFile use an UpdateAttribute processor configured with a
single property 'times-through-here' and a value of '0'

- After PutFile's failure relationship route the results to another
UpdateAttribute processor.  In it set an attribute property
'times-through-here' with a value of '${times-through-here:plus(1)}'

- After that UpdateAttribute route the data to a RouteOnAttribute.
The route on attribute should have a property of 'too many' with a
value of '${times-through-here:ge( 3 )}'.  For the 'too many'
relationship this is your stuff that has failed too many times route
it wherever you like or auto terminate.  For the 'no match'
relationship you can route it back to the PutFile

Using this same pattern and expression language capability you can
imagine a variety of powerful things you can do to control it.

If you have any questions let us know.  We could turn this into a good
FAQ type entry complete with screenshots and templates.

We need to get our expression language docs on the website.  In the
mean time you can view them in the running app through the help
screen.

Thanks
Joe

On Sun, Feb 15, 2015 at 2:18 PM, Toivo Adams <[hidden email]> wrote:

> Data must be sent to channel (PutEmail, PutFile, PutSFTP,etc)
> In case of channel failure we should retry to send again 3 times before
> giving up.
>
> How to achieve this?
>
> One possibility is to create custom router RouterRetry which will examine
> FlowFile certain attribute FailedCount to check how many times this FlowFile
> sending is failed.
> RouterRetry has 2 relationships REL_DOWORK, REL_FAILURE
>
> It seems another processor IncFailedCount is needed. Its only task is to set
> FailedCount attribute or to increment it.
>
> Let's connect channel to  RouterRetry  REL_DOWORK and channel REL_FAILURE to
> IncFailedCount.
> And  IncFailedCount relationship back to RouterRetry.
>
> Now  RouterRetry can decide whetever to send  FlowFile to  REL_DOWORK
> or to REL_FAILURE when all retries failed.
>
> Is this reasonable?
> Any other (simpler) way?
>
> Thanks
> toivo
>
>
>
>
> --
> View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Route-Based-on-Result-tp692.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Route Based on Result

Toivo Adams
Let's assume we need to send some data to client.
Different clients may have and prefer different channels (email, sms, post, portal, etc) and combination of channels.
Let's imagine we start delivering by sending data to both portal and email channel at once.
When any of them succeed we are done.
Otherwise we must pick next channel, for example ordinary post.
And so on.

Sending to channels may be combined with retries.
Ideally we need to retry only when send failed by some temporary reason.
(some channel is not usable right now because of server down or overloaded, but works ok after few hours)
I am not sure how this can be catched.

Finally we either succeed or failed.
Failed data should be collected and next day some person will manually intervene and decide how to solve undelivered data.
Successfully delivered should also collected and some nice report generated to which channel data was successfully delivered.

What do you think, can this be done using expression language and UpdateAttribute and RouteOnAttribute processors?
Or some custom router might help here?

Thank you very much!
toivo