cron updating value used by another processor

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

cron updating value used by another processor

Edgardo Vega
Trying to figure out the best way to do the following. I want to have a
flow that runs every few minutes goes to url, pulls out a piece of json,
and puts it somewhere that another processor will use for every flow file
the comes in via another stream. The other processor would be something
like RouteOnContext or ExtractText.

Any ideas?

Thanks,

Edgardo
Reply | Threaded
Open this post in threaded view
|

Re: cron updating value used by another processor

Bryan Bende
I think PutDistributedMapCache and FetchDistributedMapCache could be used
for this.

-Bryan

On Wed, Jun 1, 2016 at 8:17 PM, Edgardo Vega <[hidden email]> wrote:

> Trying to figure out the best way to do the following. I want to have a
> flow that runs every few minutes goes to url, pulls out a piece of json,
> and puts it somewhere that another processor will use for every flow file
> the comes in via another stream. The other processor would be something
> like RouteOnContext or ExtractText.
>
> Any ideas?
>
> Thanks,
>
> Edgardo
>
Reply | Threaded
Open this post in threaded view
|

Re: cron updating value used by another processor

Edgardo Vega
Bryan,

I had not even thought of using those two processor. I will try tomorrow.
Thanks for the tip.

Cheers,

Edgardo

On Wed, Jun 1, 2016 at 8:23 PM, Bryan Bende <[hidden email]> wrote:

> I think PutDistributedMapCache and FetchDistributedMapCache could be used
> for this.
>
> -Bryan
>
> On Wed, Jun 1, 2016 at 8:17 PM, Edgardo Vega <[hidden email]>
> wrote:
>
> > Trying to figure out the best way to do the following. I want to have a
> > flow that runs every few minutes goes to url, pulls out a piece of json,
> > and puts it somewhere that another processor will use for every flow file
> > the comes in via another stream. The other processor would be something
> > like RouteOnContext or ExtractText.
> >
> > Any ideas?
> >
> > Thanks,
> >
> > Edgardo
> >
>



--
Cheers,

Edgardo
Reply | Threaded
Open this post in threaded view
|

Re: cron updating value used by another processor

Edgardo Vega
Bryan,

It worked technically. The only issue I see if that I am seeing spiky
performance.  It goes from having nothing in the queue before the Fetch to
having 50k and then back down. There are only 2 items in the cache right
now.

Cheers,

Edgardo

On Wed, Jun 1, 2016 at 9:29 PM, Edgardo Vega <[hidden email]> wrote:

> Bryan,
>
> I had not even thought of using those two processor. I will try tomorrow.
> Thanks for the tip.
>
> Cheers,
>
> Edgardo
>
> On Wed, Jun 1, 2016 at 8:23 PM, Bryan Bende <[hidden email]> wrote:
>
>> I think PutDistributedMapCache and FetchDistributedMapCache could be used
>> for this.
>>
>> -Bryan
>>
>> On Wed, Jun 1, 2016 at 8:17 PM, Edgardo Vega <[hidden email]>
>> wrote:
>>
>> > Trying to figure out the best way to do the following. I want to have a
>> > flow that runs every few minutes goes to url, pulls out a piece of json,
>> > and puts it somewhere that another processor will use for every flow
>> file
>> > the comes in via another stream. The other processor would be something
>> > like RouteOnContext or ExtractText.
>> >
>> > Any ideas?
>> >
>> > Thanks,
>> >
>> > Edgardo
>> >
>>
>
>
>
> --
> Cheers,
>
> Edgardo
>



--
Cheers,

Edgardo