Request flow within a cluster

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Request flow within a cluster

Anil Rai
Team,

Is there any documentation that explains how Nifi processes a request once
it reach nifi in a cluster settings?

Regards
Anil
Reply | Threaded
Open this post in threaded view
|

Re: Request flow within a cluster

Bryan Bende
Hello,

When you say "request" are you referring to the end-user interacting
with the NiFi web UI which makes a request to the server?

Or are you referring to data entering a NiFi cluster from an external source?

Thanks,

Bryan

On Tue, Mar 7, 2017 at 2:48 PM, Anil Rai <[hidden email]> wrote:
> Team,
>
> Is there any documentation that explains how Nifi processes a request once
> it reach nifi in a cluster settings?
>
> Regards
> Anil
Reply | Threaded
Open this post in threaded view
|

Re: Request flow within a cluster

Andy LoPresto-2
In reply to this post by Anil Rai
Anil,

Are you talking about processing a flowfile or responding to a REST HTTP request? The Admin Guide section on clustering [1] offers more information but if you can clarify your question you may get better answers from the mailing list. 



Andy LoPresto
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Mar 7, 2017, at 11:48 AM, Anil Rai <[hidden email]> wrote:

Team,

Is there any documentation that explains how Nifi processes a request once
it reach nifi in a cluster settings?

Regards
Anil


signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Request flow within a cluster

Anil Rai
Thanks Andy. I am looking for the processing of a flow file.If i create a
flow, with multiple processor, how does this request flow in nifi in a
cluster.


On Tue, Mar 7, 2017 at 2:58 PM, Andy LoPresto <[hidden email]> wrote:

> Anil,
>
> Are you talking about processing a flowfile or responding to a REST HTTP
> request? The Admin Guide section on clustering [1] offers more information
> but if you can clarify your question you may get better answers from the
> mailing list.
>
> [1] https://nifi.apache.org/docs/nifi-docs/html/administration-
> guide.html#clustering
>
>
> Andy LoPresto
> [hidden email]
> *[hidden email] <[hidden email]>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Mar 7, 2017, at 11:48 AM, Anil Rai <[hidden email]> wrote:
>
> Team,
>
> Is there any documentation that explains how Nifi processes a request once
> it reach nifi in a cluster settings?
>
> Regards
> Anil
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Request flow within a cluster

Bryan Bende
Anil,

The short answer is that each node in the cluster is running the
entire data flow you see in UI, and NiFi does not transfer flow files
between nodes unless the data flow is created to do that.

This article describes approaches to distributing data across the
cluster, which is dependent on how the data is entering the cluster:
https://community.hortonworks.com/articles/16120/how-do-i-distribute-data-across-a-nifi-cluster.html

Thanks,

Bryan


On Tue, Mar 7, 2017 at 3:03 PM, Anil Rai <[hidden email]> wrote:

> Thanks Andy. I am looking for the processing of a flow file.If i create a
> flow, with multiple processor, how does this request flow in nifi in a
> cluster.
>
>
> On Tue, Mar 7, 2017 at 2:58 PM, Andy LoPresto <[hidden email]> wrote:
>
>> Anil,
>>
>> Are you talking about processing a flowfile or responding to a REST HTTP
>> request? The Admin Guide section on clustering [1] offers more information
>> but if you can clarify your question you may get better answers from the
>> mailing list.
>>
>> [1] https://nifi.apache.org/docs/nifi-docs/html/administration-
>> guide.html#clustering
>>
>>
>> Andy LoPresto
>> [hidden email]
>> *[hidden email] <[hidden email]>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Mar 7, 2017, at 11:48 AM, Anil Rai <[hidden email]> wrote:
>>
>> Team,
>>
>> Is there any documentation that explains how Nifi processes a request once
>> it reach nifi in a cluster settings?
>>
>> Regards
>> Anil
>>
>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Request flow within a cluster

Anil Rai
Thank you.
Is there any documentation on the ideal cluster set up for production
environment? I saw the 3 zoo keepers on 3 VM's and 3 nodes on 3 VM's is the
best. Could you confirm?
Also the ideal system requirement for each node and each zoo keeper.....in
terms of CPU, memory, disk space etc?
and finally archiving....logs, flow files etc and their frequency

Thanks
Anil


On Tue, Mar 7, 2017 at 3:11 PM, Bryan Bende <[hidden email]> wrote:

> Anil,
>
> The short answer is that each node in the cluster is running the
> entire data flow you see in UI, and NiFi does not transfer flow files
> between nodes unless the data flow is created to do that.
>
> This article describes approaches to distributing data across the
> cluster, which is dependent on how the data is entering the cluster:
> https://community.hortonworks.com/articles/16120/how-do-i-
> distribute-data-across-a-nifi-cluster.html
>
> Thanks,
>
> Bryan
>
>
> On Tue, Mar 7, 2017 at 3:03 PM, Anil Rai <[hidden email]> wrote:
> > Thanks Andy. I am looking for the processing of a flow file.If i create a
> > flow, with multiple processor, how does this request flow in nifi in a
> > cluster.
> >
> >
> > On Tue, Mar 7, 2017 at 2:58 PM, Andy LoPresto <[hidden email]>
> wrote:
> >
> >> Anil,
> >>
> >> Are you talking about processing a flowfile or responding to a REST HTTP
> >> request? The Admin Guide section on clustering [1] offers more
> information
> >> but if you can clarify your question you may get better answers from the
> >> mailing list.
> >>
> >> [1] https://nifi.apache.org/docs/nifi-docs/html/administration-
> >> guide.html#clustering
> >>
> >>
> >> Andy LoPresto
> >> [hidden email]
> >> *[hidden email] <[hidden email]>*
> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>
> >> On Mar 7, 2017, at 11:48 AM, Anil Rai <[hidden email]> wrote:
> >>
> >> Team,
> >>
> >> Is there any documentation that explains how Nifi processes a request
> once
> >> it reach nifi in a cluster settings?
> >>
> >> Regards
> >> Anil
> >>
> >>
> >>
>