Site-to-Site load balancing

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Site-to-Site load balancing

Mark Bean
How is load balancing across a cluster performed in Site-to-Site? Is it the
responsibility of the Cluster Coordinator to inform the remote site-to-site
client which Node to push data to? Is the Node determination simply
round-robin? or total per Node FlowFile count or byte count? or based on
some more specific load statistics of the host system?

Thanks,
Mark
Reply | Threaded
Open this post in threaded view
|

Re: Site-to-Site load balancing

Bryan Rosander-3
Hey Mark,

The Site-to-Site client attempts to load balance based on the number of
flow files each node is currently processing combined with some randomness.
[1]

Thanks,
Bryan

[1]
https://github.com/apache/nifi/blob/master/nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/PeerSelector.java#L159

On Thu, May 18, 2017 at 9:41 AM, Mark Bean <[hidden email]> wrote:

> How is load balancing across a cluster performed in Site-to-Site? Is it the
> responsibility of the Cluster Coordinator to inform the remote site-to-site
> client which Node to push data to? Is the Node determination simply
> round-robin? or total per Node FlowFile count or byte count? or based on
> some more specific load statistics of the host system?
>
> Thanks,
> Mark
>