Load issues

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Load issues

Phil H
Hi team,

My NiFi is struggling to process data (lots of back pressure in various
queues) but it is only using a very small amount (2-5%) of CPU according to
top.

Any ideas?
Reply | Threaded
Open this post in threaded view
|

Re: Load issues

Milan Das
Did you increase the no. of Concurrent Tasks and Run Duration on slow processors ?

Thanks,
Milan Das

On 10/23/18, 8:26 PM, "Phil H" <[hidden email]> wrote:

    Hi team,
   
    My NiFi is struggling to process data (lots of back pressure in various
    queues) but it is only using a very small amount (2-5%) of CPU according to
    top.
   
    Any ideas?
   


Reply | Threaded
Open this post in threaded view
|

Re: Load issues

Charlie Meyer
Also, we've seen flows automatically throttled if provenance (or other
repositories) can't keep up. Tuning the configs and switching drives (gp2
-> io1) helped

On Tue, Oct 23, 2018, 19:38 Milan Das <[hidden email]> wrote:

> Did you increase the no. of Concurrent Tasks and Run Duration on slow
> processors ?
>
> Thanks,
> Milan Das
>
> On 10/23/18, 8:26 PM, "Phil H" <[hidden email]> wrote:
>
>     Hi team,
>
>     My NiFi is struggling to process data (lots of back pressure in various
>     queues) but it is only using a very small amount (2-5%) of CPU
> according to
>     top.
>
>     Any ideas?
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Load issues

Phil H
In reply to this post by Milan Das
Yep - although it now has less throughput with 8 threads than it previously had with one.

I have seen this happen when you have a lot of errors going into the logs, but that doesn’t appear to be the case this time around

Run Schedule is set to 0 sec

> On 24 Oct 2018, at 11:38 am, Milan Das <[hidden email]> wrote:
>
> Did you increase the no. of Concurrent Tasks and Run Duration on slow processors ?
>
> Thanks,
> Milan Das
>
> On 10/23/18, 8:26 PM, "Phil H" <[hidden email]> wrote:
>
>    Hi team,
>
>    My NiFi is struggling to process data (lots of back pressure in various
>    queues) but it is only using a very small amount (2-5%) of CPU according to
>    top.
>
>    Any ideas?
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Load issues

Bryan Bende
You’ll probably need to provide some more information like hardware specs
(cpus/cores, memory, disks), number of nodes in the cluster, JVM heap size,
configuration of repositories, specific processors that are bottle necks,
etc.

On Tue, Oct 23, 2018 at 10:02 PM Phil H <[hidden email]> wrote:

> Yep - although it now has less throughput with 8 threads than it
> previously had with one.
>
> I have seen this happen when you have a lot of errors going into the logs,
> but that doesn’t appear to be the case this time around
>
> Run Schedule is set to 0 sec
>
> > On 24 Oct 2018, at 11:38 am, Milan Das <[hidden email]> wrote:
> >
> > Did you increase the no. of Concurrent Tasks and Run Duration on slow
> processors ?
> >
> > Thanks,
> > Milan Das
> >
> > On 10/23/18, 8:26 PM, "Phil H" <[hidden email]> wrote:
> >
> >    Hi team,
> >
> >    My NiFi is struggling to process data (lots of back pressure in
> various
> >    queues) but it is only using a very small amount (2-5%) of CPU
> according to
> >    top.
> >
> >    Any ideas?
> >
> >
> >
>
> --
Sent from Gmail Mobile
Reply | Threaded
Open this post in threaded view
|

Re: Load issues

Phil H
It's a single node. Neither the data rate or machine spec has changed
though. It's just running a lot slower now. With no errors being reported

On Wed, 24 Oct 2018 at 13:18, Bryan Bende <[hidden email]> wrote:

> You’ll probably need to provide some more information like hardware specs
> (cpus/cores, memory, disks), number of nodes in the cluster, JVM heap size,
> configuration of repositories, specific processors that are bottle necks,
> etc.
>
> On Tue, Oct 23, 2018 at 10:02 PM Phil H <[hidden email]> wrote:
>
> > Yep - although it now has less throughput with 8 threads than it
> > previously had with one.
> >
> > I have seen this happen when you have a lot of errors going into the
> logs,
> > but that doesn’t appear to be the case this time around
> >
> > Run Schedule is set to 0 sec
> >
> > > On 24 Oct 2018, at 11:38 am, Milan Das <[hidden email]> wrote:
> > >
> > > Did you increase the no. of Concurrent Tasks and Run Duration on slow
> > processors ?
> > >
> > > Thanks,
> > > Milan Das
> > >
> > > On 10/23/18, 8:26 PM, "Phil H" <[hidden email]> wrote:
> > >
> > >    Hi team,
> > >
> > >    My NiFi is struggling to process data (lots of back pressure in
> > various
> > >    queues) but it is only using a very small amount (2-5%) of CPU
> > according to
> > >    top.
> > >
> > >    Any ideas?
> > >
> > >
> > >
> >
> > --
> Sent from Gmail Mobile
>