connection fileQueue size

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

connection fileQueue size

pradeepbill
hello everybody, is there a way to get notified if a particular connection reaches particular size ?, I see {file size}, but not {connection.fileQueue.size} or something similar .Please advice.

Thanks
Pradeep
Reply | Threaded
Open this post in threaded view
|

Re: connection fileQueue size

Mark Payne
Hi Pradeep,

NiFi does not have any mechanism natively to provide notifications in such cases. What is available is a set
of reporting tasks that can be used to publish metrics such as queue size to ganglia or ambari. The queue sizes
are also available via the REST API, so third-party tools may provide a mechanism for querying the queue sizes
periodically and notifying you when the values exceed some threshold.

Thanks
-Mark


> On Jul 7, 2016, at 7:18 AM, pradeepbill <[hidden email]> wrote:
>
> hello everybody, is there a way to get notified if a particular connection
> reaches particular size ?, I see {file size}, but not
> {connection.fileQueue.size} or something similar .Please advice.
>
> Thanks
> Pradeep
>
>
>
> --
> View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/connection-fileQueue-size-tp12686.html
> Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: connection fileQueue size

pradeepbill
thanks Mark, that would help as well.I can send out emails from REST service as well.

Reply | Threaded
Open this post in threaded view
|

Re: connection fileQueue size

McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)
In reply to this post by Mark Payne
Pradeep,

We are using the the Ganglia reporting task with the ELK stack to monitor our NiFis.  The reporting task sends metrics to a LogStash running a ganglia plugin.  We use Kibana to visualize the metrics and Watcher to create notifications.  It was fairly easy to setup and works quite well.

Best of luck,

Chris McDermott
 
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
 


On 7/7/16, 8:51 AM, "Mark Payne" <[hidden email]> wrote:

>Hi Pradeep,
>
>NiFi does not have any mechanism natively to provide notifications in such cases. What is available is a set
>of reporting tasks that can be used to publish metrics such as queue size to ganglia or ambari. The queue sizes
>are also available via the REST API, so third-party tools may provide a mechanism for querying the queue sizes
>periodically and notifying you when the values exceed some threshold.
>
>Thanks
>-Mark
>
>
>> On Jul 7, 2016, at 7:18 AM, pradeepbill <[hidden email]> wrote:
>>
>> hello everybody, is there a way to get notified if a particular connection
>> reaches particular size ?, I see {file size}, but not
>> {connection.fileQueue.size} or something similar .Please advice.
>>
>> Thanks
>> Pradeep
>>
>>
>>
>> --
>> View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/connection-fileQueue-size-tp12686.html
>> Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
>