NIFI Requests

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

NIFI Requests

Adams, Timothy M.
Hello All,

I had a couple of requests that I believe could be valuable functions in NiFi.

One is the ability to "hide" a queue from the count displayed in the summary near the top of the GUI.  The reason for this is sometimes in a flow we have a processor that we have configured to create a backlog, or to slowly work off/deliver data.  When we look at the "health" of NiFi quickly, a large number in this summary area can be misleading when it is files that are properly queued waiting to be delivered.

Second, is the ability in putEmail processor to add headers.  For instance if we could specify that it is an HTML email we could then format the message using standard markup language.

Third, and this may be a limitation of the log back class but if it were possible to create a logger for a particular processor and not the entire class that could be beneficial as well.  For instance I can use the LogAttribute processor with an appender/logger to capture to a log all files that have been seen in a particular flow.  However, this limits me to a single LogAttribute processor for each level (INFO, DEBUG, etc).  If this isn't possible, perhaps a processor that appends attributes/data to a configurable file would help.

If I can clarify anything please let me know.  I appreciate any consideration on my requests!

Thank you,
Tim
---------------------------------
Leidos (formerly SAIC) Contractor
7120 Columbia Gateway Drive
Work: 443.654.3604 or 410.872.1627
Cell: 410.627.5217
Red Hat Certified Engineer<https://www.redhat.com/wapps/training/certification/verify.html?certNumber=111-208-377>
Reply | Threaded
Open this post in threaded view
|

Re: NIFI Requests

Andrew Purtell-2
Hello,

Have you considered filing these recommendations as separate issues in the
project issue tracking system? Please see
https://issues.apache.org/jira/browse/NIFI . Click the "Create" button to
get started. You'll probably want to use "Improvement" as the issue type.
That would make it easier for the project to consider and prioritize your
valuable suggestions.

Best regards,

     - Andy


On Mon, Feb 9, 2015 at 11:12 AM, Adams, Timothy M. <
[hidden email]> wrote:

> Hello All,
>
> I had a couple of requests that I believe could be valuable functions in
> NiFi.
>
> One is the ability to "hide" a queue from the count displayed in the
> summary near the top of the GUI.  The reason for this is sometimes in a
> flow we have a processor that we have configured to create a backlog, or to
> slowly work off/deliver data.  When we look at the "health" of NiFi
> quickly, a large number in this summary area can be misleading when it is
> files that are properly queued waiting to be delivered.
>
> Second, is the ability in putEmail processor to add headers.  For instance
> if we could specify that it is an HTML email we could then format the
> message using standard markup language.
>
> Third, and this may be a limitation of the log back class but if it were
> possible to create a logger for a particular processor and not the entire
> class that could be beneficial as well.  For instance I can use the
> LogAttribute processor with an appender/logger to capture to a log all
> files that have been seen in a particular flow.  However, this limits me to
> a single LogAttribute processor for each level (INFO, DEBUG, etc).  If this
> isn't possible, perhaps a processor that appends attributes/data to a
> configurable file would help.
>
> If I can clarify anything please let me know.  I appreciate any
> consideration on my requests!
>
> Thank you,
> Tim
> ---------------------------------
> Leidos (formerly SAIC) Contractor
> 7120 Columbia Gateway Drive
> Work: 443.654.3604 or 410.872.1627
> Cell: 410.627.5217
> Red Hat Certified Engineer<
> https://www.redhat.com/wapps/training/certification/verify.html?certNumber=111-208-377
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: NIFI Requests

Joe Witt
Tim,

Please definitely do create a ticket for each of these.

Quick response for the one about hiding queue-count for a 'known
issue' scenario where a queue is intentionally or known to be large.
So I suppose the case here is there are folks watching the system and
you don't want them to have to care about queues which are full for an
already known reason.  When you tell the system to 'hide' the queue
count for a given item do you expect to have to tell it to unhide or
do you expect it to clear once some time passes and or some floor
threshold is passed ?  I believe i understand why this is helpful I
just want to avoid the other problem which is now folks aren't seeing
important things because people accidentally forgot to 'unhide'

For the second one - totally makes sense.  Basically you're asking
that the PutEmail processor be more robust to support what email is
capable of.  I say to you long-live AsciiArt!  :-)

For the final one about logging...  My personal opinion here is that
the logs are going the way of the dodo.  I mean they'll still be
around and those truly brave souls that want to dumpster dive through
them certainly can.  But, for exactly the type of use case you
describe we want to enable reporting on the provenance data.  This
will allow your exact use case to be turned into an automated report
run on some scheduled interval.

If you do create tickets for these can you please add the respective
comments for them with it.

Thanks
Joe

On Wed, Feb 11, 2015 at 6:38 PM, Andrew Purtell <[hidden email]> wrote:

> Hello,
>
> Have you considered filing these recommendations as separate issues in the
> project issue tracking system? Please see
> https://issues.apache.org/jira/browse/NIFI . Click the "Create" button to
> get started. You'll probably want to use "Improvement" as the issue type.
> That would make it easier for the project to consider and prioritize your
> valuable suggestions.
>
> Best regards,
>
>      - Andy
>
>
> On Mon, Feb 9, 2015 at 11:12 AM, Adams, Timothy M. <
> [hidden email]> wrote:
>
>> Hello All,
>>
>> I had a couple of requests that I believe could be valuable functions in
>> NiFi.
>>
>> One is the ability to "hide" a queue from the count displayed in the
>> summary near the top of the GUI.  The reason for this is sometimes in a
>> flow we have a processor that we have configured to create a backlog, or to
>> slowly work off/deliver data.  When we look at the "health" of NiFi
>> quickly, a large number in this summary area can be misleading when it is
>> files that are properly queued waiting to be delivered.
>>
>> Second, is the ability in putEmail processor to add headers.  For instance
>> if we could specify that it is an HTML email we could then format the
>> message using standard markup language.
>>
>> Third, and this may be a limitation of the log back class but if it were
>> possible to create a logger for a particular processor and not the entire
>> class that could be beneficial as well.  For instance I can use the
>> LogAttribute processor with an appender/logger to capture to a log all
>> files that have been seen in a particular flow.  However, this limits me to
>> a single LogAttribute processor for each level (INFO, DEBUG, etc).  If this
>> isn't possible, perhaps a processor that appends attributes/data to a
>> configurable file would help.
>>
>> If I can clarify anything please let me know.  I appreciate any
>> consideration on my requests!
>>
>> Thank you,
>> Tim
>> ---------------------------------
>> Leidos (formerly SAIC) Contractor
>> 7120 Columbia Gateway Drive
>> Work: 443.654.3604 or 410.872.1627
>> Cell: 410.627.5217
>> Red Hat Certified Engineer<
>> https://www.redhat.com/wapps/training/certification/verify.html?certNumber=111-208-377
>> >
>>