[GitHub] incubator-nifi pull request: Processor Logger using same level as ...

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: Processor Logger using same level as ...

JPercivall
GitHub user brianghig opened a pull request:

    https://github.com/apache/incubator-nifi/pull/59

    Processor Logger using same level as message for Throwables

    Processor Loggers will now use the same Logger level for Throwables as is used for messages. Previously, enabling DEBUG was the only way to see stacktraces from logged throwables when running in INFO, WARN, or ERROR levels.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/brianghig/incubator-nifi ProcessorLog-ExceptionsUseSameLevels

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-nifi/pull/59.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #59
   
----
commit 29b4e56a61dc7125e10d7e5d70a0123485717b4d
Author: Brian Ghigiarelli <[hidden email]>
Date:   2015-04-22T15:32:36Z

    Merge pull request #1 from apache/develop
   
    Merging latest NiFi to my fork

commit 421ad8fb133d3bab32bc20d98011e9dfa0caff99
Author: Brian Ghigiarelli <[hidden email]>
Date:   2015-05-14T00:32:23Z

    Merge pull request #2 from apache/develop
   
    Merging latest Apache NiFi to fork

commit ce16aab6c57a5b3d771149a1720471b980cf5a1c
Author: Brian Ghigiarelli <[hidden email]>
Date:   2015-05-21T12:42:32Z

    Merge pull request #4 from apache/develop
   
    Merging latest Apache NiFi develop branch

commit 710914675f4b5cae727092c4779e82b17dd2b145
Author: Brian Ghigiarelli <[hidden email]>
Date:   2015-05-26T15:03:40Z

    Processor Loggers will now use the same Logger Level for throwables as they do for the log messages. Previously, enabling DEBUG was the only way to see stacktraces from throwables in INFO, WARN, or ERROR levels

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: Processor Logger using same level as ...

JPercivall
Github user joewitt commented on the pull request:

    https://github.com/apache/incubator-nifi/pull/59#issuecomment-105560569
 
    Brian,
   
    The issue with this proposal is that it takes away the administrators ability to control whether stack traces are provided or not.  The reason we used 'isDebugEnabled' was because that was the use case in which the stack trace was necessary (because they were debugging).   Given that you can edit logback config on the fly this seems like a reasonable approach.  But taking away their ability to control this as the effect of this proposal means they're getting the stack traces whether they want them or not.  This can cause very excessive and noisy output in the logs.
   
    Do you have an alternative proposal for how we can retain the flexibility of letting the administrator toggle the stack traces?
   
    Thanks
    Joe


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: Processor Logger using same level as ...

JPercivall
In reply to this post by JPercivall
Github user brianghig commented on the pull request:

    https://github.com/apache/incubator-nifi/pull/59#issuecomment-105625875
 
    Hi Joe,
   
    We could use filters within the Logback configuration to turn on/off Throwables or any granularity of Exceptions.  Since exceptions can be used for more than debugging, doing so would honor the caller's logging level of the throwable and allow admins to maintain full control over the logs.
   
    For example, using some built-in mechanisms from http://logback.qos.ch/manual/filters.html#evalutatorFilter, we could add something like this to the appender(s):
    ```
    <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
        <evaluator>
            <expression>java.lang.Exception.class.isInstance(throwable)</expression>
        </evaluator>
        <onMatch>DENY</onMatch>
    </filter>
    ```
   
    Thanks,
    Brian


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: Processor Logger using same level as ...

JPercivall
In reply to this post by JPercivall
Github user joewitt commented on the pull request:

    https://github.com/apache/incubator-nifi/pull/59#issuecomment-105628438
 
    ...slick.  Will need to read more on that but this could be an option.  Curious what others think?  Can you please make a JIRA for it and like that to this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: Processor Logger using same level as ...

JPercivall
In reply to this post by JPercivall
Github user joewitt commented on the pull request:

    https://github.com/apache/incubator-nifi/pull/59#issuecomment-105630245
 
    Brian - can you mention what you think are the other use cases for exception stack traces beyond debugging?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

Re: [GitHub] incubator-nifi pull request: Processor Logger using same level as ...

Adam Taft
In reply to this post by JPercivall
Sorry in advance for not cross posting this on github, but mine is more a
discussion oriented comment, not feedback on the pull request.

Just to play devil's advocate a little...  I have found the default NiFi
logging configuration to be almost always logging the wrong things.  I most
definitely always want stack traces to be logged, because it's sometimes
very hard to recreate the state by which the stacktrace was thrown.  And
it's a pain to have to change the logging level (even at runtime) just to
hope that the stacktrace happens again.

Whereas I almost never want all the existing framework INFO messages
logged.  I don't care anything about how many wali objects were garbage
collected, what the current state of each processor is, or if a heartbeat
was sent or received from a cluster configuration.  The things that one
must ignore when grepping a log often hinders the ability of finding the
thing you are looking for.

A stacktrace, by definition, is more than a DEBUG level event.  It is at
minimum a WARN if not ERROR condition in almost all standard cases.  And
you definitely can't argue that infrequent stack traces are going to bloat
the log files anymore than the exist default logging configuration, which
is already very verbose.

In short:  I want to know when things go wrong, not when they go right.

‚ÄčIf there's anything that might be done to help with stracktrace verbosity,
it would be suppress the stacktrace if it was seen x number of times in the
same period.  Thus, the first stacktrace would be logged, but all similar
stacks would be suppressed/minimized.  This strategy is used in a few
logging implementations (unsure if logback has direct support for this or
not).

Hopefully, this discussion could lead to a more balanced default logback
configuration, one with more signal and less noise.

Recommend:
  -  always log stacktraces
  -  change default org.apache.nifi.* to WARN level


Adam





On Tue, May 26, 2015 at 11:14 AM, joewitt <[hidden email]> wrote:

> Github user joewitt commented on the pull request:
>
>
> https://github.com/apache/incubator-nifi/pull/59#issuecomment-105560569
>
>     Brian,
>
>     The issue with this proposal is that it takes away the administrators
> ability to control whether stack traces are provided or not.  The reason we
> used 'isDebugEnabled' was because that was the use case in which the stack
> trace was necessary (because they were debugging).   Given that you can
> edit logback config on the fly this seems like a reasonable approach.  But
> taking away their ability to control this as the effect of this proposal
> means they're getting the stack traces whether they want them or not.  This
> can cause very excessive and noisy output in the logs.
>
>     Do you have an alternative proposal for how we can retain the
> flexibility of letting the administrator toggle the stack traces?
>
>     Thanks
>     Joe
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at [hidden email] or file a JIRA ticket
> with INFRA.
> ---
>
Reply | Threaded
Open this post in threaded view
|

Re: [GitHub] incubator-nifi pull request: Processor Logger using same level as ...

Joe Witt
It is important to keep in mind the logs have served two worlds:
1) The needs of a developer who wants to know when their stuff broke
and needs context to figure out why
2) The needs of an operations person who does at times need the
minutiae of starts/stops/statuses and how objects flowed through the
system or what given processors did even when it isn't error or warn
worthy.

I believe the comments reflect a developer centric view at the expense
of an appreciation for what operations folks have expressed needs for.
Just as the developer doesn't care about all the items mentioned the
operations person doesn't want those details drowned out by endless
streams of stack traces.  Having said this...  Our usage of the logs
and their value have diminished extensively thanks to the rise in
capability and value of the provenance functions.  The entire notion
of logging at this point could be reviewed.

On Tue, May 26, 2015 at 2:58 PM, Adam Taft <[hidden email]> wrote:

> Sorry in advance for not cross posting this on github, but mine is more a
> discussion oriented comment, not feedback on the pull request.
>
> Just to play devil's advocate a little...  I have found the default NiFi
> logging configuration to be almost always logging the wrong things.  I most
> definitely always want stack traces to be logged, because it's sometimes
> very hard to recreate the state by which the stacktrace was thrown.  And
> it's a pain to have to change the logging level (even at runtime) just to
> hope that the stacktrace happens again.
>
> Whereas I almost never want all the existing framework INFO messages
> logged.  I don't care anything about how many wali objects were garbage
> collected, what the current state of each processor is, or if a heartbeat
> was sent or received from a cluster configuration.  The things that one
> must ignore when grepping a log often hinders the ability of finding the
> thing you are looking for.
>
> A stacktrace, by definition, is more than a DEBUG level event.  It is at
> minimum a WARN if not ERROR condition in almost all standard cases.  And
> you definitely can't argue that infrequent stack traces are going to bloat
> the log files anymore than the exist default logging configuration, which
> is already very verbose.
>
> In short:  I want to know when things go wrong, not when they go right.
>
> If there's anything that might be done to help with stracktrace verbosity,
> it would be suppress the stacktrace if it was seen x number of times in the
> same period.  Thus, the first stacktrace would be logged, but all similar
> stacks would be suppressed/minimized.  This strategy is used in a few
> logging implementations (unsure if logback has direct support for this or
> not).
>
> Hopefully, this discussion could lead to a more balanced default logback
> configuration, one with more signal and less noise.
>
> Recommend:
>   -  always log stacktraces
>   -  change default org.apache.nifi.* to WARN level
>
>
> Adam
>
>
>
>
>
> On Tue, May 26, 2015 at 11:14 AM, joewitt <[hidden email]> wrote:
>
>> Github user joewitt commented on the pull request:
>>
>>
>> https://github.com/apache/incubator-nifi/pull/59#issuecomment-105560569
>>
>>     Brian,
>>
>>     The issue with this proposal is that it takes away the administrators
>> ability to control whether stack traces are provided or not.  The reason we
>> used 'isDebugEnabled' was because that was the use case in which the stack
>> trace was necessary (because they were debugging).   Given that you can
>> edit logback config on the fly this seems like a reasonable approach.  But
>> taking away their ability to control this as the effect of this proposal
>> means they're getting the stack traces whether they want them or not.  This
>> can cause very excessive and noisy output in the logs.
>>
>>     Do you have an alternative proposal for how we can retain the
>> flexibility of letting the administrator toggle the stack traces?
>>
>>     Thanks
>>     Joe
>>
>>
>> ---
>> If your project is set up for it, you can reply to this email and have your
>> reply appear on GitHub as well. If your project does not have this feature
>> enabled and wishes so, or if the feature is enabled but not working, please
>> contact infrastructure at [hidden email] or file a JIRA ticket
>> with INFRA.
>> ---
>>
Reply | Threaded
Open this post in threaded view
|

Re: [GitHub] incubator-nifi pull request: Processor Logger using same level as ...

Adam Taft
Yes, agreed.  Should have included the caveat, "this is a developer's point
of view."  But my point of view is definitely not at the expense of
"appreciating" the operations side as well.  Your point that the provenance
tooling mitigates the need for verbose logging of individual flowfiles is
spot on.  Before the provenance tracking, logging the history a flowfile
(via its uuid) across the entire flow was critical for both developers and
especially operations.  Now, not so much.

My comments were definitely made in the context of having the provenance
repository to track the life of a flowfile event.  Having the provenance
event database clearly shifts the utility of the log files way over to the
developer side and less the operations side.  Perhaps identifying what
useful operations-based logging still remains would be a good thing.

And since we're talking about provenance events, it would likely make sense
as well to provide the tooling to enable "grepping" of the provenance event
logs, so that unix style commands can be piped together in the same way
you'd do with plain text logs.  Perhaps a command line provenance client
would be useful to the operations toolkit and would further reduce the
(ab)use of the log files?

Adam



On Tue, May 26, 2015 at 3:12 PM, Joe Witt <[hidden email]> wrote:

> It is important to keep in mind the logs have served two worlds:
> 1) The needs of a developer who wants to know when their stuff broke
> and needs context to figure out why
> 2) The needs of an operations person who does at times need the
> minutiae of starts/stops/statuses and how objects flowed through the
> system or what given processors did even when it isn't error or warn
> worthy.
>
> I believe the comments reflect a developer centric view at the expense
> of an appreciation for what operations folks have expressed needs for.
> Just as the developer doesn't care about all the items mentioned the
> operations person doesn't want those details drowned out by endless
> streams of stack traces.  Having said this...  Our usage of the logs
> and their value have diminished extensively thanks to the rise in
> capability and value of the provenance functions.  The entire notion
> of logging at this point could be reviewed.
>
> On Tue, May 26, 2015 at 2:58 PM, Adam Taft <[hidden email]> wrote:
> > Sorry in advance for not cross posting this on github, but mine is more a
> > discussion oriented comment, not feedback on the pull request.
> >
> > Just to play devil's advocate a little...  I have found the default NiFi
> > logging configuration to be almost always logging the wrong things.  I
> most
> > definitely always want stack traces to be logged, because it's sometimes
> > very hard to recreate the state by which the stacktrace was thrown.  And
> > it's a pain to have to change the logging level (even at runtime) just to
> > hope that the stacktrace happens again.
> >
> > Whereas I almost never want all the existing framework INFO messages
> > logged.  I don't care anything about how many wali objects were garbage
> > collected, what the current state of each processor is, or if a heartbeat
> > was sent or received from a cluster configuration.  The things that one
> > must ignore when grepping a log often hinders the ability of finding the
> > thing you are looking for.
> >
> > A stacktrace, by definition, is more than a DEBUG level event.  It is at
> > minimum a WARN if not ERROR condition in almost all standard cases.  And
> > you definitely can't argue that infrequent stack traces are going to
> bloat
> > the log files anymore than the exist default logging configuration, which
> > is already very verbose.
> >
> > In short:  I want to know when things go wrong, not when they go right.
> >
> > If there's anything that might be done to help with stracktrace
> verbosity,
> > it would be suppress the stacktrace if it was seen x number of times in
> the
> > same period.  Thus, the first stacktrace would be logged, but all similar
> > stacks would be suppressed/minimized.  This strategy is used in a few
> > logging implementations (unsure if logback has direct support for this or
> > not).
> >
> > Hopefully, this discussion could lead to a more balanced default logback
> > configuration, one with more signal and less noise.
> >
> > Recommend:
> >   -  always log stacktraces
> >   -  change default org.apache.nifi.* to WARN level
> >
> >
> > Adam
> >
> >
> >
> >
> >
> > On Tue, May 26, 2015 at 11:14 AM, joewitt <[hidden email]> wrote:
> >
> >> Github user joewitt commented on the pull request:
> >>
> >>
> >> https://github.com/apache/incubator-nifi/pull/59#issuecomment-105560569
> >>
> >>     Brian,
> >>
> >>     The issue with this proposal is that it takes away the
> administrators
> >> ability to control whether stack traces are provided or not.  The
> reason we
> >> used 'isDebugEnabled' was because that was the use case in which the
> stack
> >> trace was necessary (because they were debugging).   Given that you can
> >> edit logback config on the fly this seems like a reasonable approach.
> But
> >> taking away their ability to control this as the effect of this proposal
> >> means they're getting the stack traces whether they want them or not.
> This
> >> can cause very excessive and noisy output in the logs.
> >>
> >>     Do you have an alternative proposal for how we can retain the
> >> flexibility of letting the administrator toggle the stack traces?
> >>
> >>     Thanks
> >>     Joe
> >>
> >>
> >> ---
> >> If your project is set up for it, you can reply to this email and have
> your
> >> reply appear on GitHub as well. If your project does not have this
> feature
> >> enabled and wishes so, or if the feature is enabled but not working,
> please
> >> contact infrastructure at [hidden email] or file a JIRA
> ticket
> >> with INFRA.
> >> ---
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: [GitHub] incubator-nifi pull request: Processor Logger using same level as ...

Joe Witt
> it would likely make sense as well to provide the tooling to enable "grepping" of the provenance event logs, so that unix style commands can be piped together in the same way you'd do with plain text logs

-- That would make for a good JIRA.


On Tue, May 26, 2015 at 3:25 PM, Adam Taft <[hidden email]> wrote:

> Yes, agreed.  Should have included the caveat, "this is a developer's point
> of view."  But my point of view is definitely not at the expense of
> "appreciating" the operations side as well.  Your point that the provenance
> tooling mitigates the need for verbose logging of individual flowfiles is
> spot on.  Before the provenance tracking, logging the history a flowfile
> (via its uuid) across the entire flow was critical for both developers and
> especially operations.  Now, not so much.
>
> My comments were definitely made in the context of having the provenance
> repository to track the life of a flowfile event.  Having the provenance
> event database clearly shifts the utility of the log files way over to the
> developer side and less the operations side.  Perhaps identifying what
> useful operations-based logging still remains would be a good thing.
>
> And since we're talking about provenance events, it would likely make sense
> as well to provide the tooling to enable "grepping" of the provenance event
> logs, so that unix style commands can be piped together in the same way
> you'd do with plain text logs.  Perhaps a command line provenance client
> would be useful to the operations toolkit and would further reduce the
> (ab)use of the log files?
>
> Adam
>
>
>
> On Tue, May 26, 2015 at 3:12 PM, Joe Witt <[hidden email]> wrote:
>
>> It is important to keep in mind the logs have served two worlds:
>> 1) The needs of a developer who wants to know when their stuff broke
>> and needs context to figure out why
>> 2) The needs of an operations person who does at times need the
>> minutiae of starts/stops/statuses and how objects flowed through the
>> system or what given processors did even when it isn't error or warn
>> worthy.
>>
>> I believe the comments reflect a developer centric view at the expense
>> of an appreciation for what operations folks have expressed needs for.
>> Just as the developer doesn't care about all the items mentioned the
>> operations person doesn't want those details drowned out by endless
>> streams of stack traces.  Having said this...  Our usage of the logs
>> and their value have diminished extensively thanks to the rise in
>> capability and value of the provenance functions.  The entire notion
>> of logging at this point could be reviewed.
>>
>> On Tue, May 26, 2015 at 2:58 PM, Adam Taft <[hidden email]> wrote:
>> > Sorry in advance for not cross posting this on github, but mine is more a
>> > discussion oriented comment, not feedback on the pull request.
>> >
>> > Just to play devil's advocate a little...  I have found the default NiFi
>> > logging configuration to be almost always logging the wrong things.  I
>> most
>> > definitely always want stack traces to be logged, because it's sometimes
>> > very hard to recreate the state by which the stacktrace was thrown.  And
>> > it's a pain to have to change the logging level (even at runtime) just to
>> > hope that the stacktrace happens again.
>> >
>> > Whereas I almost never want all the existing framework INFO messages
>> > logged.  I don't care anything about how many wali objects were garbage
>> > collected, what the current state of each processor is, or if a heartbeat
>> > was sent or received from a cluster configuration.  The things that one
>> > must ignore when grepping a log often hinders the ability of finding the
>> > thing you are looking for.
>> >
>> > A stacktrace, by definition, is more than a DEBUG level event.  It is at
>> > minimum a WARN if not ERROR condition in almost all standard cases.  And
>> > you definitely can't argue that infrequent stack traces are going to
>> bloat
>> > the log files anymore than the exist default logging configuration, which
>> > is already very verbose.
>> >
>> > In short:  I want to know when things go wrong, not when they go right.
>> >
>> > If there's anything that might be done to help with stracktrace
>> verbosity,
>> > it would be suppress the stacktrace if it was seen x number of times in
>> the
>> > same period.  Thus, the first stacktrace would be logged, but all similar
>> > stacks would be suppressed/minimized.  This strategy is used in a few
>> > logging implementations (unsure if logback has direct support for this or
>> > not).
>> >
>> > Hopefully, this discussion could lead to a more balanced default logback
>> > configuration, one with more signal and less noise.
>> >
>> > Recommend:
>> >   -  always log stacktraces
>> >   -  change default org.apache.nifi.* to WARN level
>> >
>> >
>> > Adam
>> >
>> >
>> >
>> >
>> >
>> > On Tue, May 26, 2015 at 11:14 AM, joewitt <[hidden email]> wrote:
>> >
>> >> Github user joewitt commented on the pull request:
>> >>
>> >>
>> >> https://github.com/apache/incubator-nifi/pull/59#issuecomment-105560569
>> >>
>> >>     Brian,
>> >>
>> >>     The issue with this proposal is that it takes away the
>> administrators
>> >> ability to control whether stack traces are provided or not.  The
>> reason we
>> >> used 'isDebugEnabled' was because that was the use case in which the
>> stack
>> >> trace was necessary (because they were debugging).   Given that you can
>> >> edit logback config on the fly this seems like a reasonable approach.
>> But
>> >> taking away their ability to control this as the effect of this proposal
>> >> means they're getting the stack traces whether they want them or not.
>> This
>> >> can cause very excessive and noisy output in the logs.
>> >>
>> >>     Do you have an alternative proposal for how we can retain the
>> >> flexibility of letting the administrator toggle the stack traces?
>> >>
>> >>     Thanks
>> >>     Joe
>> >>
>> >>
>> >> ---
>> >> If your project is set up for it, you can reply to this email and have
>> your
>> >> reply appear on GitHub as well. If your project does not have this
>> feature
>> >> enabled and wishes so, or if the feature is enabled but not working,
>> please
>> >> contact infrastructure at [hidden email] or file a JIRA
>> ticket
>> >> with INFRA.
>> >> ---
>> >>
>>
Reply | Threaded
Open this post in threaded view
|

Re: [GitHub] incubator-nifi pull request: Processor Logger using same level as ...

Adam Taft
You got it.

NIFI-634 and NIFI-635 created, as per output from this discussion.
Comments welcome.



On Tue, May 26, 2015 at 3:32 PM, Joe Witt <[hidden email]> wrote:

> > it would likely make sense as well to provide the tooling to enable
> "grepping" of the provenance event logs, so that unix style commands can be
> piped together in the same way you'd do with plain text logs
>
> -- That would make for a good JIRA.
>
>
> On Tue, May 26, 2015 at 3:25 PM, Adam Taft <[hidden email]> wrote:
> > Yes, agreed.  Should have included the caveat, "this is a developer's
> point
> > of view."  But my point of view is definitely not at the expense of
> > "appreciating" the operations side as well.  Your point that the
> provenance
> > tooling mitigates the need for verbose logging of individual flowfiles is
> > spot on.  Before the provenance tracking, logging the history a flowfile
> > (via its uuid) across the entire flow was critical for both developers
> and
> > especially operations.  Now, not so much.
> >
> > My comments were definitely made in the context of having the provenance
> > repository to track the life of a flowfile event.  Having the provenance
> > event database clearly shifts the utility of the log files way over to
> the
> > developer side and less the operations side.  Perhaps identifying what
> > useful operations-based logging still remains would be a good thing.
> >
> > And since we're talking about provenance events, it would likely make
> sense
> > as well to provide the tooling to enable "grepping" of the provenance
> event
> > logs, so that unix style commands can be piped together in the same way
> > you'd do with plain text logs.  Perhaps a command line provenance client
> > would be useful to the operations toolkit and would further reduce the
> > (ab)use of the log files?
> >
> > Adam
> >
> >
> >
> > On Tue, May 26, 2015 at 3:12 PM, Joe Witt <[hidden email]> wrote:
> >
> >> It is important to keep in mind the logs have served two worlds:
> >> 1) The needs of a developer who wants to know when their stuff broke
> >> and needs context to figure out why
> >> 2) The needs of an operations person who does at times need the
> >> minutiae of starts/stops/statuses and how objects flowed through the
> >> system or what given processors did even when it isn't error or warn
> >> worthy.
> >>
> >> I believe the comments reflect a developer centric view at the expense
> >> of an appreciation for what operations folks have expressed needs for.
> >> Just as the developer doesn't care about all the items mentioned the
> >> operations person doesn't want those details drowned out by endless
> >> streams of stack traces.  Having said this...  Our usage of the logs
> >> and their value have diminished extensively thanks to the rise in
> >> capability and value of the provenance functions.  The entire notion
> >> of logging at this point could be reviewed.
> >>
> >> On Tue, May 26, 2015 at 2:58 PM, Adam Taft <[hidden email]> wrote:
> >> > Sorry in advance for not cross posting this on github, but mine is
> more a
> >> > discussion oriented comment, not feedback on the pull request.
> >> >
> >> > Just to play devil's advocate a little...  I have found the default
> NiFi
> >> > logging configuration to be almost always logging the wrong things.  I
> >> most
> >> > definitely always want stack traces to be logged, because it's
> sometimes
> >> > very hard to recreate the state by which the stacktrace was thrown.
> And
> >> > it's a pain to have to change the logging level (even at runtime)
> just to
> >> > hope that the stacktrace happens again.
> >> >
> >> > Whereas I almost never want all the existing framework INFO messages
> >> > logged.  I don't care anything about how many wali objects were
> garbage
> >> > collected, what the current state of each processor is, or if a
> heartbeat
> >> > was sent or received from a cluster configuration.  The things that
> one
> >> > must ignore when grepping a log often hinders the ability of finding
> the
> >> > thing you are looking for.
> >> >
> >> > A stacktrace, by definition, is more than a DEBUG level event.  It is
> at
> >> > minimum a WARN if not ERROR condition in almost all standard cases.
> And
> >> > you definitely can't argue that infrequent stack traces are going to
> >> bloat
> >> > the log files anymore than the exist default logging configuration,
> which
> >> > is already very verbose.
> >> >
> >> > In short:  I want to know when things go wrong, not when they go
> right.
> >> >
> >> > If there's anything that might be done to help with stracktrace
> >> verbosity,
> >> > it would be suppress the stacktrace if it was seen x number of times
> in
> >> the
> >> > same period.  Thus, the first stacktrace would be logged, but all
> similar
> >> > stacks would be suppressed/minimized.  This strategy is used in a few
> >> > logging implementations (unsure if logback has direct support for
> this or
> >> > not).
> >> >
> >> > Hopefully, this discussion could lead to a more balanced default
> logback
> >> > configuration, one with more signal and less noise.
> >> >
> >> > Recommend:
> >> >   -  always log stacktraces
> >> >   -  change default org.apache.nifi.* to WARN level
> >> >
> >> >
> >> > Adam
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Tue, May 26, 2015 at 11:14 AM, joewitt <[hidden email]> wrote:
> >> >
> >> >> Github user joewitt commented on the pull request:
> >> >>
> >> >>
> >> >>
> https://github.com/apache/incubator-nifi/pull/59#issuecomment-105560569
> >> >>
> >> >>     Brian,
> >> >>
> >> >>     The issue with this proposal is that it takes away the
> >> administrators
> >> >> ability to control whether stack traces are provided or not.  The
> >> reason we
> >> >> used 'isDebugEnabled' was because that was the use case in which the
> >> stack
> >> >> trace was necessary (because they were debugging).   Given that you
> can
> >> >> edit logback config on the fly this seems like a reasonable approach.
> >> But
> >> >> taking away their ability to control this as the effect of this
> proposal
> >> >> means they're getting the stack traces whether they want them or not.
> >> This
> >> >> can cause very excessive and noisy output in the logs.
> >> >>
> >> >>     Do you have an alternative proposal for how we can retain the
> >> >> flexibility of letting the administrator toggle the stack traces?
> >> >>
> >> >>     Thanks
> >> >>     Joe
> >> >>
> >> >>
> >> >> ---
> >> >> If your project is set up for it, you can reply to this email and
> have
> >> your
> >> >> reply appear on GitHub as well. If your project does not have this
> >> feature
> >> >> enabled and wishes so, or if the feature is enabled but not working,
> >> please
> >> >> contact infrastructure at [hidden email] or file a JIRA
> >> ticket
> >> >> with INFRA.
> >> >> ---
> >> >>
> >>
>
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: Processor Logger using same level as ...

JPercivall
In reply to this post by JPercivall
Github user danbress commented on the pull request:

    https://github.com/apache/incubator-nifi/pull/59#issuecomment-105879635
 
    A couple comments
   
    1) Does this pull request have a JIRA ticket associated with it?
   
    2) As a developer I also share Adam's developer viewpoint.  I would prefer the 'operations' style log messages turned down, and the exceptions turned up.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: [NIFI-636] Processor Logger using sam...

JPercivall
In reply to this post by JPercivall
Github user brianghig commented on the pull request:

    https://github.com/apache/incubator-nifi/pull/59#issuecomment-105926818
 
    @danbress - I created NIFI-636 to cover this PR, and updated the PR's title to link it.
   
    @joewitt - the use case I'm considering for stack traces is operational with feedback to engineering teams. When something is going wrong on some far away network, we've had folks and/or tools tailing logs and reporting details about the errors to dev teams via tickets that include stack traces.  Additionally, stack traces can be useful tools for low-level auditing purposes to follow the flow of a user or piece of data through layers of the system, but as you've said, provenance does a much better job of handling that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---