Hi
I am trying to use TailFile Processor to aggregate logs. my app produce three log files : 1. app-sumo-audit.log 2. app-sumo-api.log 3. app-sumo-logging.log and corresponding rolling files app-sumo-audit.log.2017-03-11.0 app-sumo-audit.log.2017-03-11.1 app-sumo-api.log.2017-03-11.0 app-sumo-api.log.2017-03-11.1 app-sumo-logging.log.2017-03-11.0 app-sumo-logging.log.2017-03-11.1 etc. My TailFile Processor configuration as snow in the attached picture not working for rolled files. please guide me if I am doing wrong Property Value Tailing mode Multiple files File(s) to Tail app-sumo-(audit|api|logging).log Rolling Filename Pattern ${filename}.log.* Base directory /Developer/Applications/hNiFi/app/logs Initial Start Position Beginning of File State Location Local Recursive lookup false Rolling Strategy Changing name Lookup frequency 10 minutes Maximum age 24 hours logback.xml <?xml version="1.0" encoding="UTF-8"?> <configuration> <include resource="org/springframework/boot/logging/logback/base.xml" /> <statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" /> <!-- Common default settings --> <!-- You can overwrite any defined above property inside logback-<ENV_NAME>.xml --> <property name="LOGS_HOME" value="${LOGS_HOME:-./logs}" /> <property name="MAX_FILE_SIZE" value="${MAX_FILE_SIZE:-200KB}" /> <property name="MAX_HISTORY" value="${MAX_HISTORY:-5}" /> <appender name="LOCAL_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${LOGS_HOME}/app-sumo-logging.log</file> <encoder class="net.logstash.logback.encoder.LogstashEncoder"> <fieldNames> <version>[ignore]</version> </fieldNames> </encoder> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <fileNamePattern>${LOGS_HOME}/app-sumo-logging.log.%d{yyyy-MM-dd}.%i</fileNamePattern> <maxHistory>${MAX_HISTORY}</maxHistory> <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> <maxFileSize>${MAX_FILE_SIZE}</maxFileSize> </timeBasedFileNamingAndTriggeringPolicy> </rollingPolicy> </appender> <appender name="AUDIT_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${LOGS_HOME}/app-sumo-audit.log</file> <encoder class="net.logstash.logback.encoder.LogstashEncoder"> <fieldNames> <version>[ignore]</version> </fieldNames> </encoder> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <fileNamePattern>${LOGS_HOME}/app-sumo-audit.log.%d{yyyy-MM-dd}.%i</fileNamePattern> <maxHistory>${MAX_HISTORY}</maxHistory> <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> <maxFileSize>${MAX_FILE_SIZE}</maxFileSize> </timeBasedFileNamingAndTriggeringPolicy> </rollingPolicy> </appender> <appender name="API_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${LOGS_HOME}/app-sumo-api.log</file> <encoder class="net.logstash.logback.encoder.LogstashEncoder"> <fieldNames> <version>[ignore]</version> <levelValue>[ignore]</levelValue> <thread>[ignore]</thread> </fieldNames> </encoder> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <fileNamePattern>${LOGS_HOME}/app-sumo-api.log.%d{yyyy-MM-dd}.%i</fileNamePattern> <maxHistory>${MAX_HISTORY}</maxHistory> <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> <maxFileSize>${MAX_FILE_SIZE}</maxFileSize> </timeBasedFileNamingAndTriggeringPolicy> </rollingPolicy> </appender> <logger name="org.test.api" level="INFO" additivity="false"> <appender-ref ref="CONSOLE" /> <appender-ref ref="API_FILE" /> </logger> <logger name="org.test.Runner" level="INFO" additivity="false"> <appender-ref ref="CONSOLE" /> <appender-ref ref="LOCAL_FILE" /> </logger> <logger name="org.test.audit" level="INFO" additivity="false"> <appender-ref ref="CONSOLE" /> <appender-ref ref="AUDIT_FILE" /> </logger> </configuration> |
Hi Sumanth,
Your table text copy/paste did not come through very well, so could you please take a screenshot of the processor properties dialog as it is configured on your system and re-send? My initial guess is that the issue stems from having multiple dot characters in the filename (I did a brief glance at the code and the regex splits on “\\.”). Can you try a rolling log scheme like app-sumo-audit_2017-03-11-0.log just to see if the logs are now picked up? If so, please confirm and feel free to file a Jira noting the issue and requesting more lenient log capture. I think it’s technically feasible, but it may not have been caught during testing. If someone has more context on this or a historical reason this isn’t supported, please weigh in. Andy LoPresto PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
|
Free forum by Nabble | Edit this page |