Contrib Check Build - RAT and Unit Test Failure

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Contrib Check Build - RAT and Unit Test Failure

Peter Wicks (pwicks)
I'm trying to successfully run Contrib Check on my dev box.

Windows 10 x64
jdk1.8.0_91
MVN 3.3.9

I'm using IntelliJ, so my Run looks like:

Command line: -T2 -Drat.skip=true clean install
Profile: contrib-check

If I don't disable RAT (-Drat.skip=true) then I get too many RAT failures for the build to run, and the files involved aren't the ones that have changed and that I'm testing.
Then, with RAT disabled I'm getting at least one Unit Test failure, with a test that has been in place since April: https://github.com/apache/nifi/commit/0f2ac39f69c1a744f151f0d924c9978f6790b7f7.

Tests run: 12, Failures: 0, Errors: 1, Skipped: 2, Time elapsed: 11.109 sec <<< FAILURE! - in org.wali.TestMinimalLockingWriteAheadLog
testRecoverFileThatHasTrailingNULBytesAndTruncation(org.wali.TestMinimalLockingWriteAheadLog)  Time elapsed: 0.033 sec  <<< ERROR!
java.nio.channels.OverlappingFileLockException: null
                at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
                at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
                at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063)
                at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
                at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:187)
                at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:108)
                at org.wali.TestMinimalLockingWriteAheadLog.testRecoverFileThatHasTrailingNULBytesAndTruncation(TestMinimalLockingWriteAheadLog.java:472)

Are there settings I need to change on my box that are causing these failures?

Thanks,
  Peter
Reply | Threaded
Open this post in threaded view
|

Re: Contrib Check Build - RAT and Unit Test Failure

Joe Witt
Peter

I dont' believe RAT works in parallel builds (-T2 for example).

If I use RAT it is during a full clean build and activated via the
'contrib-check' profile.  'mvn clean install -Pcontrib-check'

Thanks

On Wed, Nov 22, 2017 at 9:19 PM, Peter Wicks (pwicks) <[hidden email]> wrote:

> I'm trying to successfully run Contrib Check on my dev box.
>
> Windows 10 x64
> jdk1.8.0_91
> MVN 3.3.9
>
> I'm using IntelliJ, so my Run looks like:
>
> Command line: -T2 -Drat.skip=true clean install
> Profile: contrib-check
>
> If I don't disable RAT (-Drat.skip=true) then I get too many RAT failures for the build to run, and the files involved aren't the ones that have changed and that I'm testing.
> Then, with RAT disabled I'm getting at least one Unit Test failure, with a test that has been in place since April: https://github.com/apache/nifi/commit/0f2ac39f69c1a744f151f0d924c9978f6790b7f7.
>
> Tests run: 12, Failures: 0, Errors: 1, Skipped: 2, Time elapsed: 11.109 sec <<< FAILURE! - in org.wali.TestMinimalLockingWriteAheadLog
> testRecoverFileThatHasTrailingNULBytesAndTruncation(org.wali.TestMinimalLockingWriteAheadLog)  Time elapsed: 0.033 sec  <<< ERROR!
> java.nio.channels.OverlappingFileLockException: null
>                 at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
>                 at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
>                 at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063)
>                 at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
>                 at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:187)
>                 at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:108)
>                 at org.wali.TestMinimalLockingWriteAheadLog.testRecoverFileThatHasTrailingNULBytesAndTruncation(TestMinimalLockingWriteAheadLog.java:472)
>
> Are there settings I need to change on my box that are causing these failures?
>
> Thanks,
>   Peter
Reply | Threaded
Open this post in threaded view
|

RE: [EXT] Re: Contrib Check Build - RAT and Unit Test Failure

Peter Wicks (pwicks)
Joe,

This did resolve the RAT issue, but the unit test I mentioned still fails with the same error.

--Peter

-----Original Message-----
From: Joe Witt [mailto:[hidden email]]
Sent: Thursday, November 23, 2017 10:22
To: [hidden email]
Subject: [EXT] Re: Contrib Check Build - RAT and Unit Test Failure

Peter

I dont' believe RAT works in parallel builds (-T2 for example).

If I use RAT it is during a full clean build and activated via the 'contrib-check' profile.  'mvn clean install -Pcontrib-check'

Thanks

On Wed, Nov 22, 2017 at 9:19 PM, Peter Wicks (pwicks) <[hidden email]> wrote:

> I'm trying to successfully run Contrib Check on my dev box.
>
> Windows 10 x64
> jdk1.8.0_91
> MVN 3.3.9
>
> I'm using IntelliJ, so my Run looks like:
>
> Command line: -T2 -Drat.skip=true clean install
> Profile: contrib-check
>
> If I don't disable RAT (-Drat.skip=true) then I get too many RAT failures for the build to run, and the files involved aren't the ones that have changed and that I'm testing.
> Then, with RAT disabled I'm getting at least one Unit Test failure, with a test that has been in place since April: https://github.com/apache/nifi/commit/0f2ac39f69c1a744f151f0d924c9978f6790b7f7.
>
> Tests run: 12, Failures: 0, Errors: 1, Skipped: 2, Time elapsed:
> 11.109 sec <<< FAILURE! - in org.wali.TestMinimalLockingWriteAheadLog
> testRecoverFileThatHasTrailingNULBytesAndTruncation(org.wali.TestMinimalLockingWriteAheadLog)  Time elapsed: 0.033 sec  <<< ERROR!
> java.nio.channels.OverlappingFileLockException: null
>                 at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
>                 at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
>                 at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063)
>                 at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
>                 at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:187)
>                 at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:108)
>                 at
> org.wali.TestMinimalLockingWriteAheadLog.testRecoverFileThatHasTrailin
> gNULBytesAndTruncation(TestMinimalLockingWriteAheadLog.java:472)
>
> Are there settings I need to change on my box that are causing these failures?
>
> Thanks,
>   Peter
Reply | Threaded
Open this post in threaded view
|

Re: [EXT] Re: Contrib Check Build - RAT and Unit Test Failure

Joe Witt
Ok.  You might want to check if there is already a JIRA for that.  The
tests on Windows are notoriously bad.  Many conditional ignores out
there for Windows runs of tests.  That said, this may well be an
actual windows problem and the test is possibly flagging it.

Thanks

On Wed, Nov 22, 2017 at 9:34 PM, Peter Wicks (pwicks) <[hidden email]> wrote:

> Joe,
>
> This did resolve the RAT issue, but the unit test I mentioned still fails with the same error.
>
> --Peter
>
> -----Original Message-----
> From: Joe Witt [mailto:[hidden email]]
> Sent: Thursday, November 23, 2017 10:22
> To: [hidden email]
> Subject: [EXT] Re: Contrib Check Build - RAT and Unit Test Failure
>
> Peter
>
> I dont' believe RAT works in parallel builds (-T2 for example).
>
> If I use RAT it is during a full clean build and activated via the 'contrib-check' profile.  'mvn clean install -Pcontrib-check'
>
> Thanks
>
> On Wed, Nov 22, 2017 at 9:19 PM, Peter Wicks (pwicks) <[hidden email]> wrote:
>> I'm trying to successfully run Contrib Check on my dev box.
>>
>> Windows 10 x64
>> jdk1.8.0_91
>> MVN 3.3.9
>>
>> I'm using IntelliJ, so my Run looks like:
>>
>> Command line: -T2 -Drat.skip=true clean install
>> Profile: contrib-check
>>
>> If I don't disable RAT (-Drat.skip=true) then I get too many RAT failures for the build to run, and the files involved aren't the ones that have changed and that I'm testing.
>> Then, with RAT disabled I'm getting at least one Unit Test failure, with a test that has been in place since April: https://github.com/apache/nifi/commit/0f2ac39f69c1a744f151f0d924c9978f6790b7f7.
>>
>> Tests run: 12, Failures: 0, Errors: 1, Skipped: 2, Time elapsed:
>> 11.109 sec <<< FAILURE! - in org.wali.TestMinimalLockingWriteAheadLog
>> testRecoverFileThatHasTrailingNULBytesAndTruncation(org.wali.TestMinimalLockingWriteAheadLog)  Time elapsed: 0.033 sec  <<< ERROR!
>> java.nio.channels.OverlappingFileLockException: null
>>                 at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
>>                 at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
>>                 at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063)
>>                 at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
>>                 at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:187)
>>                 at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:108)
>>                 at
>> org.wali.TestMinimalLockingWriteAheadLog.testRecoverFileThatHasTrailin
>> gNULBytesAndTruncation(TestMinimalLockingWriteAheadLog.java:472)
>>
>> Are there settings I need to change on my box that are causing these failures?
>>
>> Thanks,
>>   Peter
Reply | Threaded
Open this post in threaded view
|

RE: [EXT] Re: Contrib Check Build - RAT and Unit Test Failure

Peter Wicks (pwicks)
Joe,

I figured it out. At least on my box, there appears to be contention between two unit tests that are using the same folder to test the MinimalLockingWriteAheadLog. Folder name: target/testRecoverFileThatHasTrailingNULBytes.

Unit tests:
testRecoverFileThatHasTrailingNULBytesNoTruncation
testRecoverFileThatHasTrailingNULBytes

If I switch one of them to use a different folder name then the file is not locked when the second test runs and it's fine. I'll submit a PR.

--Peter

-----Original Message-----
From: Joe Witt [mailto:[hidden email]]
Sent: Thursday, November 23, 2017 10:48
To: [hidden email]
Subject: Re: [EXT] Re: Contrib Check Build - RAT and Unit Test Failure

Ok.  You might want to check if there is already a JIRA for that.  The tests on Windows are notoriously bad.  Many conditional ignores out there for Windows runs of tests.  That said, this may well be an actual windows problem and the test is possibly flagging it.

Thanks

On Wed, Nov 22, 2017 at 9:34 PM, Peter Wicks (pwicks) <[hidden email]> wrote:

> Joe,
>
> This did resolve the RAT issue, but the unit test I mentioned still fails with the same error.
>
> --Peter
>
> -----Original Message-----
> From: Joe Witt [mailto:[hidden email]]
> Sent: Thursday, November 23, 2017 10:22
> To: [hidden email]
> Subject: [EXT] Re: Contrib Check Build - RAT and Unit Test Failure
>
> Peter
>
> I dont' believe RAT works in parallel builds (-T2 for example).
>
> If I use RAT it is during a full clean build and activated via the 'contrib-check' profile.  'mvn clean install -Pcontrib-check'
>
> Thanks
>
> On Wed, Nov 22, 2017 at 9:19 PM, Peter Wicks (pwicks) <[hidden email]> wrote:
>> I'm trying to successfully run Contrib Check on my dev box.
>>
>> Windows 10 x64
>> jdk1.8.0_91
>> MVN 3.3.9
>>
>> I'm using IntelliJ, so my Run looks like:
>>
>> Command line: -T2 -Drat.skip=true clean install
>> Profile: contrib-check
>>
>> If I don't disable RAT (-Drat.skip=true) then I get too many RAT failures for the build to run, and the files involved aren't the ones that have changed and that I'm testing.
>> Then, with RAT disabled I'm getting at least one Unit Test failure, with a test that has been in place since April: https://github.com/apache/nifi/commit/0f2ac39f69c1a744f151f0d924c9978f6790b7f7.
>>
>> Tests run: 12, Failures: 0, Errors: 1, Skipped: 2, Time elapsed:
>> 11.109 sec <<< FAILURE! - in org.wali.TestMinimalLockingWriteAheadLog
>> testRecoverFileThatHasTrailingNULBytesAndTruncation(org.wali.TestMinimalLockingWriteAheadLog)  Time elapsed: 0.033 sec  <<< ERROR!
>> java.nio.channels.OverlappingFileLockException: null
>>                 at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
>>                 at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
>>                 at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063)
>>                 at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
>>                 at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:187)
>>                 at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:108)
>>                 at
>> org.wali.TestMinimalLockingWriteAheadLog.testRecoverFileThatHasTraili
>> n
>> gNULBytesAndTruncation(TestMinimalLockingWriteAheadLog.java:472)
>>
>> Are there settings I need to change on my box that are causing these failures?
>>
>> Thanks,
>>   Peter