TestConvertCharacterSet fails

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

TestConvertCharacterSet fails

Toivo Adams
Today I started using Windows notebook.
(Before I used mostly Linux development machine)
And test TestConvertCharacterSet failed.
TestConvertCharacterSet
org.apache.nifi.processors.standard.TestConvertCharacterSet
test(org.apache.nifi.processors.standard.TestConvertCharacterSet)
java.lang.AssertionError: FlowFile content differs from input at byte 287 with input having value 10 and FlowFile having value 13
        at org.junit.Assert.fail(Assert.java:88)
        at org.apache.nifi.util.MockFlowFile.assertContentEquals(MockFlowFile.java:252)
        at org.apache.nifi.util.MockFlowFile.assertContentEquals(MockFlowFile.java:204)
        at org.apache.nifi.util.MockFlowFile.assertContentEquals(MockFlowFile.java:192)
        at org.apache.nifi.processors.standard.TestConvertCharacterSet.test(TestConvertCharacterSet.java:45)

Test compares result using static file
/nifi-standard-processors/src/test/resources/CharacterSetConversionSamples/Converted2.txt
       
But as normal text file this file contains also newline symbols which are different in different systems.
Unix, Linux and a like systems (newer Macs also?) contain only LF as newline symbol.
But Windows use CR+LF symbols.
Converted2.txt is for Unix like systems and contain only LF (code 10)
But when using Windows reading there must be 2 symbols CR+LF (codes 13 and 10)
And test will fail.

Should I create ticket?

One solution is to create 2 Converted2.txt static files, one for Linux and one for Windows.


Just to be sure I scanned also nifi-standard-processors project and found “\n” is used in

ExecuteProcess.java
                                proxyOut.write((line + "\n").getBytes(StandardCharsets.UTF_8));

ExecuteStreamCommand.java
                    strBldr.append(line).append("\n");

InvokeHTTP.java
sb.append("\n");


LogAttribute.java
message.append("\n");
PutEmail.java
message.append("\n");

I don't know every use case, "\n" may be right solution.
But sometimes it's safer to use
http://docs.oracle.com/javase/8/docs/api/java/lang/System.html#lineSeparator--
as line separator instead of “\n”

Thanks
Toivo
Reply | Threaded
Open this post in threaded view
|

Re: TestConvertCharacterSet fails

Mark Payne
Toivo,

I think this is another fun little configuration problem with Git. You
need to be sure to set autocrlf to false:

git config --global core.autocrlf false

Then I would delete the standard-processors/src/test/resources directory
and re-pull it:

git checkout -- standard-processors/src/test/resources


If that doesn't solve the issue, please let us know!

Thanks
-Mark


------ Original Message ------
From: "Toivo Adams" <[hidden email]>
To: [hidden email]
Sent: 4/5/2015 2:26:30 PM
Subject: TestConvertCharacterSet fails

>Today I started using Windows notebook.
>(Before I used mostly Linux development machine)
>And test TestConvertCharacterSet failed.
>TestConvertCharacterSet
>org.apache.nifi.processors.standard.TestConvertCharacterSet
>test(org.apache.nifi.processors.standard.TestConvertCharacterSet)
>java.lang.AssertionError: FlowFile content differs from input at byte
>287
>with input having value 10 and FlowFile having value 13
>  at org.junit.Assert.fail(Assert.java:88)
>  at
>org.apache.nifi.util.MockFlowFile.assertContentEquals(MockFlowFile.java:252)
>  at
>org.apache.nifi.util.MockFlowFile.assertContentEquals(MockFlowFile.java:204)
>  at
>org.apache.nifi.util.MockFlowFile.assertContentEquals(MockFlowFile.java:192)
>  at
>org.apache.nifi.processors.standard.TestConvertCharacterSet.test(TestConvertCharacterSet.java:45)
>
>Test compares result using static file
>/nifi-standard-processors/src/test/resources/CharacterSetConversionSamples/Converted2.txt
>
>But as normal text file this file contains also newline symbols which
>are
>different in different systems.
>Unix, Linux and a like systems (newer Macs also?) contain only LF as
>newline
>symbol.
>But Windows use CR+LF symbols.
>Converted2.txt is for Unix like systems and contain only LF (code 10)
>But when using Windows reading there must be 2 symbols CR+LF (codes 13
>and
>10)
>And test will fail.
>
>Should I create ticket?
>
>One solution is to create 2 Converted2.txt static files, one for Linux
>and
>one for Windows.
>
>
>Just to be sure I scanned also nifi-standard-processors project and
>found
>“\n” is used in
>
>ExecuteProcess.java
>                                 proxyOut.write((line +
>"\n").getBytes(StandardCharsets.UTF_8));
>
>ExecuteStreamCommand.java
>                     strBldr.append(line).append("\n");
>
>InvokeHTTP.java
>sb.append("\n");
>
>
>LogAttribute.java
>message.append("\n");
>PutEmail.java
>message.append("\n");
>
>I don't know every use case, "\n" may be right solution.
>But sometimes it's safer to use
>http://docs.oracle.com/javase/8/docs/api/java/lang/System.html#lineSeparator--
>as line separator instead of “\n”
>
>Thanks
>Toivo
>
>
>
>
>--
>View this message in context:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/TestConvertCharacterSet-fails-tp1101.html
>Sent from the Apache NiFi (incubating) Developer List mailing list
>archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: TestConvertCharacterSet fails

Toivo Adams
Indeed, that did the trick.

Thanks
Toivo