[GitHub] incubator-nifi pull request: Fixes incorrect messages count in Pro...

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: Fixes incorrect messages count in Pro...

JPercivall
GitHub user owieboy opened a pull request:

    https://github.com/apache/incubator-nifi/pull/38

    Fixes incorrect messages count in Provenance reporter. Adds Unit test to...

    PutKafka processor creates Provenance event with
    ![selection_086](https://cloud.githubusercontent.com/assets/7962021/6724291/bddac66c-ce2f-11e4-821d-3206bdd2b114.png)
     incorrect messagesCount value when using delimiter and setting the buffer size. When buffer size is larger than that of the file size, the message count value will always be zero. On the other hand, when file size is bigger than buffer size, the last batch of buffered bytes, is not being added in the messagesSent counter .
   
    The details of the fix is in this block of PutKafka.java:
    createMessage: if ( data != null ) {
                                        // If the message has no data, ignore it.
                                        if ( data.length == 0 ) {
                                            data = null;
                                            baos.reset();
                                            break createMessage;
                                        } else {
    // send logic here
    }
    Instead of calling break createmessage whend data.length is zero, do this instead:
    if ( data != null ) {
                                        // If the message has no data, ignore it.
                                        if ( data.length != 0 ) {
                                            // send logic here
     data = null;
                                            baos.reset();
                                        }


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/owieboy/incubator-nifi putkafka-messagescount

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-nifi/pull/38.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #38
   
----
commit eb5ec703ba0d5c188822a37f6d7eed14af56a594
Author: Oscar de la Pena <[hidden email]>
Date:   2015-03-19T02:10:09Z

    Fixes incorrect messages count in Provenance reporter. Adds Unit test to verify fix

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: Fixes incorrect messages count in Pro...

JPercivall
Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-nifi/pull/38


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] incubator-nifi pull request: Fixes incorrect messages count in Pro...

JPercivall
In reply to this post by JPercivall
Github user markap14 commented on the pull request:

    https://github.com/apache/incubator-nifi/pull/38#issuecomment-83576796
 
    @owieboy : Great catch, and thanks for the contribution! I have merged this into develop, so it will be included in the 0.1.0 release. You did some really interesting stuff in the unit test with Mockito. I was not familiar with the Whitebox class, so I definitely learned something new! So thanks for that, too! :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---