InputPort (Flat files ) --> ProcessFlowfiles (combination of ExecuteScripts to filter out few fields) --> Load to Teradata(using a custom processor)
To do so I have to be sure that I process the flowfiles in certain order across the nodes so that the individual transactions are correct ( ex. An update statement executing before there is an Insert statement for that record ). I receive all the flat files via remote NiFi instance meaning they all will be distributed across nodes and there is no way for me to know in what order the files are processing. Has anyone encountered similar problem before and know a way out of this scenario?
Also, I came across this JIRA ( https://issues.apache.org/jira/browse/NIFI-4155 ). I see that there is a patch available for this already but don't see a release Version tied to this. Is there a plan to include this patch as a part of 1.5.0 or any future releases?
Another question, apart from what Koji Kawamura mentioned in the comment section for the above JIRA to use
EnforceOrder --> Wait to block only 1 FlowFile can go through --> Processors required to run serially --> Notify to release the latch
Is there any other way to enforce order across multiple nodes?