Nifi:how to handle with multiple flowfiles?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Nifi:how to handle with multiple flowfiles?

sally

I have request data which can be splitted into several flowfile for example
request data for date 2017-02-02 can be splitted to four part 2017-02-02.1
,2017-02-02.2 ....( i use this partiotion because webservice uses paging and
it can send only certain amount of data at a time (400 mb) and if there is
more than 400 mb data for this date i have to invoke it several time in this
case 4 time) after that i will put them in directory ,and then i want to
check if i don't miss any part of response data which is represented as
request data only difference is that it contains data of "2017-02-02" sent
by service,but while invoking service and getting response if i got any
error or for some reason can't get response data , i have to know if i am
missing some part of my response data. for this reason i use code below:

 var OutputStreamCallback =
    Java.type("org.apache.nifi.processor.io.OutputStreamCallback");
    var StandardCharsets = Java.type("java.nio.charset.StandardCharsets");
    var  flowFile=session.get();
    var   countAs=flowFile.getAttribute('countAs');
    var filename=flowFile.getAttribute('filename');
    var  countNumber=filename.substring(11,filename.length);
    var  countArray=countArray.push(countNumber);
    var j=0;

    for(  var  i=0;i<=;i++){
   
if(flowFile.getAttribute('filename').substring(11,flowFile.getAttribute('filename'))==countArray[i]){

    session.transfer(flowFile,REL_SUCEESS);
    }
    }
IF i don't miss anything i want to put whole reponse data in one flowfile
and then rename it to this format 2017-02-02 and save it in database but
there are several subjects i am interested in:

this code doesn't seems to work properly can you give me any better idea of
how can i check if something is missing?

2.how can i convert this splitted reponse data in one flowfile?



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/