This is my flow is working fine but i need to make this as a incremental
fetch from API ..whenever client API made changes it should reflect in our
databases .How i can process for incremental fetch in API call ?.Please
guide me to fix this problem .
It depends on the Client API on how to tell what the next response
data set should be.
That may be an additional query parameter such as last fetch
timestamp, or something like HTTP etag header in most APIs.
You can pass FlowFiles to InvokeHTTP to tell such parameters.
Also, I recommend using Record processors over splitting dataset to process.
Because Record processors work more efficiently and the data set unit
will be more meaningful in your case.
I think your flow would be something like:
InvokeHTTP (Assuming the API result contains value to make next
-> Do some JSON transformation (QueryRecord, UpdateRecord or
-> Then connect success back to InvokeHTTP to fetch next dataset
> Hi Team,
> I am new to NiFi .I have a task like .We need to fetch a data from Client
> API and stored into Postgresql
> My Flow ;
> InvokeHTTP -->>--SplitJson-->>--EvaluateJsonPath
> This is my flow is working fine but i need to make this as a incremental
> fetch from API ..whenever client API made changes it should reflect in our
> databases .How i can process for incremental fetch in API call ?.Please
> guide me to fix this problem .
> Thanks in Advance,
> Manikandan K
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/