Direct access to FlowFile Content Repository Path?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Direct access to FlowFile Content Repository Path?

Peter Wicks (pwicks)
I'm working on a custom processor. The processor calls a 3rd party library that needs the path to my FlowFile content (no streams/arrays, just file names).

I could write the content out to a temp file, but the content is already right there in the content repository... and many of the files are very large, and it would increase execution times a lot to write the data to a temp location a second time.

Is it possible, in the context of a custom processor, to get the path to the FlowFile in the content repo?

Thanks,
  Peter
Reply | Threaded
Open this post in threaded view
|

Re: Direct access to FlowFile Content Repository Path?

Joe Witt
Peter

It is not possible and if it were it would only be available for repo
implementations that use files and further would need to be limited to
cases where the content on disk was in a file by itself (unlikely) or for
processes that would honor the offsets and length, would be read only, etc..

Another option is to just invoke the 3rd party lib on the files before
pulling them into nifi.  If data arrives in list/fetch run list/execute
custom/fetch instead maybe.

Thanks

On Mon, Mar 4, 2019 at 11:43 PM Peter Wicks (pwicks) <[hidden email]>
wrote:

> I'm working on a custom processor. The processor calls a 3rd party library
> that needs the path to my FlowFile content (no streams/arrays, just file
> names).
>
> I could write the content out to a temp file, but the content is already
> right there in the content repository... and many of the files are very
> large, and it would increase execution times a lot to write the data to a
> temp location a second time.
>
> Is it possible, in the context of a custom processor, to get the path to
> the FlowFile in the content repo?
>
> Thanks,
>   Peter
>
Reply | Threaded
Open this post in threaded view
|

RE: [EXT] Re: Direct access to FlowFile Content Repository Path?

Peter Wicks (pwicks)
Thanks Joe. The data sources out of a database (ExecuteSQL). Content Repository is not an area where I'm very knowledgeable on the internals.

Thanks again.

-----Original Message-----
From: Joe Witt <[hidden email]>
Sent: Monday, March 4, 2019 10:05 PM
To: [hidden email]
Subject: [EXT] Re: Direct access to FlowFile Content Repository Path?

Peter

It is not possible and if it were it would only be available for repo implementations that use files and further would need to be limited to cases where the content on disk was in a file by itself (unlikely) or for processes that would honor the offsets and length, would be read only, etc..

Another option is to just invoke the 3rd party lib on the files before pulling them into nifi.  If data arrives in list/fetch run list/execute custom/fetch instead maybe.

Thanks

On Mon, Mar 4, 2019 at 11:43 PM Peter Wicks (pwicks) <[hidden email]>
wrote:

> I'm working on a custom processor. The processor calls a 3rd party
> library that needs the path to my FlowFile content (no streams/arrays,
> just file names).
>
> I could write the content out to a temp file, but the content is
> already right there in the content repository... and many of the files
> are very large, and it would increase execution times a lot to write
> the data to a temp location a second time.
>
> Is it possible, in the context of a custom processor, to get the path
> to the FlowFile in the content repo?
>
> Thanks,
>   Peter
>