losing property

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

losing property

anup s
Hi, I have the below workflow. I am intentionally failing the original zip file, but when I put the uncompressed file into HDFS I'm losing on the ${path} property which is lost along with the original flow file. Is there any way I can retain this property?
Uncompress


Reply | Threaded
Open this post in threaded view
|

RE: losing property

Mark Payne
Anup,

I'm not sure that I understand what you mean when you say you're "losing on the ${path} property which is lost along with the original flow file". Can you explain a bit further? The FlowFIle should be transferred to the 'success' relationship after sending to HDFS, and all attributes should be kept intact. 

Thanks
-Mark

----------------------------------------

> Date: Mon, 18 May 2015 02:44:48 -0700
> From: [hidden email]
> To: [hidden email]
> Subject: losing property
>
> Hi, I have the below workflow. I am intentionally failing the original zip
> file, but when I put the uncompressed file into HDFS I'm losing on the
> ${path} property which is lost along with the original flow file. Is there
> any way I can retain this property?
> <http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/file/n1565/nifi-_%281%29.jpg>
>
>
>
>
>
>
> --
> View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/losing-property-tp1565.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
     
Reply | Threaded
Open this post in threaded view
|

RE: losing property

anup s
Hi Mark,
   I am auto-terminating the orginal zip flow file. And on success (viz. uncompression) the unzipped file goes to HDFS and puts the file at the new location. But the directory structure isn't created, althougth the Put HDFS file location is mentioned as <HDFS loc>/${path}.

In the case, where the source is not a zip file the directory is created and moved under the directory, but this isn't working for the unpack route..
Reply | Threaded
Open this post in threaded view
|

RE: losing property

Mark Payne
OK, I think I understand now. So you want to put the unzipped file to HDFS but use the path that was assigned to the zip file?

----------------------------------------

> Date: Mon, 18 May 2015 05:20:23 -0700
> From: [hidden email]
> To: [hidden email]
> Subject: RE: losing property
>
> Hi Mark,
> I am auto-terminating the orginal zip flow file. And on success (viz.
> uncompression) the unzipped file goes to HDFS and puts the file at the new
> location. But the directory structure isn't created, althougth the Put HDFS
> file location is mentioned as <HDFS loc>/${path}.
>
> In the case, where the source is not a zip file the directory is created and
> moved under the directory, but this isn't working for the unpack route..
>
>
>
> --
> View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/losing-property-tp1565p1570.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
     
Reply | Threaded
Open this post in threaded view
|

RE: losing property

anup s
That's right Mark! I want to retain the directory structure.
Reply | Threaded
Open this post in threaded view
|

RE: losing property

Mark Payne
OK cool. Just wanted to make sure that I don't give bad advice.

So when UnpackContent unpacks a zip file, it set a new value for the path attribute based on the path within the .zip file.

So in order to avoid this, what I would do is add an UpdateAttribute before UnpackContent. Configure UpdateAttribute with a new property. For the property name, use "zipfile.path" and for the value, use "${path}". So basically you're just copying the value of "path" to "zipfile.path". Then, in PutHDFS, for the directory, you can use "${zipfile.path}". Or, if you want to also include the path of the file within the .zip, you could use "${zipfile.path}/${path}"

Does that make sense?

Thanks
-Mark


----------------------------------------

> Date: Mon, 18 May 2015 07:54:18 -0700
> From: [hidden email]
> To: [hidden email]
> Subject: RE: losing property
>
> That's right Mark! I want to retain the directory structure.
>
>
>
> --
> View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/losing-property-tp1565p1577.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
     
Reply | Threaded
Open this post in threaded view
|

RE: losing property

anup s
Thanks Mark! That worked..