PutS3Object error

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

PutS3Object error

Nabegh
Hi .. PutS3Object processor is throwing the follow error .. Any idea why?


2016-07-20 15:25:41,922 ERROR [Timer-Driven Process Thread-6] o.a.nifi.processors.aws.s3.PutS3Object
com.amazonaws.services.s3.model.AmazonS3Exception: The request signature we calculated does not match the signature you provided. Check your key and signing method. (Service: Amazon S3; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: D433C836E1584299)
        at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1219) ~[aws-java-sdk-core-1.10.32.jar:na]
        at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:803) ~[aws-java-sdk-core-1.10.32.jar:na]
        at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:505) ~[aws-java-sdk-core-1.10.32.jar:na]
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:317) ~[aws-java-sdk-core-1.10.32.jar:na]
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3595) ~[aws-java-sdk-s3-1.10.32.jar:na]
        at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1382) ~[aws-java-sdk-s3-1.10.32.jar:na]
        at org.apache.nifi.processors.aws.s3.PutS3Object$1.process(PutS3Object.java:446) ~[nifi-aws-processors-0.7.0.jar:0.7.0]
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851) ~[na:na]
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822) ~[na:na]
        at org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:400) ~[nifi-aws-processors-0.7.0.jar:0.7.0]
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-0.7.0.jar:0.7.0]
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054) [nifi-framework-core-0.7.0.jar:0.7.0]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-0.7.0.jar:0.7.0]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-0.7.0.jar:0.7.0]
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127) [nifi-framework-core-0.7.0.jar:0.7.0]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_91]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_91]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_91]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_91]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
Reply | Threaded
Open this post in threaded view
|

Re: PutS3Object error

James Wing
Nabegh,

Can you share which version of NiFi you are using?  Have you tried
GetS3Object?  I recommend also trying GetS3Object also to separate general
problems with authenticating and connecting to S3 from particular issues
putting objects, putting can be more complicated.

Thanks,

James


On Wed, Jul 20, 2016 at 12:09 PM, Nabegh <[hidden email]> wrote:

> Hi .. PutS3Object processor is throwing the follow error .. Any idea why?
>
>
> 2016-07-20 15:25:41,922 ERROR [Timer-Driven Process Thread-6]
> o.a.nifi.processors.aws.s3.PutS3Object
> com.amazonaws.services.s3.model.AmazonS3Exception: The request signature we
> calculated does not match the signature you provided. Check your key and
> signing method. (Service: Amazon S3; Status Code: 403; Error Code:
> SignatureDoesNotMatch; Request ID: D433C836E1584299)
>         at
>
> com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1219)
> ~[aws-java-sdk-core-1.10.32.jar:na]
>         at
>
> com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:803)
> ~[aws-java-sdk-core-1.10.32.jar:na]
>         at
>
> com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:505)
> ~[aws-java-sdk-core-1.10.32.jar:na]
>         at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:317)
> ~[aws-java-sdk-core-1.10.32.jar:na]
>         at
> com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3595)
> ~[aws-java-sdk-s3-1.10.32.jar:na]
>         at
>
> com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1382)
> ~[aws-java-sdk-s3-1.10.32.jar:na]
>         at
>
> org.apache.nifi.processors.aws.s3.PutS3Object$1.process(PutS3Object.java:446)
> ~[nifi-aws-processors-0.7.0.jar:0.7.0]
>         at
>
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
> ~[na:na]
>         at
>
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822)
> ~[na:na]
>         at
>
> org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:400)
> ~[nifi-aws-processors-0.7.0.jar:0.7.0]
>         at
>
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-0.7.0.jar:0.7.0]
>         at
>
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
> [nifi-framework-core-0.7.0.jar:0.7.0]
>         at
>
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-0.7.0.jar:0.7.0]
>         at
>
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-0.7.0.jar:0.7.0]
>         at
>
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
> [nifi-framework-core-0.7.0.jar:0.7.0]
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_91]
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_91]
>         at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_91]
>         at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_91]
>         at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_91]
>         at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_91]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/PutS3Object-error-tp12878.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: PutS3Object error

Nabegh
Hi James,

I'm using nifi-0.7.0-RC2. I tried ListS3 processor and I received the same error.

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: PutS3Object error

James Wing
ListS3 processor is a great place to start.  I can think of a couple of
likely causes:

* Credentials - you might want to double-check how you specified the
credentials to make sure it is right.  If you specify access key and secret
key, and the secret key is not correct, I believe you will get this error.

* Region - you should make sure the region you specify in the processors
matches the S3 bucket's region.  S3 is a bit weird about the rules for
buckets and regions, but it might help to check.

Thanks,

James

On Wed, Jul 20, 2016 at 12:57 PM, Nabegh <[hidden email]> wrote:

> Hi James,
>
> I'm using nifi-0.7.0-RC2. I tried ListS3 processor and I received the same
> error.
>
> Thanks
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/PutS3Object-error-tp12878p12880.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: PutS3Object error

Joe Skora
Were you able to resolve this?

I think James is right about the Bucket vs Region relationship, AWS
Signature Version 4 has less tolerance for bucket/region mismatches.  (See this
comment
<https://github.com/aws/aws-sdk-java/blob/1.10.32/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L518>
in the AmazonS3Client source for more detail.)

If that's not it, AWS has a page on signature troubleshooting
<http://docs.aws.amazon.com/general/latest/gr/signature-v4-troubleshooting.html#signature-v4-troubleshooting-key-signing>
.

Regards,
Joe

On Wed, Jul 20, 2016 at 6:15 PM, James Wing <[hidden email]> wrote:

> ListS3 processor is a great place to start.  I can think of a couple of
> likely causes:
>
> * Credentials - you might want to double-check how you specified the
> credentials to make sure it is right.  If you specify access key and secret
> key, and the secret key is not correct, I believe you will get this error.
>
> * Region - you should make sure the region you specify in the processors
> matches the S3 bucket's region.  S3 is a bit weird about the rules for
> buckets and regions, but it might help to check.
>
> Thanks,
>
> James
>
> On Wed, Jul 20, 2016 at 12:57 PM, Nabegh <[hidden email]> wrote:
>
> > Hi James,
> >
> > I'm using nifi-0.7.0-RC2. I tried ListS3 processor and I received the
> same
> > error.
> >
> > Thanks
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/PutS3Object-error-tp12878p12880.html
> > Sent from the Apache NiFi Developer List mailing list archive at
> > Nabble.com.
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: PutS3Object error

Nabegh
In reply to this post by James Wing
Thanks James. Problem was with secret key. It was url escaped.

It's working now.
Reply | Threaded
Open this post in threaded view
|

Re: PutS3Object error

Joe Gresock
I'm getting the same error that Nabegh got, on version 0.7.1 (though I also
got it on 0.7.0).  The strange thing is, the file reaches S3, but I still
get the error:

PutS3Object[id=2432f004-6a78-434e-97fd-76a7f85686f2] Error checking S3
Multipart Upload list for :
The request signature we calculated does not match the signature you
provided. Check your key and signing method.
(Service: Amazon S3; Status Code: 403; Error Code:
SignatureDoesNotMatch; Request ID: E439663F458AB9AE)

I'm using the same AWSCredentialsProviderControllerService that I use for
FetchS3Object, and it can download them with no errors.

On Wed, Jul 27, 2016 at 5:47 PM, Nabegh <[hidden email]> wrote:

> Thanks James. Problem was with secret key. It was url escaped.
>
> It's working now.
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/PutS3Object-error-tp12878p12901.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>



--
I know what it is to be in need, and I know what it is to have plenty.  I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want.  I can do
all this through him who gives me strength.    *-Philippians 4:12-13*