Request Signature Not Matching with Scala Stream Collector

For engineers AWS real-time pipeline
1

We’re trying to setup snowplow with kinesis and are having problems with the request signature not matching what should be calculated by AWS. Whenever we try to startup the snow plow collector we are getting the following issue:

16:12:05.429 [scala-stream-collector-akka.actor.default-dispatcher-4] DEBUG akka.event.EventStream - logger log1-Slf4jLogger started
16:12:05.431 [scala-stream-collector-akka.actor.default-dispatcher-4] DEBUG akka.event.EventStream - Default Loggers started
16:12:05.436 [main] INFO  c.s.s.c.s.sinks.KinesisSink - Creating thread pool of size 10
Exception in thread "main" com.amazonaws.AmazonServiceException: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
'POST
/

content-length:30
content-type:application/x-amz-json-1.1
host:kinesis.us-east-1.amazonaws.com
user-agent:aws-sdk-java/1.9.23 Mac_OS_X/10.12 Java_HotSpot(TM)_64-Bit_Server_VM/25.91-b14/1.8.0_91
x-amz-date:20161129T211206Z
x-amz-target:Kinesis_20131202.DescribeStream

content-length;content-type;host;user-agent;x-amz-date;x-amz-target
b775df9fd4c6333cbb46b3785440f6a881defa6b32e8eb800131b95de683b0eb'

The String-to-Sign should have been
'AWS4-HMAC-SHA256
20161129T211206Z
20161129/us-east-1/kinesis/aws4_request
d64344fb21e4a1cfd65de09ba855f86bd973330aded83bad4cea40dd615e567d'
 (Service: AmazonKinesis; Status Code: 400; Error Code: InvalidSignatureException; Request ID: f212b50a-f258-a16e-a2a5-44632eac370c)
	at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1078)
	at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:726)
	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:461)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:296)
	at com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2498)
	at com.amazonaws.services.kinesis.AmazonKinesisClient.describeStream(AmazonKinesisClient.java:861)
	at io.github.cloudify.scala.aws.kinesis.ClientImpl$$anonfun$execute$3.apply(Client.scala:88)
	at io.github.cloudify.scala.aws.kinesis.ClientImpl$$anonfun$execute$3.apply(Client.scala:85)
	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

We set the stream variables to:

stream {
        region: "us-east-1"
        good: "snowplow_good"
        bad: "snowplow_bad"
      }

both of which exist in our AWS streams. Our aws.access-key and aws.secret-key are both set to “env” and exist as environmental variables. The streams were generated by the same IAM user that snowplow has the keys for. Does anyone have any ideas on how to debug this?

2

Hi @sokser - can you check for us that the clocks on the server(s) running the collector are accurate?

3

It seemed to be a timing issue. Thanks.