I am running snowplow-kinesis-s3-0.4.0 to upload logs to s3 but i am getting an error,
“com.amazonaws.AmazonClientException: Unable to execute HTTP request: s3-http”
not sure, what is the reason of the error because l am following your setup guide step by step
Config
sink {
aws {
access-key: “key”
secret-key: “key”
}
kinesis {
in {
stream-name: “good”
initial-position: “TRIM_HORIZON”
max-records: “1000”
}
out { stream-name: "bad" }
region: "us-west-2"
app-name: "snownrich"
}
s3 {
region: “http://s3-us-west-2.amazonaws.com/”
bucket: “bucketname/folder”
max-timeout: “300”
format: “lzo”
}
buffer {
byte-limit: 4500000
record-limit: 500 # Not supported by Kafka; will be ignored
time-limit: 60000
}
logging {
level: “error”
}
monitoring {
snowplow {
collector-uri: “xx.xx.xx.xx”
collector-port: 80
app-id: “collector-monitor”
method: “GET”
}
}
}
Error
com.snowplowanalytics.snowplow.storage.kinesis.s3.S3Emitter com$snowplowanalytics$snowplow$storage$kinesis$s3$S3Emitter$$attemptEmit$1
SEVERE: S3Emitter threw an unexpected exception
com.amazonaws.AmazonClientException: Unable to execute HTTP request: s3-http
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:478)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:302)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3672)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1436)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1302)
at com.snowplowanalytics.snowplow.storage.kinesis.s3.S3Emitter.com$snowplowanalytics$snowplow$storage$kinesis$s3$S3Emitter$$attemptEmit$1(S3Emitter.scala:157)
at com.snowplowanalytics.snowplow.storage.kinesis.s3.S3Emitter$$anonfun$emit$1.apply(S3Emitter.scala:189)
at com.snowplowanalytics.snowplow.storage.kinesis.s3.S3Emitter$$anonfun$emit$1.apply(S3Emitter.scala:189)
at scala.collection.immutable.List.foreach(List.scala:318)
at com.snowplowanalytics.snowplow.storage.kinesis.s3.S3Emitter.emit(S3Emitter.scala:189)
at com.amazonaws.services.kinesis.connectors.KinesisConnectorRecordProcessor.emit(KinesisConnectorRecordProcessor.java:159)
at com.amazonaws.services.kinesis.connectors.KinesisConnectorRecordProcessor.processRecords(KinesisConnectorRecordProcessor.java:132)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.V1ToV2RecordProcessorAdapter.processRecords(V1ToV2RecordProcessorAdapter.java:42)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.ProcessTask.call(ProcessTask.java:172)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:48)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:23)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.UnknownHostException: s3-http
at java.net.InetAddress.getAllByName0(InetAddress.java:1273)
at java.net.InetAddress.getAllByName(InetAddress.java:1185)
at java.net.InetAddress.getAllByName(InetAddress.java:1119)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:44)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.resolveHostname(DefaultClientConnectionOperator.java:259)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:159)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:706)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467)
… 19 more
Feb 22, 2017 7:29:24 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 13 datums.
Feb 22, 2017 7:29:28 AM com.snowplowanalytics.snowplow.storage.kinesis.s3.S3Emitter forceShutdown
SEVERE: Shutting down application as unable to connect to S3 for over 300 ms