We are using CloudFront based Snowplow Collector for our needs. We are happy about the Logging process, but due to AWS Log delivery to S3 bucket, SLA varies from 15mints to 24hrs we see some lags in aggregating real-time logs.
We are looking for real-time collectors like Scala or Clojure based currently support snowplow collector. Alternate means requires additional infrastructure setup and engineering involvement in long-term basis pushing our decision in that direction.
Curious to know is there a way to collect Snowplow logs via AWS APIGateway -> Kinesis Stream(Sink). Adapting this way various consumers could use downstream Kinesis Stream for a different business purpose.
It would be worth you checking out this RFC on our post-Clojure Collector plans:
The summary is that we are focused on the Scala Stream Collector going forwards. This collector is now getting a lot of great GDPR- and privacy-related features, beyond anything you’d be able to setup with AWS API Gateway.