Snowflake Streaming Loader and AWS CloudWatch

For engineers
1

Hi,

I want to use Streaming loader instead of RDB loader in my Snowplow pipeline.

I am basically using a EC2 instance to run the Streaming Loader docker container . How do I expose the internal logs of the container to AWS Cloudwatch. I want to know from the logs that how many good and bad events are getting processed and what is the latency. Also, if there are self-monitoring metrics being generated I would like to transfer them to AWS Cloudwatch.

Thanks in Advance

2

Try using the awslogs log-driver.

For metrics, there’s quite a few ways to get the native StatsD metrics into Cloudwatch, probably start here.