Hi @zameer
Problem with loading bad data sounds similar to issue we had in the past.
Could you please check values for fields like: collector
, v_collector
or loaderName
in badrow JSONs stored in the S3?
If these look like snowplow-stream-collector-kinesis...
instead of ssc-...
then it may mean we violate schema definition (as table definitions in postgres are created based on JSON schemas defined in Iglu registry). Is there any error in loader logs indicated that?
The issue was fixed in 0.3.0 of terraform collector modules. It looks like quickstart-examples repository still uses 0.2.1 for AWS collector module.
We will plan necessary upgrades in quickstart repository and let you know as soon as the newest version is released. Meanwhile, you can even try to change version of snowplow-devops/collector-kinesis-ec2/aws
in terraform to 0.3.0 locally and see if it helps!