ETL RDB Loader Error

For engineers AWS batch pipeline (Legacy)
1

Having a bit of an issue with my Emr Etl Runner process.

Using: ./snowplow-emr-etl-runner run -c config/config.yml -r config/iglu_resolver.json -t config/targets

Get’s to the Elasticity Custom Jar Step: Load AWS Redshift enriched events storage target phase and fails.

ERROR: Data discovery error with following issues:
JSONPath file [com.google.analytics/private_1.json] was not found
JSONPath file [com.google.analytics/undocumented_1.json] was not found
No steps completed

I am using the Clojure connector and the Google Analytics Plugin.

My files look fine in S3, so not sure what’s happening.

2

@tliptak,

The error suggests the JSONPath files couldn’t be located on S3 hosted by Snowplow.

  • Have you tried to rerun the data load step?
  • What region are you running your ETL process from?
3

@ihor

Thanks for the reply.

Rerunning: Yes, I’ve tried same result.
US-East-2 (Ohio)

6

@tliptak,

There might be an issue with those specific files. I would advise resuming in a while when they are sorted out. Alternatively, you could try hosting them yourself for now. They could be obtained from here.

7

Ok, will try that.

Seems odd as I know others are using the Google Analytics integration and would need the same JSON files.