Hi @anton,
If I understood you correctly and as we only need to deploy the shredder, then out of (EmrEtlRunner + Shredder + Loader) we ignore EmrEtlRunner and the Loader and create a playbook and a config file to submit our shredder spark job via dataflow-runner to AWS EMR. For this we will require these 4 files:
playbook.json
emr-config.json
iglu-resolver.json {base64}
config.hocon {base64}
In this case, where can we find a sample for config.hocon required for the shredder (“s3://snowplow-hosted-assets-eu-central-1/4-storage/rdb-shredder/snowplow-rdb-shredder-0.19.0”)?
From this dataflow-runner tutorial, I assume this would be how we submit our shredder spark job:
$ ./path/to/dataflow-runner run-transient --emr-config ./path/to/emr-config.json --emr-playbook ./path/to/playbook.json
Please correct me if I am wrong.