Pass environment variables to json config files, use hocon instead?

For engineers Enrichment
1

Hey,

we are facing a problem with several snowplow modules that use json config files, e.g. resolver.json, emr-config.json. We are passing environment variables to these jsons, e.g. AWS credentials etc.
Currently we are using a hacky shell script using several sed commands in order to replace the placeholders with the variables when we are building the docker image for the resp modules.
With hocon files we do not have this problem as they have the advantage that env variables can be passed directly. Are we doing something wrong?
Is there another workaround? Or can hocon files be used instead of jsons?

2

Hi @mgloel,

I afraid you’re right and there’s no workarounds, but to use a templating mechansim right now. We’re slowly moving to HOCON in most apps (RDB Shredder, Loader, FS2 Enrich already use it) and we have plans to migrate other apps to it, but no plans for enrichment configurations and iglu resolver yet - not sure they will ever be in different format.

3 Likes