I’ve a product with 5 markets(different location) and each market specific configurations is stored in different config files. Like a_market…hocon, b_market…hocon and so forth.
I want to deploy Stream collector snowplow/scala-stream-collector-kafka:2.4.0-rc1 image. and want to pass each market specific config file as a
option 1 - pass as parameter
option 2 - Embed all the config values from config to inside the docker image and then deploy it.
could anyone please help
Hey @pramod.niralakeri the easiest way would be to mount a volume with the config you want for each container - you could create custom Docker containers for each application but its a lot of overhead just to pass in a config.
Some relevant documentation if you are doing this with Kubernetes: Configure a Pod to Use a ConfigMap | Kubernetes
It if helps, the latest collector release lets you also pass parameters using java system properties. So you can do something like:
docker run \
You can even use a mixture of hocon files and system properties. System properties take precedence over the hocon. This new feature of using system properties was only introduced in version 2.4.1.
So you might be able to have a shared common hocon file, and then override specific values for your different markets.
Hi @pramod.niralakeri, by system properties I meant the using the
-D command line flag. This is a standard flag for setting what in java applications are called system properties.
So in my example in my previous comment,
-Dcollector.port=80 means set the
collector.port system property to 80.
Alternatively, there is a way of using environment variables instead of system properties:
docker run \
-e "CONFIG_FORCE_collector_interface=0.0.0.0" \
-e "CONFIG_FORCE_collector_port=80" \
-e "CONFIG_FORCE_collector_streams_good=goodstream" \
-e "CONFIG_FORCE_collector_streams_bad=badstream" \
-e "CONFIG_FORCE_collector_streams_sink_brokers=localhost:9092" \
The syntax is slightly awkward, but it is standard behaviour described in the readme of this config-parsing library. Note it still requires the
-Dconfig.override_with_env_vars=true to make it work. Personally I prefer the syntax in my earlier post using just system properties.