Configure stream-collector, stream-enrich and iglu-server to use JSON logging


We’re using these Docker images to run snowplow and using DataDog for our application logging.

How can I configure JSON logging in each of these components?



Hi @andrew ,

Welcome to Snowplow community!

I’m not sure exactly what you mean by JSON logging, so I’ll assume that you’re talking about structured logging where each line of log would be a JSON of key value pairs, like here.

For that to be possible, the apps would need to log using StructuredArguments, which is not the case at the moment.

Why do you need to log the lines as JSONs ?

Hi @BenB

We’re using DataDog to visualise our application logs. Their documentation recommends to use JSON logging (or structured as you say).

To alleviate this complexity, configure your logging library to produce your logs in JSON format

If structured logging isn’t possible it looks like we can use grok parsing instead:

UPDATE: I used grok parsing with the following rule

JavaRule \[%{notSpace:thread}\] %{notSpace:level} %{notSpace:class} - %{data:msg}