Is there a way to get the contents of the enriched field derived_context in a non-JSON way into the enriched event? For example extra TSV columns at the end of the events?
Or should I use (in this case) a Kafka Streams Application that takes care of this? I prefer to not use an extra step of course.
This has actually been one of our largest pain points since we need to iterate over the array of contexts (both “derived contexts” and “contexts”), and flatten from there.
It’s significantly more expensive to do that than having context key-val pairs of “schema”: “payload” in a “context” object, especially when you want to start leveraging global contexts.
If anyone has sql shortcuts to parse this, I’m very interested…
Are you writing from a snowplow user perspective or from a snowplow developer one?
I’d rather have everything in JSON or everything in TSV, but the mix makes an extra step to flatten it necessary I think.