GCP: Snowplow Event Recovery

For engineers GCP pipeline
1

Hi everyone,

My team wants to use snowplow event recovery for some schema failures. The error occurred due to a missing field in one of the contexts for some of our apps. How can I insert this missing field in the recovery process or has anyone recovered something similar before?

Thanks

1 Like
2

You should test this first but assuming that you want to add a field fulfillment_type to the shopping_experience context (entity) the recovery configuration might look similar to this.

{
  "schema": "iglu:com.snowplowanalytics.snowplow/recoveries/jsonschema/4-0-0",
  "data": {
    "iglu:com.snowplowanalytics.snowplow.badrows/schema_violations/jsonschema/2-0-*": [
      {
        "name": "mainflow",
        "conditions": [{
            "op": "Test",
            "path": "$.payload.enriched.contexts",
            "value": { "regex": "iglu:com.acme/shopping_experience/jsonschema/1-0-0" }
          }],
        "steps": [
          {
            "op": "Replace",
            "path": "$.raw.parameters.cx.data.[?(@.schema=~iglu:com.acme/shopping_experience/jsonschema/1-0-0)].data",
            "match": "}$",
            "value": ", \"fulfillment_type\": \"pickup\"}"
          }
        ]
      }
    ]
  }
}
4

What was the behaviour when you tried this?

5

Hi Mike,

Thanks for your response, the method suggested helped with the recovery

1 Like