Event Recovery 0.4.0

Announcements New releases
1

Fixes and improvements release adding a first-class support for iglu webhooks and unstructured events, addition step and array multi-matching.

When in doubt, source repository hosts showcases for most of the scenarios, that’s used by our integration testing. BDP customers can use a recovery UI instead.

Add step

This recovery step allows adding values to existing JSON attributes. It can handle all JSON types with following semantics:

  • adds numbers
  • concatenates strings
  • sums booleans
  • merges objects
  • merges arrays
  • overrides null values

This means that for example, the following step will add a new parameter into an existing array without relying on regular expressions:

{
   "op": "Add",
   "path": "$.raw.parameters",
   "value": [{"name": "schema","value": "iglu:com.snowplowanalytics.com/lorem_ipsum/jsonschema/1-0-0"}]
}

Improved Iglu-webhook and unstructured event support

Previously it has been cumbersome to work with these kinds of events as they required rebuilding JSONs through regex. Which might have not been able to solve all the required cases. This addition allows using well-known DSL semantics to alter even iglu-webhook/querystring supplied url-encoded/base64 events.
Making it possible to execute the following snippet by decoding, replacing and encoding the result:

{
   "op": "Replace",
   "path": "$.raw.parameters.[?(@.name=~ue_pr)].value",
   "match": "iglu:0/video_impression/jsonschema/1-0-0",
   "value": "iglu:com.snowplowanalytics/lorem_ipsum/jsonschema/1-0-0"
}

Greedy array matcher

Previously filter matcher ([@.name=~lorem]) have only matched the first instance. Now they will match all the occurrences.

Full changelog

Add Add step (#80)
Coerce ue_pr to CollectorPayload from RawEvent (#81)
Add check transformation for url-encoded fields (closes #77)
Greedy array matcher (close #91)
Raise instead of not modifying when querystring invalid (#86)
Bump sbt-native-packager to 1.7.6 (#89)
Bump beam base image to 0.2.0 (#88)
Bump scalafmt to 1.6.1 (#85)
Use github actions for CI (#87)

6 Likes