Using snowplow-micro with custom events

For engineers Collectors
1

Hi,

I have setup snowplow-micro in a local iglu repo so I can validate the schemas before we use them in production. There is no issue testing with the default payload, but my custom payloads (which I have already validated so I know the work) do not work on my instance.

I am using the standard setup from GitHub - snowplow-incubator/snowplow-micro-examples: Examples of how to apply Snowplow Micro to popular build/test strategies

Oddly enough, I am unable to successfully test the custom payloads they provided, http://localhost:9091/com.snowplowanalytics.snowplow/tp2
fails with
“Error while extracting event(s) from collector payload and validating it/them.”,

. The posting is done in python so it looks like this

data = {
    "event": {
        "schema": "iglu:test.example.iglu/cart_action_event/jsonschema/1-0-0",
        "data": {
            "type": "add"
        }
    },
    "context": [
        {
            "schema": "iglu:test.example.iglu/product_entity/jsonschema/1-0-0",
            "data": {
                "sku": "100",
                "name": "Galaxy QUest",
                "price": 25.99,
                "quantity": 1
            }
        }
    ]
}

    headers = {
        "Content-Type": "application/json",
    }
    # Post the event to the Snowplow micro endpoint
    response = requests.post("http://localhost:9091/com.snowplowanalytics.snowplow/tp2", json=data, headers=headers)

I have no trouble accessing my iglu schemas on say this url

[type or paste code here](http://localhost:9091/micro/iglu/test.example.iglu/product_entity/jsonschema/1-0-0)

Has anyone tried to use snowplow-micro to test like this before?

2

So generally for something like this you’d typically use the Python tracker this will take care of things like setting the path, headers, encoding the body etc to ensure it’s a valid Snowplow request. In addition you’ll also get things like graceful retries, helper methods for tracking and a bunch of other useful stuff.

If you don’t want to use the Python tracker for some reason - you can still POST to the Snowplow collector but you’ll need to follow the expected JSON payload format that includes the variables as part of the tracker protocol. At this point your request would looking something a bit more like:

payload = {
      "schema": "iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4",
      "data": [
          {
              "e": "ue",
              "tv": "python-???",
              "aid": "micro",
              "cx": "base64 encoded contexts (or use co for non-encoded)",
              "ue_px": "base64 encoded event (or use ur_pr for non-encoded)"
          } 
      ]
}

    # Post the event to the Snowplow micro endpoint
    response = requests.post("http://localhost:9091/com.snowplowanalytics.snowplow/tp2", json=data)
3

If you want to send to the tp2 endpoint, they need to conform to the tracker protocol.

That means your custom event must be in the ue_px property (if base64 encoded) or stringified and in the ue_pr property.

There’s a number of raw event examples for tp2 on this page in the docs: Example Tracker Requests | Snowplow Documentation

(Or just use the Python Tracker :slight_smile: )