Unstructured event couldn't be extracted

T_P · May 13, 2016, 12:17pm

Hi guys,

I’m in the first experiments using Snowplow Mini for tracking pixel and link click events, so noob alert

Example:

<img src="http://xxx/i?e=ue&cid=53450&nid=2446468&uid=12345678" alt="Snowplow Mini" width="1" height="1" />`
and
`<a href="http://xxx/r/tp2?e=ue&cid=53450&nid=2446468&uid=12345678&u=http%3A%2F%2Fwww.xxx.com%2F%3Fname%3DThomas">Snowplow Mini Link</a>

Kibana returned it as bad result:

{"level":"error","message":"Field [eid]: [2446468] is not a valid UUID"},{"level":"error","message":"Unstructured event couldn't be extracted"}

Then I’ve submited some changes to the schemas and also replaced “eid” to “nid” since “eid” seems a reserved variable for Snowplow.

Current schema for tracking a pixel impression:

{
  "$schema": "http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#",
  "description": "Schema for open tracking event",
  "self": {
    "vendor": "com.xxx",
    "name": "open",
    "version": "1-0-1",
    "format": "jsonschema"
  },
  "type": "object",
  "properties": {
    "cid": {
      "type": "string"
    },
    "nid": {
      "type": "string"
    },
    "uid": {
      "type": "string"
    }
  },
  "additionalProperties": true
}

Now, after a few tries finding the right schema within Iglu it does still returns:

{“level”:“error”,“message”:“Unstructured event couldn’t be extracted”}

Any help?

Also, since there isn’t any DELETE method in the Iglu schema repository, is there a way to delete previous schemas uploaded?

Thanks!

josh · May 13, 2016, 12:33pm

Hi @T_P to send your own custom Unstructured Events you will need to structure your data a touch differently. The actual event JSON for your unstructured event has to be nested as a json in a key-value pair. The reason you are seeing that it cannot extract the Unstructured Event is that you are missing the ue_pr key which is where your custom event must be stored.

To send as a POST request the data payload should be sent to http://xxx/com.snowplowanalytics.snowplow/tp2 with the following JSON data:

{
  "schema": "iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4",
  "data": [
    "e": "ue",
    "ue_pr": "{\"schema\":\"iglu:com.xxx/open/jsonschema/1-0-1\",\"data\":{\"cid\":\"53450\"}}",
    "tv": "custom",
    "p": "web"
  ]
}

You will also need to provide a Content-Type header of application/json; charset=utf-8

To send as a GET request the querystring should be sent to http://xxx/i? with the following querystring:

e=ue&ue_pr=%7B%5C%22schema%5C%22%3A%5C%22iglu%3Acom.xxx%2Fopen%2Fjsonschema%2F1-0-1%5C%22%2C%5C%22data%5C%22%3A%7B%5C%22cid%5C%22%3A%5C%2253450%5C%22%7D%7D&tv=custom&p=web

The JSON for the ue_pr key has been URL Encoded.

Hope that helps!

Josh

T_P · May 13, 2016, 3:43pm

Thank you @josh
Helped for sure!

I’m passing everything as a GET request like suggested:

<img src="http://xxx/i?e=ue&ue_pr=%7B%22schema%22%3A%22iglu%3Acom.xxx%2Fopen%2Fjsonschema%2F1-0-1%22%2C%22data%22%3A%7B%22cid%22%3A%2253450%22%2C%22eid%22%3A%222446468%22%2C%22uid%22%3A%2212345678%22%7D%7D&tv=custom&p=web" alt="Snowplow Mini" width="1" height="1" />

But keeps returning errors:

"errors": [
  {
    "level": "error",
    "message": "error: Verifying schema as iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-* failed: found iglu:com.xxx/open/jsonschema/1-0-1"
  },
  {
    "level": "error",
    "message": "Unstructured event couldn't be extracted"
  }
]

Damn schemas

Do I need to wrap the current schema in another one?

josh · May 13, 2016, 4:04pm

Sorry yes you do need another layer of wrapping! This is the primary reason behind all of our Tracking SDKs… simplifies this all a great deal.

It will need to be:

"{\"schema\":\"iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-0\",\"data\":[{\"schema\":\"iglu:com.xxx/open/jsonschema/1-0-1\",\"data\":{\"cid\":\"53450\"}}]}"

Sorry about the confusion!

T_P · May 13, 2016, 4:35pm

Even before you replied, I’ve tried it, but doesn’t seem to work…

"errors": [
  {
    "level": "error",
    "message": "error: instance type (array) does not match any allowed primitive type (allowed: [\"object\"])\n    level: \"error\"\n    schema: {\"loadingURI\":\"#\",\"pointer\":\"\"}\n    instance: {\"pointer\":\"\"}\n    domain: \"validation\"\n    keyword: \"type\"\n    found: \"array\"\n    expected: [\"object\"]\n"
  }
]

josh · May 13, 2016, 9:19pm

Sorry this one should do it, the unstruct schema does not need an array:

"{\"schema\":\"iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-0\",\"data\":{\"schema\":\"iglu:com.xxx/open/jsonschema/1-0-1\",\"data\":{\"cid\":\"53450\"}}}"

If that still does not work let me know!

vjoxyodo · May 19, 2016, 5:58pm

Hello Josh,

I’m a co-worker of T_P. I’ve tried your proposal, for a click - example:

http://xxx/r/tp2?e=ue&ue_pr={"schema"%3A"iglu%3Acom.snowplowanalytics.snowplow\%2Funstruct_event\%2Fjsonschema\%2F1-0-0"%2C"data"%3A{"schema"%3A"iglu%3Acom.xxx\%2Fclick\%2Fjsonschema\%2F1-0-1"%2C"data"%3A{"cid"%3A"53450"%2C"eid"%3A"2446468"%2C"uid"%3A"12345678"}}}&tv=custom&p=web&u=redirect_url

But keep getting the following error:

"errors": [ { "level": "error", "message": "error: Could not find schema with key iglu:com.xxx/click/jsonschema/1-0-1 in any repository, tried:\n level: \"error\"\n repositories: [\"Iglu Central [HTTP]\",\"Iglu Server [HTTP]\",\"Iglu Client Embedded [embedded]\"]\n" } ]

The json schema is:

{"schema":"iglu:com.snowplowanalytics.snowplow\/unstruct_event\/jsonschema\/1-0-0","data":{"schema":"iglu:com.xxx\/click\/jsonschema\/1-0-1","data":{"cid":"53450","eid":"2446468","uid":"12345678"}}}

Any idea what it might be wrong in the schema?

Thanks!

josh · May 20, 2016, 3:23pm

Hi @vjoxyodo, that could be an issue either with:

If you send the events without it being in the Iglu Server then it will be cached as “bad” and you will need to restart your instance before it will find it again.
The script for uploading in 0.2.0 was incorrectly making the Schemas Private rather than Public. If you grab the updated script from here and upload again it should work.

Hopefully that will get it working for you!

Josh

T_P · May 23, 2016, 2:43pm

Hey @josh

We’ve tried your suggestions:

Used the newest version of iglu_server_upload.sh where “?isPublic=true” is setted and updated all existing schemas. Status 200 was returned.
In the Iglu schema repository searched for the vendor prefix and returned all updated schemas above.

Fired an event, data went flagged as “bad” and the output remained exactly the same as before:

{"level":"error","message":"error: Could not find schema with key iglu:com.xxx/click/jsonschema/1-0-1 in any repository, tried:\n level: \"error\"\n repositories: [\"Iglu Central [HTTP]\",\"Iglu Server [HTTP]\",\"Iglu Client Embedded [embedded]\"]\n"}

Is there anything else we can check?

TIA

josh · May 23, 2016, 2:59pm

Hi @T_P did you restart the Snowplow Mini instance to reset the cache?

The Stream Emitter holds an internal cache where if an event is sent with a schema that it cannot find that schema is locked into a bad cache - so we do not waste processing time attempting to find a bad schema over and over. For the moment the only way to reset this cache is to restart the instance.

If you have done the above are you certain that the schema you are sending with the event matches what is in the Iglu Server?

T_P · May 23, 2016, 3:13pm

We have “good” data. It’s working!

The restart solved it

Thank you for your patience in helping us, @josh

josh · May 23, 2016, 3:15pm

No worries @T_P! Glad it started working in the end.

Topic		Replies	Views
Unable to send event to snowplow mini For engineers	8	1865	June 30, 2020
NodeJs Unstructured Events - http://iglucentral.com/ Iglu	12	1109	October 26, 2022
Self-describing event (Unstructured)	1	1668	October 22, 2019
Tracking Custom Events Snowplow Mini For engineers	2	1671	March 13, 2018
Unstructured event Collectors	3	1647	May 19, 2016

Unstructured event couldn't be extracted

Related topics