Hi team,
We had a type of event that should fail the JSON validation at the Stream Enrich Step, however, stream enrich fail to recognize it as the malformed event.
The schema looks like this:
{
"$schema": "http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#",
"description": "Schema for customer",
"self": {
"vendor": "au.com.some.vendor",
"name": "customer",
"format": "jsonschema",
"version": "2-0-1"
},
"type": "object",
"properties": {
"customer_type": {
"type": "array",
"items": {
"type": "string",
"enum": ["residential", "unknown","unknown_verified"]
}
},
"total_claimed_profiles": {
"type": "integer"
},
"about_me": {
"type": "array",
"items": {
"type": "object",
"properties": {
"type_of_work": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
},
"agents": {
"type": "array",
"items": {
"type": "object",
"properties": {
"agency_id": {
"type": "string"
},
"agent_profile_id": {
"type": "string"
},
"permissions": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
}
},
"required": ["customer_type"],
"additionalProperties": false
}
The event payload looks like this:
{
"schema":"iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0",
"data":[
{
"schema":"iglu:au.com.some.vendor/customer/jsonschema/2-0-1",
"data":{
"customer_type":[
"unkown"
],
"about_me":[
{
"type_of_work":[
]
}
],
"total_claimed_profiles":0,
"agents":[
{
"agency_id":"BWXGOX",
"agent_profile_id":"1BDF564F",
"permission":[
"basic",
"rent_application_management"
]
}
]
}
}
]
}
Noticed that in the schema, a field called permissions
while in the event payload, it called permission
. JSON validator didn’t pick up this error.
Similarly, I change the field from agency_id
to agency_ids
in the event payload, and send the event to snowplow collector. It pass the JSON validator too.
We are using snowplow stream enrich v1.3.2
I’ve gone over the source code but could not figure out why. Any suggestions would be graceful.
Cheers