Hi. We’re trying to make use of a schema where a field is optional but, if present, has a minimum length. The problem we’re seeing is that when the field is absent, snowplow is rejecting it for violating the minLength validation criteria.
What we’re seeing is that if the optional field is left out of the context when we make the snowplow api call, that the message shows up in bad logs with a schema violation. Snowplow is reporting it as violating the minLength criteria, and the value we see in bad logs is "<NA>".
What’s the correct way to not pass an optional field that has validation criteria? Should we include the field in the context packet but set its value to NULL? Or something else?
That may work, but it seems to me there’s probably a more elegant way of doing it - if a field is already left out of required, we shouldn’t have to separately set its type as optionally null.
Null is a value, and required is a separate constraint. I see where you’re coming from, but what happens when you need to require a field, but it can be null?
Or, conversely, when a field may be missing, but may not be set to null?
JSONschema is a very generic specification, so there will often be cases that might look awkward for a given implementation but make sense when you consider the standard as a whole IMO.
That’s true! But my point is, if a field is optional, then there must be some way of sending the packet so the field is recognized as missing rather than processed against its validation criteria as though it were present. I would think that just leaving the field out would work, but it doesn’t for some reason.
I believe this is an underlying property of the way the spec has been implemented in the library that is used under the hood for JSON schema validation ( https://github.com/networknt/json-schema-validator/issues/198 ) so unfortunately the solution at this stage is to add a null type for these fields in order for it to pass validaiton.
Note to anyone who finds this thread: The problem identified does not exist in snowplow.
If a field is marked as optional, and also has a minLength, snowplow correctly does not, when the field is absent from a context packet, mark the event as invalid for failing the minimum length criteria.
The data we were seeing was caused by old javascript being cached in cloudflare.