Enricher fails to refresh updated schema

For engineers Enrichment
1

Hi,

I was wondering what the expected behaviour should be for the enricher when an existing schema is being updated:
Let’s say I have a schema version 1-0-1 with timestamp in seconds where I change the requirements for the timestamp to be in milliseconds, if I don’t update the version number and overwrite the schema with the new requirements and upload the schema in my S3 bucket (iglu repo is hosted as static website on S3), should my enricher pick up the change and apply the new requirements for my schema?
I have a cacheTTL of 900 in my resolver config file.

thanks a lot,

Alexandre

1 Like
2

Hi @Alexandre5602,

Welcome to Snowplow community !

Even though we don’t recommend to overwrite existing schemas, the way you describe should work.

Enrich would re-fetch the schema after the cache item is expired or if the application is restarted.

3

Hi Enes,

I’m afraid its not hapening somehow because it doesn’t validate new events which follow the most recent of the update in the schema, is there somewhere in the log where I should see this refresh happening ?

Thanks,

Alexandre

4

I don’t think we have logs for this refresh. But, what you can do is checking your bad rows. Are the failures for this specific timestamp unit ?

5

Well timestamp are still being sent in seconds where they should be forced in milliseconds and have a minimum higher than the timestamp value in seconds, so normal behaviour should be that those events fail to validate but they are still being validated correctly.

6

Could you send the schema part for this timestamp rule, please ? Maybe I can spot something.