I was experimenting with snowplow mini after I’ve updated some event schema versions. For some reason when I send new events they don’t appear neither in bad nor good indexes. Initially when I was sending with wrong version but new data fields it has parsed and validated against those schemas and I can see that schemas are present via rest api.
Is there any reason that could prevent events from being indexed? My only guess now is that for some reason it doesn’t pick up new versions of the schema, as all events that get indexed are version 1-0-0, though I am sending newer version.
Not a solution : I encountered a similar problem when adding schema throug the api. A reboot of the snowplow mini instance helped. I guess something is not taken into account with the iglu server <-> internal processing of events.
Hi @evaldas @bcharp - the Stream Enrich application that is embedded caches all schemas that come in. So if you send an event without first updating your Iglu Server it will cache the event as bad and all future events will fail.
Snowplow Mini is also quite fragile as we are just chaining applications with Unix pipes so it can infrequently require a reboot to get things back on track. We hope to make it more robust when support for NSQ in the stream processing applications is done.
1 Like