Hi all, we have a static repository running here: http://schemas.tripshock.com.s3-website-us-east-1.amazonaws.com which is a clone of Iglucenral plus some of our custom schemas. We take the events that come out of our Enricher, transform them from TSV to Json and send them to elastic search with Kibana. We use Snowplow inspector to look at the events we send and they all come up as Valid. See the first screenshot.
This shouldn’t be important as it is mostly Kibana / Elasticsearch complaining about not having a mapping for a field type - which is unrelated to Snowplow validation. If the validator is successfully passing and the event is being sent through the enricher to the good stream / good index that’s a guarantee that validation has passed.
Elasticsearch automatically infers a “mapping” from new fields that are sent into it. But Kibana needs to be refreshed manually to have these new fields shown correctly.
If you navigate to:
Kibana > Management > Index Patterns
Then select the “good” index and hit the “refresh” button in the top right hand corner you should see the fields counter increment and it should stop showing it as unknown.
In any case though if it is in the good index and showing up there there is nothing to worry about!
Thanks @josh. I tried exactly that last night and it worked for most fields that had ? mark but there were some nested ones like this one that we’ll have to figure out what do to with
