Is there something I’m missing or that I have misconfigured? Should there be an error message or bad record somewhere if validation fails during shredding? Is there any way to test just the shredding process outside of EMR so it doesn’t take 20-30 minutes to debug a change? Thanks!
Hi @mrosack - that’s very unusual. Did you check in the errors bucket as well - was that also empty? Every row of input should end up in either good, bad or errors. If it’s not, then that’s a bug in our shredding process.
Would you be able to share an example enriched event which disappears in shredding? We can then trace that through and figure out what is going on.
I thought the errors bucket was it at first - I didn’t have the errors bucket configured, but even after I set it up the errors bucket was empty and nothing changed. Attached are the enriched events I was testing with - if the shredding process can’t find a schema for or validate com.ferritelabs.snowplow/touchpoint_interaction/jsonschema/1-0-0 the records disappear.