I was getting duplicate rows(event_id is same) for some of the events which was problem to sending notification to the end user. due to duplicate of row, I have seen there is snowplow document to add event_fingerprint in enrichment layer: Event fingerprint enrichment | Snowplow Documentation
Sorry my bad. I was using event_fingerprint only. I have updated my description. but not getting any value for same in my bigquery table. I can see null value for event_fingerprint
Generally if you have it setup with defaults, and have rebooted the enrichment this will work without any issues. It may be worth checking failed events however to see if some warehouse constraint is being violated.
The schema already exists in Iglu Central so you shouldn’t need to deploy it yourself. And the enrichment only adjusts that atomic field, so there’s no need to reboot the loader or repeater when making these changes, only the enricher.
Do your other enrichments run correctly? (e.g. do mkt_source, mkt_clickid etc get populated from UTM/gclid parameters by campaign_attribution.json?) Maybe make sure you’re specifying the right directory in --enrichments when you start the enricher and that the files are up to date when you do it.
When it works, the value will be an MD5 hash (other hashes are avaialble) of the fields that should be the same for duplicate events, so the value should be different for every event that isn’t a duplicate, or collide if it is a duplicate (except for the configured fields, so event_id and dvce_sent_tstamp can be different but if everything else matches it will count as duplicate and have the same hash). It doesn’t remove any rows by itself, just gives you a new value to dedupe by.
Thanks for detail explaination.
Yes, I can see the value for UTM parameter by campaign_attribution.json. Infact we rebuild the image using jenkins teraform script and can see the event_fingerprint_enrichment.json image are redploying the same but still facing issue. No value in the event_fingerprint value.
Is there anything related to version? I can post my docker file of enrichment here.
