Duplicate `domain_userid` causing job failure

For engineers
1

Hi,
For the snowplow open source, we are suddenly getting duplicate records in scratch.snowplow_web_users_this_run (duplicate domain_userid) which leads to a job failure when it tries to insert those records into snowplow_web_users
Currently its a single domain_userid which has duplicates.

We are using

- package: snowplow/snowplow_utils
    version: 0.15.2
- package: snowplow/snowplow_web
    version: 0.16.2
2

Hi Sid,
To be able to help investigate the issue, I have a few questions, please: Which warehouse are you using? Did you alter the snowplow__user_identifiers or snowplow__session_identifiers variables or they are left as defaults? When you do dbt test where else does it fail apart from unique domain_userid test within snowplow_web_users_this_run? There are a few joins within that model, we need to see which is the most upstream table where the duplicates are introduced and the tests should narrow it down. It would be ideal to check the target folder in a local run then and by running the actual compiled codes separately, identify which join causes the duplication:

image
image962×296 28.2 KB