Question about ip_lookups enrichment and storage of MaxMind DB in our S3

For engineers Enrichment
1

This is possibly a really silly question, but I’m going to ask anyways, because someone else might have the same question in the future.

I was hoping to use Snowplow Mini to test enrichments, I have configured the IP anonymization and IP lookups enrichments for the Snowplow Open Source pipeline. However, I have discovered in this discourse that the IP Lookups enrichment has been disabled for Snowplow Mini. So that leaves me with a problem.

I don’t want to just deploy the new enrichments (well the IP anon one should be fine), because I do not want to break our production pipeline. But also not sure how to test the IP lookups one now.

I have downloaded the MaxMind GeoCity2 DB and put it in a private bucket, as per the instructions for the enrichment, and configured the JSON config (I don’t think there will be a problem with the actual configuration.

The question I have is: how will the enrichment server access the file in the private bucket if it’s private? Do I need to allow access from the server somehow? I’ve configured my pipeline using the provided Terraform modules.

Thanks in advance for your help.

1 Like
2

So the Terraform module uses iam for the AWS permissions for enrich. The enrich user has access to the bucket snowplow-hosted-assets so I think if I create a bucket with that name in my AWS and put the maxmind DB in there it should work. Except that it’s saying the bucket already exists, it doesn’t. So I’m stuck again.

3

Hey @angelsk ,

This module’s source has an optional input variable custom_s3_hosted_assets_bucket_name to specify a custom bucket name for your private artifacts like maxmind database.

It is available in terraform registry too.

Assuming that you cloned our quickstart repository, you can modify that terraform module to specify your bucket name using custom_s3_hosted_assets_bucket_name and re-apply.

Please let us know how it goes.

Kind regards

3 Likes
4

That’s what I need! Thanks :slight_smile: Shouldn’t have to modify the module at all, just pass through that variable.

5

just pass through that variable.

Definitely, just like another terraform input variable.

2 Likes
6

Just to follow up and say thank you again. This is all deployed and working :slight_smile:

2 Likes
7

I’m glad to hear that @angelsk ! Thanks for the follow up :slight_smile: