S3 errors for region when upgrading

For engineers
1

after getting stream enrich and EMR ETL runner up to date to latest versions, we are getting this. we have everything set to us-east-1 in enrich.conf. this worked yesterday when we had only stream enrich updated on the box and when we upgraded to latest version of EMR ETL runner this morning these started popping up. EMR ETL runner was on R100 version until this morning.

==> snowplow-stream-enrich-1.error.log <==

NonEmptyList(The authorization header is malformed; the region ‘us-east-1’ is wrong; expecting ‘eu-west-1’ (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: XXX; S3 Extended Request ID: XXX))

other things updated were these two:

[root@ip-XXX enrichments]# more ip_lookups.json
{
“schema”: “iglu:com.snowplowanalytics.snowplow/ip_lookups/jsonschema/2-0-0”,

    "data": {

            "name": "ip_lookups",
            "vendor": "com.snowplowanalytics.snowplow",
            "enabled": true,
            "parameters": {
                    "geo": {
                            "database": "GeoLite2-City.mmdb",
                            "uri": "http://snowplow-hosted-assets.s3.amazonaws.com/third-party/maxmind"
                    }
            }
    }

}
[root@ip-XXX enrichments]# more ua_parser_config.json
{
“schema”: “iglu:com.snowplowanalytics.snowplow/ua_parser_config/jsonschema/1-0-1”,
“data”: {
“vendor”: “com.snowplowanalytics.snowplow”,
“name”: “ua_parser_config”,
“enabled”: true,
“parameters”: {
“database”: “regexes-latest.yaml”,
“uri”: “s3://snowplow-hosted-assets/third-party/ua-parser/”
}
}
}

2

so i pinpointed this to the new ua_parser.json. the ip_lookups.json is fine. but when i add ua_parser.json back in and restart the process it complains.

3

Hi @mjensen,

This is likely due to the fact that you are downloading IP Lookups over HTTP and downloading the UA parser database via S3 - the later would be a bit more picky when it comes to AWS regions.

If you look at this page we have several hosted assets buckets available for the majority of the different AWS regions; for us-east-1 you should be using:

snowplow-hosted-assets-us-east-1

Or rather:

s3://snowplow-hosted-assets-us-east-1/third-party/ua-parser/

Could you check whether or not that enrichment works with the region specific bucket path?

1 Like
4

@josh thank you! that worked. forgot about the different locations for assets

1 Like
5

No worries at all @mjensen ! Glad that it is all working now.