Was wondering if it is possible to set AWS credential chain to default with the new configuration file format (enrich/config.kinesis.extended.hocon at master · snowplow/enrich · GitHub) as it previously was. Asking as we need to configure Snowplow running in EKS to use IRSA rather than node IAM role.
Thank you for the link. I’ll still post some thoughts here for others to follow in the future.
The way AWS EKS IRSA works is that on assigning a service account to the pod a web identity token is created as a file in the pod and two environment variables are set:
AWS_ROLE_ARN - the AWS IAM role arn of the role to be assumed by the pod.
AWS_WEB_IDENTITY_TOKEN_FILE - points to the web identity token to be used for authentication.
Default chain should take these env variables as the top priority if I am not mistaken.
For stream enricher it falls back to assuming node IAM role for some reason.
IRSA works for the kinesis client but fails on dynamoDb client
i’m trying to understand the code but i have no experience on scala but i’m presuming that the problem comes from how the dynamodb aws client is initialized.
seems related to lease function that uses version 2 of aws sdk software.amazon.kinesis.leases.exceptions.DependencyException: software.amazon.awssdk.services.dynamodb.model.DynamoDbException: User: arn:aws:sts::xxxx:assumed-role/xxxx/i-xxx is not authorized to perform: dynamodb:DescribeTable on resource: arn:aws:dynamodb:eu-west-3:xxxxx:table/snowplow-enrich-kinesis because no identity-based policy allows the dynamodb:DescribeTable action (Service: DynamoDb, Status Code: 400, Request ID: xxxxx)