Snowflake loader failing

For engineers Storage targets
1

We have a Snowflake Loader job that is failing with EMR error:

Unhandled runtime exception. Failed to access remote file: token expired. Please check your credentials

The transformer job is running correctly. Loader version is 0.6.0.

Any suggestion how to fix this issue?

Here is the playbook.json file:

 {
"schema":"iglu:com.snowplowanalytics.dataflowrunner/PlaybookConfig/avro/1-0-1",
           "data":{
              "region":"eu-west-1",
              "credentials":{
             "accessKeyId":xxxxxxxxxxxxx,
             "secretAccessKey":xxxxxxxxxxxxxxxxxxxxx
          },
          "steps":[
             {
                "type":"CUSTOM_JAR",
                "name":"Snowflake Transformer",
                "actionOnFailure":"CANCEL_AND_WAIT",
                "jar":"command-runner.jar",
                "arguments":[
                   "spark-submit",
                   "--deploy-mode",
                   "cluster",
                   "--class",
                   "com.snowplowanalytics.snowflake.transformer.Main",
                   "s3://snowplow-hosted-assets/4-storage/snowflake-loader/snowplow-snowflake-transformer-0.6.0.jar",
                   "--config",
                   "{{base64File "./snowflake-loader.json"}}",
                   "--resolver",
                   "{{base64File "./resolver.json"}}"
                ]
             },
             {
                "type":"CUSTOM_JAR",
                "name":"Snowflake Loader",
                "actionOnFailure":"CANCEL_AND_WAIT",
                "jar":"s3://snowplow-hosted-assets/4-storage/snowflake-loader/snowplow-snowflake-loader-0.6.0.jar",
                "arguments":[
                   "load",
                   "--base64",
                   "--config",
                   "{{base64File "./snowflake-loader.json"}}",
                   "--resolver",
                   "{{base64File "./resolver.json"}}"
                ]
             }
          ],
          "tags":[ ]
       }
    }
2

Looks like an authentication error - perhaps check whether the credentials you’re using have access to all the S3 locations they need?

4

I set up that user with full access to S3, still getting the same error

5

Hi @vladimir.ilic,

I suspect this error is about DynamoDB, wherer Transformer and Loader store their state.

6

It is fixed. Issue was role sessionDuration in snowflake-loader.json file. It was set to 900 seconds, which was not enough. Thanks.