Loader to add existing column

1

Hey,
we get this error

  * **************/listing/jsonschema/1-*-* WIDEROW
  * **************/web_page/jsonschema/1-*-* WIDEROW
  * **************/focus_form/jsonschema/1-*-* WIDEROW
  * **************/change_form/jsonschema/1-*-* WIDEROW
  * **************/vehicle_details/jsonschema/1-*-* WIDEROW
  * **************/listing_feature/jsonschema/1-*-* WIDEROW
List()
unstruct_event_com_snowplowanalytics_snowplow_focus_form_1
True for Info(s3://**************/run=2023-08-29-13-00-00-84201f43-b0f2-45a8-bfdc-ed3a48b1e944/,com.snowplowanalytics.snowplow,focus_form,1,SelfDescribingEvent)
List()
contexts_com_*****_listing_1
True for Info(s3://**************/run=2023-08-29-13-00-00-84201f43-b0f2-45a8-bfdc-ed3a48b1e944/,com.******,listing,1,Context)
List()
contexts_com_*****_vehicle_details_1
True for Info(s3://**************/run=2023-08-29-13-00-00-84201f43-b0f2-45a8-bfdc-ed3a48b1e944/,com.******,vehicle_details,1,Context)
List()
unstruct_event_com_snowplowanalytics_snowplow_change_form_1
True for Info(s3://**************/run=2023-08-29-13-00-00-84201f43-b0f2-45a8-bfdc-ed3a48b1e944/,com.snowplowanalytics.snowplow,change_form,1,SelfDescribingEvent)
List()
contexts_com_*****_listing_feature_1
True for Info(s3://**************/run=2023-08-29-13-00-00-84201f43-b0f2-45a8-bfdc-ed3a48b1e944/,com.*****,listing_feature,1,Context)
List()
contexts_com_snowplowanalytics_snowplow_web_page_1
True for Info(s3://**************/run=2023-08-29-13-00-00-84201f43-b0f2-45a8-bfdc-ed3a48b1e944/,com.snowplowanalytics.snowplow,web_page,1,Context)
INFO Migration: Migrating unstruct_event_com_snowplowanalytics_snowplow_focus_form_1 (pre-transaction)
ERROR Error executing transaction. Sleeping for 30 seconds for the first time
net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:
column 'UNSTRUCT_EVENT_COM_SNOWPLOWANALYTICS_SNOWPLOW_FOCUS_FORM_1' already exists

… in Dynamodb, we can’t find the manifest , how can we create the manifest inside DynamoDB

2

Hi @abdalaziz can you please share details about your deployment such as:

  1. Snowflake Loader version you are using?
  2. Shredder version you are using?
  3. Shredder type you are using?
  4. Any other details on the deployment that would help add context to this?
3

Thank you @josh for your answer

  1. Snowflake Loader version you are using?
  • loader version using this image: snowplow/rdb-loader-snowflake:5.7.2
  1. Shredder version you are using?
  2. Shredder type you are using?
  • we are using transformer `snowplow/transformer-kinesis:5.7.2
  1. Any other details on the deployment that would help add context to this?
    we are using default configuration (minimal for each component) for snowflake
4

Thanks @josh, we figure out the issue we had a bad configuration in a loader service related to snowflake scheme, it should be the schema name only for ex: SCHEMA_NAME but we was adding the name of the DB before it DB_NAME. SCHEMA_NAME

1 Like
5

Thanks for letting us know @abdalaziz !