Databricks RDB Loader: Table or view not found for 'SHOW COLUMNS'

10

My earlier comment is what you’re gonna need for partioning on a date instead, which is actually the recommended method following the 4.1.0 release: Databricks RDB Loader: Table or view not found for 'SHOW COLUMNS' - #3 by PaulBoocock

Basically, create your table as:

CREATE TABLE IF NOT EXISTS snowplow.events (
  app_id                      VARCHAR(255),
  collector_tstamp            TIMESTAMP       NOT NULL,
  event_name                  VARCHAR(1000),
  -- Lots of other fields go here

  -- Collector timestamp date for partitioning
  collector_tstamp_date       DATE GENERATED ALWAYS AS (DATE(collector_tstamp))
)
PARTITIONED BY (collector_tstamp_date, event_name);
1 Like