Snowplow-media-player 0.3.1 dbt package released

Announcements New releases
1

We are very happy to announce the release of the snowplow-media-player v.0.3.1 dbt package.

This release optimises performance in Databricks for incremental models in the derived schemas. In addition it clarifies some documentation around how to set up your dbt packages to work optimally with Databricks depending on whether or not you have the Unity Catalog enabled and which dbt connector you are using.

Features

  • Optimize performance in Databricks for incremental models (Close #14)
  • Improve the GitHub workflow (Close #15)

Upgrading

To upgrade simply bump the snowplow-media-player version in your packages.yml file. If you want to take advantage of the performance optimisations in already existing incremental tables in your Databricks environment, run the following commands against Databricks for all of the incremental tables in your derived schema:

ALTER TABLE {derived_schema}.{table_name} SET TBLPROPERTIES (delta.autoOptimize.optimizeWrite = true, delta.autoOptimize.autoCompact = true);

By default, these are the snowplow_media_player_media_stats.sql, and snowplow_media_player_base tables in the derived schema.

2 Likes