Snowplow-mobile 0.5.3 dbt package released

Announcements New releases
1

We are very happy to announce the release of the snowplow-mobile v.0.5.3 dbt package.

This release optimises performance in Databricks for incremental models in the derived schemas. In addition it clarifies some documentation around how to set up your dbt packages to work optimally with Databricks depending on whether or not you have the Unity Catalog enabled and which dbt connector you are using.

Features

  • Optimize performance in Databricks for incremental models (Close #40)

Upgrading

To upgrade simply bump the snowplow-mobile version in your packages.yml file. If you want to take advantage of the performance optimisations in already existing incremental tables in your Databricks environment, run the following commands against Databricks for all of the incremental tables in your derived and manifest schemas:

ALTER TABLE {derived_schema}.{table_name} SET TBLPROPERTIES (delta.autoOptimize.optimizeWrite = true, delta.autoOptimize.autoCompact = true);
ALTER TABLE {manifest_schema}.{table_name} SET TBLPROPERTIES (delta.autoOptimize.optimizeWrite = true, delta.autoOptimize.autoCompact = true);

By default, these are the snowplow_mobile_screen_views, snowplow_mobile_sessions, and snowplow_mobile_users tables in the derived schema, and the snowplow_mobile_incremental_manifest, and snowplow_mobile_base_sessions_lifecycle_manifest tables in the manifest schema.

2 Likes