Snowplow Unified 0.3.0 and Snowplow Utils 0.16.2 Released!

Announcements New releases
1

We are very happy to announce the release of snowplow-unified v0.3.0 and snowplow-utils v0.16.2. These releases add some great new features to Unified such as custom aggregations as well as a new table grants options. Read below for more details about what’s included in each release!

Snowplow Unified 0.3.0

Summary

This release adds one major new feature, which is custom aggregations on the views, sessions, and users tables. You can read more about it in our docs here. We also added the ability to manage grants to all tables in the package via the snowplow__grant_select_to variable.

Under the hood we did a lot of small tweaks and improvements including prefixing all macro calls for easier custom models, moved cluster by fields to macros, ensure that the manifest tables are only full refreshed when snowplow__allow_refresh is set to true AND there is a full refresh flag on the run, and added a few context fields to the derived tables that were being discarded previously.

:rotating_light: Breaking Changes :rotating_light:

  • We have changed the behavior of the allow_refresh macro so now if snowplow__allow_refresh is set to true it will only refresh the manifest models if the --full-refresh flag is also set. If you require the old behavior where it would refresh the manifest models on an incremental run when snowplow__allow_refresh was set to true, please overwrite this macro. See the Overriding Macros guide for more details.
  • Renamed snowplow__page_view_passthroughs to snowplow__view_passthroughs to be consistent with the rest of the package
  • Minimum snowplow-utils version is now 0.16.2

Features

  • Add new passthrough aggregations to the views, sessions, and users table, enabled using snowplow__view/session/user_aggregations
  • Reorder and add some additional context fields to derived tables (non-breaking change)
  • Add snowplow__custom_sql to allow adding custom sql to the snowplow_unified_base_events_this_run and snowplow_unified_events_this_run models
  • Add macro to define cluster-by for tables to allow users to overwrite this if required
  • Add check for --full-refresh flag before allowing refresh of manifest models when snowplow__allow_refresh is set to true.
  • Add ability to grant select to a list of users, principals or roles on tables created by the package using snowplow__grant_select_to for all warehouses except BigQuery
  • Add auto-grant of usage on schemas to snowplow__grant_select_to, can be disabled using snowplow__grant_schema_usage (see docs at https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-dbt/package-features/table-grants/)

Fixes

  • Fix a bug where if you ran the package in a period with no data, and had list all events enabled, the package would error rather than complete
  • Fix incorrect tagging in app errors module tables

Under the hood

  • Prefix all macro calls with package name for easier customization
  • Use macros for grouped fields (e.g. contexts) where possible
  • Bump actions version numbers

Upgrading

Bump the snowplow-unified version in your packages.yml file, paying attention to the breaking changes above.

Snowplow Utils 0.16.2

Summary

This releases adds two new sets of macros, one is parse_agg_dict that generates aggregation sql, and the other is apply_grants which allows the usage of the snowplow__grant_select_to variable to grant select on models to any user/role in that list (does not support BigQuery). We also fix a rare edge case where custom user identifiers made from sql could lead to duplicate sessions in the lifecycle table.

Features

  • Add new parse_agg_dict macro for use to generate aggregation sql in other packages
  • Overwrite default dbt apply_grants macro to enable using a variable to define grant targets
  • Add new default__grant_usage_on_schemas_built_into macro to add as a post-hook in package to grant usage for schemas
  • Ensure only one user id even when a custom user sql is provided for the lifecycles manifest table

Upgrading

To upgrade, bump the package version in your packages.yml file.

1 Like