Using the snowplow unified DBT package incremental logic how can I build a derived model, of another derived model incrementally?
For instance I have 2 staging models that are off the events_this_run and views_this run table that i merge into an intermediate model where the intermediate model looks like:
-- depends_on: [{{ ref('snowplow_unified_events_this_run') }}, {{ ref('snowplow_unified_views_this_run') }}]
{% set union_incremental -%}
{{ snowplow_utils.is_run_with_new_events('snowplow_unified') }}
{% if is_incremental() %}
{%- set events_lower_limit, events_upper_limit = snowplow_utils.return_limits_from_model(ref('snowplow_unified_events_this_run'), 'derived_tstamp', 'derived_tstamp') %}
{%- set views_lower_limit, views_upper_limit = snowplow_utils.return_limits_from_model(ref('snowplow_unified_views_this_run'), 'derived_tstamp', 'derived_tstamp') %}
-- This incremental uses two different start limits, because we pull from both the
-- snowplow view model run and the snowplow run event model run and these can be at different limits
and
(
(
_dbt_source_relation ilike '%stg_content_open_events'
and start_tstamp >= date({{ events_lower_limit }})
and start_tstamp <= {{ dateadd(datepart="day", interval=1, from_date_or_timestamp=events_upper_limit) }}
)
or
(
_dbt_source_relation ilike '%stg_web_first_party_content_page_views'
and start_tstamp >= date({{ views_lower_limit }})
and start_tstamp <= {{ dateadd(datepart="day", interval=1, from_date_or_timestamp=views_upper_limit) }}
)
)
{% endif %}
{%- endset %}
-- Pull in the custom content_open events, and the web first party content page views
with unioned_data as (
{{
dbt_utils.union_relations(
relations=[ref('stg_content_open_events'), ref('stg_web_first_party_content_page_views')],
where=union_incremental
)
}}
),
How can I then build an incremental model off of the intermediate model efficiently?