We’re happy to announce version 2.1.0 of Snowplow RDB Loader.
This release comes with automation, stability and observability improvements on multiple fronts.
RDB Loader now is long-running (or “self-healing”) application with proper retry logic, which makes an attempt to retry on transient (such as connection lost) and alerts on permanent (such as JSONPath file missing) issues. Apart from that on stability front, there’s additional backup retry queue, DB healthchecks and several other low-level fixes, such as new connection pool.
Another new feature worth mentioning is no-op schedule, allowing to stop loader on predefined schedules, such as maintenance window or expensive monthly reporting.
As of Shredder, it has two important changes: DAG/performance optimisation and an option to improve or disable synthetic deduplication (use with caution!).
Upgrading
There’s no required changes in 2.1.0, but make sure you’ve upgraded your pipeline to 2.0.0, which includes configuration file changes.
All new features are optional and can be found in our reference.hocon config file.