We are pleased to announce Version 0.3.0 of the Snowplow Postgres Loader.
Even though Postgres Loader isn’t considered as production-ready yet, we are getting closer to that target with every new release. In this release, we are adding new features, fixing some bugs, and bumping a few dependencies.
Ability to output bad rows to topics
In the previous version, bad rows are just printed to stdout. With this release, it is possible to send bad rows to Kinesis and Pubsub. You can find the necessary changes on the config to activate this feature in the config documentation page.
Local file system as an event source
Local file system is added as a new event source in this release. This feature makes debugging and quick trials much easier.
Improvements
- Records that come from streams are started to be acknowledged after inserting them into database in order to reduce the risk of losing events in case of an application crash.
- Database pool size is made configurable.
Bug fixes
- Fixed the bug which crashes application when Kinesis retrieval mode is FanOut.
- Changed database schema to make possible to load events enriched with pii pseudoanonymization.
Upgrading from 0.2.0
Changes on config
We made some changes to the general config structure to make it more in line with other Snowplow products. You can find the description of the new structure here. Also, you can find example configs here.
Changing some of the column types
The database schemas created with the previous version isn’t working well with events enriched with pii pseudoanonymization. Therefore, we needed to change types of the some of the columns. You need to apply this migration script to your database.