Upgrade shredder from 0.19.0 to 2.0.0 decoding failure in shredding job

Hi @mgloel,

The problem that you have is with config HOCON. I think yours is a bit outdated. It’s for 1.x series, whereas 2.x has separated Shredder and Loader configs. You can find minimal and reference (full) example in our repo or follow the upgrade guide. Also you might be interested in recent 2.1.0 release, although most of its changes are in Loader, not Shredder.

Another potetnial problem is with resolver - it seems you don’t have any Iglu Servers, which might be ok if you use only Snowplow-authored schemas. But if you have any custom schemas that need to be migrated you’d need an Iglu Server with them. More info here:

Is --duplicate-storage-config <>
Base64-encoded Events Manifest JSON config optional?

Yes, it is optional.

1 Like