Adding optional field to existing schema, RDBLoader failure

Hi @mgloel,

With JSON format - yes, unfortunately you still have to manage tables by yourself, including creation and migration.

We recommend to use TSV to let Loader managing the tables, but then you need to deploy an Iglu Server and also it requires a bit extra versioning circumspection, i.e. it seems you’ve patched your schema by adding the column, but we generally strictly discouraging from doing so - it’s a very classic minor version bump, so should be plain 1-0-1 as @mike has mentioned.

I think it’s generally fine to leave it as is if you already did so (technically all historical data is still valid against the patched version), but it can cause some problems next time.

Anyways, in order to fix it now - you just need to make sure that your table and JSONPath fully match, we have a tutorial for it:

2 Likes