Igluctl 0.11.0 released

Announcements
1

Igluctl 0.11.0 released

We are pleased to announce version 0.11.0 of igluctl.

Igluctl is a CLI application for common Iglu registry tasks. You can find out more about igluctl on our docs site here.

JSONPaths deprecation

We are removing support of JSONPaths generation starting with this release. Our JSON Schema to warehouse inference
capabilities drastically improved in the recent year. Challenges that the JSONPaths were meant to solve are now
automatically handled by the transformer and schema-ddl library. Therefore we are will be gradually removing
JSONPaths support from our components starting with igluctl.

This release drops JSONPaths generation from static generate along with other options. This command should only be
used for troubleshooting RDB Loader related issues or as a guide for migration from JSONPaths tables. As a result, many of
the arguments were hardcoded to align with the transformer’s logic:

  • owner - removed and set to None
  • varcharSize- removed and set to 4096
  • withJsonPaths - removed and set to false
  • rawMode - removed and set to false
  • splitProduct - removed and set to false
  • noHeader - removed and set to true

static generate will now issues warnings when new schema version migration is impossible.

Older versions of this tool with JSONPaths support are still available for legacy use-cases.

Deprecation of migrate command

With the upcoming changes to RDB Loader, column order in tables will not matter for loading. The COPY statement will explicitly specify column names. For this reason, the rdbms migrate command was removed, as it
only dealt with column ordering.

Adding parquet-verify command

We have added the parquet-verify command to detect breaking changes in parquet (Databricks) schema migrations.
igluctl static verify-parquet <schema_dir>

This is similar to the new static generate functionality. But it doesn’t produce SQL files, it just prints out the
warnings about the breaking changes.

Example output:
'com.test/test/jsonschema/1-0-1'. Changes: Incompatible type change String to Double at /id"

2 Likes