Igluctl 0.8.0 released

We are happy to announced the 0.8.0 release of igluctl

Igluctl is the cli tool for performing most common tasks on a Iglu repository. The igluctl documentation has details on obtaining and using the new version.

The new release changes introduce some change in behaviour of the app compared to previous versions.

1. More strict about schemas directory

For commands like igluctl static s3cp and igluctl static push, every file in the directory must contain a valid iglu schema. Or else it fails early and does not upload anything.

2. Uploads schema lists by default.

If myschemas directory contains these files:

myschemas
└── com.example
    β”œβ”€β”€ schema1
    β”‚   └── jsonschema
    β”‚       β”œβ”€β”€ 1-0-0
    β”‚       └── 1-0-1
    └── schema2
        └── jsonschema
            β”œβ”€β”€ 1-0-0
            └── 1-1-0

Then the command igluctl static s3cp myschemas mybucket will upload the following files to s3

s3://mybucket/schemas/com.example/schema1/jsonschema/1-0-0
s3://mybucket/schemas/com.example/schema1/jsonschema/1-0-1
s3://mybucket/schemas/com.example/schema2/jsonschema/1-0-0
s3://mybucket/schemas/com.example/schema2/jsonschema/1-1-0
s3://mybucket/schemas/com.example/schema1/jsonschema/1
s3://mybucket/schemas/com.example/schema2/jsonschema/1
s3://mybucket/schemas

…where the latter 3 files are lists of the schemas that were uploaded. This is so that the s3 iglu repository can mimic the β€œlist” endpoints of a full iglu server.

List files can be turned off using the --skip-schema-lists command line option.

A list file is skipped if the schemas cannot be placed unambiguously in order of when they were created. For example, schemas β€œ1-0-1” and β€œ1-1-0” have ambiguous order, so a list containing both these versions would be skipped.

3. Strips whitespace when uploading schemas to s3.

Previously the command igluctl static s3cp myschemas mybucket would upload schema files without making any formatting changing. In the new version, the schema files are read and interpreted as json before upload. We stringify the json object with the nospaces option, and then upload it to s3.

4. More opinionated about s3 paths.

Imagine your schemas are in a directory called myschemas. Previously the command igluctl static s3cp myschemas mybucket would upload files to s3://mybucket/myschemas/. In the new version we upload it to s3://mybycket/schemas, i.e. we rename the top level directory to always be schemas. This forces the directory structure to conform to iglu repository.

The --s3path command line option now consistently refers to the prefix before the /schemas directory in s3. So the command igluctl static s3cp --s3path myprefix myschemas mybucket would upload files to s3://mybucket/myprefix/schemas/.

This change in behaviour is most noticeable when uploading a single schema. Consider this command:

igluctl static s3cp --s3path myprefix myschemas/com.example/schema1/jsonschema/ mybucket

In the old version, this would upload a file to s3://mybucket/myprefix/jsonschema/1-0-0. In the new version it is uploaded to s3://mybucket/myprefix/schemas/com.example/schema1/jsonschema/1-0-0.

5. Hocon configuration format for the deploy command

Previously the igluctl static deploy command accepted a configuration file with self-describing json. It still accepts the old format, for legacy users, but we suggest to start using the hocon style input. These days we prefer hocon for application configuration because:

  • Easier to add new optional fields without making breaking change for users
  • The hocon parsing library supports environment variables and system properties out of the box.
4 Likes