Iglu Scala Client 2.2.0 and 1.3.0 released

Announcements
1

Iglu Scala Client 2.2.0 and 1.3.0 released

We are very pleased to announce versions 2.2.0 and 1.3.0 of Iglu Scala Client. We keep maintaining both 2.x and 1.x
branches for Cats Effect (CE) 3 and CE2 respectively.

Iglu scala client is a client implementation for the Iglu schema repository.
It is extensively used by Snowplow pipeline components for accessing the Iglu server though HTTP, as well as for test purposes
with static iglu repositories.

Allow building caches based on the resolver’s content

This feature was available in earlier release for 1.x branch in 1.2.0. But it is new for the 2.x branch.

In iglu-scala-client, the client provide two core functionalities:

  1. The resolver, which can lookup schemas (or a list of schemas)
  2. The validator, which can validate data against resolved schema

We already had a cache for the resolver. Here, we introduce a cache in the validator.

Technical details

Currently, for every validation we convert a schema returned from the resolver to the JsonSchema instance from the com.networknt library as part of validating the schema. This operation proved to be expensive in our tests, so we wanted to cache it. For a given schema key, if the corresponding schema from the resolver does not change, then neither will the the result of converting the schema to a JsonSchema instance from the com.networknt library.

However, in some rare cases, schemas do change even though our recommendation remains that if a schema changes, the schema’s version should be updated. See our docs on versioning here.

So, we have introduced a pattern that will enable us to create caches that need to eventually update if the corresponding schema cached by the resolver has been updated. This pattern has been described in more detail here. This will allow us to introduce caches to improve performance even in cases where that cache needs to keep up with updates to schemas.

Schema List Improvements

In the anticipation to the upcoming changes to the transformer, we added the schema listing capability for embedded
repositories.

With schema caching introduced in 1.2.0, the client would return a stale version of schema list when a new schema was added.
This happened because it had no means to invalidate the cache. This could cause events to fail validation whenever they used a new schema
version that was not yet in the cache. We added a safe method that invalidates the cache when a referenced schema
version is not found. This makes sure new schema versions are not missed.

Upgrading

If you are already using version older version bump libraryDependencies in your build.sbt:

libraryDependencies += "com.snowplowanalytics" %% "iglu-scala-client" % "1.3.0"

Or if you using Cats Effect 3:

libraryDependencies += "com.snowplowanalytics" %% "iglu-scala-client" % "2.2.0"
5 Likes