Local Iglu registry RepoFailure with Snowplow Mirco

For engineers
1

I’m looking into using Snowplow Micro with a local docker-based Iglu schema registry for testing - with a view to using some of this for automated testing. I have the following setup:

  • iglu_postgres docker container running postgres for Iglu back-end storage
  • iglu_server docker container running snowplow/iglu-server
  • snowplow_micro docker container running snowplow/snowplow-micro:0.14.0
  • Some Python code firing test events

This setup works for IgluCentral events (e.g. common GA4 events), but I’m getting an error resolving custom schemas:

[WARN] EventLog - BAD {
  "schemaKey" : "iglu:XXX/XXX/jsonschema/1-0-0",
  "error" : {
    "error" : "ResolutionError",
    "lookupHistory" : [
      {
        "repository" : "Iglu Central",
        "errors" : [
          {
            "error" : "NotFound"
          }
        ],
        "attempts" : 1,
        "lastAttempt" : "2025-01-17T22:09:17.991Z"
      },
      {
        "repository" : "Iglu Client Embedded",
        "errors" : [
          {
            "error" : "NotFound"
          }
        ],
        "attempts" : 1,
        "lastAttempt" : "2025-01-17T22:09:17.832Z"
      },
      {
        "repository" : "Local Iglu Server",
        "errors" : [
          {
            "error" : "RepoFailure",
            "message" : "Unexpected exception fetching: java.lang.IllegalArgumentException: unsupported URI http://iglu_server:8080/api/schemas/XXX/XXX/jsonschema/1-0-0"
          }
        ],
        "attempts" : 4,
        "lastAttempt" : "2025-01-17T22:09:23.178Z"
      }
    ]
  }
}

My Iglu resolver config is:

{
    "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-3",
    "data": {
      "cacheSize": 10,
      "cacheTtl": 10,
      "repositories": [
        {
          "name": "Iglu Central",
          "priority": 0,
          "vendorPrefixes": [ "com.snowplowanalytics" ],
          "connection": {
            "http": {
              "uri": "http://iglucentral.com"
            }
          }
        },
        {
          "name": "Local Iglu Server",
          "priority": 1,
          "vendorPrefixes": [ "XXX" ],
          "connection": {
            "http": {
              "uri": "http://iglu_server:8080/api",
              "apikey": "XXX"
            }
          } 
        }
      ]
    }
  }

I’m mounting the iglu resolver config into the snowplow_micro container and have confirmed that it’s being picked up. Changing the uri results in the uri in the error updating, so the config is definitely being used.

I can docker exec into the snowplow_micro container and run a curl command generated by the Swagger UI in my local Iglu to validate that the two containers can talk to each other, and that the schema I’m looking for can be accessed:

curl -X 'GET' \
  'http://iglu_server:8080/api/schemas/XXX/XXX/jsonschema/1-0-0?repr=Canonical' \
  -H 'accept: application/json' \
  -H 'apikey: XXX'

This is successful and returns the schema. I’ve tried a lot of things and read a lot of documentation and threads, but can’t figure out why Snowplow micro isn’t able to access the schema or what the “unsupported URI” error means.

I’m running the Snowplow micro image using the following docker command:

docker run -it --rm --network iglu_network --name snowplow_micro -p 9090:9090 \
  -v /path/to/local/iglu.json:/config/iglu.json \
  snowplow/snowplow-micro:0.14.0 \
  --iglu /config/iglu.json

I’m mentioning this as I see that failed schema lookups are cached. I have been restarting the container, and since it has an -rm option it should be a fresh container each time, so caching should not be the issue.

Any help would be appreciated, thanks.

3

I read many many posts on discourse.snowplow.io and couldn’t believe that there was an issue or bug in Snowplow mini or Iglu itself, so I started looking further afield - specifically into the Java error. That led me to this StackOverflow post:

It turns out that the issue is with Java not liking underscore characters _ in hostnames, and I was using iglu_server as my docker image name. Updating this to be iglu-server (- instead of _) solved my issue!

Sharing here so others may find sanity.

4

Thanks for posting this follow up - that is indeed an odd one that I’m not sure we’ve seen before!