Composition using $ref

For engineers Troubleshooting
1

Can iglu schemas be composed using $ref in Snowplow?

The 2018 blog on snowplow.io ‘Modelling page view events as a graph’ shows how to use iglu references to compose schemas:

"relationships": { "type": "object", "properties": { "visited": { "$ref": "iglu:com.snowplowanalytics.graph/visited/jsonschema/1-0-0" } } }

On Snowplow Micro and Mini I get an error when I try this:

items/$ref: Reference iglu:com.snowplowanalytics.snowplow/identify/jsonschema/1-0-0 cannot be resolved

    "slates": {
      "type": "array",
      "description": "Recommended corpus slates",
      "items": {
        "$ref": "iglu:com.snowplowanalytics.snowplow/identify/jsonschema/1-0-0"
      }
    }

Our Snowplow Micro is configured to resolve to Iglu Central:

{
  "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-2",
    "data": {
      "cacheSize": 500,
      "repositories": [
        {
          "name": "Iglu Central",
          "priority": 0,
          "vendorPrefixes": [ "com.snowplowanalytics" ],
          "connection": {
            "http": {
              "uri": "http://iglucentral.com"
            }
          }
        },
        {
          "name": "DEV Iglu",
          "priority": 0,
          "vendorPrefixes": [
            "com.pocket"
          ],
          "connection": {
            "http": {
              "uri": "https://com-getpocket-dev.iglu.snplow.net/api",
              "apikey": "######"
            }
          }
        }
      ]
    }
}

I get the same error when trying to reference an entity defined in our private iglu server hosted on iglu.snplow.net.

2

I got a step further by discovering a 2018 page_view schema. This works for iglucentral:

    "slates": {
      "type": "array",
      "description": "Recommended corpus slates",
      "items": {
        "$ref": "http://iglucentral.com/schemas/com.snowplowanalytics.snowplow/identify/jsonschema/1-0-0"
      }
    }

My goal is to compose custom entities that we defined. Is there a way to reference schemas on snplow.net, given that it requires an api key? I get the following ValidationError if I replace the above $ref with a custom entity:

java.io.FileNotFoundException: https://com-getpocket-dev.iglu.snplow.net/api/schemas/com.pocket/corpus_slate/jsonschema/3-0-0

3

Hi @mmiermans, did you manage to get it working?

We want to do similar stuff in our repo

{
  "$schema": "http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#",
  "description": "Dummy",
  "self": {
    "vendor": "com.foo",
    "name": "bar",
    "format": "jsonschema",
    "version": "1-0-0"
  },
  "type": "object",
  "allOf": [
    {
      "$ref": "iglu:com.foo/baz/jsonschema/1-0-0"
    }
  ]
}

EDIT: Seems like it’s not possible yet be able to resolve iglu schemas in validator · Issue #210 · snowplow/iglu-scala-client · GitHub