Error: Could not find schema with key

For engineers Troubleshooting
1

We recently added a new jsonschema and it appears that the iglu resolver is unable to locate it.

It seems as though we’ve done everything correctly based on the the instructions on Github and the other posts I’ve investigated here on the Snowplow Discourse.

Here’s an example context:

{"schema":"iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0","data":[{"schema":"iglu:com.managedbyq/hivy_user/jsonschema/1-0-0","data":{"user_id":"BySwZTqAf","is_admin":false,"is_employee":true,"is_contractor":false}},{"schema":"iglu:com.managedbyq/hivy_company/jsonschema/1-0-0","data":{"company_id":"Vk_1-Hb5z","company_slug":"redacted"}},{"schema":"iglu:com.managedbyq/hivy_slack/jsonschema/1-0-0","data":{"action":"product","source":"command"}}]}

error message:

error: Could not find schema with key iglu:com.managedbyq/hivy_slack/jsonschema/1-0-0 in any repository, tried: level: "error" repositories: ["Iglu Central [HTTP]","Iglu Client Embedded [embedded]","Q Iglu [HTTP]"]

our iglu resolver:

{
  "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-0",
  "data": {
    "cacheSize": 500,
    "repositories": [
      {
        "name": "Iglu Central",
        "priority": 0,
        "vendorPrefixes": [ "com.snowplowanalytics" ],
        "connection": {
          "http": {
            "uri": "http://iglucentral.com"
          }
        }
      },
      {
        "name": "Q Iglu",
        "priority": 1,
        "vendorPrefixes": [ "com.managedbyq" ],
        "connection": {
          "http": {
            "uri": "http://q-snowplow-iglu.s3-website-us-east-1.amazonaws.com"
          }
        }
      }
    ]
  }
}

For example, this url works: http://q-snowplow-iglu.s3-website-us-east-1.amazonaws.com/schemas/com.managedbyq/hivy_slack/jsonschema/1-0-0

We have other schemas stored in the same S3 bucket (q-snowplow-iglu) that are working as expected.

Any tips on debugging?

2

Is this issue cropping up during the stream (real time) enrichment process or the batch enrichment process?

Overall your setup looks good but in the stream enrich process it is possible for this to occur if an event referencing that schema is sent before the schema is available on S3. The enrich process will record this as a cache miss so future events will go to bad as long as that reference is in the cache. The quick solution here is to manually evict the cache. There’s some more information on the cache and eviction in this thread.

3

Thanks for the response, @mike !

I was able to figure out the issue, the problem is that there was another error and I was only seeing this one. The other error was that that the schema itself was invalid due to required: []. We need to either exclude required from the schema or make at least one field required.

We were using the batch enrichment process, BTW.

Thanks again!

4

No worries. That’s interesting about the empty required array causing events to fail. It might be a good prospect for igluctl to pick up and raise as a warning during the linting process!