Snowplow schema violation error for custom schema

For engineers Troubleshooting
1

Hi, I’m facing this error {“lookuphistory”:[{“lastattempt”:“2022-07-26T07:27:02.756Z”,“repository”:“Iglu Central”,“errors”:[{“error”:“NotFound”}],“attempts”:“2”},{“lastattempt”:“2022-07-26T07:27:03.014Z”,“repository”:“Iglu Central - GCP Mirror”,“errors”:[{“error”:“NotFound”}],“attempts”:“2”},{“lastattempt”:“2022-07-26T07:20:05.530Z”,“repository”:“Iglu Client Embedded”,“errors”:[{“error”:“NotFound”}],“attempts”:“1”},{“lastattempt”:“2022-07-26T07:20:05.533Z”,“repository”:“local Iglu repository”,“errors”:[{“error”:“RepoFailure”,“message”:“ParsingFailure: exhausted input”}],“attempts”:“1”}],“error”:“ResolutionError”}

I’m passing the custom context schema to snowplow-enrichment and I was able to collect and enrich the custom context data, but suddenly now I’m seeing the above issue.

Can anybody tell me where the above error may have been created? Is it on the application side or the enrichment process side? And how to understand this error to fix it asap.

2

Hi @cheluvesha ,

Welcome to Snowplow community !

I assume you’re seeing this error in a bad row emitted by enrich.

This error means that enrich isn’t able to find your custom schema in the list of Iglu registries that you’ve provided.

{“lastattempt”:“2022-07-26T07:20:05.533Z”,“repository”:“local Iglu repository”,“errors”:[{“error”:“RepoFailure”,“message”:“ParsingFailure: exhausted input”}],“attempts”:“1”}],“error”:“ResolutionError”}

It seems that there is a problem in the configuration of your local registry (ResolutionError). Could you share the file configuring your resolvers please?

3 
{
  "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-1",
  "data": {
    "cacheSize": 500,
    "repositories": [
      {
        "name": "Iglu Central",
        "priority": 0,
        "vendorPrefixes": [ "com.snowplowanalytics" ],
        "connection": {
          "http": {
            "uri": "http://iglucentral.com"
          }
        }
      },
      {
        "name": "Iglu Central - GCP Mirror",
        "priority": 1,
        "vendorPrefixes": [ "com.snowplowanalytics" ],
        "connection": {
          "http": {
            "uri": "http://mirror01.iglucentral.com"
          }
        }
      }
    ]
  }
}

Hi @BenB Thank you for the response, Is this what you asked? ?

4

Hi @cheluvesha , I was asking for your file, not the example. I’d like to see the config for your local Iglu repository.

5

@BenB I’m sorry, I hope this is the correct one

6

Not yet @cheluvesha .

When you run enrich you need to specify a file listing the Iglu repositories to use, similar to the example. That’s the file I’d like to see.

7

@BenB Again I’m sorry,

{
  "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-1",
  "data": {
    "cacheSize": 500,
    "repositories": [
      {
        "name": "Iglu Central",
        "priority": 0,
        "vendorPrefixes": [ "com.snowplowanalytics" ],
        "connection": {
          "http": {
            "uri": "http://iglucentral.com"
          }
        }
      },
      {
        "name": "Iglu Central - GCP Mirror",
        "priority": 1,
        "vendorPrefixes": [ "com.snowplowanalytics" ],
        "connection": {
          "http": {
            "uri": "http://mirror01.iglucentral.com"
          }
        }
      },
      {
        "name": "local Iglu repository",
        "priority": 5,
        "vendorPrefixes": [ "com.clarivate" ],
        "connection": {
          "embedded": {
            "path": "/iglu-repo"
          }
        }
      }
    ]
  }
}

I hope this time I have sent the right one.

8

Hi @cheluvesha ,

Yes this is the one, thank you !

The issue comes from this:

          "embedded": {
            "path": "/iglu-repo"
          }

Embedded schemas need to be accessible in the CLASSPATH of the app.

In your case you probably want to use an HTTP repository. What you could do is run a simple http server in /iglu-repo (e.g. with python3 -m http.server) and then point to it in your resolvers:

            "uri": "http://localhost:8000"

(not tested)

9

@BenB Can you please give me a clear and step by step instructions whats need to be done, I have not integrated from the App side, dedicated application teams are integrating snowplow with their applications and my job is to process the data to Data warehouse, so please give me an instruction I will try to follow it and Thanks a lot for your help.

10

Hi @cheluvesha ,

  1. Run python3 -m http.server in the path where you store your Iglu schemas (/iglu-repo I believe)
  2. In the config file listing your Iglu repositories, replace
      {
        "name": "local Iglu repository",
        "priority": 5,
        "vendorPrefixes": [ "com.clarivate" ],
        "connection": {
          "embedded": {
            "path": "/iglu-repo"
          }
        }
      }

with

      {
        "name": "local Iglu repository",
        "priority": 1,
        "vendorPrefixes": [ "com.clarivate" ],
        "connection": {
          "http": {
            "uri": "http://localhost:8000"
          }
        }
      }
  1. Restart enrich
1 Like