Getting `Iglu Client Embedded due [NotFound] after 1 attempt` in Event Recover Prcess

1

Hi All,
Getting this error

Screenshot 2023-05-11 at 18.46.46
Screenshot 2023-05-11 at 18.46.463298×226 36.3 KB

  • XYZ due [Not Found] after 1 attempt

this is my resolver.json file:

{
    "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-0",
    "data": {
        "cacheSize": 500,
        "repositories": [
            {
                "name": "xyz Iglu Repo",
                "priority": 0,
                "vendorPrefixes": [
                    "au.com.xyz"
                ],
                "connection": {
                    "http": {
                        "uri": "https://snowplow-schema-lphvsidewdq-uc.a.run.app"
                    }
                }
            }
        ]
    }
}

The docker image was tagged into the cloud run job along with the config
[both config and resolver were base64 encoded]

config.json:

{
  "schema": "iglu:com.snowplowanalytics.snowplow/recovery_config/jsonschema/1-0-0",
  "data": {
    "iglu:com.snowplowanalytics.snowplow.badrows/schema_violations/jsonschema/2-0-0": [
      {
        "name":"ReplaceInBase64FieldInBody",
        "conditions":[
          {
            "op": "Test",
            "path":"$.payload.enriched.page_title", 
            "value":{
              "eq":"Budget 2023 Winners and losers"
            }
          }
        ],
        "steps":[
          {
            "op":"Replace",
            "path":"$.raw.parameters.cx.data.[?(@.schema=~iglu:au.com.xyz/metadata/jsonschema/1-0-7)].data.page.authors",
            "match":"dwdq dwdqd",
            "value":["dwdq dwdqd"] 
          }
        ]

      },
      {
        "name": "PassThrough",
        "conditions": [],
        "steps": []
      }
    ]  
  }

}

I was trying to convert the authors name which comes in as a string to an array.

Any help will be appreciated.

Thanks.

2

@emeritus_motet , for recovery, you need Iglu Central in your resolver, no need for custom Iglu.

{
  "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-2",
  "data": {
    "cacheSize": 500,
    "cacheTtl": 600,
    "repositories": [
      {
        "name": "Iglu Central",
        "priority": 1,
        "vendorPrefixes": [
          "com.snowplowanalytics"
        ],
        "connection": {
          "http": {
            "uri": "http://iglucentral.com"
          }
        }
      },
      {
        "name": "Iglu Central - Mirror 01",
        "priority": 1,
        "vendorPrefixes": [
          "com.snowplowanalytics"
        ],
        "connection": {
          "http": {
            "uri": "http://mirror01.iglucentral.com"
          }
        }
      }
    ]
  }
}
2 Likes
3

Hey @ihor
Thanks for the response,
I did add in the iglu resolver into the resolver file but keep getting the same error

Screenshot 2023-05-12 at 10.02.14
Screenshot 2023-05-12 at 10.02.141920×193 33.7 KB

4

What schema Iglu URI is it trying to resolve?

5 
{
  "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-2",
  "data": {
    "cacheSize": 500,
    "cacheTtl": 600,
    "repositories": [
      {
        "name": "Iglu Central",
        "priority": 1,
        "vendorPrefixes": [
          "com.snowplowanalytics"
        ],
        "connection": {
          "http": {
            "uri": "http://iglucentral.com"
          }
        }
      },
      {
        "name": "Iglu Central - Mirror 01",
        "priority": 1,
        "vendorPrefixes": [
          "com.snowplowanalytics"
        ],
        "connection": {
          "http": {
            "uri": "http://mirror01.iglucentral.com"
          }
        }
      }
    ]
  }
}
6

So this is the resolver file which is used by enrich to try and find where the schema is stored. At the moment it is searching for a Iglu URI (not sure which one, perhaps you can specify or you have it in your error message) but it is saying that it can’t find this URI in the repos you’ve specified (Iglu Central and the Iglu Central mirror). If you are using a custom schema then this may be the reason why it is not finding the definition in Iglu Central.

7

Hey Mike,
These are the two uri’s I am currently using along with the custom schema separately pointing to the local registry.

Wasn’t having this issue previously.
Logs don’t specify much expect for the above errors.