Adapter failure error

For engineers
1

Hi, does anybody know what does the below error means ?

{"schema":"iglu:com.snowplowanalytics.snowplow.badrows/adapter_failures/jsonschema/1-0-0","data":{"processor":{"artifact":"snowplow-enrich-kafka","version":"3.9.0"},"failure":{"timestamp":"2025-01-13T11:11:06.515902Z","vendor":"cdn-cgi","version":"trace","messages":[{"field":"vendor/version","value":"cdn-cgi/trace","expectation":"vendor/version combination is not supported"}]},"payload":{"vendor":"cdn-cgi","version":"trace","querystring":[],"contentType":null,"body":null,"collector":"ssc-3.0.1-kafkasink","encoding":"UTF-8","hostname":null,"timestamp":"2025-01-13T11:11:05.967Z","ipAddress":"178.128.54.126","useragent":"Mozilla/5.0","refererUri":null,"headers":["X-FORWARDED-PROTO: http","X-FORWARDED-PORT: 80","X-Forwarded-For: 178.128.54.126:43170","X-Original-URL: /cdn-cgi/trace","Connection: keep-alive","X-AppGW-Trace-Id: 943321b21ce5170d6eb01a3d47baa531","Host: 10.0.20.5:8080","X-ORIGINAL-HOST: speed.cloudflare.com","User-Agent: Mozilla/5.0","Accept-Encoding: gzip"],"networkUserId":"56666c6a-8888-4cc0-b5df-cd08cc9ff863"}}}

2

Hi @Jaspreet_Singh,
the official documentation is pretty clear about this failure type, have you already found it? Understanding failed events | Snowplow Documentation

I can confirm the quite a lot of these failed events are caused by bots.

3

Hi @davidher_mann , How to resolve this issue ?

4

Its not really an issue so to speak - nothing is wrong with the pipeline - it is correctly rejecting invalid data being pushed to it and sending it to the “failed” queue.

If you want to prevent that kind of data hitting the pipeline at all then investment in a proxy upstream / WAF is likely the way to go so you can filter our illegitimate traffic.

5

@Jaspreet_Singh We have encountered this in our pipelines. Our pipelines write failed event json to s3 and then load to redshift, then, to handle these events caused by bots, we delete anything from the failed events table that has a null unstruct_event

6

@josh @Alex_Kudlick @davidher_mann This is my schema :

{
  "$schema": "iglu:com.relevancestudio/request-event/jsonschema/1-0-0",
  "self": {
    "vendor": "com.relevancestudio",
    "name": "request-event",
    "format": "jsonschema",
    "version": "1-0-0"
  },
  "type": "object",
  "properties": {
    "endpoint": {
      "type": "string"
    },
    "method": {
      "type": "string"
    },
    "headers": {
      "type": "object"
    }
  },
  "required": [
    "endpoint",
    "method",
    "headers"
  ],
  "additionalProperties": true
}

Can you tell me if there’s something wrong with my schema that’s causing this error? Could it be due to the vendor or version?

7

The metaschema (the $schema property) should not be changed from http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#

8

@mike even if i am using custom schema also?

9

Yes - if you are using a custom schema you should not customise the metaschema (the pipeline relies on this for the structure of both Iglu and custom schemas).

10

@mike is there a way to delete or remove schemas ?

11

@also how to add new schemas ot update existing one ?

12

This page covers the API interface for Iglu Server that includes adding, removing and patching schemas: Iglu Server | Snowplow Documentation

13

@mike I am still facing issue.Can you check if am wrong somewhere ?
this is my collector url:

http://snowplow-collector-lb-2366c511-4b8c-ebc6-c0ee-613c6c9786dc.canadacentral.cloudapp.azure.com/

here is the data that is being send to the tracker

 data = {'endpoint': 'http://127.0.0.1:8000/tenant/tenant-config/', 'method': 'POST', 'headers': {'Content-Length': '77', 'Content-Type': 'application/json', 'Host': '127.0.0.1:8000', 'Connection': 'keep-alive', 'Sec-Ch-Ua': '"Not)A;Brand";v="99", "Google Chrome";v="127", "Chromium";v="127"', 'Accept': 'application/json', 'Sec-Ch-Ua-Platform': '"Linux"', 'Sec-Ch-Ua-Mobile': '?0', 'X-Csrftoken': 'zar3vbMxG89EMFinGpRe0iRKPQlfeb6lVx32nE6Ir4irTLaeWvQY7SGO0VPFS9hY', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36', 'Origin': 'http://127.0.0.1:8000', 'Sec-Fetch-Site': 'same-origin', 'Sec-Fetch-Mode': 'cors', 'Sec-Fetch-Dest': 'empty', 'Referer': 'http://127.0.0.1:8000/apidoc/v2/', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Accept-Language': 'en-GB,en-US;q=0.9,en;q=0.8', 'Cookie': 'csrftoken=wxM92DulV6jXhg21qg9UhKZelfEAO8lN'}}

here is the tracker:

    schema = "iglu:com.relevancestudio/request-event/jsonschema/1-0-1"
    track_link_click(tracker=self.tracker, schema=schema, data=data)

   def track_link_click(tracker, schema, data):
    link_click = SelfDescribing(
            SelfDescribingJson(schema,data)
        )
    tracker.track(link_click

)

this is my updated schema:

{
  "$schema": "http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#",
  "self": {
    "vendor": "com.relevancestudio",
    "name": "request-event",
    "format": "jsonschema",
    "version": "1-0-1"
  },
  "type": "object",
  "properties": {
    "endpoint": {
      "type": "string"
    },
    "method": {
      "type": "string"
    },
    "headers": {
      "type": "object",
      "additionalProperties": {
        "type": "string"
      }
    }
  },
  "required": ["endpoint", "method", "headers"],
  "additionalProperties": true
}
14

When you say issue - what issue exactly are you running into?

15

@mike By issue I mean I am still get getting adapter failure error message.By the way what’s wrong with the data variable ?

16

What is the adapter failure error message?

17

@mike This error

{
   "schema":"iglu:com.snowplowanalytics.snowplow.badrows/adapter_failures/jsonschema/1-0-0",
   "data":{
      "processor":{
         "artifact":"snowplow-enrich-kafka",
         "version":"3.9.0"
      },
      "failure":{
         "timestamp":"2025-01-23T05:41:45.437356Z",
         "vendor":"portal",
         "version":"redlion",
         "messages":[
            {
               "field":"vendor/version",
               "value":"portal/redlion",
               "expectation":"vendor/version combination is not supported"
            }
         ]
      },
      "payload":{
         "vendor":"portal",
         "version":"redlion",
         "querystring":[
            
         ],
         "contentType":null,
         "body":null,
         "collector":"ssc-3.0.1-kafkasink",
         "encoding":"UTF-8",
         "hostname":null,
         "timestamp":"2025-01-23T05:41:14.478Z",
         "ipAddress":"172.169.206.224",
         "useragent":"Mozilla/5.0 zgrab/0.x",
         "refererUri":null,
         "headers":[
            "X-FORWARDED-PROTO: http",
            "X-FORWARDED-PORT: 80",
            "X-Forwarded-For: 172.169.206.224:44926",
            "X-Original-URL: /portal/redlion",
            "Connection: keep-alive",
            "X-AppGW-Trace-Id: 115fb47ad2ff3a6967d711a0dec7edd0",
            "Host: 10.0.20.5:8080",
            "X-ORIGINAL-HOST: 4.205.199.97",
            "User-Agent: Mozilla/5.0 zgrab/0.x",
            "Accept: */*",
            "Accept-Encoding: gzip"
         ],
         "networkUserId":"644c7d24-6f82-4ee8-b9f0-5a34d817439c"
      }
   }
}
18

What url are you sending data to (including the path)? This looks like bot traffic rather than a legitimate event.

19

@mike this is my collector url http://snowplow-collector-lb-2366c511-4b8c-ebc6-c0ee-613c6c9786dc.canadacentral.cloudapp.azure.com/

and this is my schema: iglu:com.relevancestudio/request-event/jsonschema/1-0-1

is this you are asking ?

20

That is the hostname, rather than the path. I’d recommend testing first locally with either Snowplow Micro or Snowplow Local to confirm that your events are being captured and validated correctly before moving into production.