Inconsistent Good/Bad Event Labeling with Snowplow Micro

For engineers
1

Hi, I’ve tried to implement the JavaScript tracker on a sample website that I’m hosting through localhost. I’ve sent a trackPageView event and a button-click event that I’ve implemented as a structured event. I’m using snowplow micro.

However, both events are labeled as good and bad at varying times. I haven’t added any custom schema and followed the instructions for basic usage when setting up micro. How do I fix this issue?

For reference, this is my tracker code:

function(p,l,o,w,i,n,g){
if(!p[i]){
p.GlobalSnowplowNamespace=p.GlobalSnowplowNamespace||;
p.GlobalSnowplowNamespace.push(i);
p[i]=function(){
(p[i].q=p[i].q||).push(arguments);
};
p[i].q=p[i].q||;
n=l.createElement(o);
g=l.getElementsByTagName(o)[0];
n.async=1;
n.src=w;
g.parentNode.insertBefore(n,g);
}
}(window,document,“script”,“sp.js”,“snowplow”));

snowplow('newTracker', 'my-tracker', 'http://localhost:8081', {
  appId: 'my-webapp',
  contexts: {
    webPage: true,
    performanceTiming: true
  },
  schema: {
    vendor: 'com.snowplowanalytics.iglu',
    name: 'resolver-config',
    format: 'jsonschema',
    version: '1-0-3'
  },
  iglu: {
    resolver: {
    uri: "https://iglucentral.com"
  }
}

});

// Track a page view event
snowplow('trackPageView');

// Track a button click event
function trackButtonClick() {
  snowplow('trackStructEvent', 'Button', 'Click', 'Button1', null, 0.0, 'iglu:com.example/my-schema/jsonschema');
}
2

Hi @sven,

Could you post the output from Micro? What does the BAD log say? Can it be that one of your events is consistently bad, but the fact that the order of output in Micro is not deterministic hides it? (The tracker can batch events as it sends them out, so things can get reordered.)

A few bits in your code caught my attention.

This does not seem right, or at least I don’t see these settings in the tracker initialization options. What were you hoping for them to achieve?

  schema: {
    vendor: 'com.snowplowanalytics.iglu',
    name: 'resolver-config',
    format: 'jsonschema',
    version: '1-0-3'
  },
  iglu: {
    resolver: {
    uri: "https://iglucentral.com"
  }

This also does not seem right. This method takes an optional argument which is an array of entities to attach to the event, however you are providing a string. (Same question — what were you hoping to achieve?)

, 'iglu:com.example/my-schema/jsonschema')

3

This is the output from the collector:
2023-06-29 11:05:21 [WARN] EventLog - BAD Error while extracting event(s) from collector payload and validating it/them.
2023-06-29 11:12:32 [WARN] EventLog - BAD Error while extracting event(s) from collector payload and validating it/them.

This is after one page view and one button click.

The two parts of my code you pointed out were me trying to fix the bad events, because this problem was there even before adding. I tried adding a resolver configuration that wasn’t created by default (iglucentral.com), since I thought that perhaps the absence of an explicit resolver was causing it. I also tried adding a custom schema (iglu:com.example/my-schema/jsonschema) to my button click events so that they would be paged as such and not just struct events. I did this when the events were marked as good, but it isn’t of much use now.

Neither of these ended up working.

4

Could you also paste the output from Micro API? At localhost:8081/micro/bad.

Trackers don’t validate the events against schemas, so you can’t add a resolver to a tracker. Resolvers are only relevant for backend apps that process the events (Enrich, Loaders, Micro, …).

Structured events can’t have schemas, only self-describing (aka unstructured) events can. See https://docs.snowplow.io/docs/understanding-your-pipeline/events/#structured-events and the section below it, I think that might clarify some doubts.

5

@sven , I’m not quite following your tracking of the structured event

snowplow('trackStructEvent', 'Button', 'Click', 'Button1', null, 0.0, 'iglu:com.example/my-schema/jsonschema');

What is the purpose of the last argument passed to trackStructEvent? Did you mean to attach a context (entity)?

6

Sure, this is from my API:

[
{
“collectorPayload”:{
“api”:{
“vendor”:“com.snowplowanalytics.snowplow”,
“version”:“tp2”
},
“querystring”:[

     ],
     "contentType":"application/json",
     "body":"{\"schema\":\"iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4\",\"data\":[{\"e\":\"se\",\"eid\":\"a02a8084-71f6-4c45-9592-f709b6071ae1\",\"tv\":\"js-3.13.0\",\"tna\":\"my-tracker\",\"aid\":\"my-webapp\",\"p\":\"web\",\"cookie\":\"1\",\"cs\":\"windows-1252\",\"lang\":\"en-US\",\"res\":\"1280x800\",\"cd\":\"24\",\"tz\":\"America/Phoenix\",\"dtm\":\"1688062352112\",\"vp\":\"1280x689\",\"ds\":\"1280x689\",\"vid\":\"21\",\"sid\":\"96c9255c-e9c9-4078-bb4c-229467e5d47a\",\"duid\":\"2a64c3f3-8d7a-4b80-b81b-865ed530af1f\",\"url\":\"http://localhost:90/\",\"cx\":\"eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpjb20uc25vd3Bsb3dhbmFseXRpY3Muc25vd3Bsb3cvd2ViX3BhZ2UvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsiaWQiOiJjYjdhOWYyMi04ZTRkLTQ0NTQtODJlOC04ZTE1NzhiZDg2NTMifX0seyJzY2hlbWEiOiJpZ2x1Om9yZy53My9QZXJmb3JtYW5jZVRpbWluZy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6eyJuYXZpZ2F0aW9uU3RhcnQiOjE2ODgwNjE5MjA0MDcsInJlZGlyZWN0U3RhcnQiOjAsInJlZGlyZWN0RW5kIjowLCJmZXRjaFN0YXJ0IjoxNjg4MDYxOTIwNDExLCJkb21haW5Mb29rdXBTdGFydCI6MTY4ODA2MTkyMDQxNCwiZG9tYWluTG9va3VwRW5kIjoxNjg4MDYxOTIwNDE0LCJjb25uZWN0U3RhcnQiOjE2ODgwNjE5MjA0MTQsInNlY3VyZUNvbm5lY3Rpb25TdGFydCI6MCwiY29ubmVjdEVuZCI6MTY4ODA2MTkyMDQxNCwicmVxdWVzdFN0YXJ0IjoxNjg4MDYxOTIwNDE0LCJyZXNwb25zZVN0YXJ0IjoxNjg4MDYxOTIwNDE3LCJyZXNwb25zZUVuZCI6MTY4ODA2MTkyMDQxNywidW5sb2FkRXZlbnRTdGFydCI6MCwidW5sb2FkRXZlbnRFbmQiOjAsImRvbUxvYWRpbmciOjE2ODgwNjE5MjA0MjAsImRvbUludGVyYWN0aXZlIjoxNjg4MDYxOTIwNDMzLCJkb21Db250ZW50TG9hZGVkRXZlbnRTdGFydCI6MTY4ODA2MTkyMDQzMywiZG9tQ29udGVudExvYWRlZEV2ZW50RW5kIjoxNjg4MDYxOTIwNDMzLCJkb21Db21wbGV0ZSI6MTY4ODA2MTkyMDQ4MCwibG9hZEV2ZW50U3RhcnQiOjE2ODgwNjE5MjA0ODAsImxvYWRFdmVudEVuZCI6MTY4ODA2MTkyMDQ4MH19XX0\",\"stm\":\"1688062352118\"}]}",
     "source":{
        "name":"snowplow-micro-1.7.1-stdout$",
        "encoding":"UTF-8",
        "hostname":"localhost"
     },
     "context":{
        "timestamp":"2023-06-29T18:12:32.131Z",
        "ipAddress":"172.17.0.1",
        "useragent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36",
        "refererUri":"http://localhost:90/",
        "headers":[
           "Timeout-Access: <function1>",
           "Host: localhost:8081",
           "Connection: keep-alive",
           "sec-ch-ua: \"Not.A/Brand\";v=\"8\", \"Chromium\";v=\"114\", \"Google Chrome\";v=\"114\"",
           "sec-ch-ua-platform: \"Windows\"",
           "sec-ch-ua-mobile: ?0",
           "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36",
           "Accept: */*",
           "Origin: http://localhost:90",
           "Sec-Fetch-Site: same-site",
           "Sec-Fetch-Mode: cors",
           "Sec-Fetch-Dest: empty",
           "Referer: http://localhost:90/",
           "Accept-Encoding: gzip, deflate, br",
           "Accept-Language: en-US, en;q=0.9",
           "Cookie: micro=26500323-2adf-4a56-a88e-309753992f82; _sp_ses.1fff=*; _sp_id.1fff=2a64c3f3-8d7a-4b80-b81b-865ed530af1f.1687544513.21.1688062352.1688055006.96c9255c-e9c9-4078-bb4c-229467e5d47a.30173119-cd64-4d5a-9bbb-20282160226a.f50f833c-134b-4d30-b315-c5a5fb8a20f5.1688061920473.2",
           "application/json"
        ],
        "userId":"26500323-2adf-4a56-a88e-309753992f82"
     }
  },
  "rawEvent":null,
  "errors":[
     "Error while extracting event(s) from collector payload and validating it/them.",
     "{\"schema\":\"iglu:com.snowplowanalytics.snowplow.badrows/tracker_protocol_violations/jsonschema/1-0-0\",\"data\":{\"processor\":{\"artifact\":\"snowplow-micro\",\"version\":\"1.7.1\"},\"failure\":{\"timestamp\":\"2023-06-29T18:12:32.441587Z\",\"vendor\":\"com.snowplowanalytics.snowplow\",\"version\":\"tp2\",\"messages\":[{\"schemaKey\":\"iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4\",\"error\":{\"error\":\"ResolutionError\",\"lookupHistory\":[{\"repository\":\"Iglu Central\",\"errors\":[{\"error\":\"NotFound\"}],\"attempts\":2,\"lastAttempt\":\"2023-06-29T18:12:32.437Z\"},{\"repository\":\"Iglu Client Embedded\",\"errors\":[{\"error\":\"NotFound\"}],\"attempts\":1,\"lastAttempt\":\"2023-06-29T18:05:20.654Z\"}]}}]},\"payload\":{\"vendor\":\"com.snowplowanalytics.snowplow\",\"version\":\"tp2\",\"querystring\":[],\"contentType\":\"application/json\",\"body\":\"{\\\"schema\\\":\\\"iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4\\\",\\\"data\\\":[{\\\"e\\\":\\\"se\\\",\\\"eid\\\":\\\"a02a8084-71f6-4c45-9592-f709b6071ae1\\\",\\\"tv\\\":\\\"js-3.13.0\\\",\\\"tna\\\":\\\"my-tracker\\\",\\\"aid\\\":\\\"my-webapp\\\",\\\"p\\\":\\\"web\\\",\\\"cookie\\\":\\\"1\\\",\\\"cs\\\":\\\"windows-1252\\\",\\\"lang\\\":\\\"en-US\\\",\\\"res\\\":\\\"1280x800\\\",\\\"cd\\\":\\\"24\\\",\\\"tz\\\":\\\"America/Phoenix\\\",\\\"dtm\\\":\\\"1688062352112\\\",\\\"vp\\\":\\\"1280x689\\\",\\\"ds\\\":\\\"1280x689\\\",\\\"vid\\\":\\\"21\\\",\\\"sid\\\":\\\"96c9255c-e9c9-4078-bb4c-229467e5d47a\\\",\\\"duid\\\":\\\"2a64c3f3-8d7a-4b80-b81b-865ed530af1f\\\",\\\"url\\\":\\\"http://localhost:90/\\\",\\\"cx\\\":\\\"eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpjb20uc25vd3Bsb3dhbmFseXRpY3Muc25vd3Bsb3cvd2ViX3BhZ2UvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsiaWQiOiJjYjdhOWYyMi04ZTRkLTQ0NTQtODJlOC04ZTE1NzhiZDg2NTMifX0seyJzY2hlbWEiOiJpZ2x1Om9yZy53My9QZXJmb3JtYW5jZVRpbWluZy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6eyJuYXZpZ2F0aW9uU3RhcnQiOjE2ODgwNjE5MjA0MDcsInJlZGlyZWN0U3RhcnQiOjAsInJlZGlyZWN0RW5kIjowLCJmZXRjaFN0YXJ0IjoxNjg4MDYxOTIwNDExLCJkb21haW5Mb29rdXBTdGFydCI6MTY4ODA2MTkyMDQxNCwiZG9tYWluTG9va3VwRW5kIjoxNjg4MDYxOTIwNDE0LCJjb25uZWN0U3RhcnQiOjE2ODgwNjE5MjA0MTQsInNlY3VyZUNvbm5lY3Rpb25TdGFydCI6MCwiY29ubmVjdEVuZCI6MTY4ODA2MTkyMDQxNCwicmVxdWVzdFN0YXJ0IjoxNjg4MDYxOTIwNDE0LCJyZXNwb25zZVN0YXJ0IjoxNjg4MDYxOTIwNDE3LCJyZXNwb25zZUVuZCI6MTY4ODA2MTkyMDQxNywidW5sb2FkRXZlbnRTdGFydCI6MCwidW5sb2FkRXZlbnRFbmQiOjAsImRvbUxvYWRpbmciOjE2ODgwNjE5MjA0MjAsImRvbUludGVyYWN0aXZlIjoxNjg4MDYxOTIwNDMzLCJkb21Db250ZW50TG9hZGVkRXZlbnRTdGFydCI6MTY4ODA2MTkyMDQzMywiZG9tQ29udGVudExvYWRlZEV2ZW50RW5kIjoxNjg4MDYxOTIwNDMzLCJkb21Db21wbGV0ZSI6MTY4ODA2MTkyMDQ4MCwibG9hZEV2ZW50U3RhcnQiOjE2ODgwNjE5MjA0ODAsImxvYWRFdmVudEVuZCI6MTY4ODA2MTkyMDQ4MH19XX0\\\",\\\"stm\\\":\\\"1688062352118\\\"}]}\",\"collector\":\"snowplow-micro-1.7.1-stdout$\",\"encoding\":\"UTF-8\",\"hostname\":\"localhost\",\"timestamp\":\"2023-06-29T18:12:32.131Z\",\"ipAddress\":\"172.17.0.1\",\"useragent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36\",\"refererUri\":\"http://localhost:90/\",\"headers\":[\"Timeout-Access: <function1>\",\"Host: localhost:8081\",\"Connection: keep-alive\",\"sec-ch-ua: \\\"Not.A/Brand\\\";v=\\\"8\\\", \\\"Chromium\\\";v=\\\"114\\\", \\\"Google Chrome\\\";v=\\\"114\\\"\",\"sec-ch-ua-platform: \\\"Windows\\\"\",\"sec-ch-ua-mobile: ?0\",\"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36\",\"Accept: */*\",\"Origin: http://localhost:90\",\"Sec-Fetch-Site: same-site\",\"Sec-Fetch-Mode: cors\",\"Sec-Fetch-Dest: empty\",\"Referer: http://localhost:90/\",\"Accept-Encoding: gzip, deflate, br\",\"Accept-Language: en-US, en;q=0.9\",\"Cookie: micro=26500323-2adf-4a56-a88e-309753992f82; _sp_ses.1fff=*; _sp_id.1fff=2a64c3f3-8d7a-4b80-b81b-865ed530af1f.1687544513.21.1688062352.1688055006.96c9255c-e9c9-4078-bb4c-229467e5d47a.30173119-cd64-4d5a-9bbb-20282160226a.f50f833c-134b-4d30-b315-c5a5fb8a20f5.1688061920473.2\",\"application/json\"],\"networkUserId\":\"26500323-2adf-4a56-a88e-309753992f82\"}}}"
  ]

},
{
“collectorPayload”:{
“api”:{
“vendor”:“com.snowplowanalytics.snowplow”,
“version”:“tp2”
},
“querystring”:[

     ],
     "contentType":"application/json",
     "body":"{\"schema\":\"iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4\",\"data\":[{\"e\":\"pv\",\"url\":\"http://localhost:90/\",\"page\":\"My Web Page\",\"eid\":\"f50f833c-134b-4d30-b315-c5a5fb8a20f5\",\"tv\":\"js-3.13.0\",\"tna\":\"my-tracker\",\"aid\":\"my-webapp\",\"p\":\"web\",\"cookie\":\"1\",\"cs\":\"windows-1252\",\"lang\":\"en-US\",\"res\":\"1280x800\",\"cd\":\"24\",\"tz\":\"America/Phoenix\",\"dtm\":\"1688061920473\",\"cx\":\"eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpjb20uc25vd3Bsb3dhbmFseXRpY3Muc25vd3Bsb3cvd2ViX3BhZ2UvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsiaWQiOiJjYjdhOWYyMi04ZTRkLTQ0NTQtODJlOC04ZTE1NzhiZDg2NTMifX0seyJzY2hlbWEiOiJpZ2x1Om9yZy53My9QZXJmb3JtYW5jZVRpbWluZy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6eyJuYXZpZ2F0aW9uU3RhcnQiOjE2ODgwNjE5MjA0MDcsInJlZGlyZWN0U3RhcnQiOjAsInJlZGlyZWN0RW5kIjowLCJmZXRjaFN0YXJ0IjoxNjg4MDYxOTIwNDExLCJkb21haW5Mb29rdXBTdGFydCI6MTY4ODA2MTkyMDQxNCwiZG9tYWluTG9va3VwRW5kIjoxNjg4MDYxOTIwNDE0LCJjb25uZWN0U3RhcnQiOjE2ODgwNjE5MjA0MTQsInNlY3VyZUNvbm5lY3Rpb25TdGFydCI6MCwiY29ubmVjdEVuZCI6MTY4ODA2MTkyMDQxNCwicmVxdWVzdFN0YXJ0IjoxNjg4MDYxOTIwNDE0LCJyZXNwb25zZVN0YXJ0IjoxNjg4MDYxOTIwNDE3LCJyZXNwb25zZUVuZCI6MTY4ODA2MTkyMDQxNywidW5sb2FkRXZlbnRTdGFydCI6MCwidW5sb2FkRXZlbnRFbmQiOjAsImRvbUxvYWRpbmciOjE2ODgwNjE5MjA0MjAsImRvbUludGVyYWN0aXZlIjoxNjg4MDYxOTIwNDMzLCJkb21Db250ZW50TG9hZGVkRXZlbnRTdGFydCI6MTY4ODA2MTkyMDQzMywiZG9tQ29udGVudExvYWRlZEV2ZW50RW5kIjoxNjg4MDYxOTIwNDMzLCJkb21Db21wbGV0ZSI6MCwibG9hZEV2ZW50U3RhcnQiOjAsImxvYWRFdmVudEVuZCI6MH19XX0\",\"vp\":\"1280x689\",\"ds\":\"1280x689\",\"vid\":\"21\",\"sid\":\"96c9255c-e9c9-4078-bb4c-229467e5d47a\",\"duid\":\"2a64c3f3-8d7a-4b80-b81b-865ed530af1f\",\"stm\":\"1688061920479\"}]}",
     "source":{
        "name":"snowplow-micro-1.7.1-stdout$",
        "encoding":"UTF-8",
        "hostname":"localhost"
     },
     "context":{
        "timestamp":"2023-06-29T18:05:20.523Z",
        "ipAddress":"172.17.0.1",
        "useragent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36",
        "refererUri":"http://localhost:90/",
        "headers":[
           "Timeout-Access: <function1>",
           "Host: localhost:8081",
           "Connection: keep-alive",
           "sec-ch-ua: \"Not.A/Brand\";v=\"8\", \"Chromium\";v=\"114\", \"Google Chrome\";v=\"114\"",
           "sec-ch-ua-platform: \"Windows\"",
           "Sec-Purpose: prefetch;prerender",
           "sec-ch-ua-mobile: ?0",
           "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36",
           "Accept: */*",
           "Origin: http://localhost:90",
           "Purpose: prefetch",
           "Sec-Fetch-Site: same-site",
           "Sec-Fetch-Mode: cors",
           "Sec-Fetch-Dest: empty",
           "Referer: http://localhost:90/",
           "Accept-Encoding: gzip, deflate, br",
           "Accept-Language: en-US, en;q=0.9",
           "Cookie: micro=26500323-2adf-4a56-a88e-309753992f82; _sp_ses.1fff=*; _sp_id.1fff=2a64c3f3-8d7a-4b80-b81b-865ed530af1f.1687544513.21.1688061920.1688055006.96c9255c-e9c9-4078-bb4c-229467e5d47a.30173119-cd64-4d5a-9bbb-20282160226a.f50f833c-134b-4d30-b315-c5a5fb8a20f5.1688061920473.1",
           "application/json"
        ],
        "userId":"26500323-2adf-4a56-a88e-309753992f82"
     }
  },
  "rawEvent":null,
  "errors":[
     "Error while extracting event(s) from collector payload and validating it/them.",
     "{\"schema\":\"iglu:com.snowplowanalytics.snowplow.badrows/tracker_protocol_violations/jsonschema/1-0-0\",\"data\":{\"processor\":{\"artifact\":\"snowplow-micro\",\"version\":\"1.7.1\"},\"failure\":{\"timestamp\":\"2023-06-29T18:05:21.219373Z\",\"vendor\":\"com.snowplowanalytics.snowplow\",\"version\":\"tp2\",\"messages\":[{\"schemaKey\":\"iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4\",\"error\":{\"error\":\"ResolutionError\",\"lookupHistory\":[{\"repository\":\"Iglu Central\",\"errors\":[{\"error\":\"NotFound\"}],\"attempts\":1,\"lastAttempt\":\"2023-06-29T18:05:21.203Z\"},{\"repository\":\"Iglu Client Embedded\",\"errors\":[{\"error\":\"NotFound\"}],\"attempts\":1,\"lastAttempt\":\"2023-06-29T18:05:20.654Z\"}]}}]},\"payload\":{\"vendor\":\"com.snowplowanalytics.snowplow\",\"version\":\"tp2\",\"querystring\":[],\"contentType\":\"application/json\",\"body\":\"{\\\"schema\\\":\\\"iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4\\\",\\\"data\\\":[{\\\"e\\\":\\\"pv\\\",\\\"url\\\":\\\"http://localhost:90/\\\",\\\"page\\\":\\\"My Web Page\\\",\\\"eid\\\":\\\"f50f833c-134b-4d30-b315-c5a5fb8a20f5\\\",\\\"tv\\\":\\\"js-3.13.0\\\",\\\"tna\\\":\\\"my-tracker\\\",\\\"aid\\\":\\\"my-webapp\\\",\\\"p\\\":\\\"web\\\",\\\"cookie\\\":\\\"1\\\",\\\"cs\\\":\\\"windows-1252\\\",\\\"lang\\\":\\\"en-US\\\",\\\"res\\\":\\\"1280x800\\\",\\\"cd\\\":\\\"24\\\",\\\"tz\\\":\\\"America/Phoenix\\\",\\\"dtm\\\":\\\"1688061920473\\\",\\\"cx\\\":\\\"eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpjb20uc25vd3Bsb3dhbmFseXRpY3Muc25vd3Bsb3cvd2ViX3BhZ2UvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsiaWQiOiJjYjdhOWYyMi04ZTRkLTQ0NTQtODJlOC04ZTE1NzhiZDg2NTMifX0seyJzY2hlbWEiOiJpZ2x1Om9yZy53My9QZXJmb3JtYW5jZVRpbWluZy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6eyJuYXZpZ2F0aW9uU3RhcnQiOjE2ODgwNjE5MjA0MDcsInJlZGlyZWN0U3RhcnQiOjAsInJlZGlyZWN0RW5kIjowLCJmZXRjaFN0YXJ0IjoxNjg4MDYxOTIwNDExLCJkb21haW5Mb29rdXBTdGFydCI6MTY4ODA2MTkyMDQxNCwiZG9tYWluTG9va3VwRW5kIjoxNjg4MDYxOTIwNDE0LCJjb25uZWN0U3RhcnQiOjE2ODgwNjE5MjA0MTQsInNlY3VyZUNvbm5lY3Rpb25TdGFydCI6MCwiY29ubmVjdEVuZCI6MTY4ODA2MTkyMDQxNCwicmVxdWVzdFN0YXJ0IjoxNjg4MDYxOTIwNDE0LCJyZXNwb25zZVN0YXJ0IjoxNjg4MDYxOTIwNDE3LCJyZXNwb25zZUVuZCI6MTY4ODA2MTkyMDQxNywidW5sb2FkRXZlbnRTdGFydCI6MCwidW5sb2FkRXZlbnRFbmQiOjAsImRvbUxvYWRpbmciOjE2ODgwNjE5MjA0MjAsImRvbUludGVyYWN0aXZlIjoxNjg4MDYxOTIwNDMzLCJkb21Db250ZW50TG9hZGVkRXZlbnRTdGFydCI6MTY4ODA2MTkyMDQzMywiZG9tQ29udGVudExvYWRlZEV2ZW50RW5kIjoxNjg4MDYxOTIwNDMzLCJkb21Db21wbGV0ZSI6MCwibG9hZEV2ZW50U3RhcnQiOjAsImxvYWRFdmVudEVuZCI6MH19XX0\\\",\\\"vp\\\":\\\"1280x689\\\",\\\"ds\\\":\\\"1280x689\\\",\\\"vid\\\":\\\"21\\\",\\\"sid\\\":\\\"96c9255c-e9c9-4078-bb4c-229467e5d47a\\\",\\\"duid\\\":\\\"2a64c3f3-8d7a-4b80-b81b-865ed530af1f\\\",\\\"stm\\\":\\\"1688061920479\\\"}]}\",\"collector\":\"snowplow-micro-1.7.1-stdout$\",\"encoding\":\"UTF-8\",\"hostname\":\"localhost\",\"timestamp\":\"2023-06-29T18:05:20.523Z\",\"ipAddress\":\"172.17.0.1\",\"useragent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36\",\"refererUri\":\"http://localhost:90/\",\"headers\":[\"Timeout-Access: <function1>\",\"Host: localhost:8081\",\"Connection: keep-alive\",\"sec-ch-ua: \\\"Not.A/Brand\\\";v=\\\"8\\\", \\\"Chromium\\\";v=\\\"114\\\", \\\"Google Chrome\\\";v=\\\"114\\\"\",\"sec-ch-ua-platform: \\\"Windows\\\"\",\"Sec-Purpose: prefetch;prerender\",\"sec-ch-ua-mobile: ?0\",\"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36\",\"Accept: */*\",\"Origin: http://localhost:90\",\"Purpose: prefetch\",\"Sec-Fetch-Site: same-site\",\"Sec-Fetch-Mode: cors\",\"Sec-Fetch-Dest: empty\",\"Referer: http://localhost:90/\",\"Accept-Encoding: gzip, deflate, br\",\"Accept-Language: en-US, en;q=0.9\",\"Cookie: micro=26500323-2adf-4a56-a88e-309753992f82; _sp_ses.1fff=*; _sp_id.1fff=2a64c3f3-8d7a-4b80-b81b-865ed530af1f.1687544513.21.1688061920.1688055006.96c9255c-e9c9-4078-bb4c-229467e5d47a.30173119-cd64-4d5a-9bbb-20282160226a.f50f833c-134b-4d30-b315-c5a5fb8a20f5.1688061920473.1\",\"application/json\"],\"networkUserId\":\"26500323-2adf-4a56-a88e-309753992f82\"}}}"
  ]

}
]

I understand your point about the structured events. As for resolvers, is there no reason to manually include one then? I assume Micro already has a default one.

7

I meant to add a link to a custom schema that I could use to validate my button-clicking events, because they show up as just type struct. It looks like I can’t do this though since I’ve defined it as a structured event, so I might make it unstructured instead.

8

The reason for bad data is inability to locate a JSON schema on the Iglu server

{
  "error": "ResolutionError",
  "lookupHistory": [
    {
      "repository": "Iglu Central",
      "errors": [
        {
          "error": "NotFound"
        }
      ],
      "attempts": 2,
      "lastAttempt": "2023-06-29T18:12:32.437Z"
    },
    {
      "repository": "Iglu Client Embedded",
      "errors": [
        {
          "error": "NotFound"
        }
      ],
      "attempts": 1,
      "lastAttempt": "2023-06-29T18:05:20.654Z"
    }
  ]
}

Perhaps it cannot be accessed via HTTPS from your environment? Are you using http://iglucentral.com in your resolver?

9

Yes, you only need a custom resolver if you have a custom Iglu repository where you placed you custom schema(s).

To @ihor’s point, it sounds like your Micro resolver is actually malformed, as Micro struggles to locate the default schemas it needs to process events.

How are you running Micro? What configuration are you passing to it? For your example, the simple docker command at the very top of https://docs.snowplow.io/docs/getting-started-with-micro/basic-usage/#running should be sufficient.

10

I used the instructions in the basic usage section, so just this:

docker run -p 8081:9090 snowplow/snowplow-micro:1.7.1

11

I have a resolver file that includes it, but I didn’t use it to run the container. Here are the contents:

{
“schema”: “iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-3”,
“data”: {
“cacheSize”: 500,
“repositories”: [
{
“name”: “Iglu Central”,
“priority”: 0,
“vendorPrefixes”: [ “com.snowplowanalytics” ],
“connection”: {
“http”: {
“uri”: “http://iglucentral.com”
}
}
}
]
}
}

12

My problem actually started when I set up Snowplow Micro using the basic usage instructions. All of the custom things I added were attempts to fix the problem, but nothing worked.

I’m not sure if the issue lies with the Snowplow Micro setup or with my tracker code.

13

An update: the events have now switched to good, and here is what the log for a page view event looks like:

{
“rawEvent”:{
“api”:{
“vendor”:“com.snowplowanalytics.snowplow”,
“version”:“tp2”
},
“parameters”:{
“e”:“pv”,
“duid”:“2a64c3f3-8d7a-4b80-b81b-865ed530af1f”,
“vid”:“28”,
“eid”:“fdd20b4b-b532-402a-90af-4afeade07c7b”,
“url”:“http://localhost:90/”,
“aid”:“my-webapp”,
“cx”:“eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpjb20uc25vd3Bsb3dhbmFseXRpY3Muc25vd3Bsb3cvd2ViX3BhZ2UvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsiaWQiOiJiZjJlZDUyYy1iNTQ4LTQzMDYtOWZkYy1kOTI1NDJmZDlmZmQifX0seyJzY2hlbWEiOiJpZ2x1Om9yZy53My9QZXJmb3JtYW5jZVRpbWluZy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6eyJuYXZpZ2F0aW9uU3RhcnQiOjE2ODgyNjU4OTgwNDksInJlZGlyZWN0U3RhcnQiOjAsInJlZGlyZWN0RW5kIjowLCJmZXRjaFN0YXJ0IjoxNjg4MjY1ODk4MDUyLCJkb21haW5Mb29rdXBTdGFydCI6MTY4ODI2NTg5ODA1MiwiZG9tYWluTG9va3VwRW5kIjoxNjg4MjY1ODk4MDUyLCJjb25uZWN0U3RhcnQiOjE2ODgyNjU4OTgwNTIsInNlY3VyZUNvbm5lY3Rpb25TdGFydCI6MCwiY29ubmVjdEVuZCI6MTY4ODI2NTg5ODA1MiwicmVxdWVzdFN0YXJ0IjoxNjg4MjY1ODk4MDU1LCJyZXNwb25zZVN0YXJ0IjoxNjg4MjY1ODk4MDU2LCJyZXNwb25zZUVuZCI6MTY4ODI2NTg5ODA1NywidW5sb2FkRXZlbnRTdGFydCI6MTY4ODI2NTg5ODA1OSwidW5sb2FkRXZlbnRFbmQiOjE2ODgyNjU4OTgwNTksImRvbUxvYWRpbmciOjE2ODgyNjU4OTgwNjEsImRvbUludGVyYWN0aXZlIjoxNjg4MjY1ODk4MDY2LCJkb21Db250ZW50TG9hZGVkRXZlbnRTdGFydCI6MTY4ODI2NTg5ODA2NiwiZG9tQ29udGVudExvYWRlZEV2ZW50RW5kIjoxNjg4MjY1ODk4MDY2LCJkb21Db21wbGV0ZSI6MCwibG9hZEV2ZW50U3RhcnQiOjAsImxvYWRFdmVudEVuZCI6MH19XX0”,
“tna”:“my-tracker”,
“cs”:“windows-1252”,
“cd”:“24”,
“page”:“My Web Page”,
“stm”:“1688265898086”,
“tz”:“America/Los_Angeles”,
“tv”:“js-3.13.0”,
“vp”:“725x689”,
“ds”:“725x689”,
“res”:“1280x800”,
“cookie”:“1”,
“p”:“web”,
“dtm”:“1688265898085”,
“lang”:“en-US”,
“sid”:“a6df13bc-5624-40ab-a25b-dfab73586721”
},
“contentType”:“application/json”,
“source”:{
“name”:“snowplow-micro-1.7.1-stdout$”,
“encoding”:“UTF-8”,
“hostname”:“localhost”
},
“context”:{
“timestamp”:“2023-07-02T02:44:58.112Z”,
“ipAddress”:“172.17.0.1”,
“useragent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”,
“refererUri”:“http://localhost:90/”,
“headers”:[
“Timeout-Access: ”,
“Host: localhost:8081”,
“Connection: keep-alive”,
“sec-ch-ua: "Not.A/Brand";v="8", "Chromium";v="114", "Google Chrome";v="114"”,
“sec-ch-ua-platform: "Windows"”,
“sec-ch-ua-mobile: ?0”,
“User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”,
“Accept: /”,
“Origin: http://localhost:90”,
“Sec-Fetch-Site: same-site”,
“Sec-Fetch-Mode: cors”,
“Sec-Fetch-Dest: empty”,
“Referer: http://localhost:90/”,
“Accept-Encoding: gzip, deflate, br”,
“Accept-Language: en-US, en;q=0.9”,
“Cookie: micro=26500323-2adf-4a56-a88e-309753992f82; _sp_ses.1fff=*; _sp_id.1fff=2a64c3f3-8d7a-4b80-b81b-865ed530af1f.1687544513.28.1688265898.1688256278.a6df13bc-5624-40ab-a25b-dfab73586721.e3d837e4-81f2-463f-923a-98e9b28f99a8.30608784-5fed-4741-8115-d176e2cf7886.1688264722499.7”,
“application/json”
],
“userId”:“26500323-2adf-4a56-a88e-309753992f82”
}
},
“eventType”:“page_view”,
“schema”:“iglu:com.snowplowanalytics.snowplow/page_view/jsonschema/1-0-0”,
“contexts”:[
“iglu:com.snowplowanalytics.snowplow/web_page/jsonschema/1-0-0”,
“iglu:org.w3/PerformanceTiming/jsonschema/1-0-0”
],
“event”:{
“app_id”:“my-webapp”,
“platform”:“web”,
“etl_tstamp”:“2023-07-02T02:44:58.115Z”,
“collector_tstamp”:“2023-07-02T02:44:58.112Z”,
“dvce_created_tstamp”:“2023-07-02T02:44:58.085Z”,
“event”:“page_view”,
“event_id”:“fdd20b4b-b532-402a-90af-4afeade07c7b”,
“txn_id”:null,
“name_tracker”:“my-tracker”,
“v_tracker”:“js-3.13.0”,
“v_collector”:“snowplow-micro-1.7.1-stdout$”,
“v_etl”:“snowplow-micro-1.7.1”,
“user_id”:null,
“user_ipaddress”:“172.17.0.1”,
“user_fingerprint”:null,
“domain_userid”:“2a64c3f3-8d7a-4b80-b81b-865ed530af1f”,
“domain_sessionidx”:28,
“network_userid”:“26500323-2adf-4a56-a88e-309753992f82”,
“geo_country”:null,
“geo_region”:null,
“geo_city”:null,
“geo_zipcode”:null,
“geo_latitude”:null,
“geo_longitude”:null,
“geo_region_name”:null,
“ip_isp”:null,
“ip_organization”:null,
“ip_domain”:null,
“ip_netspeed”:null,
“page_url”:“http://localhost:90/”,
“page_title”:“My Web Page”,
“page_referrer”:null,
“page_urlscheme”:“http”,
“page_urlhost”:“localhost”,
“page_urlport”:90,
“page_urlpath”:“/”,
“page_urlquery”:null,
“page_urlfragment”:null,
“refr_urlscheme”:null,
“refr_urlhost”:null,
“refr_urlport”:null,
“refr_urlpath”:null,
“refr_urlquery”:null,
“refr_urlfragment”:null,
“refr_medium”:null,
“refr_source”:null,
“refr_term”:null,
“mkt_medium”:null,
“mkt_source”:null,
“mkt_term”:null,
“mkt_content”:null,
“mkt_campaign”:null,
“contexts”:{
“schema”:“iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0”,
“data”:[
{
“schema”:“iglu:com.snowplowanalytics.snowplow/web_page/jsonschema/1-0-0”,
“data”:{
“id”:“bf2ed52c-b548-4306-9fdc-d92542fd9ffd”
}
},
{
“schema”:“iglu:org.w3/PerformanceTiming/jsonschema/1-0-0”,
“data”:{
“navigationStart”:1688265898049,
“redirectStart”:0,
“redirectEnd”:0,
“fetchStart”:1688265898052,
“domainLookupStart”:1688265898052,
“domainLookupEnd”:1688265898052,
“connectStart”:1688265898052,
“secureConnectionStart”:0,
“connectEnd”:1688265898052,
“requestStart”:1688265898055,
“responseStart”:1688265898056,
“responseEnd”:1688265898057,
“unloadEventStart”:1688265898059,
“unloadEventEnd”:1688265898059,
“domLoading”:1688265898061,
“domInteractive”:1688265898066,
“domContentLoadedEventStart”:1688265898066,
“domContentLoadedEventEnd”:1688265898066,
“domComplete”:0,
“loadEventStart”:0,
“loadEventEnd”:0
}
}
]
},
“se_category”:null,
“se_action”:null,
“se_label”:null,
“se_property”:null,
“se_value”:null,
“unstruct_event”:null,
“tr_orderid”:null,
“tr_affiliation”:null,
“tr_total”:null,
“tr_tax”:null,
“tr_shipping”:null,
“tr_city”:null,
“tr_state”:null,
“tr_country”:null,
“ti_orderid”:null,
“ti_sku”:null,
“ti_name”:null,
“ti_category”:null,
“ti_price”:null,
“ti_quantity”:null,
“pp_xoffset_min”:null,
“pp_xoffset_max”:null,
“pp_yoffset_min”:null,
“pp_yoffset_max”:null,
“useragent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”,
“br_name”:null,
“br_family”:null,
“br_version”:null,
“br_type”:null,
“br_renderengine”:null,
“br_lang”:“en-US”,
“br_features_pdf”:null,
“br_features_flash”:null,
“br_features_java”:null,
“br_features_director”:null,
“br_features_quicktime”:null,
“br_features_realplayer”:null,
“br_features_windowsmedia”:null,
“br_features_gears”:null,
“br_features_silverlight”:null,
“br_cookies”:true,
“br_colordepth”:“24”,
“br_viewwidth”:725,
“br_viewheight”:689,
“os_name”:null,
“os_family”:null,
“os_manufacturer”:null,
“os_timezone”:“America/Los_Angeles”,
“dvce_type”:null,
“dvce_ismobile”:null,
“dvce_screenwidth”:1280,
“dvce_screenheight”:800,
“doc_charset”:“windows-1252”,
“doc_width”:725,
“doc_height”:689,
“tr_currency”:null,
“tr_total_base”:null,
“tr_tax_base”:null,
“tr_shipping_base”:null,
“ti_currency”:null,
“ti_price_base”:null,
“base_currency”:null,
“geo_timezone”:null,
“mkt_clickid”:null,
“mkt_network”:null,
“etl_tags”:null,
“dvce_sent_tstamp”:“2023-07-02T02:44:58.086Z”,
“refr_domain_userid”:null,
“refr_dvce_tstamp”:null,
“derived_contexts”:{

  },
  "domain_sessionid":"a6df13bc-5624-40ab-a25b-dfab73586721",
  "derived_tstamp":"2023-07-02T02:44:58.111Z",
  "event_vendor":"com.snowplowanalytics.snowplow",
  "event_name":"page_view",
  "event_format":"jsonschema",
  "event_version":"1-0-0",
  "event_fingerprint":null,
  "true_tstamp":null

}
}

However, I expect the same event to turn bad later. Can anyone use the information here to help me with what might be causing the issue?

14

Why do you expect this event to turn bad later? It looks like a well-formed and validating event so in theory it shouldn’t go bad unless the contents of the event is modified (or the configuration of Micro).

15

I have a suspicion that there was some problem with Iglu Central schema repository. We’ll add a mirror or two to the default Micro config to prevent this issue in the future.

16

Hi! We’ve solved our issue, it looks like we had to deal with occasional firewalls when accessing Iglu Central, and it was during those times that our events would go bad, so we had to make sure that access was allowed each time. Everything is good otherwise!

17

Thanks for the update, glad it’s all working for you now!

1 Like