Getting an error parsing Iglu schema

[waspesi]

We are getting this error when trying to pass the following schema:

https://datascience-iglu-repo.cxawsnprd.massmutual.com/com.snowplowanalytics.mm/click_event/jsonschema/1-0-0

“error",“message”:“error: Problem parsing iglu:com.snowplowanalytics.mm/click_event/jsonschema/1-0-0 as JSON in HTTP Iglu repository Custom Schemas: Unexpected character (’<’ (code 60)): expected a valid value (number, String, array, object, ‘true’, ‘false’ or ‘null’) at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@xxxxxx; line: 1, column: 2]\n level: “error”\n”},{“level”:“error”,“message”:“error: Problem parsing iglu:com.snowplowanalytics.mm/click_event/jsonschema/1-0-0 as JSON in HTTP Iglu repository Custom Schemas: Unexpected character (’<’ (code 60)): expected a valid value (number, String, array, object, ‘true’, ‘false’ or ‘null’) at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@xxxxxx; line: 1, column: 2]\n level: “error”\n”},{“level”:“error”,“message”:"error: Problem parsing iglu:com.snowplowanalytics.mm/click_event/jsonschema/1-0-0 as JSON in HTTP Iglu repository Custom Schemas: Unexpected character (’<’ (code 60)): expected a valid value (number, String, array, object, ‘true’, ‘false’ or ‘null’) at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@xxxxxx; line: 1, column: 2]\n level: "error”

here is the cx output which is being sent to the bad folder and not being enriched:

{
“schema”:“iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0”,
“data”:[ <------
{
“schema”:“iglu:com.snowplowanalytics.mm/click_event/jsonschema/1-0-0”,
“data”:{
“pageName”:“Home”,
“sc_audience”:“prospect”,
“authentication_status”:“never”,
“sc_logged_in_status”:“logged out”,
“tealium_session_id”:“1512472817916”,
“tealium_visitor_id”:“01602667b0fc0022adcd00eef33c0c048001e00d009d4”,
“content_type”:“Home”,
“site_id”:“MM-Dotcom”
}
}
]
}

Is this an issue that its looking for the string to be an object vs an array?

Regards,
Wayne

[Colm]

Hi Wayne,

This part of the error message:

“message”:"error: Problem parsing iglu:com.snowplowanalytics.mm/click_event/jsonschema/1-0-0 as JSON in HTTP Iglu repository Custom Schemas: Unexpected character (’<’ (code 60)): expected a valid value (number, String, array, object, ‘true’, ‘false’ or ‘null’

tells me it expected a value, but it got a < character. In this case it looks like you just need to send the JSON data object with no extraneous characters.

I would check the tracker for how the data is being sent. There should be no extraneous characters inside the []. IN this case it looks like these characters are being sent too: <------.

Best,
colm

[waspesi]

Sorry the <------ was added by mistake on my side and really not part of the call. I’ve attached the tracker config im using, please take a look. I used also tested the schema vs the json out put expected and did not see any error.

Tealium Config:

//~~tv:20010.20140827
//~~tc: Tealium Custom Container

/*
Tealium Custom Container Notes:

• Add sending code between “Start Tag Sending Code” and “End Tag Sending Code”.
• Add JavaScript tag library code between “Start Tag Library Code” and “End Tag Library Code”.
• Add JavaScript code only, do not add HTML code in this file.
• Remove any tags from the code you place in this file.

Loading external JavaScript files (Loader):

• If you need to load an additional external JavaScript file, un-comment the singe-line JavaScript comments ("//") within the following Loader sections near the bottom of this file:
• “Start Loader Function Call”
• “End Loader Function Call”
• “Start Loader Callback Function”
• “End Loader Callback Function”
• After un-commenting, insert the path to the external JavaScript file you want to load.
• Finally, within the Loader callback function, insert the JavaScript code that should run after the external JavaScript file has loaded.
  */

/* Start Tag Library Code /
/ End Tag Library Code */

//tealium universal tag - utag.sender.custom_container ut4.0.##UTVERSION##, Copyright ##UTYEAR## Tealium.com Inc. All Rights Reserved.
try {
(function (id, loader) {
var u = {};
utag.o[loader].sender[id] = u;

// Start Tealium loader 4.32
// Please do not modify
if (utag === undefined) { utag = {}; } if (utag.ut === undefined) { utag.ut = {}; } if (utag.ut.loader === undefined) { u.loader = function (o) { var a, b, c, l; a = document; if (o.type === "iframe") { b = a.createElement("iframe"); b.setAttribute("height", "1"); b.setAttribute("width", "1"); b.setAttribute("style", "display:none"); b.setAttribute("src", o.src); } else if (o.type === "img") { utag.DB("Attach img: " + o.src); b = new Image(); b.src = o.src; return; } else { b = a.createElement("script"); b.language = "javascript"; b.type = "text/javascript"; b.async = 1; b.charset = "utf-8"; b.src = o.src; } if (o.id) { b.id = o.id; } if (typeof o.cb === "function") { if (b.addEventListener) { b.addEventListener("load", function () { o.cb(); }, false); } else { b.onreadystatechange = function () { if (this.readyState === "complete" || this.readyState === "loaded") { this.onreadystatechange = null; o.cb(); } }; } } l = o.loc || "head"; c = a.getElementsByTagName(l)[0]; if (c) { utag.DB("Attach to " + l + ": " + o.src); if (l === "script") { c.parentNode.insertBefore(b, c); } else { c.appendChild(b); } } }; } else { u.loader = utag.ut.loader; }
// End Tealium loader

u.ev = {'view' : 1, 'link': 1};

u.initialized = false;

##UTGEN##
  
  u.send = function(a, b) {
if (u.ev[a] || u.ev.all !== undefined) {
  //##UTENABLEDEBUG##utag.DB("send:##UTID##");
  
  var c, d, e, f, i, g = {};
  
  u.data = {
    /* Initialize default tag parameter values here */
    /* Examples: */
    "collector_uri" : "collector.cxawsnprd.massmutual.com",
    "base_url" : "//d1fc8wv8zag5ca.cloudfront.net/2.6.2/sp.js",
    "tracker_name" : "tealium_snowplow",
    "search_term" : ""
    /* A value mapped to "account_id" or "base_url" in TiQ will replace these default values. */
  };
  
  
  /* Start Tag-Scoped Extensions Code */
  /* Please Do Not Edit This Section */
  ##UTEXTEND##
    /* End Tag-Scoped Extensions Code */
    
    
    /* Start Mapping Code */
    for (d in utag.loader.GV(u.map)) {
      if (b[d] !== undefined && b[d] !== "") {
	e = u.map[d].split(",");
	for (f = 0; f < e.length; f++) {
	  if (e[f].indexOf('trackPageView.')>-1) {
	    // schema key-value pair
	    g[e[f].split('.')[1]] = b[d];
	  } else {
	    u.data[e[f]] = b[d];
	  }
	}
      }
    }
  /* End Mapping Code */
  
  
  /* Start Tag Sending Code */
  
  // Insert your tag sending code here.
  
  
  /* End Tag Sending Code */
  
  
  /* Start Loader Callback Function */
  /* Un-comment the single-line JavaScript comments ("//") to use this Loader callback function. */
  
  u.loader_cb = function () {
    
    /* Start Loader Callback Tag Sending Code */
    
    // Tracker Name
    var tracker = window[u.data.tracker_name];
    
    // "view"
    if (a === 'view') {
      
      // Initialize
      if (!u.initialized) {
	u.initialized = true;
	tracker('newTracker', 'tealium', u.data.collector_uri, { // Initialise a tracker
	  appId: u.data.appId || "tealium_0",
	  cookieDomain: u.data.cookieDomain || ".massmutual.com"
	});
      }
      
      // Link tracking
      //tracker('enableLinkClickTracking');
      
      // Page pings (time period on page)
      tracker('enableActivityTracking', 30, 10);
      
      // Site Search 
      if (u.data.search_term) {
	var array_of_search_terms = u.data.search_term.split('+');
	tracker('trackSiteSearch', array_of_search_terms);// results displayed on first page
      }
      
      // Page View
      if (u.data.schema) {
      	tracker('trackPageView', null, [{"schema":u.data.schema, data: g}]);
      } else {
      	tracker('trackPageView');
      }
    }
    
    // "link"
    else if (a === 'link') {
      // Structured Event w/o Custom Context (no Schema) - category and action required
      if (u.data.category && u.data.action) {
	tracker('trackStructEvent', u.data.category, u.data.action, u.data.label || "", u.data.property || "", u.data.value || "");
      }
      
      //trackLinkClick Method
      u.data.event_name = u.data.event_name || b.event_name || "";
      if (u.data.event_name && u.data.event_name === 'link') {
	
	u.data.link_text = u.data.link_text || b.link_text || "";
	u.data.link_url = u.data.link_url || b.link_url || "";
	u.data.link_id = u.data.link_id || b.link_id || "";
	u.data.link_class = u.data.link_class || b.link_class || [];
	
	tracker('trackLinkClick', u.data.link_url, u.data.link_id, u.data.link_class, '', u.data.link_text);
      }
    }
    /* End Loader Callback Tag Sending Code */
  };
  
  /* End Loader Callback Function */
  
  
  /* Start Loader Function Call */
  /* Un-comment the single-line JavaScript comments ("//") to use Loader. */
  
  if (!u.initialized) {
    ;(function(p,l,o,w,i,n,g){if(!p[i]){p.GlobalSnowplowNamespace=p.GlobalSnowplowNamespace||[]; p.GlobalSnowplowNamespace.push(i);p[i]=function(){(p[i].q=p[i].q||[]).push(arguments);};p[i].q=p[i].q||[];}}(window,document,"script",u.data.base_url,u.data.tracker_name));
    //u.loader({"type" : "iframe", "src" : u.data.base_url + c.join(u.data.qsp_delim), "cb" : u.loader_cb, "loc" : "body", "id" : 'utag_##UTID##' });
    u.loader({"type" : "script", "src" : u.data.base_url, "cb" : u.loader_cb, "loc" : "script", "id" : 'utag_##UTID##' });
  } else {
    u.loader_cb();
  }
  
  //u.loader({"type" : "img", "src" : u.data.base_url + c.join(u.data.qsp_delim) });
  
  /* End Loader Function Call */
  
  
  //##UTENABLEDEBUG##utag.DB("send:##UTID##:COMPLETE");
}
  };
utag.o[loader].loader.LOAD(id);

})("##UTID##", “##UTLOADERID##”);
} catch (error) {
utag.DB(error);
}
//end tealium universal tag

[alex]

Thanks @waspesi - can you share your updated Iglu error message now that you have removed the erroneous <------?

[ehubbard]

I work with @waspesi the “<------” was never actually sent by tracker, I added the arrow to highlight what I suspected was the issue with the data since its an array, and not an object as the schema is structured. We have since validated the json, (screen shot provided above), so I presume this is not the actual issue. Please take a look at the following resolver.json, schema, and error. It appears something is wrong with the configuration of the enricher.

resolver.json

  "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-1",
  "data": {
    "cacheSize": 500,
    "repositories": [
      {
        "name": "Iglu Central",
        "priority": 0,
        "vendorPrefixes": [ "com.snowplowanalytics" ],
        "connection": {
          "http": {
            "uri": "http://iglucentral.com"
          }
        }
      },
      {
        "name": "Custom Schemas",
        "priority": 1,
        "vendorPrefixes": [ "com.snowplowanalytics.mm" ],
        "connection": {
          "http": {
            "uri": "http://datascience-iglu-repo.cxawsnprd.massmutual.com"
          }
        }
      }
    ]
  }
}

Schema

{
    "$schema": "https://datascience-iglu-repo.cxawsnprd.massmutual.com/com.snowplowanalytics.mm/click_event/jsonschema/1-0-0#",
    "description": "Schema for click_event",
    "self": {
        "vendor": "com.snowplowanalytics.mm",
        "name": "click_event",
        "format": "jsonschema",
        "version": "1-0-0"
    },
    "type": "object",
    "properties": {
        "linkName": {
            "type": "string"
        },
        "linkType": {
            "type": "string"
        },
        "pageName": {
            "type": "string"
        },
        "sc_audience" : {
            "type": "string"
        },
        "url" : {
            "type": "string"
        },
        "authentication_status": {
            "type": "string"
        },
        "sc_logged_in_status" : {
            "type": "string"
        },
        "lead_id" : {
            "type": "string"
        },
        "tealium_session_id" : {
            "type": "string"
        },
        "tealium_visitor_id" : {
            "type": "string"
        },
        "sc_calculator_results" : {
            "type": "string"
        },
        "sc_portal_login" : {
            "type": "string"
        },
        "sc_navigation_actions" : {
            "type": "string"
        },
        "query_string" : {
            "type": "string"
        },
        "calculator_name" : {
            "type": "string"
        },
        "calculator_category" : {
            "type": "string"
        },
        "calculator_quantity" : {
            "type": "string"
        },
        "calculator_value" : {
            "type": "string"
        },
        "calculator_finished_id" : {
            "type": "string"
        },
        "content_type" : {
            "type": "string"
        },
        "site_id" : {
            "type": "string"
        },
        "visitor_id220892" : {
            "type": "string"
        },
        "search_term" : {
            "type": "string"
        },
        "advisor" : {
            "type": "string"
        },
        "agency" : {
            "type": "string"
        },
        "state" : {
            "type": "string"
        },
        "city" : {
            "type": "string"
        },
        "field_interaction" : {
            "type": "string"
        },
        "event_name" : {
            "type": "string"
        },
        "homepage_tile_text" : {
            "type": "string"
        },
        "author" : {
            "type": "string"
        },
        "scope" : {
            "type": "string"
        },
        "results_count" : {
            "type": "string"
        },
        "module_section" : {
            "type": "string"
        },
        "pdf_name" : {
            "type": "string"
        },
        "filters" : {
            "type": "string"
        }
    },
    "minProperties": 1,
    "required": [],
    "additionalProperties": true
}

Data coming (note I redacted values).
{  
   "schema":"iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0",
   "data":[  
      {  
         "schema":"iglu:com.snowplowanalytics.mm/click_event/jsonschema/1-0-0",
         "data":{  
            "pageName":"Home",
            "sc_audience":"prospect",
            "authentication_status":"never",
            "sc_logged_in_status":"logged out",
            "tealium_session_id":"redacted",
            "tealium_visitor_id":"redacted",
            "content_type":"Home",
            "site_id":"MM-Dotcom"
         }
      }
   ]
}

Error
&cv=clj-1.1.0-tom-0.2.0&nuid=aaae56ed-2637-485f-af4f-9dbd516e586e\t-\t-\t-\t-\t-","errors":[{"level":"error","message":"error: Could not find schema with key iglu:com.snowplowanalytics.mm/click_event/jsonschema/1-0-0 in any repository, tried:\n    level: \"error\"\n    repositories: [\"Iglu Client Embedded [embedded]\",\"Custom Schemas [HTTP]\",\"Iglu Central [HTTP]\"]\n"},{"level":"error","message":"error: Problem parsing iglu:com.snowplowanalytics.mm/click_event/jsonschema/1-0-0 as JSON in HTTP Iglu repository Custom Schemas: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null') at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@xxxxxx; line: 1, column: 2]\n    level: \"error\"\n"}],"failure_tstamp":"2017-12-05T14:11:38.885Z"}
{"line":"2017-12-05\t11:20:33\t-\t13\t10.109.1.84\tGET\t10.109.1.84\t/nginx-health\t404\t-\tcurl%2F7.51.0\t&cv=clj-1.1.0-tom-0.2.0&nuid=-\t-\t-\t-\t-\t-","errors":[{"level":"error","message":"Request path /nginx-health does not match (/)vendor/version(/) pattern nor is a legacy /i(ce.png)

[alex]

Right, the error is:

Could not find schema with key iglu:com.snowplowanalytics.mm/click_event/jsonschema/1-0-0 in any repository

Your schemas should be a in sub-folder, /schemas/, thus:

http://datascience-iglu-repo.cxawsnprd.massmutual.com/schemas

I suspect that’s the problem.

This is more of a style point, but if your organisation is massmutual.com, then a much better vendor for your events would be:

com.massmutual{.snowplow}

Think of it as a namespace that you “control”. By contrast, we at Snowplow could add schemas into com.snowplowanalytics.mm (multimedia? mass-market?) in Iglu Central at any time, which would be super confusing for your team.

[ehubbard]

@alex

Could you please provide an example? It i’s still failing to enrich. I presume I made the changes in the wrong place. I tried using the path you provided in the enricher and it failed.

This is the location of the schema on our repo
/var/www/html/repos/schemas/com.snowplowanalytics.mm/click_event/jsonschema

I also tried changing the root in nginx to
root /var/www/html/repos/schemas;

I also tried changing the root to (please see below), and it still failed with the same error.
root /var/www/html/repos/schemas/com.snowplowanalytics.mm;

[mike]

There’s a few things that look a little unusual about this (although the schemas now look like they are in the correct location).

1. The schema you’ve provided doesn’t validate with igluctl (you can check this by running igluctl lint schemas/. There’s two changes that need to be made the first is that the $schema value is self-referential whereas you want to point $schema to http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0# which can be thought of as a JSON metaschema. You shouldn’t need to change this for schemas.

2. The required property will only validate if 1 more more properties are included in the array (error: array must have at least one element). If you make the change to the $schema reference and the required property your schema should now validate with igluctl.

3. I’m not sure how you’re serving the schemas over the web but it looks like you’ve uploaded them to a local filesystem and put nginx in front of it. I’d recommend using one of the Iglu repository options instead. The lowest friction one is the static repository which allows you to upload and serve schemas from S3.

After making these changes and testing on Snowplow Mini I was able to get that schema running without issue.

[ehubbard]

Thank you Mike, and everyone else who helped. The issue has been resolved. I appreciate it!

[alex]

What was the problem @ehubbard? This will help others in the community who find this thread.