Unable to set up Iglu repo

For data modelers & consumers Iglu
1

Hi folks,

I having a problem coming up when setting up my Iglu repo for json schema. When using json schema from iglu central it coming fine.

My current stack is something like this shown below: ->

tracker -> collector -> kafka(good and bad topic) -> enricher(using resolver config) -> kafka (enriched_data and badenriched_data)

My tracker code for firing event is:

window.snowplow_name_here('trackUnstructEvent', {
    schema: 'iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-0',
    data: {
  cacheSize: 1,
  repositories: [{
  	name: "Shakti Gupta",
  	priority: 2,
  	vendorPrefixes: [	
  	],
  	connection: {
  		embedded: {
  			path: "Some value"
  		}
  	}
  }]
}
});

Whlle enrichment is am using resolver file contains schema like -

{
  "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-0",
  "data": {
    "cacheSize": 500,
    "repositories": [
      {
        "name": "Iglu Central",
        "priority": 0,
        "vendorPrefixes": [ "com.snowplowanalytics" ],
        "connection": {
          "http": {
            "uri": "http://iglucentral.com"
          }
        }
      }
    ]
  }
}

Everything is fine upto here.

app1	mob	2017-05-19 14:18:55.542	2017-05-19 14:18:55.525	2017-05-19 14:18:55.502	unstruct	b7dd2aba-6646-4927-88e8-ed94a0189b3c		cf	js-2.6.2	ssc-0.9.0-kafka	kinesis-0.10.0-common-0.24.0		127.0.0.1	404893028	cb3de8ea-e92d-428b-91a0-b608c8798f7a	10	8b27c03b-02e6-429e-8762-a6573b022c7b												http://localhost/snowplow.html			http	localhost	80/snowplow.html																					{"schema":"iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-0","data":{"schema":"iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-0","data":{"cacheSize":1,"repositories":[{"name":"Shakti Gupta","priority":2,"vendorPrefixes":[],"connection":{"embedded":{"path":"Some value"}}}]}}}					Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0						en-US							124	1366	621				Asia/Kolkata			1366	768	UTF-8	1366	621								2017-05-19 14:18:55.505				7a44c8a9-0358-435c-b936-a2c4de789182	2017-05-19 14:18:55.522	com.snowplowanalytics.iglu	resolver-config	jsonschema1-0-0

Issue arise when is try to host a json schema in my localhost. I have my whole setup of snowplow stack on my local ubuntu machine(for testing).

> root@shakti:/opt/snowplow/scala_stream_enrich# ./snowplow-stream-enrich-0.10.0 --config scala_enrich.conf --resolver file:iglu_resolver.json Exception in thread "main" java.lang.RuntimeException: NonEmptyList(error: Resolver configuration failed JSON Schema validation    level: "error", error: Verifying schema as iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-1 failed: found iglu:com.shopclues/product/jsonschema/1-0-0    level: "error")	at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$$anonfun$7.apply(KinesisEnrichApp.scala:137)	at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$$anonfun$7.apply(KinesisEnrichApp.scala:137)	at scalaz.Validation$class.fold(Validation.scala:64)	at scalaz.Failure.fold(Validation.scala:330)	at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$delayedInit$body.apply(KinesisEnrichApp.scala:136)	at scala.Function0$class.apply$mcV$sp(Function0.scala:40)	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)	at scala.App$$anonfun$main$1.apply(App.scala:71)	at scala.App$$anonfun$main$1.apply(App.scala:71)	at scala.collection.immutable.List.foreach(List.scala:318)	at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)	at scala.App$class.main(App.scala:71)	at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$.main(KinesisEnrichApp.scala:71)	at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp.main(KinesisEnrichApp.scala)[INFO] [05/19/2017 14:48:45.343] [snowplow-scala-tracker-akka.actor.default-dispatcher-4] [akka://snowplow-scala-tracker/user/IO-HTTP/group-0] Message [spray.can.Http$Connect] from Actor[akka://snowplow-scala-tracker/user/IO-HTTP/host-connector-0/0#1236322060] to Actor[akka://snowplow-scala-tracker/user/IO-HTTP/group-0#1384988627] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-le

Kindly help in creating the directory structure for the the iglu schema.

I am creating json schema file named as 1-0-0 inside a directory custom_event and my document root is /var/www/html/iglurepo and my json schema is accessible using company.com/custom_event/1-0-0 .

snowplow schema.png1366×768 38.9 KB

Help me here how to use this schema in my tracker code as well as in resolver config schema.

2

Hey! Your JSON Schema should be on this path:

company.com/custom_event/jsonschema/1-0-0

Hope this helps.

3

Hi @shaktigupta200,

Besides of what Alex said (wrong path), it’s also unclear for me what event you’re trying to track. Argument of your trackUnstructEvent contains resolver config, which makes no sense - it should contain actual event data - tracker should know nothing about resolver configuration or address of your Iglu registry.

1 Like
4

Thanks for the help. Iglu repo has been created and working fine.

5

schemas/company.com/custom_event/jsonschema/1-0-0

Thanks alex. adding schemas in directory structure solved the issue.