Hi,
I am currently working on a project in Django/Django Rest and would like to implement event tracking at the API level using Snowplow. I am seeking guidance on how to properly integrate Snowplow with my Django application to track events effectively.
I would also like your advice on which event type to use. Given my requirements, should I go with StructuredEvent, PageView, PagePing, ScreenView, or SelfDescribing events? Your recommendation would be greatly appreciated.
I think this is going to depend a lot on your application and what you are trying to achieve but if it’s a Django web based app I’d steer towards things like page views, page pings and self-describing events.
Structured events aren’t deprecated but they are considered an older way of tracking things and screen views tend to be useful for tracking on mobile apps rather than the web.
If you are instrumenting an API in addition, self-describing is likely the best way to go here where you don’t necessarily have direct user interactions in a UI.
Can you also tell me which schema I should use, or should I create a custom schema?
This is the schema and data I am using:
data = {
“endpoint”: request.build_absolute_uri(),
“method”: request.method,
“headers”: dict(request.headers)
}
schema = “http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#”
context = SelfDescribingJson(schema=schema, data=data)
You should be looking to create a custom schema - the com.snowplowanalytics.self-desc schema is a metaschema, so you don’t need to reference this directly - instead reference the vendor / name / format / version of your custom schema.
Hi,
Could you kindly guide me on how to create a custom schema? Your expertise would be greatly appreciated.
The tutorial in our documentation might be helpful
Hi,
My snowplow tracker is sending events to the Snowplow collector endpoint successfully.
But I don’t see any files in the Azure blob storage.Why?
Default schema is working fine but when I my custom schema it is not storing any data in azure blob.
def track_link_click(tracker, schema, data):
link_click = SelfDescribing(
SelfDescribingJson(schema,data)
)
link_click2 = tracker.track(link_click)
data = {
“endpoint”: request.build_absolute_uri(),
“method”: request.method,
“headers”: dict(request.headers)
}
schema = "my_custom_schema_blob_url"
track_link_click(tracker=self.tracker, schema=schema, data=data)
response = self.get_response(request)
resp = self.tracker.flush()
My custom schema is this :
{
“$schema”: “my_custom_schema_blob_url”,
“self”: {
“vendor”: “com.relevancestudio”,
“name”: “request-event”,
“format”: “jsonschema”,
“version”: “1-0-0”
},
“type”: “object”,
“properties”: {
“endpoint”: {
“type”: “string”
},
“method”: {
“type”: “string”
},
“headers”: {
“type”: “object”
}
},
“required”: [“endpoint”, “method”, “headers”],
“additionalProperties”: true
}
It might be worth trying this on Snowplow Micro ( GitHub - snowplow-incubator/snowplow-micro: Standalone application to automate testing of trackers ) first.
At an initial glance my guess would be you’ve changed the metaschema for your schema (to my custom_schema_blob_url). This is a fixed value - so you should leave it as default e.g., your schema becomes:
{
“$schema”: “http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#”,
“self”: {
“vendor”: “com.relevancestudio”,
“name”: “request-event”,
“format”: “jsonschema”,
“version”: “1-0-0”
},
“type”: “object”,
“properties”: {
“endpoint”: {
“type”: “string”
},
“method”: {
“type”: “string”
},
“headers”: {
“type”: “object”
}
},
“required”: [“endpoint”, “method”, “headers”],
“additionalProperties”: true
}
Hi,
I tried above schema, it is not working.I have created a custom schema do i need to publish it on iglu server,if yes then how ?
I am also receiving validation error on custom schema
What is your value for schema?
this is the value for my schema https://rssnowplowstorage.blob.core.windows.net/iglu-schemas/schemas/com.relevancestudio/request-event/jsonschema/1-0-0.json
and this is my custom schema
{
“$schema”: “https://rssnowplowstorage.blob.core.windows.net/iglu-schemas/schemas/com.relevancestudio/request-event/jsonschema/1-0-0.json”,
“self”: {
“vendor”: “com.relevancestudio”,
“name”: “request-event”,
“format”: “jsonschema”,
“version”: “1-0-0”
},
“type”: “object”,
“properties”: {
“endpoint”: {
“type”: “string”
},
“method”: {
“type”: “string”
},
“headers”: {
“type”: “object”
}
},
“required”: [“endpoint”, “method”, “headers”],
“additionalProperties”: true
}
You you try the value suggested above? Integration Support - #8 by mike
I tried it didn’t work without pushing schema to iglu server.
Again when I try to publish schema on iglu central I am receiving this error:
-X PUT http://localhost:8080/api/schemas/iglu:com.relevancestudio/request-event/jsonschema/1-0-0 -H “Content-Type: application/json” -H "apikey: “” -d @/home/algo/Desktop/request-event-schema.json
{“message”:“Authentication error: not authorized”}
Hi,
Do I need to publish schema on iglu central or not ?
I had uploaded schema in my server but tracker is trying to find schema on iglu sever.
How to make sure my tracker gets schema from iglu server ?
Iglu Central is managed by Snowplow directly so you don’t need to go and push anything here - just your own personal Iglu registry.
As part of your enrichment process you will have a resolver.json file that tells the enrichment process where to look for schemas ( Iglu Resolver | Snowplow Documentation ). Make sure that your Iglu Server is defined in this file and that will ensure enrich can fetch the schema for you.
The tracker itself isn’t aware of where the schemas are hosted - it just needs the URI that you provide as part of the tracking, and then the lookup and resolution itself is the responsibility of the enricher.
Hi, I have a few questions:
resolver.json file, or does it already exist?I have downloaded this github repo GitHub - snowplow/quickstart-examples: Examples of how to automate creating a Snowplow Community Edition pipeline for pipeline setup.
Do I need to make changes in this file or somewhere else ?
Hi, @mike anything ?
Ah - if you are using quickstart it will be within Terraform itself (in quickstart-examples/terraform/gcp/pipeline/default/main.tf at c0d170b074b94f322e2132a28700016d98273fe4 · snowplow/quickstart-examples · GitHub ) and it will define your Iglu Server for you - so you shouldn’t need to modify anything.
If you’ve pushed the schema it’s worth checking that it exists on the server - you can use the API to do this.
Hi @Mike, Few doubts:
locals {
custom_iglu_resolvers = [
{
name = "Iglu Server"
priority = 0
uri = "${var.iglu_server_dns_name}/api"
api_key = var.iglu_super_api_key
vendor_prefixes = []
}
]
}
curl with schema in response:
curl -X GET http://snowplow-iglu-lb-6b13e842-3cd6-7a6a-65bc-3166334c27c1.canadacentral.cloudapp.azure.com/api/schemas/com.relevancestudio/request-event/jsonschema/1-0-0 -H "apikey: api-key"
{"$schema":"iglu:com.relevancestudio/request-event/jsonschema/1-0-0","self":{"vendor":"com.relevancestudio","name":"request-event","format":"jsonschema","version":"1-0-0"},"type":"object","properties":{"endpoint":{"type":"string"},"method":{"type":"string"},"headers":{"type":"object"}},"required":["endpoint","method","headers"],"additionalProperties":true}
Hi Jaspreet,
- I am using Azure, not GCP.Is the process the same for Azure?
Yes, that’s correct.
- Schema is already present in my iglu server but still facing issue.Why ?
The metaschema value is incorrect (see below).
- Do I need to provide some value in vendor ?
You do need a value for vendor in your schema, you do not need a value in vendor_prefixes however - this is optional.
As above, you’ll need to ensure the metaschema ($schema) is a fixed value not request-event.