Python Tracker: Tracking date-time doesn't work (works in Micro though)

For engineers
1

Hi Team

I’m using date-time fields in my self-describing schema:

    "gp_created": {
        "description": "GP Created Datum",
        "type": ["string", "null"],
        "format": "date-time"
    },
    "gp_updated": {
        "description": "GP Updated Datum",
        "type": ["string", "null"],
        "format": "date-time"
    }

When I send an example tracking event to Micro, it works. When I send it to my production pipeline, no event is recorded and no error event is recorded either?! (The sending itself works… when I comment out the date-time fields in my payload the event is recorded)

I’m using the python tracker and my code looks like this:

"gp_created": datetime.datetime.now(datetime.timezone.utc).isoformat(),
"gp_updated": datetime.datetime.now(datetime.timezone.utc).isoformat(),

Is this a bug or am I missing something?

Thanks,
Moritz

2

Update: I got it working.

I checked what payload my javascript tracker sends and adapted the date-time format accordingly. So in python you have to do this to send a valid date-time:

>>> iso8601_time = current_time.replace().isoformat() + 'Z'
>>> iso8601_time
'2023-10-31T09:58:56.977004Z'

I wonder why Micro accepted my payload though and snowplow didn’t create an error when I tried to send it the wrong way? :thinking:

3

I believe “date-time” implements this specification. That is, it has to be in the form full-date "T" full-time, where “full-date” is in the form date-fullyear "-" date-month "-" date-mday and “full-time” is partial-time time-offset. The “time-offset” in its turn has to be either “Z” for UTC or time-numoffset, which is in the form ("+" / "-") time-hour ":" time-minute with “time-hour” having values 00-23 and time-minute are limited to 00-59.

Examples of the valid timestamps are in this section of the RFC.

1 Like
4

I wonder why Micro accepted my payload though and snowplow didn’t create an error when I tried to send it the wrong way? :thinking:

Hi @Moritz_Bauer, I think if you update your Micro docker image to the latest version, then you will find Micro will also invalidate those events.

Enrich had an update approximately 1 year ago (version 3.6.1) which made some of the date parsing slightly more strict. We try to update Micro so it is always in-sync with how Enrich does validation. But you might find you are running and older version of Micro but a newer version of Enrich.

The latest Micro version is 2.0.0

docker pull snowplow/snowplow-micro:2.0.0
1 Like