Can I use snowplow micro with externally hosted website?

Hi all,

I am investigating adding snowplow micro to our existing front end automation tests. I was wondering whether it was possible to point snowplow micro at an external url, rather than using localhost?

My only experience testing snowplow is with the snowplow analytics debugger extension in chrome. I’m guessing the events that snowplow micro sees are different to those found with the browser extension.

Thanks
Kieran

I’m not the best to answer this, so someone else might jump in and tell you otherwise, but I just wanted to make sure I understood your question.

Snowplow micro is a destination for your events, used to help test new schemas or changes to your site before pushing those events through your full pipeline (to avoid bad events).

My understanding of the chrome extension (even less my area of expertise) is that is just looks at the events being sent from your client to a snowplow collector (which you can see on the network tab) and reports these to you in a nice visual way.

So it’s not that you would point micro at an external URL, you’d need to host/run micro in a way that it was accessible from your testing environment and send events to micro.

If you have some frontend test environment you could possibly send events from that environment to an instance of micro, instead of the production pipeline, but someone else is best to advise how you might go about doing that.

2 Likes

I think that makes more sense, so the browser extension reads the event as it gets sent off to the snowplow collector. Whilst snowplow micro is the collector at the other end, which is running on web server itself. I can’t think off the top of my head how I would make this work without redirecting the test environment to the snowplow micro collector or creating a new environment and pointing snowplow events to go to micro instead.

That’s it, yes! A typical approach would be to run Micro within the tests themselves, using Docker. We have a guide on that: Automated testing with Snowplow Micro | Snowplow Documentation.

Do I need to change where the snowplow events are sent to in the application code? I’m not involved with the creation of the events themselves as I only test the finished product.

Also do events hit both snowplow micro and production snowplow collector or do they only hit snowplow micro?

I’m thinking for this to work, I will need another machine hosting the staging environment of our website, I put the tests and snowplow micro onto that machine and it should work without changing any of the application code.

What kind of testing are you thinking to do? You mentioned that you are involved with testing the finished product — does it mean you would look at the data in the warehouse?

Snowplow Micro is most useful for quick debugging (sending a few events and seeing them in the logs or outputting to a TSV file) and for automatic testing (a script that sends events and checks the Micro API to validate the events are there). If this is sufficient for you, then you would have two setups:

  • One that runs on a dedicated machine (Continuous Integration infrastructure or even your laptop) and sends events to Micro (the tracking code must point to Micro).
  • One that’s your actual production site.

If you want to look at the data in the warehouse, then having a fully fledged staging pipeline (writing to a separate schema in the warehouse) is a better option, although more costly from the infrastructure point of view.

So I don’t think I want to go as far as checking the data in the warehouse, I think micro itself should tell me if the data is what we expect:

  • There are no bad events.
  • The data sent is as expected.
  • The right event data is sent with each event.
  • The right entities / contexts are attached to each event.
  • The right values are sent with each event.

I’m thinking micro will enable us to have an automated regression pipeline so we can check events are being fired correctly automatically before every release. We have had issues in the past where events were sent when they shouldn’t have been and vice versa and I think micro is the tool to help mitigate this?

Yes, you should be able to validate that with Micro!

2 Likes