Porting Snowplow to Microsoft Azure

This is pretty exciting stuff.

Here’s some things that come to mind:

  1. Should the implementations be tailored to each individual platform or is there (and should there) be an abstraction above each computing platform such that each platform and each product become interchangeable components*?

  2. Does it make sense to favour platform-agnostic components (like Kafka) over platform specific components (like Kinesis)? This might reduce effort to migrate but may not necessarily ofter the same flexibility/performance as tools built by individual platforms (e.g., features in Event Hubs that aren’t in Kinesis).

  3. Should it be possible for products on different platforms to run in not just sequence but also in parallel? An example of this might be events from a collector getting pushed into Kinesis and Event Hubs.

These questions aren’t really Azure specific but are more around the most popular platforms that aren’t AWS (Azure, Google, OpenStack).

  • I don’t think these two options are mutually exclusive but there’s obviously a trade off involved.
3 Likes