Realtime GCP pipeline

Hey @tziegler
I always like to refer people to this article when this topic pops up, it’s AWS oriented but the same concepts apply: Real-time reporting using AWS Lambda and DynamoDB: a tutorial to compute the number of players in a game level on the Snowplow event stream (1/2)

Using the Snowplow Analytics SDKs will help you parse the TSV event from the enriched pubsub stream in real time. Using Cloud Functions is a good starting point, or building your own consumers directly as a pubsub subscription. I’d also look at the new pubsub attribute filtering that you can now do with pubsub subscriptions, this has a potential to reduce the load as you might not need all your events going to your real time app.