Google Kubernetes Engine scaling


Our Snowplow open source pipeline runs on Google Kubernetes Engine, using PubSub for messages, and I’m wondering how each individual service can be scaled up horizontally, if that’s possible ?
Can each service be scaled horizontally without concurrency issue ?


Hi @Ltsrc ,

Kubernetes handles horizontal auto-scaling, and the strategy for scaling up up down depends on how you configure it.

Yes. PubSub consumers from a same subscription automatically share the consumption of the records, each record going to one consumer only.

1 Like

Thanks for the reply !

That’s clear for the PubSub consumers services, but how about the collector ? I’m certain there is no issue scaling this service either, but I’d rather make sure now to avoid further issues :smile: