We are very pleased to announce version 2.5.0 of the Snowplow Collector.
Graceful shutdown
Graceful shutdowns are important in deployments with autoscaling enabled. Collectors will frequently start up and terminate automatically in response to traffic volume. Typically an orchestration platform will initiate a termination sequence by first sending a trappable signal (e.g. a SIGINT or SIGTERM) to tell the collector that forced termination is imminent.
Starting from version 2.5.0, the collector makes the following actions when it receives one of those termination signals:
- Trap the signal and initially wait for a configurable amount of time. We call this the pre-termination period. During this period, the collector continues to accept connections, respond to requests, and sink events.
- Shutdown the http server. When the pre-termination period is over, then the collector stops accepting new TCP connections. The collector continues to respond to any pending request, for a configurable amount of time.
- Flush any outstanding events to the sink. For example, the output kinesis stream or PubSub topic.
The default values for the pre-termination and termination periods is 10 seconds each. With a certain amount of trial and error, you can make sure that the pre-termination period is long enough such that your load balancer is no longer routing requests to the collector by the time the termination period starts. The details depend on which load balaner you are using, but a sensible value in production might be a couple of minutes.
Disable warnings for illegal headers
You might have noticed your collector’s stdout logs are always filled with unhelpful warning messages like this:
WARN akka.actor.ActorSystemImpl - Illegal header: Illegal 'user-agent' header: Invalid input '/', expected OWS, 'EOI', tchar, product-or-comment or comment (line 1, column 190): Mozilla/5.0 ...... etc.
In version 2.5.0 we disabled logging for this particular warning because we find it unhelpful, and also because it leaks customer-sent headers into the logs, which we try to avoid.
We hope you appreciate the new cleaner easy-to-read logs containing helpful messages only.
Increase defaults for PubSub backoff policy
In previous versions of the PubSub collector, events that could not be written to PubSub within 10 seconds would be dropped and lost. This did not happen too often at all because of how PubSub is configured, but generally at Snowplow we aim to never drop any event under any circumstances.
In version 2.5.0 we remove the time limit (well, technically we set the limit to 292 years) so events will be retried to PubSub indefinitely until they are written successfully.
Use shorter application name
Previous versions of collector used longer application name in an event payload e.g.: snowplow-stream-collector-kinesis-2.4.5-kinesis which didn’t comply with some of a limits set in bad rows schemas. In version 2.5.0 the used name is shortened (based on provided example it would be ssc-2.4.5-kinesis), hence solving schema compliance issues.
Upgrading to 2.5.0
If you are already using a 2.4.x version then you can simply start using the new docker images without making any changes:
docker pull snowplow/scala-stream-collector-kinesis:2.5.0
docker pull snowplow/scala-stream-collector-pubsub:2.5.0
docker pull snowplow/scala-stream-collector-kafka:2.5.0
Optionally, if you want to fine-tune the new features, then check out the new configuration options for graceful shutdown and pubsub backoff
Our docs site has instructions on setting up the collector and a full list of all configuration options