Issue when parallelizing multiple components


We had an issue when trying to parallelize multiple components (enrich, storage s3, storage es). It seems the components put themselves in a paused state, while we added a new one.
The issue didn’t show up with the collector component though.

Here is the stacktrace which showed up when we had two instanciations of a component in parallel :

[RecordProcessor-0005] DEBUG - Lease update failed for lease with key shardId-000000000018 because the lease counter was not 9164846
[RecordProcessor-0005] INFO - Worker 64bc947f0df14e80:-1ff1f730:174bebcb283:-8000 lost lease with key shardId-000000000018 - discovered during update
[RecordProcessor-0005] ERROR - Can't update checkpoint - instance doesn't hold the lease for this shard

Everything is deployed in AWS ECS with Kinesis as queues

Versions of components used:

  • snowplow-stream-collector-kinesis : 1.0.0
  • snowplow_stream_enrich : 0.13.0
  • snowplow_elasticsearch_loader : 1.0.0
  • snowplow_s3_loader : 0.7.0

Do you have any feedback on this ? Anyone already had this issue ?

Thanks a lot,