Scala Stream Collector - java.net.SocketException: Broken pipe (Write failed)

If some messages fail to be inserted (because of throughput limits), you should see a message along the lines of:

Successfully wrote XYZ out of ABC records
Retrying all failed records in X milliseconds...

Also, Kinesis limits vary between regions but I think it’s like 500-1,000 records per second per shard.

However, there is also a limit on the size of the record – 1MB per second per shard. So it might indeed be the case that the batches are too big… Now that I said that, I notice the your byteLimit is 10MBs. Can you try with 1MB and see if that fixes things?