SASL Kafka Configuration - AWS MSK - Resolved

Hi All,

I am using AWS MSK for Managed Kafka and trying to run Snowplow. I get the following error:

[] INFO com.snowplowanalytics.snowplow.collectors.scalastream.KafkaCollector$ - REST interface bound to /
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 ([MYCLUSTERHOST]( terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue.

My Config is:

sink {
  enabled = kafka
  brokers = ""
  retries = 0

  producerConf {
    "sasl.jaas.config" = " required username='myuser' password='mypassword';"
    "security.protocol" = "SASL_SSL"
    "sasl.mechanism" = "SCRAM-SHA-512"

I am able to produce and consumer with these details and network connectivity exists e.g.

nc -zv 9096
Connection to 9096 port [tcp/*] succeeded!

Any ideas. I saw a previous post similar issue - but that was a typo they had on sasl.mechanism. Mine is set correctly.


Answering my own question here - user error.

The issue was I didn’t update sasl.jaas.config from plain.PlainLoginModule to scram.ScramLoginModule. Only when I compared my Kafka Connect Configs did I notice difference


Nice, good to hear you got it sorted. Sorry we didn’t get to this before, but thanks for following up!