Hi, I have a problem with getting correct user ip addresses in user_ipaddress field in following setting.
There is Snowplow connector behind AWS Api Gateway and Application Load Balancer. Api Gateway puts X-Forwarded-For value received from the client (with correct user ip address) into a new forwarded header and it sets ip address of the api gateway into X-Forwarded-For header. ALB is just passing through the values intact. I verified this behaviour using http_header_extractor_config enrichment and I can see correct client’s IP address in forwarded header. The enrichment is then just picking up the from X-Forwarded-For header which is IP address of the api gateway.
Is there any way how to access forwarded header value when enriching user_ipaddress field and passing it down into IP Lookup enrichment? Thank you.
Unfortunately not without changing the behaviour of the collector as to which headers it looks at.
The rewrite of X-Forwarded-For is a little unusual here as the generally the accepted behaviour is to append to this array with the proxies rather than to overwrite it and remove the client IP address.
As this behaviour isn’t particularly easy to change in the collector itself (without a source code change) I’d be tempted to use a mapping template in API gateway to either override the X-Forwarded-For header (or create a new Remote-Address header) with the client IP address. Setting this new header should cause the collector to look in this value first before looking in X-Forwarded-For or if modifying X-Forwarded-For it should prioritise looking at the client IP rather than any of the proxies.
Thank you very much for mentioning Remote-Address. I configured it in the api gateway, verified using http_header_extractor_config enrichment but the collector is not picking it up. Maybe because it expects it as Remote-Address and I can only have it as remote-address in api gateway. Headers should be case insensitive, but it is case sensitive probably?
I think it is case insensitive but I may have given you the incorrect information here, apologies (this header would work but requires changing the Akka defaults to pass the remote address attribute through which is off by default).
Could you try the x-real-ip header instead (lowercase is fine)?
I’ve tested this on the collector and it seems to pass this through correctly without any issues (both as the lowercase version and the X-Real-IP version).
Thanks for all the help. I set x-real-ip header, I see it in http_header_extractor_config enrichment with correct client IP address, but the collector is still picking up api gateway ip address from X-Forwarded-For header. I am on terraform snowplow-devops/collector-kinesis-ec2/aws version 0.3.0. I will try to somehow debug it in the collector source code.