Unknown host exception in Snowplow open source Quick Start

Alessandro_Arpetti · June 9, 2022, 5:56pm

Hi, I’ve installed the snowplow open source quick start version on AWS and the test event sent to it stop in the enrich server, with an unknown host exception.

This is the error log:

Caused by: java.net.UnknownHostException: ip-10-10-3-119.us-west-2.compute.internal: Name or service not known
	at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
	at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
	at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1515)
	at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
	at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
	at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1637)
	... 3 more
[main] INFO nl.basjes.parse.useragent.AbstractUserAgentAnalyzerDirect - - Loaded 69 files in 1575 msec using expression: classpath*:UserAgents/**/*.yaml
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - 
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - /-----------------------------------------------------------\
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | Yauaa 5.23 (v5.23 @ 2021-03-05T11:05:38Z)                 |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - +-----------------------------------------------------------+
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | For more information: https://yauaa.basjes.nl             |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | Copyright (C) 2013-2021 Niels Basjes - License Apache 2.0 |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - \-----------------------------------------------------------/
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - 
[main] INFO nl.basjes.parse.useragent.AbstractUserAgentAnalyzerDirect - Building all matchers for all possible fields.
Exception in thread "main" java.net.UnknownHostException: ip-10-10-3-119.us-west-2.compute.internal: ip-10-10-3-119.us-west-2.compute.internal: Name or service not known
	at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1642)
	at com.snowplowanalytics.snowplow.enrich.stream.sources.KinesisSource.run(KinesisSource.scala:147)
	at com.snowplowanalytics.snowplow.enrich.stream.KinesisEnrich$.main(KinesisEnrich.scala:97)
	at com.snowplowanalytics.snowplow.enrich.stream.KinesisEnrich.main(KinesisEnrich.scala)
Caused by: java.net.UnknownHostException: ip-10-10-3-119.us-west-2.compute.internal: Name or service not known
	at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
	at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
	at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1515)
	at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
	at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
	at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1637)
	... 3 more
[main] INFO nl.basjes.parse.useragent.AbstractUserAgentAnalyzerDirect - - Loaded 69 files in 1430 msec using expression: classpath*:UserAgents/**/*.yaml
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - 
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - /-----------------------------------------------------------\
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | Yauaa 5.23 (v5.23 @ 2021-03-05T11:05:38Z)                 |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - +-----------------------------------------------------------+
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | For more information: https://yauaa.basjes.nl             |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | Copyright (C) 2013-2021 Niels Basjes - License Apache 2.0 |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - \-----------------------------------------------------------/
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - 
[main] INFO nl.basjes.parse.useragent.AbstractUserAgentAnalyzerDirect - Building all matchers for all possible fields.
Exception in thread "main" java.net.UnknownHostException: ip-10-10-3-119.us-west-2.compute.internal: ip-10-10-3-119.us-west-2.compute.internal: Name or service not known
	at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1642)
	at com.snowplowanalytics.snowplow.enrich.stream.sources.KinesisSource.run(KinesisSource.scala:147)
	at com.snowplowanalytics.snowplow.enrich.stream.KinesisEnrich$.main(KinesisEnrich.scala:97)
	at com.snowplowanalytics.snowplow.enrich.stream.KinesisEnrich.main(KinesisEnrich.scala)
Caused by: java.net.UnknownHostException: ip-10-10-3-119.us-west-2.compute.internal: Name or service not known
	at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
	at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
	at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1515)
	at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
	at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
	at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1637)
	... 3 more
[main] INFO nl.basjes.parse.useragent.AbstractUserAgentAnalyzerDirect - - Loaded 69 files in 1413 msec using expression: classpath*:UserAgents/**/*.yaml
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - 
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - /-----------------------------------------------------------\
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | Yauaa 5.23 (v5.23 @ 2021-03-05T11:05:38Z)                 |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - +-----------------------------------------------------------+
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | For more information: https://yauaa.basjes.nl             |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | Copyright (C) 2013-2021 Niels Basjes - License Apache 2.0 |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - \-----------------------------------------------------------/
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - 
[main] INFO nl.basjes.parse.useragent.AbstractUserAgentAnalyzerDirect - Building all matchers for all possible fields.
Exception in thread "main" java.net.UnknownHostException: ip-10-10-3-119.us-west-2.compute.internal: ip-10-10-3-119.us-west-2.compute.internal: Name or service not known
	at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1642)
	at com.snowplowanalytics.snowplow.enrich.stream.sources.KinesisSource.run(KinesisSource.scala:147)
	at com.snowplowanalytics.snowplow.enrich.stream.KinesisEnrich$.main(KinesisEnrich.scala:97)
	at com.snowplowanalytics.snowplow.enrich.stream.KinesisEnrich.main(KinesisEnrich.scala)
Caused by: java.net.UnknownHostException: ip-10-10-3-119.us-west-2.compute.internal: Name or service not known
	at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
	at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
	at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1515)
	at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
	at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
	at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1637)
	... 3 more
[main] INFO nl.basjes.parse.useragent.AbstractUserAgentAnalyzerDirect - - Loaded 69 files in 1275 msec using expression: classpath*:UserAgents/**/*.yaml
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - 
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - /-----------------------------------------------------------\
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | Yauaa 5.23 (v5.23 @ 2021-03-05T11:05:38Z)                 |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - +-----------------------------------------------------------+
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | For more information: https://yauaa.basjes.nl             |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - | Copyright (C) 2013-2021 Niels Basjes - License Apache 2.0 |
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - \-----------------------------------------------------------/
[main] INFO nl.basjes.parse.useragent.utils.YauaaVersion - 
[main] INFO nl.basjes.parse.useragent.AbstractUserAgentAnalyzerDirect - Building all matchers for all possible fields.
Exception in thread "main" java.net.UnknownHostException: ip-10-10-3-119.us-west-2.compute.internal: ip-10-10-3-119.us-west-2.compute.internal: Name or service not known
	at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1642)
	at com.snowplowanalytics.snowplow.enrich.stream.sources.KinesisSource.run(KinesisSource.scala:147)
	at com.snowplowanalytics.snowplow.enrich.stream.KinesisEnrich$.main(KinesisEnrich.scala:97)
	at com.snowplowanalytics.snowplow.enrich.stream.KinesisEnrich.main(KinesisEnrich.scala)
Caused by: java.net.UnknownHostException: ip-10-10-3-119.us-west-2.compute.internal: Name or service not known
	at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
	at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
	at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1515)
	at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
	at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
	at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1637)

Any help for this problem?

enes_aldemir · June 10, 2022, 7:24am

Hey @Alessandro_Arpetti,

There was an AWS outage yesterday. I suspect error you got might be related with that. Could you try to deploy it again, please ? Let us know if you encounter same error.

Alessandro_Arpetti · June 10, 2022, 11:06am

Hi @enes_aldemir, thank you for your reply.

I tested several times, and today too, but the problem persists.

It maybe could be a network problem? I’ve implemented a VPC with two privates and two public subnets, a NAT and an internet gateway and I’ve configured the variable in terraform like that:

### VPC MODULE VARIABLES

vpc_cidr             = "10.10.0.0/16"
public_subnets_cidr  = ["10.10.0.0/24", "10.10.1.0/24"]
private_subnets_cidr = ["10.10.3.0/24", "10.10.4.0/24"]

josh · June 11, 2022, 7:29am

Hi @Alessandro_Arpetti - would recommend SSHing into one of the nodes you have setup and checking for internet connectivity.

dumky · April 12, 2023, 11:54am

Hi @arpetti,

Sorry to bring up an old topic, but I was wondering if you ever resolved this issue? I have pretty much the same setup and same issue. I’ve tried adding the instance hostname to the /etc/hosts file but no luck unfortunately.

dumky · April 12, 2023, 2:00pm

Apparently I made an error earlier, because I just managed to resolve the issue by adding the local hostname to the /etc/hosts file. (e.g. adding 127.0.0.1 ip-10-0-0-1). However I’m still having trouble getting this to work through Terraform as I think it’s not currently possible to extend the user_data (startup) scripts for the ec2 instances managed through the Snowplow modules.

Edit: I’ve created an issue on Github explaining this in more detail: java.net.UnknownHostException due to AWS EC2 instance hostname not in /etc/hosts · Issue #30 · snowplow-devops/terraform-aws-enrich-kinesis-ec2 · GitHub

Topic		Replies	Views
UnknownHostException when running stream enrich AWS real-time pipeline	1	2090	January 24, 2017
UnknownHostException on Enrichment	1	763	October 26, 2022
User_ipaddress unknown for some events Enrichment	7	963	April 12, 2022
IP Lookup Beam Enrich Error Enrichment	3	1168	August 16, 2019
Snowplow ETL runner failing at the step: [enrich] spark: Enrich Raw Events with spark exception Enrichment	6	2305	August 8, 2019

Unknown host exception in Snowplow open source Quick Start

Related topics