Setting up scala collector

For engineers Collectors
21

Ok, i think Intellij has a funny way of dealing with scala. After going through some debug threads I now get this error. This seems to be more general interoperability exception between scala and java.

Exception in thread “main” java.lang.NoClassDefFoundError: scala/collection/IterableOnce
Caused by: java.lang.ClassNotFoundException: scala.collection.IterableOnce
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
… 1 more

22

Would you believe it: SOLVED!!!

  1. changed scala settings in intellij as given here: https://stackoverflow.com/questions/26390219/intellij-idea-java-lang-nosuchmethoderror-scala-collection-immutable-coloncol

  2. The above “NoClassdefFoundErr” comes because of some changes between scala 2.12 and 2.13, so i went back to 2.12 for the scala sdk.

I now get a fully formatted , beautiful to look at, event in json format

{

“app_id” : “{{MY-SITE-ID}}”,
“platform” : “web”,
“etl_tstamp” : “2020-10-24T18:17:55.689Z”,
“collector_tstamp” : “2020-10-24T18:17:55.486Z”,
“dvce_created_tstamp” : “2020-10-24T18:17:55.477Z”,
“event” : “page_ping”,
“event_id” : “83e1a202-c4a8-4d54-ad44-672ec8dae9fe”,
“txn_id” : null,
“name_tracker” : “sp”,
“v_tracker” : “js-2.16.2”,
“v_collector” : “ssc-1.0.0-kafka”,
“v_etl” : “stream-enrich-1.4.0-common-1.4.0”,
…
…
…

Haven’t gotten this kick for a while.

Million thanks for getting me going on my project @PaulBoocock @Colm

I will be regular on this forum for sure and wishing this team to conquer worlds!

1 Like
23

Glad we could help!

For future reference, when using Scala libraries, you’ll notice they end with the Scala version. For example: snowplow-scala-analytics-sdk_2.12. We haven’t published the Analytics SDK for 2.13 but it’s due out soon.

I’m not sure exactly how you solved it with intelliJ in the end but I’m happy you found a way. As a side note though, and perhaps for future readers, you can also shade only the dependencies that the SDK requires, so you don’t need to build a fat jar including lots of other depedencies too.

I just popped this example repo up on Github which does just that: https://github.com/snowplow-incubator/snowplow-scala-analytics-sdk-java-example

Take a look at the pom.xml: https://github.com/snowplow-incubator/snowplow-scala-analytics-sdk-java-example/blob/05a00b0f2840ef656df9d32e48c1de45c3e2e9a0/pom.xml

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-shade-plugin</artifactId>
	<version>2.3</version>
	<executions>
		<execution>
			<phase>package</phase>
			<goals>
				<goal>shade</goal>
			</goals>
			<configuration>
				<transformers>
					<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
						<mainClass>net.paulboocock.core.App</mainClass>
					</transformer>
				</transformers>
				<artifactSet>
					<includes>
						<include>com.snowplowanalytics:*</include>
						<include>org.typelevel:*</include>
						<include>io.circe:*</include>
						<include>com.chuusai:*</include>
						<include>org.scala-lang:*</include>
					</includes>
				</artifactSet>
			</configuration>
		</execution>
	</executions>
</plugin>
1 Like