Hi,
Here is the thing. I used Beam Enrich for a long time with the version of 1.2.3. Today, when I tried to upgrade to the latest (3.0.0-rc10), there were some errors coming. The following is the startup script and error log. I am not quite sure the reason. And I also tried version 2.0.2, it worked. Where do I need to change if I still want to upgrade to version3?
And another question is, I found there is a new way called Enrich PubSub. What is the difference between Enrich PubSub and Beam Enrich? Which one do you suggest if my website will have a large amount of data flow? Thanks!
/$ sudo docker run \
-v /snowplow/config:/snowplow/config \
-e GOOGLE_APPLICATION_CREDENTIALS=/snowplow/config/credentials.json \
snowplow/beam-enrich:$enrich_version \
--runner=DataFlowRunner \
--project=$project_id \
--region=$region \
--streaming=true \
--gcpTempLocation=gs://$temp_dir/temp-files/ \
--job-name=beam-enrich \
--raw=projects/$project_id/subscriptions/good-sub \
--enriched=projects/$project_id/topics/enriched-good \
--bad=projects/$project_id/topics/enriched-bad \
--pii=projects/$project_id/topics/pii-good \
--resolver=/snowplow/config/iglu_resolver.json \
--enrichments=/snowplow/config/enrichments/ \
--workerMachineType=$dataflow_type
params: /opt/docker/bin/beam-enrich --runner=DataFlowRunner --project=snowplow-test --region=us-west1 --streaming=true --gcpTempLocation=gs://snowplow-test/temp/temp-files/ --job-name=beam-enrich --raw=projects/snowplow-test/subscriptions/good-sub --enriched=projects/snowplow-test/topics/enriched-good --bad=projects/snowplow-test/topics/enriched-bad --pii=projects/snowplow-test/topics/pii-good --resolver=/snowplow/config/iglu_resolver.json --enrichmen
ts=/snowplow/config/enrichments/ --workerMachineType=n1-standard-4 threshold: 5 delay: 3 GOOGLE_APPLICATION_CREDENTIALS: /snowplow/config/credentials.json
Activated service account credentials for: [datamodeling2@snowplow-test.iam.gserviceaccount.com]
gs://snowplow-test/temp/temp-files/credentials.json
Bucket gs://snowplow-test/temp/temp-files/ exists! Proceeding.
Exception in thread "main" java.lang.NoClassDefFoundError: io/netty/internal/tcnative/AsyncSSLPrivateKeyMethod
at io.netty.handler.ssl.SslContext.newClientContextInternal(SslContext.java:831)
at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:611)
at com.spotify.scio.pubsub.PubSubAdmin$GrpcClient$.newChannel(PubSubAdmin.scala:43)
at com.spotify.scio.pubsub.PubSubAdmin$GrpcClient$.publisher(PubSubAdmin.scala:58)
at com.spotify.scio.pubsub.PubSubAdmin$.topic(PubSubAdmin.scala:88)
at com.snowplowanalytics.snowplow.enrich.beam.Enrich$.checkTopicExists(Enrich.scala:327)
at com.snowplowanalytics.snowplow.enrich.beam.Enrich$.$anonfun$main$2(Enrich.scala:77)
at scala.util.Either.flatMap(Either.scala:341)
at com.snowplowanalytics.snowplow.enrich.beam.Enrich$.main(Enrich.scala:75)
at com.snowplowanalytics.snowplow.enrich.beam.Enrich.main(Enrich.scala)
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.AsyncSSLPrivateKeyMethod
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 10 more