I’m using scala stream collector, stream enrich, s3loader and emretlrunner with redshift. This morning I began setting up a simple custom event to capture large chunks of page content in order to build part of a training data set. So far so good-ish.
Everything seems to be working except for the fact that I don’t see any records in the new custom event table after the emretlrunner job successfully completes.
By the way I do see the basic struct and page_view events the events table. That part is still working.
I followed this guide and read through the docs quite thoroughly… still, maybe I missed something.
I’ve worked through all the various errors that I’ve bumped into and I now see my custom events in the s3 stream (enrichment output) bucket and the archives for both enrich and shredded. There are no errors during emretlrunner’s job. There are no errors in the Redshift console.
I suspect that it’s something to do with the jsonpaths file. By the way, I generated it and the ddl using schema-guru and validated the actual schema using igluctl.
Where would I see jsonpaths related errors if there are any?
I looked into the EMR rdb_loader step’s stderr output and see this exception:
Exception in thread "Thread-2" java.lang.ExceptionInInitializerError at com.snowplowanalytics.snowplow.scalatracker.emitters.AsyncEmitter$$anon$1.run(AsyncEmitter.scala:55) Caused by: java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66) at java.lang.Runtime.addShutdownHook(Runtime.java:211) at com.snowplowanalytics.snowplow.scalatracker.emitters.RequestUtils$.<init>(RequestUtils.scala:78) at com.snowplowanalytics.snowplow.scalatracker.emitters.RequestUtils$.<clinit>(RequestUtils.scala) ... 1 more
…I’m not sure what to make of that.