Enrich PubSub - "Cannot construct Input.PubSub from topic" error

For engineers GCP pipeline
1

Hey all! I am in the process of implementing a full pipeline in GCP and have gotten stuck while trying to get Enrich PubSub up and running.

Can anyone provide any clarity on this error message I am getting?

[pool-1-thread-1] INFO com.snowplowanalytics.snowplow.enrich.pubsub.Main - Initialising resources for Enrich job
[pool-1-thread-1] INFO com.snowplowanalytics.snowplow.enrich.common.fs2.io.FileSystem - Files found in /snowplow/enrichments: /snowplow/enrichments/cookie_extractor_config.json, /snowplow/enrichments/event_fingerprint_config.json, /snowplow/enrichments/referer_parser.json, /snowplow/enrichments/ua_parser_config.json, /snowplow/enrichments/yauaa_enrichment_config.json
[pool-1-thread-2] ERROR com.snowplowanalytics.snowplow.enrich.pubsub.Main - Cannot initialise enrichment resources
Cannot construct Input.PubSub from snowplow-collected-good-sub

My PubSub topics and subscriptions have been created and I can’t find anything wrong with them.

Is there any documentation available for the error messages and what they mean for Enrich PubSub and the various other pipeline components’ docker images? I am finding setup quite arduous due to the opacity and lack of detail in the error messages to help me debug.

2

This is resolved. I found by grep-ing for my error message in the source code that the config parser wanted the PubSub topic name to be in a very particular format – not just the topic name, as it is for the Stream Collector’s config file, but in the format projects/<my-gcp-project>/topics/<my-topic-name>. The example config file gives a hint about this in the example topic name it includes, however there is no comment describing this particular requirement. I typically delete all example names without a second glance when I’m writing my config files, so this is a big area for improvement of the documentation for this component.

3

Thanks for the heads up @mariah_rogers , the error message is indeed not very helpful in this instance. I’ve opened an issue to address this in a future release: Pubsub: make error thrown by incorrect topic format more explicit · Issue #483 · snowplow/enrich · GitHub