Suggestions about GCP Setup Documentation

For engineers
1

Hello,

I am following this(below link) tutorial but I think it’s a bit outdated. Below, I’ll suggest some changes.

  • There is an item Under Source filter pick “Allow from any source”, there is no " allow from any source" option now, people should write 0.0.0.0/0 .
  • There is an item Then click “Upload Files” and upload your configuration file but which files? It needs some clarification. Also why do I need Storage Bucket since I have VM ?

Best,

2

Hi @Valdym,

Thanks a lot for your feedback. We updated the page with your suggestions. Please do not hesitate to let us know if you have any more comments / suggestions.

There is an item Then click “Upload Files” and upload your configuration file but which files? It needs some clarification. Also why do I need Storage Bucket since I have VM ?

The only purpose of the bucket is to host the configuration file of the collector, whose template can be found here. The VM is where the collector will run, but it still needs to be configured. For instance this is in the configuration file that we tell the collector which PubSub topics to read from/write to.

Cheers !

3

Hi Ben,

Thank you, I think this helps future data engineers to build their pipeline.

Also, Simo Ahava maked awesome tutorial about pipeline setup. I strongly recommend for newcomers

4

For sure we really aim at having awesome documentation!

Thanks for sharing, that’s great!