Quick Start on GCP - Iglu Server instance group creation timeout

For engineers Troubleshooting
1

Hi,

I’ve been working on the Snowplow Quick Start on GCP set up and I have an issue with creating Iglu Server instance group. I get a timeout error while executing ‘terraform apply’ command. The instance is unhealthy. In the instance logs I found only one error regarding ssh key. The message says that the ssh key format is invalid. I think I did everything right according to the guide and checked it multiple times. Could you please help me find the solution to this problem?
Also, has anyone successfully finished Quick Start on GCP set up after new release with BQ loader?

Best,
Ula

2

There was a similar issue here (which ended up being API related) that might be the issue Timeout Apply Iglu Server - #4 by cristhians13

3

Thank you for the answer! Unfortunately, all of the listed APIs are enabled in my project and the error still occurs.

4

Hi @ulakr I have noticed that SSH message myself and the service should still startup correctly. Not sure exactly why GCP no longer likes the input for SSH keys but it should not prevent the service startup-script from running properly (have tested this very recently).

If the error concerns you another option is to simply remove any SSH keys entirely and use the GCP UI to SSH into a particular node as and when needed.

Would you mind sharing the full logs from the deployment + the tfvars inputs you have used here please? Please redact anything sensitive!

5

Hi @josh,

Thank you for your help. I found the reason - the instance startup script didn’t work at all due to CRLF end of line sequence. After changing it, terraform apply completed without errors both for Iglu Server and the BQ pipeline.

But now, during testing I have some concerns. I’ve sent some test events and I can only find the messages which appear in enrich-server Pub/Sub subscription, nothing more.

1 Like
6

Hi @ulakr could you explain a little more what exactly you changed to get it working? Would be great to either make the quickstart more robust or add to the docs!

On your other point do you see any error messages in the logs for the BQ Loader applications?

7

Hi, Can you supply more details about what was the script you changed and what was the change ?

8

In windows had to convert all the .tf files to unix line endings with dos2unix.
Otherwise startup scripts do not run

9

Hi @Dimitry_Behak is this an issue with Windows specifically in how GitHub clones the files down or is there something mis-configured on our end? I don’t have a windows machine available to validate that unfortunately so if there is something we should change please do shout!

10

Hi, I am also facing the issue. Health check is getting timed out for me causing timeout in iglu server instance group creation. I need help.