Send Test request and verify it was published into Pub/Sub

[Elena_Paraschiv]

Hi, I am trying to send a test request using curl and verify it was published into Pub/Sub.

I configured my project using GCP and I am following the article of Simo to set it up.
I get stuck in Step 6: Send a test request and verify it was published into Pub/Sub.

The command I run in the terminal:
curl -d "&e=pv&page=curl-test&url=http%3A%2F%2Fjust-testing.com&aid=snowplow-test" -X POST http://<EXTERNAL_IP>:8080/com.snowplowanalytics.iglu/v1

gives me the error in the screenshot.
Can someone help me identify the issue ?

[Colm]

Hi @Elena_Paraschiv,

I think this is likely down to one of three things, so let’s see if any of the following works:

1. The endpoint you’re pointing at is the endpoint for the Iglu Webhook. There’s a more generic endpoint we could try - try replacing com.snowplowanalytics.iglu/v1 with com.snowplowanalytics.snowplow/tp2 in your endpoint.

2. I think it used to be required to include the port number in the URL, but now it’s not. Try removing :8080 (so eg. http://<EXTERNAL_IP>/com.snowplowanalytics.snowplow/tp2).

3. If neither of those work, then my main suspicion would be the network rules for the IP. Indeed in your screenshot ‘Connect’ seems to be set to ‘SSH’.

I’m not massively familiar with where to find/how to configure network rules on GCP, but the collector endpoint’s IP address should be configured to accept traffic from the internet. Typically, it’s configured with ‘allow all’ - since it needs to receive data from the users’ devices, which could come from anywhere.

Honestly, my suspicion as to what’s going on is stronger on point 3 since you’re getting timeouts. I would expect 1 and 2 to return a 4XX if misconfigured - however those will be easier to check quickly.

Best,

[mike]

@Colm

I think it used to be required to include the port number in the URL, but now it’s not. Try removing :8080 (so eg. http://<EXTERNAL_IP>/com.snowplowanalytics.snowplow/tp2 ).

This will depend on the port the collector is running on (e.g., 80 or 8080 in the setup). By default if you select HTTP / HTTPS access access will work over 80 / 443 but not port 8080.

If the collector is being served on 8080 you’ll need to modify your network tags for the VM (and add a firewall rule) which will then allow you to accept traffic on this port.

To check what port the collector is running on you can always SSH into the instance directly and run something like:

curl -XGET 127.0.0.1:8080/i

or

curl -XGET 127.0.0.1/i

and you should hopefully get a response from one of these depending on the port.

[Elena_Paraschiv]

Thanks Colm.

I have tried approach 1. and 2. , but it failed ( screenshot 1)

Regarding approach 3. here is a screenshot of the network rules in GCP. (

.
I am following this config file uploaded to GCP Buckets with my own values.I can send snippets of it if relevant

[Elena_Paraschiv]

@mike

curl -XGET 127.0.0.1/i

Hi Mike,

Thank you for your reply. I have tried the commands in the VM instance

The firewall rule is added

[mike]

Hi Elena,

Based on the article from Simo the collector should be running on port 8080. What was the command you used on the instance to start the collector?

[Elena_Paraschiv]

Hi Mike,

I ran the following commands:

pretty much same as in Simo’s article,but I replaced application.config to this https://github.com/snowplow/snowplow/blob/master/2-collectors/scala-stream-collector/examples/config.hocon.sample