Upgrading collector

[volderette]

Hi,

do you have any best practices on upgrading the collector on Google Cloud? Basically I would try to do a rolling update of my compute engine and replace the startup script + application.conf. Since this is a critical part, do you have any advice on how to ensure proper rollout?

Thanks
volderette

[volderette]

Specifically some more questions on the update to Madara Rider:

• Do I have to update the JavaScript-Tracker as well to ensure compatibility of the scala collector?

• For Safari ITP, what are the ideal settings? I’ve read the blog posts, but there are still open questions:
  This should be ok:

  secure = true
  httpOnly = true

But what to do with sameSite? What are you suggesting as setting?

• What would be the best way to validate a new config file?

• Is it sufficient to overwrite the config or do I have to restart the collector process?

• Custom collector paths are a new feature I see in the config option in the hacon file. Do I have to setup anything in the Javascript? I guess though?

Thank you so much for your feedback!

[dilyan]

Hi @volderette,

any advice on how to ensure proper rollout

It’s best to do a rolling update, where you deprecate old nodes in favour of new nodes 1 by 1. If you notice any issues after each update, you roll back.

It’s also worth testing the configuration before deploying. You can use for example the stdout collector so you can send requests with cURL and verify that all looks good locally.

Do I have to update the JavaScript-Tracker as well to ensure compatibility of the scala collector?

It might be necessary, eg if you plan to use custom paths or if you want to set a collector cookie on multiple domains from the same collector. (See the post linked below for specifics.)

For Safari ITP, what are the ideal settings?

It might be best to set SameSite=None. (See the post linked below for specifics.)

What would be the best way to validate a new config file?

The easiest way currently is to try and launch an stdout collector with the new config. If there are any issues, that will fail.

Is it sufficient to overwrite the config or do I have to restart the collector process?

You have to re-deploy the collector, using the new artefact. If you’re planning on using multiple domains, you’ll also have to configure the proper endpoints (which is not done via the hocon config and depends on which cloud you are).

Custom collector paths are a new feature I see in the config option in the hacon file. Do I have to setup anything in the Javascript?

Yes, this part of the documentation explains how to set the path to be used in the JavaScript tacker. For more details on how to set it up in the collector, see the post linked below.

We’ve tried to compile a list of known tricky issues when upgrading to Madara Rider. Please review and see if it answers your outstanding questions:

How to properly configure the Scala Stream Collector v0.16.0

With the recent release of Snowplow R116 Madara Rider, we introduced a host of new features to the Scala Stream Collector. Users now have greater control over the attributes attached to the Set-Cookie header of the collector response. However this means that the configuration file has had to become even more complex. If you’re migrating from an earlier version, it’s now easier to break stuff by simply copying the old hocon over. Also, you now have to be more aware about the specific implications…

[volderette]

Sorry for the late reply and thank you very much for the information! This is really appreciated!