Snowplow Mini 0.13 released

Announcements New releases
1

We are pleased to announce version version 0.13 of Snowplow Mini.

There have been a number of 0.13 releases in quick succession.

UPDATED: We recommend using version 0.13.5
See new release post for information on the latest releases: Snowplow Mini 0.13.4 and 0.13.5 released

This release upgrades pipeline components to latest versions and introduces new Control Plane API endpoints to manage telemetry setting of the collector, and to restart individual services and to reset Elasticsearch. There is also mitigation in place regarding the Log4Shell vulnerability.

Upgraded components

Mini 0.13.3 is now available.

Mini 0.13.2 comes with:

  • Caddy 2.4.6
  • Snowplow Stream Collector NSQ 2.4.2
  • Snowplow Stream Enrich NSQ 2.0.3
  • Snowplow Elasticsearch Loader 1.0.0
  • Snowplow Iglu Server 0.6.1
  • Postgres 9.5
  • NSQ v1.2.1
  • Elasticsearch-OSS 6.8.20
  • Kibana-OSS 6.8.20
  • cAdvisor 0.43.0

New endpoints for Control Plane API

  • Details of the new endpoint to enable/disable telemetry:

    Available at GET/PUT <mini-address>/control-plane/telemetry path

    HTTP GET to get current configuration

    curl -L -X GET '<mini-address>/control-plane/telemetry' -H 'Authorization: Basic <token>'

    HTTP PUT to set it (use true or false as value of key disable)

    curl -L -X PUT '<mini-address>/control-plane/telemetry' -H 'Authorization: Basic <token>' -H 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'disable=false'

  • Details of the new endpoint to reset Elasticsearch & Kibana:

    Available at POST <mini-address>/control-plane/reset-service path

    curl -L \
-X POST '<mini-address>/control-plane/reset-service' \
-H 'Authorization: Basic <token>' \
-H 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'service_name=elasticsearch'

  • Details of the new endpoint to restart services individually:

    Available at PUT <mini-address>/control-plane/restart-service path

    curl -L -X PUT '<mini-address>/control-plane/restart-service' -H 'Authorization: Basic <token>' -H 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'service_name=<service_name>'

    where service_name can be one of the following: collector, enrich, esLoaderGood, esLoaderBad, iglu, kibana, elasticsearch.

Reset vs Restart of Elasticsearch

We recommend you reset Elasticsearch whenever you find your events are not getting loaded into Elasticsearch. While restarting restarts Elasticsearch only, resetting Elasticsearch deletes indexes with their data and index patterns from Kibana, creates new Elasticsearch indexes from scratch and the corresponding index patterns in Kibana. This addresses the problem that occurs when a new schema version is published which is incompatible with previous versions; a common scenario for Mini users when designing new tracking.

Telemetry

To better understand how Snowplow Mini is used, we have introduced some basic telemetry with this release. This change will help us answer some key questions that we don’t currently have visibility of, such as - how popular is Mini, and how much usage is the new version of the application seeing?

You can find the context which we send with each heartbeat event here.

Enhanced password security

The passwords used to access Snowplow Mini have a new stength requirement. Password strength is determined by the zxcvbn library. A good idea to meet the new password strength requirements is to generate a password of at least 15 characters, with a mix of upper and lower case characters, numbers and special characters. Although there are no hard requirements, it needs to be complex and secure.

CHANGELOG

  • Upgrade axios to 0.21.2 (#313)
  • Upgrade ca-certificates upstream in the setup playbook (#312) (thanks @eldarshamukhamedov)
  • Make it possible to refresh Elasticsearch index mapping (#192)
  • Add dedicated endpoint to restart individual services (#310)
  • Bump Caddy to 2.4.6 (#303)
  • Bump cadvisor to 0.43.0 (#311)
  • Make Elasticsearch index creation fault-tolerant (#309)
  • Configure a base path for nsqadmin (#217)
  • Bump NSQ to 1.2.1 (#300)
  • Add cronjob to crop nsq data (#297)
  • Add new endpoint to manage telemetry (#307)
  • Bump Stream Collector to 2.4.2 (#314)
  • Bump Stream Enrich to 2.0.3 (#305)
  • Bump Elastic stack to 6.8.20 (#306)
  • Fix invalid vagrant ip address (#302)
  • Fix JAVA_OPTS environment variable (#316)
  • Use -Dlog4j2.formatMsgNoLookups=true flag (#317)

Checkout the documentation to learn how to setup and use latest Mini on AWS and GCP.

Project repository can be found here and community contributions are always welcome!

2 Likes
2

Hi. I’m having issues bringing up Vagrant with this latest release (brand new to Snowplow Mini) - it fails on the npm install step. I created a bug in the repo with the error log, but wondering if there’s a step I’m missing that’s not documented?

3

Hey @angelsk I was just responding to you on Github :slight_smile:

1 Like