We are pleased to announce version version 0.13 of Snowplow Mini.
There have been a number of 0.13 releases in quick succession.
UPDATED: We recommend using version 0.13.5
See new release post for information on the latest releases: Snowplow Mini 0.13.4 and 0.13.5 released
This release upgrades pipeline components to latest versions and introduces new Control Plane API endpoints to manage telemetry setting of the collector, and to restart individual services and to reset Elasticsearch. There is also mitigation in place regarding the Log4Shell vulnerability.
Mini 0.13.2 comes with:
- Caddy 2.4.6
- Snowplow Stream Collector NSQ 2.4.2
- Snowplow Stream Enrich NSQ 2.0.3
- Snowplow Elasticsearch Loader 1.0.0
- Snowplow Iglu Server 0.6.1
- Postgres 9.5
- NSQ v1.2.1
- Elasticsearch-OSS 6.8.20
- Kibana-OSS 6.8.20
- cAdvisor 0.43.0
New endpoints for Control Plane API
Details of the new endpoint to enable/disable telemetry:
HTTP GET to get current configuration
curl -L -X GET '<mini-address>/control-plane/telemetry' -H 'Authorization: Basic <token>'
HTTP PUT to set it (use true or false as value of key
curl -L -X PUT '<mini-address>/control-plane/telemetry' -H 'Authorization: Basic <token>' -H 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'disable=false'
Details of the new endpoint to reset Elasticsearch & Kibana:
curl -L \ -X POST '<mini-address>/control-plane/reset-service' \ -H 'Authorization: Basic <token>' \ -H 'Content-Type: application/x-www-form-urlencoded' \ --data-urlencode 'service_name=elasticsearch'
Details of the new endpoint to restart services individually:
curl -L -X PUT '<mini-address>/control-plane/restart-service' -H 'Authorization: Basic <token>' -H 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'service_name=<service_name>'
service_namecan be one of the following:
Reset vs Restart of Elasticsearch
We recommend you reset Elasticsearch whenever you find your events are not getting loaded into Elasticsearch. While restarting restarts Elasticsearch only, resetting Elasticsearch deletes indexes with their data and index patterns from Kibana, creates new Elasticsearch indexes from scratch and the corresponding index patterns in Kibana. This addresses the problem that occurs when a new schema version is published which is incompatible with previous versions; a common scenario for Mini users when designing new tracking.
To better understand how Snowplow Mini is used, we have introduced some basic telemetry with this release. This change will help us answer some key questions that we don’t currently have visibility of, such as - how popular is Mini, and how much usage is the new version of the application seeing?
You can find the context which we send with each heartbeat event here.
Enhanced password security
The passwords used to access Snowplow Mini have a new stength requirement. Password strength is determined by the
zxcvbn library. A good idea to meet the new password strength requirements is to generate a password of at least 15 characters, with a mix of upper and lower case characters, numbers and special characters. Although there are no hard requirements, it needs to be complex and secure.
- Upgrade axios to 0.21.2 (#313)
- Upgrade ca-certificates upstream in the setup playbook (#312) (thanks @eldarshamukhamedov)
- Make it possible to refresh Elasticsearch index mapping (#192)
- Add dedicated endpoint to restart individual services (#310)
- Bump Caddy to 2.4.6 (#303)
- Bump cadvisor to 0.43.0 (#311)
- Make Elasticsearch index creation fault-tolerant (#309)
- Configure a base path for nsqadmin (#217)
- Bump NSQ to 1.2.1 (#300)
- Add cronjob to crop nsq data (#297)
- Add new endpoint to manage telemetry (#307)
- Bump Stream Collector to 2.4.2 (#314)
- Bump Stream Enrich to 2.0.3 (#305)
- Bump Elastic stack to 6.8.20 (#306)
- Fix invalid vagrant ip address (#302)
- Fix JAVA_OPTS environment variable (#316)
- Use -Dlog4j2.formatMsgNoLookups=true flag (#317)
Checkout the documentation to learn how to setup and use latest Mini on AWS and GCP.
Project repository can be found here and community contributions are always welcome!