Hi,
we are using the Elasticsearch-Loader to store our events (snowplow-elasticsearch-loader-http-0.10.1.jar).
We had the case now, that our ES storage ran out, causing the loader to write numerous error messages to the syslog including the data of each event which couldnt be stores in ES. Hence, within a short range of time the storage of the instance where the loader runs on ran out of space.
I am aware that we neglected some things here and that we have to put measures in place that this doesnt happen again. Among others, i would like to ask if there is a possibility to configure the es-loader in terms of what he logs to syslog. I havent really found any documentation on this.
For example i would like to now that X events coudnt be stored to ES but i dont need the data of those events in the syslog.
Would be very glad for some advice.
Thanks a lot.