Has anyone hooked up Redshift to Snowplow Mini? Curious what your approach was.
Hi @AlbertZerok,
Hooking Redshift up to Snowplow Mini is difficult at the moment, because a core part of the load code can only run right now on Hadoop (the Scala Hadoop Shred component). But the good news is that part of the recent RFC for migrating the Snowplow batch jobs to Spark proposes moving this Redshift load process to Spark/Spark Streaming, which would make it much easier to embed in Snowplow Mini.
Obviously a lot to do first but I wanted to share the possible direction…