Just a small addition to this awesome thread. Yesterday we’ve released R102 Afontova Gora with new EmrEtlRunner’s Stream Enrich mode, which makes our Lambda architecture much simpler and more efficient. Now you don’t need to perform enrichment twice (in Spark Enrich and Stream Enrich) nor deal with complicated Dataflow Runner configurations.
4 Likes