RDB Loader 1.1.0 docs refer to Shredding / EMR

1

Hi… I would edit the page, but being new to Snowplow the past year I’m not sure if what I’m seeing is a documentation bug.

On https://docs.snowplow.io/docs/getting-started-on-snowplow-open-source/setup-snowplow-on-aws/setup-destinations/setup-redshift/rdb-loader-1-1-0/, it states in step 2:

Shredder is run as an EMR step. It reads the directory from step 1. Step 1 and 2 are orchestrated by Dataflow Runner (or any other orchestration tool).

As of RDB Loader version R35, Shredding was no longer run from EMR, so I think this is outdated. Also, should all appearances of “Shredder” on this page be replaced with “Transformer” as that is now the new separate Transformer/Loader application?

2

Hi @pt-mike,

You are correct and this is indeed outdated! The required changes, however, are a bit more substantial. We need to replace the entire 1.1.0, 1.1.0 and pre-R35 pages with a fresh guide for the most recent version of RDB Loader. I feel that might be a bit too much to ask for an external contribution :slight_smile: Unfortunately, we haven’t gotten around to this ourselves yet, but it’s definitely on the radar.

1 Like
3

Hi @stanch – thanks for confirming! I understand the struggles with documentation :slight_smile:

1 Like