Hello. We use igluctl to generate DDL for Redshift and noticed that when generating a root_tstamp column, the compression method is specified, but at the same time, amazon recommends not using compression for sort keys. Is there any way to remove this from DDL generation in igluctl and loader for Redshift? Or perhaps there is a reason why compression is needed for the sort key in this case?
It was also discussed here
I’m not too sure why this is the case - it probably make the most sense to have this as either ENCODE RAW or ENCODE AUTO.
If you are using rdb loader this creates (and mutates) the tables automatically so it’s worth noting that you should not need to use igluctl to do this.
The loader generates the same DDL. The question is not how best to generate, the question is why what is generated does not correspond to the best practices from Amazon
We’re going to look at doing some performance testing on a RAW / AUTO encoding and based on those performance tests we’ll look at making this change in schema-ddl.