I have a question regarding the use of the Iglu resolver in the DB Loader for Databricks. Given that the Transformer already resolves the schema and converts the data into Parquet format before storing it in S3, what is the purpose of having another Iglu resolver in the DB Loader?
I understand it might relate to validation and schema evolution, but I’d appreciate a detailed explanation of why this additional step is necessary.