Dataflow Runner 0.4.1 released

We are pleased to announce Dataflow Runner 0.4.1:

This release centers around the run-transient command, which, as a reminder, allows you to spin up an EMR cluster, run steps and shut it down as one command.

This release modifies its behaviour so that it eagerly evaluates and stores in memory both the cluster and playbook configurations. This way you can launch a run-transient and remove the configurations straight away without issues.

1 Like