We have released an updated Google Cloud Storage loader library. The update features critical version updates and a change to default shard configuration.
For performance reasons the default shard number (--numShards
) configuration is set to follow runner’s settings. This results with a better performance in autoscalable runtimes such as GCP’s Dataflow. To preserve the old behaviour set --numShards=1
when running the job.