GCS Dataflow Job Error

For engineers GCP pipeline
1

Hey,

I have a GCS Loader running as a Dataflow job and faced the following issue.

image
image1484×179 54.3 KB

The job says its streaming but the backlog in the pubsub is increasing and there is no data being written into the bucket.

For the time being I have restarted the job.

The following are the job parameters

image

Is there a solution for this?

Thank you!

2

Hey @siv,

Could you tell me what kind of data you are sending to GCS Loader ? Also, could you give us full error stack trace if it is possible ?

1 Like
3

Hi @enes_aldemir,

I am using multiple GCS loaders actually. The following GCS Loader gave the above mentioned error:

  • GCS Loader to write enricher bad records to bucket
  • GCS Loader to write collector good records to bucket

This is all I can grab from the logs:

image
image1307×283 59.1 KB

4

Thanks for the additional information @siv. Could you confirm that you have some other GCS Loader instances which works fine but this two is giving the mentioned error ?

5

All of them give errors, but these two have greater load comparatively, hence the error count is higher.