Hi @arihantsurana,
I’m not sure this is the reason for your troubles, but the key difference in bad rows for the stream pipeline is that the partition structure changes. The batch pipeline outputs a structure partitioned by run, but since the stream pipeline doesn’t run on a schedule in the same way, that’s not possible.
There’s a recent tutorial here on querying bad rows specifically for the real-time format. Note that if you have a lot of data in bad rows, it’s worth copying a sample to another bucket and querying that at first, to avoid running up charges.
Is it possible that your queries produce an undecipherable output because of this change in format?
Best,