22 Aug 18 · Bozhack-miller ·       Add to Favorites   Report

BigQuery - How to find the error stream?

While using the bq command to load data into Bigquery, you will often have reject/error records based on bad data in the columns or even mismatch in the number of columns. 

The Error message usually looks like the below:

Error While reading data, error message: CSV table encountered too many 
errors, giving up, Rows: 3894; errors:1 Please look into the error 
stream for more details

Where to find the error stream?

Run the following command in the terminal:

bq --format=prettyjson show -j <JobID>

It returns a JSON with more details. In my case it was:

"message": "Error while reading data, error message: Could not parse '16.66666666666667' 
as int for field Course_Percentage (position 46) starting at location 1717164"


posted on 22 Aug 18

Read great educational content like this and a lot more !

Members get free exclusive access to content, new courses, and discounts. Signup for a free account to write a post / comment / upvote posts. Creating an account takes less than 5 seconds and you can start earning badges & points too