How to log Google API Errors in Python
Errors are common for any application, it's just that we need to log and handle them appropriately. It looks like just about every error I get back from BigQuery is a apiclient.errors.HttpError, for example
<HttpError 409 when requesting https://www.googleapis.com/bigquery/v2/projects/some_id/datasets/some_dataset/tables?alt=json returned "Already Exists: Table some_id:some_dataset.some_table">
BigQuery is a REST API, the errors it uses follow standard HTTP error conventions.
In python, an HttpError has a resp.status field that returns the HTTP status code. This can be: notFound, duplicate, accessDenied, invalidQuery, backendError, resourcesExceeded, invalid, quotaExceeded, rateLimitExceeded, timeout, etc.
from googleapiclient.errors import HttpError try: ... except HttpError as err: # If the error is a rate limit or connection error, # wait and try again. if err.resp.status in [403, 500, 503]: time.sleep(5) else: raise
The response is also a JSON object, an even better way is to parse the JSON and read the error reason field:
if err.resp.get('content-type', '').startswith('application/json'): reason = json.loads(err.content).get('error').get('errors').get('reason')
Enjoy great content like this and a lot more !
Signup for a free account to write a post / comment / upvote posts. Its simple and takes less than 5 seconds