09 August 2018 · victor ·       Add to Favorites   Report

How to log Google API Errors in Python

Errors are common for any application, it's just that we need to log and handle them appropriately. It looks like just about every error I get back from BigQuery is a apiclient.errors.HttpError, for example

<HttpError 409 when requesting https://www.googleapis.com/bigquery/v2/projects/some_id/datasets/some_dataset/tables?alt=json returned "Already Exists: Table some_id:some_dataset.some_table">

BigQuery is a REST API, the errors it uses follow standard HTTP error conventions.

In python, an HttpError has a resp.status field that returns the HTTP status code. This can be: notFound, duplicate, accessDenied, invalidQuery, backendError, resourcesExceeded, invalid, quotaExceeded, rateLimitExceeded, timeout, etc.

For example:

from googleapiclient.errors import HttpError
except HttpError as err:
  # If the error is a rate limit or connection error,
  # wait and try again.
  if err.resp.status in [403, 500, 503]:
  else: raise

The response is also a JSON object, an even better way is to parse the JSON and read the error reason field:

if err.resp.get('content-type', '').startswith('application/json'):
    reason = json.loads(err.content).get('error').get('errors')[0].get('reason')

Read great educational content like this and a lot more !

Members get free exclusive access to content, new courses, and discounts. Signup for a free account to write a post / comment / upvote posts. Creating an account takes less than 5 seconds and you can start earning badges & points too