06 November 2018 · Atori ·       Add to Favorites   Report

BigQuery Invalid Dataset ID in Cloud Shell

While querying from the google cloud shell, and try to access the dataset and table through python

from google.cloud import bigquery

client = bigquery.Client()
dataset_id = 'mytest-0001:reports_test'
table_id = 'test_data'
dataset_ref = client.dataset(dataset_id)
dataset = client.get_dataset(dataset_ref)
table_ref = dataset_ref.table(table_id)
table = client.get_table(table_ref)
print('Dataset ID: '.format(dataset_id))
print('Description: '.format(dataset.description))
print(table.schema)
print(table.description)
print(table.num_rows)

It throws the below error:

google.api_core.exceptions.BadRequest: 400 GET https://www.googleapis.com/bigquery/v2/projects/gcd-my-reporting/datasets/mytest-0001:reports_test: Invalid dataset ID "mytest-0001:reports_test". Dataset IDs must be al
  phanumeric (plus underscores, dashes, and colons) and must be at most 1024 characters long.
With some reason I can't modified the dataset id, Any idea to fixed this?

Solution:

Make sure you pass,

  • The Project name while initializing the client
  • And specify the dataset id without the project prefix

The code will be:

from google.cloud import bigquery

client = bigquery.Client(project='mytest-0001')
dataset_id = 'reports_test'

Atori

posted on 06 November 2018

Read great educational content like this and a lot more !

Members get free exclusive access to content, new courses, and discounts. Signup for a free account to write a post / comment / upvote posts. Creating an account takes less than 5 seconds and you can start earning badges & points too

Copied