30 Apr 19 · Atori ·       Add to Favorites  

Bigquery Error: Resources exceeded during query execution: Table will have 2501 partitions when the job finishes, exceeding limit 2500..'}]

Are you getting below error while moving data from unpartitioned to partitioned table.

"Resources exceeded during query execution: Table mydataset.mytable$20101213 
will have 2501 partitions when the job finishes, exceeding limit 2500..'}]"

Solution:

Per Bigquery docs, Currently, you cannot have more than 4000 partitions per table. You'll need to split your data/partitions across multiple tables (each of around 7 years of data) or redesign your partition keys to make sure you are within the limits.

Each partitioned table can have up to 4000 partitions.

Atori

posted on 30 Apr 19

Enjoy great content like this and a lot more !

Signup for a free account to write a post / comment / upvote posts. Its simple and takes less than 5 seconds

Copied