Bigquery : Resources exceeded during query execution: Table will have 2501 partitions when the job finishes, exceeding limit 2500
Are you getting below error while moving data from unpartitioned to partitioned table.
"Resources exceeded during query execution: Table mydataset.mytable$20101213 will have 2501 partitions when the job finishes, exceeding limit 2500..'}]"
Solution:
Per Bigquery docs, Currently, you cannot have more than 4000 partitions per table. You'll need to split your data/partitions across multiple tables (each of around 7 years of data) or redesign your partition keys to make sure you are within the limits.
Each partitioned table can have up to 4000 partitions.
Atori
posted on 30 Apr 19Enjoy great content like this and a lot more !
Signup for a free account to write a post / comment / upvote posts. Its simple and takes less than 5 seconds
Post Comment