Bigquery Error: Resources exceeded during query execution: Table will have 2501 partitions when the job finishes, exceeding limit 2500..'}]
Are you getting below error while moving data from unpartitioned to partitioned table.
"Resources exceeded during query execution: Table mydataset.mytable$20101213 will have 2501 partitions when the job finishes, exceeding limit 2500..'}]"
Per Bigquery docs, Currently, you cannot have more than 4000 partitions per table. You'll need to split your data/partitions across multiple tables (each of around 7 years of data) or redesign your partition keys to make sure you are within the limits.
Each partitioned table can have up to 4000 partitions.
Read great educational content like this and a lot more !
Members get free exclusive access to content, new courses, and discounts. Signup for a free account to write a post / comment / upvote posts. Creating an account takes less than 5 seconds and you can start earning badges & points too