2

I've been trying to start my existing GCP VM that has an NVIDIA T4 GPU attached to it, for almost a month at this time. It has been working fine before but now I am constantly getting the error message:

The zone '***' does not have enough resources available to fulfill the request. Try a different zone, or try again later."

Which indicates that there are no GPUs available.

Starting any VM with a GPU in another zone does not work either, nor can I start other existing VMs in other projects. Starting VMs without any GPUs attached works perfectly fine however.

All evidence points towards GCP just not having any available GPUs but I cannot believe this would be the case for almost a month at this point.

Any insight into this?

masus04
  • 121
  • 4
  • 3
    There are only three possible answers you can get here: "I have the same problem", "it works ok for me" and *"ask GCP support"*. – Massimo Jan 11 '22 at 17:22

2 Answers2

3

Yes, the error indicates that there is not enough free resources in the zone to start your instance. Lots of users are facing this nowadays.

I was just able to spin up a T4 attached N1-standard-1 instance in zone southamerica-east1-c.

Try moving your instance to this zone and start it there. You can find instructions here.

Consider reserving a GPU if you plan to keep the instance running for a long time so that you will not face this issue again.

Erhard Czving
  • 141
  • 5
  • Our team is restricted to using resources in europe, and none of the european zones have had availability whenever we tried. Any idea how long these issues usually persist? Our workloads kind of depend on the promised flexibility of getting a GPU whenever we need them. – masus04 Jan 14 '22 at 14:07
  • No idea. Usually does not apply here I believe. With the ongoing chip shortage there is much less GPU available than the demand for it. Gamers and probably cloud providers struggle as well. The GPU prices are sky high and you may not be able to just buy more. My suggestion is to try to get approval to use resources in other regions as I am sure EU regions have a much higher demand for GPUs. – Erhard Czving Jan 14 '22 at 15:56
0

Had the same issue. no zone let me create a gpu machine due to not enough resources. However, i managed to create one trying every zone that has gpus available:

$ for zone in {\
asia-east1-a,\
asia-east1-b,\
asia-east1-c,\
asia-northeast1-a,\
asia-northeast1-c,\
asia-northeast3-b,\
asia-northeast3-c,\
asia-south1-a,\
asia-south1-b,\
asia-southeast1-a,\
asia-southeast1-b,\
asia-southeast1-c,\
asia-southeast2-a,\
asia-southeast2-b,\
asia-southeast2-c,\
australia-southeast1-b,\
australia-southeast1-c,\
europe-west1-b,\
europe-west1-c,\
europe-west1-d,\
europe-west2-a,\
europe-west2-b,\
europe-west3-b,\
europe-west4-a,\
europe-west4-b,\
europe-west4-c,\
northamerica-northeast1-a,\
northamerica-northeast1-b,\
northamerica-northeast1-c,\
southamerica-east1-c,\
us-central1-a,\
us-central1-b,\
us-central1-c,\
us-central1-f,\
us-east1-b,\
us-east1-c,\
us-east1-d,\
us-east4-a,\
us-east4-b,\
us-east4-c,\
us-west1-a,\
us-west1-b,\
us-west2-b,\
us-west2-c,\
us-west4-a,\
us-west4-b\
}; do gcloud compute instances create gpu-machine-$zone --project=xxxx --zone=$zone --machine-type=......; done
netbrain
  • 703
  • 1
  • 6
  • 11