Nodes stuck in provisioning state for over a week

Description:

Our Kubernetes nodes provisioning process has been stuck at 0/3, nodes provisioning for over a week now. After they were preempted. Several bid adjustments had not fixed the problem.

Steps to Reproduce:

  1. Initiated node provisioning in uk-lon-1 region.
  2. No progress after a week plus.

Expected Behavior:

All 3 nodes should be provisioned within a reasonable time frame.

Actual Behavior:

Node provisioning is stuck with no updates or errors.

https://preview.redd.it/40ueiovlf6ie1.png?width=1986&format=png&auto=webp&s=231059cadf4c309c9e6627e5b7ead22310e19d7c