1

Since last night App Engine has been throwing 503 errors for deploy, delete, and traffic migration requests. This is happening from both the CLI and the console.

App Engine is still serving traffic, but it seems all the admin functions are not working. Others have reported this issue as well on the GCP Slack

Error Message with --verbosity=debug

DEBUG: Response returned status 503, retrying
DEBUG: Retrying request to url https://appengine.googleapis.com/v1/apps/horatio-project/services/stage/versions?alt=json after exception HttpError accessing <https://appengine.googleapis.com/v1/apps/horatio-project/services/stage/versions?alt=json>: response: <{'status': '503', 'content-length': '122', 'x-xss-protection': '0', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'vary': 'Origin, X-Origin, Referer', 'server': 'ESF', '-content-encoding': 'gzip', 'cache-control': 'private', 'date': 'Mon, 09 Mar 2020 18:57:17 GMT', 'x-frame-options': 'SAMEORIGIN', 'alt-svc': 'quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000', 'content-type': 'application/json; charset=UTF-8'}>, content <{
  "error": {
    "code": 503,
    "message": "The service is currently unavailable.",
    "status": "UNAVAILABLE"
  }
}
>
DEBUG: Response returned status 503, retrying
DEBUG: Retrying request to url https://appengine.googleapis.com/v1/apps/horatio-project/services/stage/versions?alt=json after exception HttpError accessing <https://appengine.googleapis.com/v1/apps/horatio-project/services/stage/versions?alt=json>: response: <{'status': '503', 'content-length': '122', 'x-xss-protection': '0', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'vary': 'Origin, X-Origin, Referer', 'server': 'ESF', '-content-encoding': 'gzip', 'cache-control': 'private', 'date': 'Mon, 09 Mar 2020 18:57:57 GMT', 'x-frame-options': 'SAMEORIGIN', 'alt-svc': 'quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000', 'content-type': 'application/json; charset=UTF-8'}>, content <{
  "error": {
    "code": 503,
    "message": "The service is currently unavailable.",
    "status": "UNAVAILABLE"
  }
}
>
DEBUG: (gcloud.app.deploy) UNAVAILABLE: The service is currently unavailable.
Traceback (most recent call last):
  File "/home/thesandlord/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 981, in Execute
    resources = calliope_command.Run(cli=self, args=args)
  File "/home/thesandlord/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 807, in Run
    resources = command_instance.Run(args)
  File "/home/thesandlord/google-cloud-sdk/lib/surface/app/deploy.py", line 110, in Run
    default_strategy=flex_image_build_option_default))
  File "/home/thesandlord/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 643, in RunDeploy
    ignore_file=args.ignore_file)
  File "/home/thesandlord/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 433, in Deploy
    extra_config_settings)
  File "/home/thesandlord/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/appengine_api_client.py", line 172, in DeployService
    extra_config_settings)
  File "/home/thesandlord/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/appengine_api_client.py", line 249, in _CreateVersion
    return self.client.apps_services_versions.Create(create_request)
  File "/home/thesandlord/google-cloud-sdk/lib/googlecloudsdk/third_party/apis/appengine/v1/appengine_v1_client.py", line 826, in Create
    config, request, global_params=global_params)
  File "/home/thesandlord/google-cloud-sdk/lib/third_party/apitools/base/py/base_api.py", line 729, in _RunMethod
    http, http_request, **opts)
  File "/home/thesandlord/google-cloud-sdk/lib/third_party/apitools/base/py/http_wrapper.py", line 346, in MakeRequest
    check_response_func=check_response_func)
  File "/home/thesandlord/google-cloud-sdk/lib/third_party/apitools/base/py/http_wrapper.py", line 402, in _MakeRequestNoRetry
    check_response_func(response)
  File "/home/thesandlord/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/apis.py", line 273, in _CheckResponseForApiEnablement
    http_wrapper.CheckResponse(response)
  File "/home/thesandlord/google-cloud-sdk/lib/third_party/apitools/base/py/http_wrapper.py", line 223, in CheckResponse
    raise exceptions.BadStatusCodeError.FromResponse(response)
BadStatusCodeError: HttpError accessing <https://appengine.googleapis.com/v1/apps/horatio-project/services/stage/versions?alt=json>: response: <{'status': '503', 'content-length': '122', 'x-xss-protection': '0', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'vary': 
'Origin, X-Origin, Referer', 'server': 'ESF', '-content-encoding': 'gzip', 'cache-control': 'private', 'date': 'Mon, 09 Mar 2020 18:58:45 GMT', 'x-frame-options': 'SAMEORIGIN', 'alt-svc': 'quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000', 'content-type': 'application/json; charset=UTF-8'}>, content <{
  "error": {
    "code": 503,
    "message": "The service is currently unavailable.",
    "status": "UNAVAILABLE"
  }
}
>
ERROR: (gcloud.app.deploy) UNAVAILABLE: The service is currently unavailable.
Sandeep Dinesh
  • 111
  • 2

1 Answers1

0

The 503 error is on Google's side and its considered as a transient error (goes away without user intervention) the only thing that can be done when this happens is try again after a while, if the error last for too long like you mentioned it's recommended to reach out to Google support so they can assist.

Andres S
  • 186
  • 3