2

I have a metric that I’m using for a HPA, the problem is that the metric has spikes and to avoid so, I’m using an average over time in a record rule on prometheus, but to export it to stackdriver(on gke) I’m using stackdriver-prometheus-sidecar and it does not send recorded metrics to stackdriver.

Any idea on how to export this metric? Other solution that I found was to use v2beta2 hpa and change the behaviour to change the stabilization window on the scale up, but this is only available on v1.18 on kubernetes and GCP only has version 1.17 on the stable channel, any idea?

Uwe Keim
  • 2,370
  • 4
  • 29
  • 46
José Pedro Machado
  • 21
  • 3
  • Hello. As you pointed the workaround would be to use Kubernetes version `1.18`. Currently this version is available only in `Rapid` channel. Is it feasible for you to use the `Rapid` channel till the version `1.18` will be available in `Regular` channel? Also you could create a feature request to backport the HPA `v2beta2` to `GKE` `1.17`. You can do it by following this link to [Issuetracker.google.com](https://issuetracker.google.com/issues/new?component=187077&template=0). It's already in correct component. – Dawid Kruk Dec 21 '20 at 15:44
  • It is possible on my dev cluster, but for the production one I need something more stable, Google says in the documentation that a version gets available on the regular channel two months after being available on the Rapid but the 1.18 got available in September and still nothing, how many time it can take? – José Pedro Machado Dec 22 '20 at 12:13
  • You can check the [GKE Release notes](https://cloud.google.com/kubernetes-engine/docs/release-notes) for any information about updates. – Dawid Kruk Dec 24 '20 at 15:29

1 Answers1

1

Posting this answer as a community wiki. Feel free to expand it.

Focusing on the part of scaling behavior:

As said in the question. Support for configurable scaling behavior is starting from the Kubernetes version 1.18:

Support for configurable scaling behavior

Starting from v1.18 the v2beta2 API allows scaling behavior to be configured through the HPA behavior field. Behaviors are specified separately for scaling up and down in scaleUp or scaleDown section under the behavior field. A stabilization window can be specified for both directions which prevents the flapping of the number of the replicas in the scaling target. Similarly specifying scaling policies controls the rate of change of replicas while scaling.

Kubernetes.io: Docs: Tasks: Run application: Horizontal pod autoscale: Support for configurable scaling behavior

Kubernetes version 1.18 which includes the support for above scaling is included in the release notes of a Regular channel:

January 8, 2021 (2021-R1)

Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in master upgrades and node upgrades for existing clusters. See these instructions for more information on the Kubernetes versioning scheme.

Regular channel

Note: Your clusters might not have these versions available. Rollouts begin on the day of the note and take four or more business days to be completed across all Google Cloud zones. For more information, see the Rollout schedule.

  • Version 1.17.14-gke.400 is now available in the Regular channel. This version is now the default.
  • Version 1.18.12-gke.1201 is now available in the Regular channel.
  • Version 1.17.13-gke.2600 is no longer available in the Regular channel.

Auto-upgrading nodes in the Regular channel automatically upgrade from version 1.17 to version 1.17.14-gke.400 with this release.

I encourage everyone to review the Release channels documentation and the properties of each channel(trade-off between feature availability and update churn):

You could also make a feature request for backporting this functionality to be included in previous GKE versions here:

Dawid Kruk
  • 588
  • 2
  • 8