3

We're starting to manage Kubernetes resources with Helm, and we have some users who are used to managing resources with kubectl edit. We'd like Helm to sanitize the deployed resources every time it's run, bringing them back into a known-good state.

I've observed that helm upgrade does not overwrite my ConfigMaps. Instead, it merges attributes between the deployed ConfigMap and the Helm template, giving me pieces of my templated ConfigMap and pieces of the hand-edited one. If there were no changes in the Helm ConfigMap template, Helm does not reset any part of my deployed ConfigMap back to a know-good state.

How can I instruct Helm to always reset my entire Kubernetes resource to the Helm-templated versions?

spiffytech
  • 1,043
  • 2
  • 11
  • 16
  • 1
    This is not a direct answer to your question, but you might want to consider terraform as an alternative to helm if you want this kind of behavior – Matt Zimmerman Sep 09 '19 at 19:10

1 Answers1

0

This isn't a design goal of Helm, and continues to not be. It is not within Helm's purview to manipulate objects that were not created by Helm, or to disregard out-of-band changes to objects under its "control" - this is a feature that makes Helm compatible with other systems that may change manifests, and makes Helm safe to use. Remember that it is a YAML rendering system, not a configuration management system like Puppet or SaltStack. It does not "enforce" configurations.

If you wish to accomplish the behavior you describe, you could delete the objects in question prior to re-creating them with Helm. You can potentially use Kubernetes namespaces to accomplish this reliably by giving each chart its own namespace, and deleting the namespace prior to chart installation as a form of "hard reset".

Or keep those users who are used to editing manifests with kubectl from doing that, and instead direct them to pushing those changes into charts or values. It doesn't seem reasonable to discard their changes if they needed to be made - this is exactly why Helm performs three-way merge operations on manifests to begin with. I understand the desire to persist configuration in a single point of orchestration, but don't expect to accomplish this by not using of a single point of orchestration.

This problem comes up consistently when applying higher level orchestration practices to teams that are accustomed to lower level orchestration methods, and is in no way conceptually constrained to Helm. Helm is good at handling the situation gracefully by performing merge operations, but results in a functioning mess of orchestration methodologies (at least it doesn't break things or simply fail). Start by not making the mess, or constraining the mess to a scratch environment and scooping it up into Helm charts for production.

Spooler
  • 7,016
  • 16
  • 29
  • I can appreciate this as a design tradeoff for the reasons you gave, though I'm disappointed as it's not the appropriate tradeoff for the teams I work in. In my experience, trusting that "this git repo accurately represents our infrastructure" far outweighs the costs of stomping on someone's out-of-band changes. – spiffytech Sep 30 '20 at 15:18
  • 1
    I agree, wholeheartedly. Keeping the state of infrastructure flowing in the correct direction is critical to reasonably maintaining a complex system. Terraform is capable of throwing YAML into Kubernetes easily with the toYaml function and the various resources available via the Kubernetes provider, and has a much stronger concept of overriding enforcement. This has other tradeoffs; Terraform doesn't have a strong concept of packaging like Helm does, and is prone to freak out with errors if it can't assert dominance on the subject cluster. It's hard to get around having to make people do stuff – Spooler Sep 30 '20 at 18:20