I'm updating a Kubernetes cluster from version 1.17 to 1.18 by using kOps (v1.18) with the command kops update cluster --yes
Just before the end of update operation update fails with a message:
I0924 08:00:09.277510 22260 executor.go:111] Tasks: 0 done / 119 total; 42 can run
I0924 08:00:09.956582 22260 executor.go:111] Tasks: 42 done / 119 total; 26 can run
I0924 08:00:10.385824 22260 executor.go:111] Tasks: 68 done / 119 total; 49 can run
I0924 08:00:11.259824 22260 executor.go:111] Tasks: 117 done / 119 total; 2 can run
W0924 08:00:11.618475 22260 executor.go:136] error running task "AutoscalingGroup/master-eu-central-1b.masters.cluster.my_domain.org" (9m59s remaining to succeed): error detaching TargetGroups: ValidationError: Trying to remove Target Groups that are not part of the group: AccessPoint(accessPointName=master-eu-central-1b.masters.cluster.my_domain.org-kube-ing-TG-1H011E1FCW114, vpcId=null, accessPointType=TARGET_GROUP)
status code: 400, request id: ebf9f3d0-83e6-48de-8c74-662dc6cae075
W0924 08:00:11.618514 22260 executor.go:136] error running task "AutoscalingGroup/nodes.cluster.my_domain.org" (9m59s remaining to succeed): error detaching TargetGroups: ValidationError: Trying to remove Target Groups that are not part of the group: AccessPoint(accessPointName=nodes.cluster.my_domain.org-kube-ing-TG-1H011E1FCW114, vpcId=null, accessPointType=TARGET_GROUP)
status code: 400, request id: a763a98e-e93f-4f45-a91b-6a8592a21266
kops update cluster command informs:
I0924 10:29:16.363679 6443 executor.go:111] Tasks: 0 done / 119 total; 42 can run
I0924 10:29:17.060157 6443 executor.go:111] Tasks: 42 done / 119 total; 26 can run
I0924 10:29:17.560326 6443 executor.go:111] Tasks: 68 done / 119 total; 49 can run
I0924 10:29:18.567699 6443 executor.go:111] Tasks: 117 done / 119 total; 2 can run
I0924 10:29:18.661992 6443 executor.go:111] Tasks: 119 done / 119 total; 0 can run
Will modify resources:
AutoscalingGroup/master-eu-central-1b.masters.cluster.my_domain.org
TargetGroups [name:master-eu-central-1b.masters.cluster.my_domain.org-kube-ing-TG-1H011E1FCW114 id:arn:aws:elasticloadbalancing:eu-central-1:381749774323:targetgroup/kube-ing-TG-1H011E1FCW114/c4f3719c67327c8b] -> []
AutoscalingGroup/nodes.cluster.my_domain.org
TargetGroups [name:nodes.cluster.my_domain.org
-kube-ing-TG-1H011E1FCW114 id:arn:aws:elasticloadbalancing:eu-central-1:381749774323:targetgroup/kube-ing-TG-1H011E1FCW114/c4f3719c67327c8b] -> []
Why update operation tries to delete those target groups and why it fails?
The previous update (1.16 to 1.17) was successful without any problems.
