Low IOPS on GKE

Question

We are installing a MongoDB in Kubernetes Engine and it performs very poorly. We have installed it using Helm with 3 replicas (in a replicaset) and storage-class=premium-rwo (pd-ssd) with 500GB on each. It takes many seconds to just insert 500 documents. Google promises IOPS of about 15.000 but when looking at the monitoring we only see about 30 IOPS. What are we doing wrong?

The problem is that we do not receive the promised IOPS from Google - 500GB should give us 15.000 and we receive very few... does it only "kick in" when the system comes under stress?

Currently we are not confident to move our MongoDB from Atlas to our GKE cluster.

Does this answer your question? [Can you help me with my capacity planning?](https://serverfault.com/questions/384686/can-you-help-me-with-my-capacity-planning) — djdomi, May 16 '22 at 16:32
I suggest to edit your question and add the minimum redroducible information, like your helm charts, or if you followed a tutorial paste the link. — Gabriel Robledo Ahumada, May 17 '22 at 14:45

score 0 · Answer 1 · answered May 17 '22 at 06:34

0

Are you able to share more configuration detail, hard to provide much assistance with what is provided.

I hope your nodes have at least 100GB of PD-Balanced storage attached otherwise you can have performance issues.

answered May 17 '22 at 06:34

ServerMonkey

247
4
12

We only have 25GB on each node - I will try to increase that to 200GB – Oscar Orellana May 17 '22 at 06:56
Now running N2 with 8GB of ram, 2vCPU and 150GB of pd-blanaced storage – Oscar Orellana May 17 '22 at 07:03
Still very slow... – Oscar Orellana May 17 '22 at 10:37
Current setup: N2 with 8GB of ram, 2vCPU and 150GB SSD on each node (3 nodes). MongoDB is deployed with 300GB SSD PVC. Still very slow. – Oscar Orellana May 17 '22 at 10:38
Would it make any difference to deploy its own cluster, instead of its own node-pool? Currently, we have a backend node-pool with cluster scaling and a mongodb node-pool without. – Oscar Orellana May 17 '22 at 10:50
What about interactively connecting to a pod and stressing the disks to confirm whether it's disk IOPs or application configuration? – ServerMonkey May 18 '22 at 07:12

Low IOPS on GKE

1 Answers1