I can run the exact same command over and over again and sometimes it works, sometimes it doesn't:
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
ERROR 2002 (HY000): Can't connect to MySQL server on '10.245.54.251' (115)
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
#mysql50#lost+found
busman
demo_busman
information_schema
mysql
performance_schema
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
#mysql50#lost+found
busman
demo_busman
information_schema
mysql
performance_schema
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
#mysql50#lost+found
busman
demo_busman
information_schema
mysql
performance_schema
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
#mysql50#lost+found
busman
demo_busman
information_schema
mysql
performance_schema
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
#mysql50#lost+found
busman
demo_busman
information_schema
mysql
performance_schema
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
#mysql50#lost+found
busman
demo_busman
information_schema
mysql
performance_schema
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
ERROR 2002 (HY000): Can't connect to MySQL server on '10.245.54.251' (115)
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
ERROR 2002 (HY000): Can't connect to MySQL server on '10.245.54.251' (115)
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
ERROR 2002 (HY000): Can't connect to MySQL server on '10.245.54.251' (115)
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
ERROR 2002 (HY000): Can't connect to MySQL server on '10.245.54.251' (115)
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
#mysql50#lost+found
busman
demo_busman
information_schema
mysql
performance_schema
root@bak-rrk9m:/# mysql -Bsuroot -p"$MYSQL_ROOT_PASSWORD" -h"$KMB_MARIADB_SERVICE_SERVICE_HOST" -e "show databases"
ERROR 2002 (HY000): Can't connect to MySQL server on '10.245.54.251' (115)
My website, OTOH, doesn't seem to be getting any errors.
What's going on? How do I fix this?
It's the same issue whether I use the hostname or internal/cluster IP address.
Got some logs from the MariaDB pod. This might be it:
2020-06-15 0:51:51 12069 [Warning] Aborted connection 12069 to db: 'demo_kmbookings' user: 'root' host: '10.244.0.84' (Got an error writing communication packets)
Found some article with some suggestions but nothing concreate.
Thought maybe it was running out of RAM while creating those dumps since I cheaped out and only gave it 1 GiB or so. Fortunately Kubernetes made it easy to rebuild my whole cluster on a bigger node, but that didn't help.
Noticed now that my MariaDB service selector
didn't actually match my deploy template. I fixed that and now it's running successfully. Will have to try a few more times to be sure, but it makes me wonder how this ever worked.