I'm migrating a Galera cluster from MySQL to MariaDB 10.3.27, and as part of the process I'd like to change the backup method from the somewhat fiddly (and blocking) garbd+rsync to garbd+mariabackup. Changing the SST method to mariabackup worked fine from the start, so I suppose the nodes are set up correctly, but the way I call garbd is wrong:
garbd --address 'gcomm://10.2.96.145:4567,10.2.96.146:4567?gmcast.listen_addr=tcp://0.0.0.0:4444' --group m1t-shared-mariadb --donor m1t-shared-db-5 --sst mariabackup
Port 4444 is copy/pasted from documentation, I think it pertains to the local garbd's listening port? Anyway it doesn't matter whether I use 4444 or the rest of the cluster's 4567.
On the donor node I see:
2021-01-25 17:34:26 0 [Note] WSREP: Member 0.0 (garb) requested state transfer from 'm1t-shared-db-5'. Selected 3.0 (m1t-shared-db-5)(SYNCED) as donor.
2021-01-25 17:34:26 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 19280263)
2021-01-25 17:34:26 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2021-01-25 17:34:26 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'donor' --address '' --socket '/var/lib/mysql/mysql.sock' --datadir '/data/mysql/' --gtid 'c12b9eef-1780-11eb-bac6-da564ff9aacc:19280263' --gtid-domain-id '0' --mysqld-args --basedir=/usr'
2021-01-25 17:34:26 2 [Note] WSREP: sst_donor_thread signaled with 0
2021-01-25 17:34:26 0 [Note] WSREP: 0.0 (garb): State transfer from 3.0 (m1t-shared-db-5) complete.
WSREP_SST: [INFO] Streaming with xbstream (20210125 17:34:26.604)
WSREP_SST: [INFO] Using socat as streamer (20210125 17:34:26.606)
WSREP_SST: [ERROR] Upgrade joiner to 5.6.21 or higher for backup locks support (20210125 17:34:26.609)
WSREP_SST: [ERROR] The joiner is not supported for this version of donor (20210125 17:34:26.611)
WSREP_SST: [ERROR] Cleanup after exit with status:93 (20210125 17:34:26.613)
WSREP_SST: [INFO] Cleaning up temporary directories (20210125 17:34:26.617)
2021-01-25 17:34:26 0 [ERROR] WSREP: Failed to read from: wsrep_sst_mariabackup --role 'donor' --address '' --socket '/var/lib/mysql/mysql.sock' --datadir '/data/mysql/' --gtid 'c12b9eef-1780-11eb-bac6-da564ff9aacc:19280263' --gtid-domain-id '0' --mysqld-args --basedir=/usr
2021-01-25 17:34:26 0 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'donor' --address '' --socket '/var/lib/mysql/mysql.sock' --datadir '/data/mysql/' --gtid 'c12b9eef-1780-11eb-bac6-da564ff9aacc:19280263' --gtid-domain-id '0' --mysqld-args --basedir=/usr: 93 (Protocol not supported)
The error Upgrade joiner to 5.6.21 or higher for backup locks support seems to be a red herring stemming from the fact that the required version is parsed from the --address parameter and that's conspicuously empty.
Grasping for clues I took to reading MariaDB's sql/wsrep_sst.cc file to find out how --address is filled before calling the wsrep_sst_* plugins. If the wsrep_sst_receive_address variable is set to anything but "AUTO", it's supposed to just use that; otherwise it uses wsrep_node_address or, failing that, some guessing procedure (and yes, that's just as documented). However, that doesn't seem to work. --address is always empty, no matter what I set wsrep_sst_receive_address to. Also, an error should be logged if wsrep_sst_receive_address is "AUTO" and wsrep_node_address is not a valid IP address. I tried to provoke that to find out what code path is taken but wasn't able to either.
Any idea what's going wrong here?
