Percona on 2nd node will not start

Question

I have setup two Vagrant VMs to simulate Percona XtraDB Cluster for our development needs. The first VM comes online just fine, and everything works. However, when I try to start Percona on the 2nd VM we get errors when requesting state transfer.

This is the error we get on db1:

rsync: failed to connect to 10.0.2.15: Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
rsync returned code 10:
130416 17:19:07 [ERROR] WSREP: Failed to read from: wsrep_sst_rsync --role 'donor' --address '10.0.2.15:4444/rsync_sst' --auth '(null)' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/data/' --defaults-file '/etc/my.cnf' --gtid '684d0eda-a6e5-11e2-0800-75b8bb33fbfb:3'
130416 17:19:07 [Note] WSREP: Provider resumed.
130416 17:19:07 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'donor' --address '10.0.2.15:4444/rsync_sst' --auth '(null)' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/data/' --defaults-file '/etc/my.cnf' --gtid '684d0eda-a6e5-11e2-0800-75b8bb33fbfb:3': 255 (Unknown error 255)
130416 17:19:07 [Warning] WSREP: 1 (db1.ellis.dev.liquidcompass.net): State transfer to 0 (db2.ellis.dev.liquidcompass.net) failed: -1 (Operation not permitted)

What I'm finding interesting is that it's trying to connect to 10.0.2.15, when the IP of db2 is 192.168.160.211. Running ifconfig on db1 show the following network configuration:

eth0      Link encap:Ethernet  HWaddr 08:00:27:42:B3:4F
          inet addr:10.0.2.15  Bcast:10.0.2.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe42:b34f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:668 errors:0 dropped:0 overruns:0 frame:0
          TX packets:371 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:55869 (54.5 KiB)  TX bytes:63186 (61.7 KiB)

eth1      Link encap:Ethernet  HWaddr 08:00:27:90:38:2A
          inet addr:192.168.160.80  Bcast:192.168.160.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe90:382a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2863 errors:0 dropped:0 overruns:0 frame:0
          TX packets:137 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:286053 (279.3 KiB)  TX bytes:18650 (18.2 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:4 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:200 (200.0 b)  TX bytes:200 (200.0 b)

So that is telling me that Percona is trying to use eth0 when rsyncing. Shouldn't it be using eth1?

Regards,
Andrew

Can you add the routing tables for both **db1** and **db2**, the output of ifconfig for **db2** and clarify which machine **db1** is supposed to rsync with? — , Apr 17 '13 at 00:01

score 3 · Accepted Answer · answered Apr 29 '13 at 08:46

The problem is that Percona by default uses eth0 unless overridden.

It uses the following command to work out the ip address it should communicate on:

/sbin/ifconfig | grep -E '^[[:space:]]+inet addr:' | grep -m1 -v 'inet addr:127' | sed 's/:/ /' | awk '{ print $3 }'

Which results in 10.0.2.15.

The wsrep_node_address variable can be set which tells it to use an alternate ip:

wsrep_node_address=192.168.160.80:4567

We had the same problem and it fixed it for us.

James

Thank you! I posted this on my old account and just saw the answer. Now I have replication working in my dev environment. — Andrew Ellis, Jun 18 '13 at 03:55

Percona on 2nd node will not start

1 Answers1