I'm trying to restore a full backup taken from one node of a three-node percona cluster (percona cluster 5.5, galera 2.1, wsrep sst method is rsync, innodb tables only).
Backup was taken like this:
rm -rf /tmp/backup/mysqldb
innobackupex --user=bkpuser --password=xxxx --galera-info --no-timestamp /tmp/backup/mysqldb/
innobackupex --apply-log --use-memory=2G /tmp/backup/mysqldb/
The restore procedure I'm attempting is like this:
- shut down mysql on all three nodes
- on first node
- delete contents of mysql data dir
- delete mysql redologs/binlogs, doublewrite file etc (they are located in a separate folder)
- copy back database tablespace files for my database instance
- copy back database tablespace files for mysql database
- redologs/binlogs, doublewrite file, etc
- start mysql with
wsrep_urls = gcomm://
to initialize cluster
- on second and third node
- delete mysql redologs/binlogs, doublewrite file etc
- remove
galera.cache
andgrastate.dat
- start mysql (
wsrep_urls = gcomm://firstnode:port,gcomm://secondnode:port,gcomm://thirdnode:port
)
- once the cluster is all synced, restart first node with full
wsrep_urls
configuration
(I know wsrep_urls is deprecated, but I haven't gotten galera to work with wsrep_cluster_address yet.)
My problem is that the second and third nodes report errors after SST. For each table, I see this error:
130225 15:44:43 [ERROR] Cannot find or open table myTestDb/settings from
the internal data dictionary of InnoDB though the .frm file for the
table exists. Maybe you have deleted and recreated InnoDB data
files but have forgotten to delete the corresponding .frm files
of InnoDB tables, or you have moved .frm files to another database?
or, the table contains indexes that this version of the engine
doesn't support.
See http://dev.mysql.com/doc/refman/5.5/en/innodb-troubleshooting.html
how you can resolve the problem.
Mysql show tables
shows the tables as existing, but when attempting to select from them, it reports error Table 'myTestDb.settings' does not exist
...
I have tried to delete the local tablespace files before starting up mysql and requesting sst, same result.
How should I perform the restore? Should I copy back the backup files to all nodes?