I've got a MariaDB database (version 10.2) that I'm attempting to back up with Mariabackup (the official fork of Percona's XtraBackup). Both the DB server and the backup tool are installed from the MariaDB Ubuntu repository for 10.2.
After moving the backup job to another server due to a hardware failure, mariabackup is now failing to back up a particular table. From the stderr logs:
[00] 2019-09-10 01:00:01 Connecting to MySQL server host: localhost, user: root, password: set, port: 3306, socket: /var/run/mysqld/mysqld.sock
[00] 2019-09-10 01:00:01 Using server version 10.2.21-MariaDB-10.2.21+maria~trusty-log
mariabackup based on MariaDB server 10.2.23-MariaDB debian-linux-gnu (x86_64)
[00] 2019-09-10 01:00:01 uses posix_fadvise().
[00] 2019-09-10 01:00:01 cd to /var/lib/mysql/
[00] 2019-09-10 01:00:01 open files limit requested 65535, set to 65535
[00] 2019-09-10 01:00:01 mariabackup: using the following InnoDB configuration:
[00] 2019-09-10 01:00:01 innodb_data_home_dir =
[00] 2019-09-10 01:00:01 innodb_data_file_path = ibdata1:12M:autoextend
[00] 2019-09-10 01:00:01 innodb_log_group_home_dir = ./
[00] 2019-09-10 01:00:01 InnoDB: Using Linux native AIO
[00] 2019-09-10 01:00:01 using O_DIRECT
2019-09-10 1:00:01 140221145274240 [Note] InnoDB: Number of pools: 1
[00] 2019-09-10 01:00:01 mariabackup: Generating a list of tablespaces
…big snip…
2019-09-10 1:42:47 140220995024640 [Note] InnoDB: Read redo log up to LSN=59053785766400
[00] 2019-09-10 01:42:47 >> log scanned up to (59053785785093)
[00] 2019-09-10 01:42:48 >> log scanned up to (59053785831385)
[03] 2019-09-10 01:42:49 Database page corruption detected at page 993055, retrying...
[03] 2019-09-10 01:42:49 Database page corruption detected at page 993055, retrying...
[03] 2019-09-10 01:42:49 Database page corruption detected at page 993055, retrying...
[00] 2019-09-10 01:42:49 >> log scanned up to (59053785894073)
[03] 2019-09-10 01:42:49 Database page corruption detected at page 993055, retrying...
[03] 2019-09-10 01:42:50 Database page corruption detected at page 993055, retrying...
[03] 2019-09-10 01:42:50 Database page corruption detected at page 993055, retrying...
[03] 2019-09-10 01:42:50 Database page corruption detected at page 993055, retrying...
[03] 2019-09-10 01:42:50 Database page corruption detected at page 993055, retrying...
[03] 2019-09-10 01:42:50 Database page corruption detected at page 993055, retrying...
[03] 2019-09-10 01:42:50 Error: failed to read page after 10 retries. File ./database_name/table_name.ibd seems to be corrupted.
2019-09-10 1:42:50 140220969846528 [Note] InnoDB: Page dump in ascii and hex (8192 bytes):
…snipped page dump…
InnoDB: End of page dump
2019-09-10 1:42:50 140220969846528 [Note] InnoDB: Compressed page type (11); stored checksum in field1 1097107227; calculated checksums for field1: crc32 1097107227, innodb 1095451288, none 3735928559; page LSN 57324874439498; page number (if stored to page already) 993055; space id (if stored to page already) 4854
InnoDB: Page may be a compressed BLOB page
[03] 2019-09-10 01:42:50 mariabackup: xtrabackup_copy_datafile() failed.
[00] FATAL ERROR: 2019-09-10 01:42:50 failed to copy datafile.
I've verified the integrity of this replica against the primary database using pt-table-checksum, which returned no errors, so I don't think the table is actually corrupted.
The table in question is compressed (from SHOW CREATE TABLE: ENGINE=InnoDB AUTO_INCREMENT=… DEFAULT CHARSET=utf8 ROW_FORMAT=COMPRESSED). I found an open bug against Xtrabackup that seems to match this situation, but the MariaDB documentation on page compression seem to say that this has been fixed in mariabackup:
Note that Percona XtraBackup (as of 2.4) will not work with MariaDB compression. However, MariaDB's fork, MariaDB Backup, will work with compression.
This isn't the only compressed table in the database, but it appears to be the first one the backup tool got to. I would rather not have to rebuild these tables as uncompressed.
This seems like a bug in Mariabackup, but I'm open to other suggestions as well.
EDIT: I've opened a ticket: MDEV-20588.
