MySQL Test Suite使用
MySQL自动测试套件（The MySQL Test Suite）用于对MySQL程序进行测试，包括各种功能与存储引擎。包含于MySQL与MariaDB版本代码中，位于mysql-test目录下。总体测试脚本为mysql-test-run.pl。该perl脚本通过调用各种已有的测试脚本，将测试结果与预置的result文件做对比来判断测试是否通过。（任何一点差异都会导致测试失败，包括预期之外的warning）
一、主要目录介绍
- include
包含.inc文件，用于测试开始时判断是否满足测试条件，在测试用通过source命令引入
如hava_innodb.inc:
1 2 3 4 | if (`SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.ENGINES WHERE engine = 'innodb' AND support IN ('YES', 'DEFAULT', 'ENABLED')`) { --skip Test requires InnoDB. } |
如不满足，则会跳过需要该验证的测试。
- suite
包含所有测试suite，每一个suite为一个测试用例集，具体用例集在2节介绍
- t
1、测试文件
例：1st.test
1 2 3 4 5 | # # Check that we haven't any strange new tables or databases # show databases; show tables in mysql; |
上面内容即为1st这个测试用例的测试脚本
2、配置文件
例：innodb_bug53674-master.opt
–loose-innodb-locks-unsafe-for-binlog –binlog-format=mixed
为该测试需要的特殊配置项
3、脚本文件，在测试开始之前执行的脚本
例：rpl_misc_functions-slave.sh
rm -f $MYSQLTEST_VARDIR/master-data/test/rpl_misc_functions.outfile
- r
结果文件，判断是否通过的依据
使用上面的例子：1st.result
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | show databases; Database information_schema mtr mysql performance_schema test show tables in mysql; Tables_in_mysql column_stats columns_priv db event func general_log gtid_slave_pos help_category help_keyword help_relation help_topic host index_stats innodb_index_stats innodb_table_stats plugin proc procs_priv proxies_priv roles_mapping servers slow_log table_stats tables_priv time_zone time_zone_leap_second time_zone_name time_zone_transition time_zone_transition_type user |
执行完1st用例会严格比对该文件。完全一致则测试通过。
- std_date
有些测试需要用到一些标准数据。
- var
测试默认使用该文件夹下的my.cnf文件
二、测试用例集
./mysql-test-run.pl不带任何参数表示执行所有测试用例集。使用—suite=suitename来指定单独执行suitename目录下的所有测试用例。
测试用例集包括：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | main：主要测试用例 archive binlog csv encryption：加密功能 engines federated funcs_1：额外功能（包括视图、存储过程、INFORMATION_SCHEMA等） funcs_2：额外功能（字符集等） galera galera_3nodes handler heap innodb innodb_fts：innodb全文索引 innodb_zip：innodb压缩 jp：日本字符相关测试 large_tests： maria mtr mtr2 multi_source ndb optimizer_unfixed_bugs parts percona perfschema perfschema_stress plugins roles rpl storage_engine stress sys_vars unit vcol wsrep |
三、主要参数
force：默认情况下，只要遇到一个用例出错，测试程序就会退出，在指定force的情况下，测试程序会继续执行下面的测试（但是最多发现10个错误还是会退出）
suite：指定使用的测试suite
do-test：会以输入的字符串进行匹配用例执行
skip-test：会以输入的字符串进行匹配跳过用例执行
big-test：执行标记为big的测试用例。因为用例较大、耗时较长，标记为big的用例默认不会执行。输入两遍big-test则只执行标记为big的测试用例
recored：重新生成结果文件
mysqld：传递启动参数给mysqld，每个参数需要分别指定一个mysqld
其他还有很多选项，可以使用./mtr –help查看
如果需要在服务启动前执行一些脚本，可以写在 t/testname.sh文件中，由mtr自动执行。
四、测试举例
例1：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | [lzk@redhat64-26 mysql-test]$ ./mtr --force --big-test --suite=archive Logging: ./mtr --force --big-test --suite=archive vardir: /home/lzk/mysql-test/var Checking leftover processes... Removing old var directory... Creating var directory '/home/lzk/mysql-test/var'... Checking supported features... MariaDB Version 10.1.12-MariaDB - SSL connections supported Using suites: archive Collecting tests... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 archive.archive_debug [ skipped ] Requires debug build archive.partition_archive [ pass ] 74 archive.archive_no_symlink [ pass ] 5 archive.archive-big [ pass ] 298619 archive.archive [ pass ] 417 archive.archive_bitfield [ pass ] 37 archive.archive_gis [ pass ] 66 archive.archive_symlink [ pass ] 11 archive.discover [ pass ] 47 archive.discover_5438 [ pass ] 7 archive.mysqlhotcopy_archive [ pass ] 1035 archive.repair [ pass ] 3 archive.archive_plugin [ pass ] 6 -------------------------------------------------------------------------- The servers were restarted 3 times Spent 300.327 of 309 seconds executing testcases Completed: All 12 tests were successful. 1 tests were skipped, 0 by the test itself. |
可以看出共完成12个测试用例，全部通过。1个用例必须在debug模式下测试，所以跳过。测试过程中服务重启3次。测试默认使用16000-16019端口。
例2：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | [lzk@redhat64-26 mysql-test]$ ./mtr binlog_innodb_row Logging: ./mtr binlog_innodb_row vardir: /home/lzk/mysql-test/var Checking leftover processes... - found old pid 28610 in 'mysqld.1.pid', killing it... process did not exist! Removing old var directory... Creating var directory '/home/lzk/mysql-test/var'... Checking supported features... MariaDB Version 10.1.12-MariaDB - SSL connections supported klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_550) Collecting tests... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 binlog.binlog_innodb_row 'innodb_plugin,row' [ pass ] 3361 binlog.binlog_innodb_row 'row,xtradb' [ pass ] 3349 -------------------------------------------------------------------------- The servers were restarted 1 times Spent 6.710 of 17 seconds executing testcases Completed: All 2 tests were successful. |
每个测试结束前，mtr会检索error日志，如果发现warning或error，则测试失败。
再来看两个失败用例：
（截取部分日志）
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | innodb.innodb_bug53674 'innodb_plugin' [ fail ] Found warnings/errors in server log file! Test ended at 2016-03-24 20:57:36 line 2016-03-24 20:57:35 7fd0fc6fc7e0 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. ^ Found warnings in /home/lzk/mysql-test/var/log/mysqld.1.err ok - saving '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-innodb_plugin/' to '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-innodb_plugin/' innodb.innodb_bug53674 'xtradb' [ fail ] Found warnings/errors in server log file! Test ended at 2016-03-24 20:57:37 line 2016-03-24 20:57:36 7f274c58b7e0 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. ^ Found warnings in /home/lzk/mysql-test/var/log/mysqld.1.err ok - saving '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-xtradb/' to '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-xtradb/' |
分析日志可以看出是在测试过程中在错误日志中发现了warnings/errors。原因是使用了废弃的参数innodb_locks_unsafe_for_binlog导致warning。
找到并修改配置文件：innodb_bug53674-master.opt
将–loose-innodb-locks-unsafe-for-binlog –binlog-format=mixed
修改为：–transaction-isolation=READ-COMMITTED –binlog-format=mixed
重新测试：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | [lzk@redhat64-26 mysql-test]$ ./mtr innodb_bug53674 Logging: ./mtr innodb_bug53674 vardir: /home/lzk/mysql-test/var Checking leftover processes... Removing old var directory... Creating var directory '/home/lzk/mysql-test/var'... Checking supported features... MariaDB Version 10.1.12-MariaDB - SSL connections supported klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_550) Collecting tests... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 innodb.innodb_bug53674 'innodb_plugin' [ pass ] 211 innodb.innodb_bug53674 'xtradb' [ pass ] 205 -------------------------------------------------------------------------- The servers were restarted 1 times Spent 0.416 of 10 seconds executing testcases Completed: All 2 tests were successful. |
不再有warning，测试通过。