- Updated lib/DSNParser.pm, so it sets binmode to character set utf8 since
MySQL 8.0 if character set is not specified in the DSN
- Added test case
- Run update-modules
- Added check for Perl binary in lib/bash/parse_options.sh
- Updated tools
- Updated t/lib/bash/parse_options.sh, so it includes required package alt_cmds.sh
- Fixed sporadic pt-tc failures on rolled back SHOW MASTER STATUS code
- Added better diagnostic info for sporadically failing t/pt-table-checksum/basics.t
- Decreased chunk size it t/pt-table-checksum/progress.t, so this test
has less chances to fail, because pt-osc finishes faster than expected
* PT-2109 - pt-mysql-summary throws an error when using sql_mode="ANSI_QUOTES"
- Fixed issues still visible with SQL mode ANSI_QUOTES
- Added test cases for this mode
* PT-2109 - pt-mysql-summary throws an error when using sql_mode="ANSI_QUOTES"
- Removed keyring plugin check in the beginning of the test, because
reported error would show up even on servers that do not have keyring
plugin installed. Same applies to wsrep_on check and Percona Server
features checks.
- Removed temporary dir setup, because not needed for this test.
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node
- Proof of concept
- Fixed regular expression in lib/TableParser.pm mistakenly chaged in the tool's code
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node
- Added basic test case for PT-2168
- Added more details for replica lag information
- Disconnecting replica if lag is not checked. This prevents "Too many
connections" error
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node
- Implemented option --wait-lost-replicas for pt-osc, added test case
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node
- Added more tests for situations where connection to the replica can
fail
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node
- Removed extra checks for wait_no_die variable
- Added test cases for SQL queries that pt-osc sends to replicas
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node
- Allow to reload dsns table while waiting for missed replica if
--recursion-method is dsn
- Fixed logic in replica rediscovery, so it works with replicas on the
same host but with different ports
- Renamed option wait-lost-replicas to fail-on-stopped-replication, so
it is in line with pt-table-checksum
- Adjusted tests
- Removed debug code for PT-1760
- Added test case for PT-1760
- Added exception for variable Open_tables_with_triggers in
lib/bash/collect.sh due to failed test in Percona Server 8.0.34+
- Updated pt-stalk
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node
- Updated modules
- Fixed typo in t/pt-table-sync/bidirectional.t
- Removed trailing whitespaces in lib/MasterSlave.pm
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node
- Help for option --fail-on-stopped-replication
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node
- Added check for availability of the simple_rewrite_plugin in t/pt-online-schema-change/pt-2168.t
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node
- Added link to the simple_rewrite_plugin source code
- Removed tests for code that runs only in the beginning of pt-osc
action, so should not be affected by the option fail-on-stopped-replication
* Fix typos in lib/ directory
* Update generated bin/ files
* PR 655 - War on typos Act 1 #655
- Updated modules in tools that were not updated
- Fixed tests to reflect proposed changes
---------
Co-authored-by: Sveta Smirnova <sveta.smirnova@percona.com>
* PT-2156 - Fix tests for lib
Fixed tests, broken by putting fix for PT-1554 into the library code, mostly by updating checksums.
Added AutoCommit option for test lib/QueryReview.t
Reset SQL mode to empty to allow TableChunker to work with invalid and zero dates
* PT-2156 - Fix tests for lib
Adjusted t/lib/SchemaIterator.t to work with version 8.0
* PT-2156 - Fix tests for lib
Adjusted b/t/lib/Processlist.t, so it reflects fix for PT-981
* PT-2156 - Fix tests for lib
Adjusted t/lib/HTTP/Micro.t, so it works with different order of parameters, returned at v.percona.com/
Adjusted test for empty files in t/lib/bash/collect.t
Disabled mysqladmin debug test in t/lib/bash/collect.t, because of PT-2242
* PT-2156 - Fix tests for lib
Added LC_NUMERRIC=POSIX into t/lib/bash/report_system_info.sh, so reports in tests are not environment-dependent
Updated expected results in t/lib/bash/report_system_info.sh, so they reflect new information, collected by pt-summary
* PT-2156 - Fix tests for lib
- Improved fix for PT-76, so it handles inline comments
- Added test case for PT-76
- Improved fix for PT-1720, so it ignores unrecognizable option only if it comes from the toolkit-wide files and still errors out if wrong option was passed via command line or the tool-specific option file.
* PT-2156 - Fix tests for lib
- Improved fix for PT-2102, so it finds running instance configuration file using PID and also does not break t/lib/bash/report_mysql_info.t test
- Removed unnecessary copy-paste from t/pt-mysql-summary/pt-2102.t test
- Adjusted number of collected files in t/lib/bash/collect_mysql_info.sh
* PT-2156 - Fix tests for lib
- Fix for PT-1543 and MyRocks collection were originally put only into lib/bash/report_mysql_info.shthat broke the logic of collecting data first, then formatting report from this data. This, in its turn, broke test t/lib/bash/report_mysql_info.sh/t/lib/bash/report_mysql_info.t, because CMD_MYSQL is not defined in this library. I rewrote these fixes, so they follow original logic of the tool (pt-mysql-summary)
- Added tests for keyring plugin, encrypted tables, and MyRocks for t/lib/bash/report_mysql_info.sh and t/pt-mysql-summary/pt-mysql-summary_encryption.t
* PT-2156 - Fix tests for lib
- Added FLUSH TABLES to t/lib/bash/collect.sh, so it does not fail opentables tests if run when more than 1000 tables open in the sandbox environment
- Changed number of expected sample files to reflect keyring colletion file
* PT-2156 - Fix tests for lib
- Added skip to some tests in lib that file when run with PXC, because not supported to work with PXC
- Adjusted configuration files for PXC, so they allow LOAD DATA/SELECT INTO OUTFILE commands
- Adjusted data samples, so they do not depend on auto increment values
- Fixed lib tests, failing with PXC
* PT-2156 - Fix tests for lib
Updated tests for pt-online-schema-change, so they work with PXC and skipped if designed for semi-synchronous replication setup
* PT-2156 - Fix tests for lib
- Added cluster-specific samples for t/lib/SchemaIterator.t
- Removed extra debugging print from t/pt-table-checksum/pt-1728.t
* PT-2156 - Fix tests for lib
Evgeniy's review on July 20
* Update t/lib/TableSyncer.t
Removed comment left after debugging
* Update t/pt-mysql-summary/pt-mysql-summary_encryption.t
Removed extra debug output
* PT-2156 - Fix tests for lib
Tabs to spaces
* Version updated
* PT-2232 Release Notes 3.5.4 (#633)
modified: Changelog
modified: Makefile.PL
modified: config/sphinx-build/conf.py
modified: docs/release_notes.rst
* Updated version in the second place
---------
Co-authored-by: Alina Derkach <alina.derkach@percona.com>
HI, i am back
An error occurred when I migrated data from MySQL to OceanBase:【 DBD::mysql::db do failed: You have an error in your SQL syntax; check the manual that corresponds to your OceanBase version for the right syntax to use near ':= @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE /' at line 1 [for Statement "/!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */"] at /usr/bin/pt-archiver line 1932.】
I'm not a Perl programmer. But by reading the source code and making some changes, the migration was done successfully, so I submit this report
- [x] The contributed code is licensed under GPL v2.0
- [x] Contributor Licence Agreement (CLA) is signed
- [x] util/update-modules has been ran
- [x] Not involved Documention updated
- [x] Not involved Test suite update
* PT-2151 - Fix tests for pt-query-digest
Put fix for PT-1908 into the proper place
* PT-2151 - Fix tests for pt-query-digest
Put fix for PT-1554 into the proper place
* PT-2151 - Fix tests for pt-query-digest
Adjusted expected results for the default test t/pt-table-checksum/basics.t, so they do not depend on number of rows in the help tables
* PT-2151 - Fix tests for pt-query-digest
Added additional check for both replicas into t/pt-online-schema-change/preserve_triggers.t to avoid deadlock when ->ok() is doing CHECKSUM
* PT-2151 - Fix tests for pt-query-digest
Added delay to t/pt-table-sync/wait.t, so it waits for the child process to start lagging
* PT-2151 - Fix tests for pt-query-digest
Updated t/pt-query-digest/samples/issue_1196-output-8.0.txt, so it reflects changes in the latest 8.0
* PT-2151 - Fix tests for pt-query-digest
Updated queries against query history table, so they don't fail after e2cf183762
* PT-2151 - Fix tests for pt-query-digest
Fixed PT-813 by comparing query text. Since order itself does not matter, it is not essential to compare by the fingerprint or use any other function that changes the query.
* PT-2151 - Fix tests for pt-query-digest
Adjusted samples files which now should have consistent order after fix for PT-813
* PT-2151 - Fix tests for pt-query-digest
Fixed typo in the SELECT query in the QueryReview package
* PT-2151 - Fix tests for pt-query-digest
Fix for PT-981
* PT-2151 - Fix tests for pt-query-digest
Updated modules for pt-index-usage and fixed tests, because checksum is calculated differently after fix for PT-1554
* PT-2151 - Fix tests for pt-query-digest
Updated modules for pt-diskstats, pt-fk-error-logger, pt-heartbeat, pt-online-schema-change, pt-slave-delay, pt-slave-find, pt-table-checksum, pt-table-sync, pt-upgrade
* PT-2151 - Fix tests for pt-query-digest
Updated lib/IndexUsage.pm, so fix for pt-index-usage is in the correct place
* PT-2156 - Fix tests for lib
- Fixed tests broken for lib/TableParser.pm after fix for PT-1059
- Updated tests for lib/TableParser.pm that are broken due to SHOW CREATE TABLE output format change in 8.0
- Updated modules for all tools that use lib/TableParser.pm
* Revert "Fixed pt-archiver tests"
This reverts commit a3ab87b12e.
This commit wa needed, because removed code in sandbox/slave_channels.sql broked the test. Proper fix would be to do not remove channel names rather than chaging the test. So revertig it.
* PT-2156 - Fix tests for lib
- Updated test t/lib/TableChecksum.t so it reflects changes, introduced in the fix for PT-2016
- Updated test t/lib/RowChecksum.t so it reflects changes added to in the fix for PT-2138: UTF8 support
- Uncommented SQL in sandbox/slave_channels.sql that made t/lib/MasterSlave.t to fail
- Added check for undef into t/pt-archiver/archive_using_channels.t
- Updated lib/Cxn.pm so it uses $dbh->{Active} after issue with the ping() call, reported at https://github.com/perl5-dbi/DBD-mysql/issues/306
* PT-2156 - Fix tests for lib
- Impoved the fix for PT-2016, so it does not files with keys with USING keyword
- Added brackets to expression in lib/TableNibbler.pm, so it does not crap query wit many indexes with OR keyword
- Adjusted test t/lib/TableNibbler.t, so it reflects above chages
- Modified lib/Cxn.pm, so it has workaround for https://github.com/perl5-dbi/DBD-mysql/issues/306 , introduced in DBD::mysql 4.0.50
- Updated tests: added debugging code and cleanups
- Updated modules for tools
* PT-2160 - Fix tests for pt-online-schema-change
Re-enabled tests after MySQL bug #89441 fix in 8.0.14 and later
Updated t/pt-online-schema-change/samples/basic_no_fks_innodb.sql, so its unique index is NOT NULL
Added synchonizations when tests are waiting for too long due to slave catch up
Added innodb_directories option for t/pt-online-schema-change/pt-244.t
* PT-2160 - Fix tests for pt-online-schema-change
PT-2048 - pt-osc spans excessive connections to the replica when executing in the source
t/pt-online-schema-change/slave_lag.t was failing due to PT-2048. I added code that reuses already created connections when checks for slave in the wait loop. It fixes slave_lag.t too.
I also added that prints more detailed error when fails to connect to the replica.
* PT-2160 - Fix tests for pt-online-schema-change
Modified t/pt-online-schema-change/slave_lag.t, so it runs and clean ups faster.
* PT-2160 - Fix tests for pt-online-schema-change
Modified t/pt-online-schema-change/pt-1455.t, so it runs and clean ups faster.
* PT-2160 - Fix tests for pt-online-schema-change
Fixed search for HASH and BTREE keys broken by fix for PT-2123
Updated modules for pt-archiver
Fixed die message for sandbox/start-sandbox
Added global_grants table to list of exceptions for the ok function in lib/Sandbox.pm
Added wait_for_slaves to bugs.t, so changes on the source are copied to replica before pt-osc starts working
Updated PXC tests
Added wait_for_slaves to pt-1455.t
Fixed regular expression in pt-229.t, so it works with both 5.x and 8.0 versions
Added innodb_directories option for pt-244.t, so it does not fail on 8.0
Modified slave_lag.t, so it is more stable and slow enough, so pt-osc can print message about delayed replicas
* PT-2102 pt-mysql-summary should support include* in config file
includedir directories are now printed in the order of which they appear in root my.cnf
* PT-2102 pt-mysql-summary should support include* in config file
!include was not supported. Changed 'for' to 'while' to properly
test for '!includedir' and '!include'. Supports extra spaces between
files and !include.
* PT-2102 pt-mysql-summary should support include* in config file
Updates modules, added test case for !include and !includedir
Changed find_my_cnf function to make it try every regexes and only
returned when it find one
* Update pt-mysql-summary
Identation fix as suggested by review
* Update collect_mysql_info.sh
Identation as suggested by review
* PT-2102 pt-mysql-summary should support include* in config file
Fix formatting after review
---------
Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>
* PT-2150 Fix tests for pt-mysql-summary
- Added POSIX environment to tests, so they do not depend on the local user environment
- Enabled keyring plugin on 8.0
- Fixed encryption information collection in 8.0, see also PT-1588
* PT-2150 Fix tests for pt-mysql-summary
- Added support of the environment variable DISABLE_KEYRING, so we can diable keyring plugin in sandbox/servers/start
* PT-2150 Fix tests for pt-mysql-summary
- Reverted changes in the expected output for tests 006 and 007 made by commit 9190b5e6ac
Because the tool now works properly again and can parse Percona XtraDB Cluster data
* PT-2150 Fix tests for pt-mysql-summary
- Added comment '--read-samples' back to the test cases
* PT-1059 - Tools can't parse index names containing newlines
Fixed regular expressions in TableParser.
Added test case, including test for new lines in the column name
* PT-1059 - Tools can't parse index names containing newlines
Disabled pt-1637.t until PT-2174 is fixed.
Updated number of tables in b/t/pt-table-checksum/issue_1485195.t
* Patch newlines in table columns (#369)
Will accept this change as part of the fix for PT-1059 - Tools cannot parse index names containing new lines. We will later fix the issue with the patch ourselves.
mysql 5.6.40 allows newlines in column names however the following code:
my @defs = $ddl =~ m/^(\s+`.*?),?$/gm;
breaks due to it detecting newlines as line ends. The 'm' argument at the end does this by auto-detecting lines by newline characters.
To correct this issue I've made use of zero-length assertions known as " positive lookback"
https://www.regular-expressions.info/lookaround.html
what does it do?
m/(?:(?<=,\n)|(?<=\(\n))(\s+`(?:.|\n)+?`.+?),?\n/g;
TLDR:
Treat the string as one long string and don't treat \n as the end of a line.
look for (\s+`(?:.|\n)+?`.+?),?\n
if one of those matches look at what precedes the string
if it's ',\n' or ')\n' the string matches. Only save what's in (\s+`(?:.|\n)+?`.+?),?\n
m/ is declaring this a matching regex.
(?:(?<=,\n)|(?<=(\n)) This is an OR statement including two look-behind clauses. The ?: tells the enclosing parentheses to not store the result as a variable. I've put the two look-behinds in this OR statement below this line:
(?<=,\n) Look behind the matched string for a comma followed by a newline, the comma must be there for this look behind to match.
(?<=(\n) Look behind the matched string for a open parentheses followed by a newline, the open parentheses must be there.
(\s+`(?:.|\n)+?`.+?),?\n This is the actual match. Match newline character followed by one or more spaces followed by back-tick followed by a character which can be any character or a newline one or more times, but don't be greedy and take the rest of the match into consideration. Followed by a back tick and any character one or more times. This match stops where there is a comma or failing that a newline following a back tick and some characters.
,?\n match a comma that may not be there followed by a newline.
/g don't stop if this pattern matches keep looking for more patterns to the end of the string.
* PT-1059 - Tools can't parse index names containing newlines
Placed fix from PR-369 into proper place and created test case for this fix.
---------
Co-authored-by: geneguido <31323560+geneguido@users.noreply.github.com>
* PT-2084 Readding SHOW SLAVE STATUS outputs to pt-stalk
Before, they were only gathered if version was 5.6 or prior.
Now, they are always gathered, no matter what version.
Since ${mysql_version} is only major.minor, we can't yet add
SHOW REPLICA STATUS, because it starts on 8.0.22 and we have
no way to check that. We will need to wait for 8.1 release.
* PT-2084 Ran update-modules to generate binaries with changes.
* PT-2084 Added checks for preemptive 8.1 SHOW SLAVE STATUS removal.
* PT-2084 Adding test case for SHOW SLAVE STATUS
* Update t/pt-stalk/pt-stalk-replication.t
---------
Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>
* RM-1153 - Percona Toolkit 3.5.1
Updated version for Perl files to 3.5.1
Updated Perl module Percona::Toolkit to version 3.5.1 and updated all scripts using this module
Manually updated Go programs
Moved ONLY_UPDATE_VERSION after definition of DATE and SERIES variables, so the script does not fail because of undefined variables
* RM-1153 - Percona Toolkit 3.5.1
Updated Changelog
Updated release date in docs/percona-toolkit.pod
* PT-2162 Release Notes 3.5.1 (#571)
modified: Makefile.PL
modified: config/sphinx-build/conf.py
modified: docs/release_notes.rst
new file: docs/rn.3-5-1.txt
Co-authored-by: Alina Derkach <“alina.derkach@percona.com”>
* PT-7 Fix syntax (#573)
Co-authored-by: Alina Derkach <81975178+alina-derkach-oaza@users.noreply.github.com>
Co-authored-by: Alina Derkach <“alina.derkach@percona.com”>
Co-authored-by: EvgeniyPatlan <evgeniy.patlan@percona.com>
* PT-2138 - Fix tests for pt-table-checksum
- Updated t/pt-table-checksum/samples/default-results-8.0.txt and
t/pt-table-checksum/samples/static-chunk-size-results-8.0.txt
to support latest MySQL 8.0 version. Tests are now incompatible with elder 8.0 releases.
- Put fix for PT-136 into package RowChecksum
- Added execution bit for pt-online-schema-change
* PT-2138 - Fix tests for pt-table-checksum
- Fixed percona_test.load_data, so it really tests if LOAD DATA LOCAL INFILE is enabled
- Fixed option --[no]create-replicate-table, broken by commit c9836d5962
* PT-2138 - Fix tests for pt-table-checksum
- Enabled t/pt-table-checksum/error_handling.t for MySQL 8.0
- Fixed test t/pt-table-checksum/fnv_64.t and it's samples file
t/pt-table-checksum/samples/fnv64-sakila-city.txt
to reflect new function name convention and
changes after 62d84e5dba
* PT-2138 - Fix tests for pt-table-checksum
- Fixed t/pt-table-checksum/issue_1485195.t, so it checks only one table and
isn't get broken when we add tables into percona_test database
- Fixed typo in error output of bin/pt-table-checksum
- Skipped issue_47.t in 8.0 until https://jira.percona.com/browse/PT-1805 is fixed
* PT-2138 - Fix tests for pt-table-checksum
- Disabled pt-131.t for 8.0, because it does not have the QUERY_RESPONSE_TIME plugins
- Added SLOW_TESTS check to pt-1616.t
- Updated pt-226.t to include the fix for PT-1766
- For replication_filters.t: excluded false positive expression for tests 10 and 11 and added sys schema to the list of checked databases for 8.0
- Changed get_slaves in lib/MasterSlave.pm, so it returns slave's parent, required for wait_for_slaves in pt-table-checksum to work properly with chained slaves
* PT-2138 - Fix tests for pt-table-checksum
- Modified pt-204.t to support 8.0 and diffs in system tables due to timestamps
Moved the fix for PT-1616 into the proper place: lib/NibbleIterator.pm
* PT-2138 - Fix tests for pt-table-checksum
- pxc.t -added mysql.proxies_priv into ignore list, because its timestamp is different on node
- pxc.t - removed FORK=pxc from statup options for slave (non-cluster) nodes
- pxc.t - disabled wsrep replication with help of the variable wsrep_on: sql_log_bin doesn't disable wsrep replication anymore. See https://jira.percona.com/browse/PXC-3464 for details
- Removed data.tar.gz from 5.7 sandbox configuration, because it has an outdated definition for Performance Schema
- Disabled pxc.t for version 8.0 until PT-1699 is fixed
- start-sandbox - removed the first line (ALTER USER) from the init file, because it was rewritten by the next echo command, and then repeated later.
* PT-2138 - Fix tests for pt-table-checksum
- Adopted issue_1485195.t and basics.t for MyRocks-enabled setup
- replication_filters.t - added sys schema to the list of expected schemas for 5.7 and 8.0
- issue_1485195.t - added checks for the existence of mysql.plugin, func, and proxies_priv tables
- added samples/pt-131-wipe.sql that uninstalls QRT plugin if it was earlier installed by this test
-adjusted return code in pt-204.t, because expected differences in mysql.proxies_priv
* Update lib/PerconaTest.pm
removed diagnostic code
Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>
Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>
Sveta Smirnova