Commit Graph

295 Commits

Author SHA1 Message Date
Sveta Smirnova
c49c58db2b Merge pull request #941 from percona/PT-2355_pt-online-schema-change_should_not_resume_a_job_with_empty_boundaries
PT-2355 - pt-online-schema-change should not resume a job with empty …
2025-03-27 21:13:43 +03:00
Sveta Smirnova
88367c1dea PT-2407 - pt-online-schema-change exit status(return code) is 0 even if it does NOT succeed
- Changed die call for the case when ALTER TABLE fails for the new table
- Removed stalled comment from Frank
- Added test case
2025-03-26 17:51:35 +03:00
Sveta Smirnova
840ba6926b PT-2355 - pt-online-schema-change should not resume a job with empty boundaries
- The tool should still fail, so I improved error message and added test case
2025-03-22 00:01:56 +03:00
Sveta Smirnova
4bf48d864f PT-2422 - pt-online-schema-change race overwrites new_table_name
- Added test case
2025-03-13 21:30:00 +03:00
Sveta Smirnova
5c16d37020 PT-2389 - pt-online-schema-change resume functionality doesn't work with ADD/DROP column
- Modified pt-1717-resume.t test case, so it passes now
2025-03-12 17:31:43 +03:00
Perry Harrington
888af5f5ef Updated test for PT-1717 to perform meaningful alter so fix for PT-2389 would be exercised 2025-01-28 11:51:03 -08:00
Sveta Smirnova
f57d63ab00 PT-2340 - Support MySQL 8.4
- Added sleep to t/lib/MasterSlave.t, so it does not fail on fast machines
- Disabled SSL tests on versions earlier than 8.0
2024-11-14 22:48:06 +03:00
Sveta Smirnova
3786d4e4ce PT-2340 - Support MySQL 8.4
- Added SSL to 8.4 tests
2024-11-14 03:20:43 +03:00
Sveta Smirnova
33086769a1 PT-2340 - Support MySQL 8.4
- Added test for replica lag check for pt-archiver
- Re-added deprecated slave- options
- Added tests for deprecation warnings and for legacy options for pt-archiver
- Removed practically not supported options --replica-user and --replica-password from pt-archiver, pt-kill, pt-query-digest
- Added legacy source/replica options (master/slave) variants and tests for pt-heartbeat, pt-online-schema-change, pt-replica-find, pt-replica-restart, pt-table-checksum, pt-table-sync
- Updated modules after lib/MasterSlave.pm change
2024-10-04 18:27:00 +03:00
Sveta Smirnova
5d55904436 PT-2340 - Support MySQL 8.4
- Tests for the minimal SSL support
- Updated util/update-modules, so they don't skip tools with modules, not defined in lib
2024-09-18 15:28:45 +03:00
Sveta Smirnova
e3e352d1d6 PT-2340 - Support MySQL 8.4
- Fixed comparison operators for VersionCheck calls
2024-08-27 13:45:37 +03:00
Sveta Smirnova
b332537481 PT-2340 - Support MySQL 8.4
- Fixed staff I broke for 8.0
2024-08-23 16:03:53 +03:00
Sveta Smirnova
76c1202cb5 PT-2340 - Support MySQL 8.4
- Updated modules and tests for pt-online-schema-change
- Removed typo from lib/MasterSlave.pm
2024-07-30 18:20:50 +03:00
Sveta Smirnova
aaa258d209 Added master server restart in the cleanup for t/pt-online-schema-change/pt-2168-cancel.t test, so next test is not affected by not properly closed connections in 5.7 2024-06-07 22:12:44 +03:00
Sveta Smirnova
f8d2f59e62 PT-2119 - pt-osc aborts in 8.0.15 even if no FK exists
- Adjusted test case, so it is skipped on 5.7
2024-05-30 03:38:20 +03:00
Sveta Smirnova
c5cafbb486 PT-2231 - pt-osc + PTDEBUG=1 fails with Use of uninitialized value in concatenation (.) or string at ./pt-online-schema-change line 4309.
- Re-implemented fix for PT-1799 properly
- Added test case
- Updated modules
2024-04-11 21:47:11 +03:00
Sveta Smirnova
a9a4bc5b6c Merge pull request #792 from percona/PT-2119_pt-osc_aborts_in_8.0.15_even__if_no_FK_exists
PT-2119 - pt-osc aborts in 8.0.15 even if no FK exists
2024-04-08 18:40:11 +03:00
Sveta Smirnova
50bc80c243 Merge pull request #783 from percona/PT-1717_resume_pt-online-schema-change_if_interrupted
PT-1717 resume pt-online-schema-change if interrupted
2024-04-08 18:39:17 +03:00
Sveta Smirnova
c87a5da5ba PT-2119 - pt-osc aborts in 8.0.15 even if no FK exists
- Moved check after we identified child tables
- Added test case
2024-04-01 23:36:54 +03:00
Sveta Smirnova
e9ec7213f4 Merge pull request #769 from percona/PT-2295_Fix_tests_on_distributions
PT-2295 - fix tests on distributions
2024-03-15 16:24:29 +03:00
Sveta Smirnova
57e4008a2f PT-1717 - resume pt-online-schema-change if it's interrupted
- dash in options --no-drop-new-table and --no-drop-triggers
2024-03-14 22:19:16 +03:00
Sveta Smirnova
2d4f5c290a PT-1717 - resume pt-online-schema-change if it's interrupted
- Now --resume dies if new table or triggers do not exist
- New table and triggers are not removed if --resume fails during these
  checkups
- Added sample file and more tests for --resume
2024-03-14 21:20:50 +03:00
Sveta Smirnova
8ce3b94a1c PT-1717 - resume pt-online-schema-change if it's interrupted
- Added test case for failures and recovery
- Implemented option --resume
2024-03-05 18:46:14 +03:00
Sveta Smirnova
a111ed0b70 PT-1717 - resume pt-online-schema-change if it's interrupted
- Updating lower and upper boundaries in the history table
2024-02-28 18:25:39 +03:00
Sveta Smirnova
c8c1b2483a PT-1717 - resume pt-online-schema-change if it's interrupted
- Insert row with Job ID into the history table
- Update job row with progress, set 'done' to 'yes' when finished
  succesfully
2024-02-27 18:08:41 +03:00
Sveta Smirnova
ab6e5aa1bb PT-1717 - resume pt-online-schema-change if it's interrupted
- Added options --history, --history-table, and --binary-index
- Added code that creates history table to store pt-osc progress
2024-02-20 20:08:20 +03:00
Sveta Smirnova
d820b1b4ac PT-2295 - Fix tests on distributions
- Better diagnostic code in t/pt-online-schema-change/pt-169.t for
  failures
2024-02-01 22:14:02 +03:00
Sveta Smirnova
cbc00ad761 PT-1751 - adds --where param to pt-online-schema-change
- Option --where added
- Added test case
2024-01-30 17:55:15 +03:00
Sveta Smirnova
643113540f PT-2268 - pt-online-schema-change 3.5.5 breaks on mysql5.7 with server_id 0
- Better check if server_id is defined
- Added test case
2023-12-22 23:11:31 +03:00
Sveta Smirnova
9f5fc7e50a Merge pull request #388 from cswingler/check_undefined_skip_check_slave_lag_schema_change
Check undefined skip check slave lag schema change

Contirbutor signed corporate SLA as stated in https://github.com/percona/percona-toolkit/pull/230#issuecomment-308564024
2023-11-12 23:31:30 +03:00
Sveta Smirnova
3e1e9b8425 PT-1860 make pt-osc respect case insesitive lookup on Windows and osx #516
- Run update-modules
- Adjusted fix, so it does not create extra variable to hold $EVAL_ERROR
- Adjusted test case, so it runs on Linux
2023-11-09 17:16:43 +03:00
Sveta Smirnova
e4583b0542 Merge branch '3.x' into HEAD 2023-11-09 13:43:17 +03:00
Ivan Kruglov
8d38d0900a PT-1860 make pt-osc respect case insesitive lookup on Windows and osx 2023-11-08 20:16:31 +01:00
Sveta Smirnova
3d4751f428 PR-388 - Check undefined skip check slave lag schema change
- Merge branch '3.x' into HEAD
- Improve the fix, so it does not fail when $slave_to_skip->{P} is not
  defined while $slave->{dsn}->{P} is defined
- Improve the fix for pt-table-checksum (PR-230)
- Added test cases for this fix for pt-table-checksum and pt-osc
- Adjusted replica info repository options in test for pt-1760.t, so it
  does not break regression tests for pt-osc on 5.7
2023-11-08 19:42:35 +03:00
Ilaria Migliozzi
8e4824813f added new hook before_die (#509)
* added new hook before_die

before die, the script calls this hook in oder to execute extra user's 
operations

* PR-509 -  added new hook before_die

- Added test cases

---------

Co-authored-by: Sveta Smirnova <sveta.smirnova@percona.com>
2023-11-06 18:22:14 +03:00
Sveta Smirnova
46fd033994 PT-2258 - Pass tbl as argument for on_copy_rows_after_nibble (#682)
- Implemented the feature
- Created test case
2023-09-22 13:35:29 +03:00
Sveta Smirnova
f8c43118bd PT-2168 pt-osc shouldnt fail while unable to monitor a replica node (#676)
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Proof of concept
- Fixed regular expression in lib/TableParser.pm mistakenly chaged in the tool's code

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Added basic test case for PT-2168
- Added more details for replica lag information
- Disconnecting replica if lag is not checked. This prevents "Too many
  connections" error

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Implemented option --wait-lost-replicas for pt-osc, added test case

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Added more tests for situations where connection to the replica can
  fail

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Removed extra checks for wait_no_die variable
- Added test cases for SQL queries that pt-osc sends to replicas

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Allow to reload dsns table while waiting for missed replica if
  --recursion-method is dsn
- Fixed logic in replica rediscovery, so it works with replicas on the
  same host but with different ports
- Renamed option wait-lost-replicas to fail-on-stopped-replication, so
  it is in line with pt-table-checksum
- Adjusted tests
- Removed debug code for PT-1760
- Added test case for PT-1760
- Added exception for variable Open_tables_with_triggers in
  lib/bash/collect.sh due to failed test in Percona Server 8.0.34+
- Updated pt-stalk

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Updated modules
- Fixed typo in t/pt-table-sync/bidirectional.t
- Removed trailing whitespaces in lib/MasterSlave.pm

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Help for option --fail-on-stopped-replication

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Added check for availability of the simple_rewrite_plugin in t/pt-online-schema-change/pt-2168.t

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Added link to the simple_rewrite_plugin source code
- Removed tests for code that runs only in the beginning of pt-osc
  action, so should not be affected by the option fail-on-stopped-replication
2023-09-19 23:31:33 +03:00
Viktor Szépe
2bd40d8c39 Remove trailing spaces (#665)
* Remove trailing spaces

* PR-665 -  Remove trailing spaces

- Updated not stable test t/pt-online-schema-change/preserve_triggers.t
- Updated utilities in bin directory

* PR-665 -  Remove trailing spaces

- Fixed typos

* PR-665 -  Remove trailing spaces

- Fixed typos

---------

Co-authored-by: Sveta Smirnova <sveta.smirnova@percona.com>
2023-09-06 01:15:12 +03:00
Sveta Smirnova
9755074cd8 PT-2156 Fix tests for lib, part2 (#650)
* PT-2156 - Fix tests for lib

Fixed tests, broken by putting fix for PT-1554 into the library code, mostly by updating checksums.
Added AutoCommit option for test lib/QueryReview.t
Reset SQL mode to empty to allow TableChunker to work with invalid and zero dates

* PT-2156 - Fix tests for lib

Adjusted t/lib/SchemaIterator.t to work with version 8.0

* PT-2156 - Fix tests for lib

Adjusted b/t/lib/Processlist.t, so it reflects fix for PT-981

* PT-2156 - Fix tests for lib

Adjusted t/lib/HTTP/Micro.t, so it works with different order of parameters, returned at v.percona.com/
Adjusted test for empty files in t/lib/bash/collect.t
Disabled mysqladmin debug test in t/lib/bash/collect.t, because of PT-2242

* PT-2156 - Fix tests for lib

Added LC_NUMERRIC=POSIX into t/lib/bash/report_system_info.sh, so reports in tests are not environment-dependent
Updated expected results in t/lib/bash/report_system_info.sh, so they reflect new information, collected by pt-summary

* PT-2156 - Fix tests for lib

- Improved fix for PT-76, so it handles inline comments
- Added test case for PT-76
- Improved fix for PT-1720, so it ignores unrecognizable option only if it comes from the toolkit-wide files and still errors out if wrong option was passed via command line or the tool-specific option file.

* PT-2156 - Fix tests for lib

- Improved fix for PT-2102, so it finds running instance configuration file using PID and also does not break t/lib/bash/report_mysql_info.t test
- Removed unnecessary copy-paste from t/pt-mysql-summary/pt-2102.t test
- Adjusted number of collected files in t/lib/bash/collect_mysql_info.sh

* PT-2156 - Fix tests for lib

- Fix for PT-1543 and MyRocks collection were originally put only into lib/bash/report_mysql_info.shthat broke the logic of collecting data first, then formatting report from this data. This, in its turn, broke test t/lib/bash/report_mysql_info.sh/t/lib/bash/report_mysql_info.t, because CMD_MYSQL is not defined in this library. I rewrote these fixes, so they follow original logic of the tool (pt-mysql-summary)
- Added tests for keyring plugin, encrypted tables, and MyRocks for t/lib/bash/report_mysql_info.sh and t/pt-mysql-summary/pt-mysql-summary_encryption.t

* PT-2156 - Fix tests for lib

- Added FLUSH TABLES to t/lib/bash/collect.sh, so it does not fail opentables tests if run when more than 1000 tables open in the sandbox environment
- Changed number of expected sample files to reflect keyring colletion file

* PT-2156 - Fix tests for lib

- Added skip to some tests in lib that file when run with PXC, because not supported to work with PXC
- Adjusted configuration files for PXC, so they allow LOAD DATA/SELECT INTO OUTFILE commands
- Adjusted data samples, so they do not depend on auto increment values
- Fixed lib tests, failing with PXC

* PT-2156 - Fix tests for lib

Updated tests for pt-online-schema-change, so they work with PXC and skipped if designed for semi-synchronous replication setup

* PT-2156 - Fix tests for lib

- Added cluster-specific samples for t/lib/SchemaIterator.t
- Removed extra debugging print from t/pt-table-checksum/pt-1728.t

* PT-2156 - Fix tests for lib

Evgeniy's review on July 20

* Update t/lib/TableSyncer.t

Removed comment left after debugging

* Update t/pt-mysql-summary/pt-mysql-summary_encryption.t

Removed extra debug output

* PT-2156 - Fix tests for lib

Tabs to spaces
2023-08-02 14:50:21 +03:00
PinoCao
91f9e27255 PT-2241 - fix(pt-online-schema-change): using --skip-check-slave-lag multiple times does not work as expected (#622)
* fix(pt-online-schema-change): Fix the issue where using --skip-check-slave-lag multiple times does not work as expected.

* PT-2241: add test cases for pt-online-schema-change

* Update t/pt-online-schema-change/pt-2241.t

fix():reset the delay on replicas

Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>

---------

Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>
2023-07-31 16:59:09 +03:00
Marcelo HP Ferreira
8ece30ff2f Add after nibble hook to pt-online-schema-change (#645)
* Include hook after nibble on copy rows

* Update tests with new hook for pt-online-schema-change
2023-07-31 16:30:24 +03:00
Sveta Smirnova
25c969542b PT-2151 fix tests for pt-query-digest (#631)
* PT-2151 - Fix tests for pt-query-digest

Put fix for PT-1908 into the proper place

* PT-2151 - Fix tests for pt-query-digest

Put fix for PT-1554 into the proper place

* PT-2151 - Fix tests for pt-query-digest

Adjusted expected results for the default test t/pt-table-checksum/basics.t, so they do not depend on number of rows in the help tables

* PT-2151 - Fix tests for pt-query-digest

Added additional check for both replicas into t/pt-online-schema-change/preserve_triggers.t to avoid deadlock when ->ok() is doing CHECKSUM

* PT-2151 - Fix tests for pt-query-digest

Added delay to t/pt-table-sync/wait.t, so it waits for the child process to start lagging

* PT-2151 - Fix tests for pt-query-digest

Updated t/pt-query-digest/samples/issue_1196-output-8.0.txt, so it reflects changes in the latest 8.0

* PT-2151 - Fix tests for pt-query-digest

Updated queries against query history table, so they don't fail after e2cf183762

* PT-2151 - Fix tests for pt-query-digest

Fixed PT-813 by comparing query text. Since order itself does not matter, it is not essential to compare by the fingerprint or use any other function that changes the query.

* PT-2151 - Fix tests for pt-query-digest

Adjusted samples files which now should have consistent order after fix for PT-813

* PT-2151 - Fix tests for pt-query-digest

Fixed typo in the SELECT query in the QueryReview package

* PT-2151 - Fix tests for pt-query-digest

Fix for PT-981

* PT-2151 - Fix tests for pt-query-digest

Updated modules for pt-index-usage and fixed tests, because checksum is calculated differently after fix for PT-1554

* PT-2151 - Fix tests for pt-query-digest

Updated modules for pt-diskstats, pt-fk-error-logger, pt-heartbeat, pt-online-schema-change, pt-slave-delay, pt-slave-find, pt-table-checksum, pt-table-sync, pt-upgrade

* PT-2151 - Fix tests for pt-query-digest

Updated lib/IndexUsage.pm, so fix for pt-index-usage is in the correct place
2023-06-23 15:48:21 +03:00
Sveta Smirnova
249b5b61e9 PT-2048 - pt-osc spans excessive connections to the replica when executing in the source (#613)
* PT-2048 - pt-osc spans excessive connections to the replica when executing in the source

The bug itself is fixed by PT-2160: 'fix tests for pt online schema change'

* PT-2048 - pt-osc spans excessive connections to the replica when executing in the source

Suggestions by Saikumar

* PT-2048 - pt-osc spans excessive connections to the replica when executing in the source

Typos in the test file
2023-05-09 14:22:29 +03:00
Sveta Smirnova
bfc00b93a8 PT-2160 fix tests for pt online schema change (#602)
* PT-2160 - Fix tests for pt-online-schema-change

Re-enabled tests after MySQL bug #89441 fix in 8.0.14 and later
Updated t/pt-online-schema-change/samples/basic_no_fks_innodb.sql, so its unique index is NOT NULL
Added synchonizations when tests are waiting for too long due to slave catch up
Added innodb_directories option for t/pt-online-schema-change/pt-244.t

* PT-2160 - Fix tests for pt-online-schema-change
PT-2048 - pt-osc spans excessive connections to the replica when executing in the source

t/pt-online-schema-change/slave_lag.t was failing due to PT-2048. I added code that reuses already created connections when checks for slave in the wait loop. It fixes slave_lag.t too.
I also added that prints more detailed error when fails to connect to the replica.

* PT-2160 - Fix tests for pt-online-schema-change

Modified t/pt-online-schema-change/slave_lag.t, so it runs and clean ups faster.

* PT-2160 - Fix tests for pt-online-schema-change

Modified t/pt-online-schema-change/pt-1455.t, so it runs and clean ups faster.

* PT-2160 - Fix tests for pt-online-schema-change

Fixed search for HASH and BTREE keys broken by fix for PT-2123
Updated modules for pt-archiver
Fixed die message for sandbox/start-sandbox
Added global_grants table to list of exceptions for the ok function in lib/Sandbox.pm
Added wait_for_slaves to bugs.t, so changes on the source are copied to replica before pt-osc starts working
Updated PXC tests
Added wait_for_slaves to pt-1455.t
Fixed regular expression in pt-229.t, so it works with both 5.x and 8.0 versions
Added innodb_directories option for pt-244.t, so it does not fail on 8.0
Modified slave_lag.t, so it is more stable and slow enough, so pt-osc can print message about delayed replicas
2023-02-27 01:19:11 +03:00
Sveta Smirnova
01de987154 PT-2141 fix tests for pt-archiver (#581)
* PT-2141 Fix tests for pt-archiver

Fixed tests t/pt-archiver/bulk_insert.t, t/pt-archiver/issue_1225.t, t/pt-archiver/issue_1229.t
Their failures were due to change of utf8 alias from utf8mb3 in MySQL 5.7 to utf8mb4 in MySQL 8.0.
And also due to character set match check between connection ad source table.

* PT-2141 Fix tests for pt-archiver

Removed test for PT-1898, because it was broken from the beginning:
- it called pt-online-schema-change instead of pt-archiver
- it announced it will run 6 tests while was running only 2
- the test case itself did not test the issue, described in PT-1898
2023-02-02 17:09:34 +03:00
yoku0825
cceb48619f Use full_output to avoid to die (PT-1967) (#494) 2022-09-28 02:55:30 +03:00
yoku0825
dcf237ef8c Fix test PT-1966 (#493)
* Detect raising error for PT-1966

* Fix non-duplicated column name for PT-1966

* More good way to catch an error. (PT-1966)

- https://github.com/percona/percona-toolkit/pull/493#pullrequestreview-678426876

* Add Changelog (PT-1966)

- https://github.com/percona/percona-toolkit/pull/493
2021-06-17 10:28:54 -03:00
Carlos Salguero
0b97e1f471 PT-1747 pt-online-schema-change was bringing the database into a broken state (#491)
* PT-1717 Updated behavior for FKs

* Fixed rename_columns.t

* Fixed pt-244.t

* fixed drop swap test

* Fixed test for MySQL 8.0

* Updated test for MySQL 8

* Updated test for MySQL 8
2021-06-03 12:00:43 -03:00
Carlos Salguero
e570cecbab PT-1914 Column data lost when 'Generated' is in the column comment (#482)
* PMM-1914 Fixed column parsing having generated

Fixed table parser code that errouneously considered a column as
generated when the default was empty DEFAULT '' and the COMMENT had
the word 'Generated'.

* PMM-1914 Updated TableParser in all programs

* PT-1914 Updated changelog

* PT-1914 Added test
2021-03-09 11:46:05 -03:00
Carlos Salguero
4f78d5450c PT-1919 drop-swap can drop triggers (#483)
* PT-1919 drop-swap can drop triggers

* PT-1919 Update changelog
2021-03-05 11:32:03 -03:00