Commit Graph

362 Commits

Author SHA1 Message Date
Sveta Smirnova
ef2dbd887f PT-2129 - pt-online-schem-change and other tools fail on non-readable version check file
- Implemented fix in lib/VersionCheck.pm
- Updated modules in all tools
- Added test case to t/pt-query-digest/version_check.t
2025-03-29 23:22:44 +03:00
Sveta Smirnova
c92d95bc38 Merge pull request #903 from Robertoh98/PT-2410
PT-2410 - Fixing the export with output-format=csv when there are null values
2025-03-18 15:02:03 +03:00
Henning Poettker
1e581be06a PT-2378 - extended FP precision in pt-table-sync
pt-table-sync now uses up to 17 decimal digits when writing
floating point numbers in the generated SQL statements.
This is necessary to prevent unintended data changes.
2025-01-13 09:31:51 +01:00
Roberto de Bem
d5ec5d9ca8 PT-2410 - Removing the enclosed by on \N(Null) values and removing the space existent after the comma 2024-12-21 15:00:37 -03:00
Sveta Smirnova
02ada0ddc9 Release 3.7.0
- Updated version
2024-12-19 18:41:01 +03:00
Sveta Smirnova
bb66ab0845 PT-2340 - Support MySQL 8.4
- Implemented functions get_replica_name and get_source_name in lib/MasterSlave.pm I did not implement similar functions in other places, because they set other variables as well, not only replica_name or source_name.
2024-11-13 02:04:39 +03:00
Sveta Smirnova
a2102efb86 PT-2340 - Support MySQL 8.4
- s/slave lag/replica lag/ in reports
2024-10-07 15:53:46 +03:00
Sveta Smirnova
86fe021434 PT-2340 - Support MySQL 8.4
- Updated modules
2024-10-04 19:06:01 +03:00
Sveta Smirnova
33086769a1 PT-2340 - Support MySQL 8.4
- Added test for replica lag check for pt-archiver
- Re-added deprecated slave- options
- Added tests for deprecation warnings and for legacy options for pt-archiver
- Removed practically not supported options --replica-user and --replica-password from pt-archiver, pt-kill, pt-query-digest
- Added legacy source/replica options (master/slave) variants and tests for pt-heartbeat, pt-online-schema-change, pt-replica-find, pt-replica-restart, pt-table-checksum, pt-table-sync
- Updated modules after lib/MasterSlave.pm change
2024-10-04 18:27:00 +03:00
Sveta Smirnova
d03bd604a1 PT-2340 - Support MySQL 8.4
- Fixed typos
- Added tests for MyRocks and TokuDB clustered indexes into t/pt-duplicate-key-checker/clustered_keys.t
- Updated modules
2024-09-04 18:32:58 +03:00
Sveta Smirnova
94bcfd2b69 PT-2340 - Support MySQL 8.4
- Updated documentation
2024-09-03 20:05:26 +03:00
Sveta Smirnova
e3e352d1d6 PT-2340 - Support MySQL 8.4
- Fixed comparison operators for VersionCheck calls
2024-08-27 13:45:37 +03:00
Sveta Smirnova
b332537481 PT-2340 - Support MySQL 8.4
- Fixed staff I broke for 8.0
2024-08-23 16:03:53 +03:00
Sveta Smirnova
5c999ca3e0 PT-2340 - Support MySQL 8.4
- Removed runtime.txt after discussion with Anastasia Alexandrova
- Added "use VersionParser" into tests in t/lib when needed
- Removed word master from tests for pt-archiver, pt-config-diff, pt-deadlock-logger, pt-duplicate-key-checker, pt-find, pt-fk-error-logger, pt-heartbeat, pt-index-usage, pt-ioprofile, pt-kill, pt-mysql-summary
- Removed word slave from tests for pt-archiver, pt-config-diff, pt-deadlock-logger, pt-duplicate-key-checker, pt-find, pt-fk-error-logger, pt-heartbeat, pt-index-usage, pt-ioprofile, pt-kill, pt-mysql-summary
- Updated modules for pt-archiver, pt-config-diff, pt-deadlock-logger, pt-duplicate-key-checker, pt-find, pt-fk-error-logger, pt-heartbeat, pt-index-usage, pt-ioprofile, pt-kill, pt-mysql-summary
- Changed mysql_ssl patch, so it is now short option s
- Added a check for existing zombies in t/pt-kill/execute_command.t
- Added bin/pt-galera-log-explainer to .gitignore
2024-07-27 01:59:52 +03:00
Sveta Smirnova
65914c4c87 Version and modules updated for version 3.6.0 2024-05-25 16:13:43 +03:00
Michael Coburn
2a2160fff7 pgk -> pkg for OpenSolaris installer 2024-04-23 16:58:21 -06:00
Sveta Smirnova
c5cafbb486 PT-2231 - pt-osc + PTDEBUG=1 fails with Use of uninitialized value in concatenation (.) or string at ./pt-online-schema-change line 4309.
- Re-implemented fix for PT-1799 properly
- Added test case
- Updated modules
2024-04-11 21:47:11 +03:00
Sveta Smirnova
e946d26a45 Merge pull request #788 from percona/PT-2264_pt-query-digest_Pipeline_process_11_--output_slowlog_caused_an_error_Wide_character
PT-2264 - pt-query-digest Pipeline process 11 (--output slowlog) caused an error: Wide character
2024-03-29 10:48:16 +03:00
Sveta Smirnova
d68f56c924 PT-2284 - pt-kill 3.5.5: "Wide character in printf at /usr/bin/pt-kill line 7553."
- Set binmode to utf8 for the Daemon log files
- Added test case
- Run update-modules
2024-03-23 03:23:48 +03:00
Sveta Smirnova
37ca3d2d49 PT-1205 - LP #1275034: pt-table-sync reports 'wide character in print' and terminates
- Added binmode :utf8 for STDERR
- Added test case for PT-1205
- Run update-modules
2024-03-22 01:42:14 +03:00
Sveta Smirnova
1ad44e04da PT-2264 - pt-query-digest Pipeline process 11 (--output slowlog) caused an error: Wide character
- Updated lib/DSNParser.pm, so it sets binmode to character set utf8 since
  MySQL 8.0 if character set is not specified in the DSN
- Added test case
- Run update-modules
2024-03-21 13:01:03 +03:00
Sveta Smirnova
f136116826 PT-2295 - Fix tests on distributions
- Added test result adjustment on slow machines for
  t/lib/bash/collect.sh, because no other solution worked
- Updated modules for all tools
2024-02-13 20:03:32 +03:00
Sveta Smirnova
325809e86f PT-2295 - Fix tests on distributions
- Added regular expressions that match MariaDB Connector/C syntax for
  errors 2006 (Server has gone away) and 2013 (Lost connection)
2024-02-01 22:10:32 +03:00
Sveta Smirnova
48f062b37d Update copyright to 2024
- Added option ONLY_UPDATE_COPYRIGHT_YEAR to util/build-packages
- Fixed update_copyright_year function in util/build-packages, so it
  works again
- Updated copyright year for tools and docs/percona-toolkit.pod
2024-01-18 21:03:18 +03:00
Sveta Smirnova
5b61ec79b5 Version and release notes updated 2023-12-22 23:34:57 +03:00
Sveta Smirnova
bab154be03 Updated modules and version 2023-12-22 23:11:31 +03:00
Sveta Smirnova
fa41e20e83 PT-2083 - Pt-archiver --charset option is not working for MySQL8.0
- Run util/update-modules
2023-12-22 23:11:31 +03:00
Sveta Smirnova
cadf35abc4 PT-2083 - Pt-archiver --charset option is not working for MySQL8.0
- Changed logic, introduced by commit 1d1c13f
- Added test case
2023-12-22 23:11:31 +03:00
Sveta Smirnova
9b347dd466 Revert "PT-2083 - Pt-archiver --charset option is not working for MySQL8.0"
This reverts commit 284a1f945e.
2023-12-22 23:11:31 +03:00
Sveta Smirnova
1682aa202c PT-2083 - Pt-archiver --charset option is not working for MySQL8.0
- Changed logic, introduced by commit 1d1c13fbcb
- Added test case
2023-12-22 23:11:31 +03:00
Sveta Smirnova
a4cf465087 Merge pull request #706 from percona/Follow_PR-429_unstop_option_with_testcase
Follow PR-429 unstop option with testcase
2023-12-22 23:11:31 +03:00
Sveta Smirnova
2b440638c4 Merge pull request #604 from mikrobi/patch-1
Only rollback destination if it's defined
2023-11-12 23:28:13 +03:00
Sveta Smirnova
52cc6c11de Follow PR-429 unstop option with testcase
- Review by Evgeniy - print path to the sentinel file in case of error
2023-11-12 23:26:14 +03:00
Sveta Smirnova
c1b751927c Follow_PR-429_unstop_option_with_testcase
- Added information message about removing sentinel file
- Added cross-links for options --stop and --unstop in docs
- Added test case
2023-11-07 16:25:50 +03:00
Francois Lallart
db8207b1ed Add --unstop option to remove sentinel file. 2023-11-07 16:07:47 +03:00
Sveta Smirnova
f0af7114e7 PT-2279 - Option --charset is confusing when set to utf8mb4
- Added pattern check for utf8 character set instead of eq
- Added test cases
2023-11-06 23:26:54 +03:00
Sveta Smirnova
6ac26fd1d8 Merge branch '3.x' into HEAD 2023-11-06 18:59:44 +03:00
Sveta Smirnova
817673755f PT-2207 - pt-archiver doesn't work when ANSI_QUOTES is set in sql_mode (#694)
- Implemented suggestion from the bug report
- Added test case
2023-11-02 15:06:46 +03:00
Sveta Smirnova
df73533d29 Changelog for v3.5.5 (#684)
* PT-2255 Release notes for Percona Toolkit version 3.5.5
	modified:   Makefile.PL
	modified:   config/sphinx-build/conf.py
	modified:   docs/release_notes.rst

* PT-2255 Release notes for Percona Toolkit version 3.5.5

- Added missed contributions

* PT-2255 Release notes for Percona Toolkit version 3.5.5

- Added PT-2258

* Changelog for version 3.5.5

Added Changelog, updated version

* Update Changelog

Co-authored-by: Alina Derkach <alina.derkach@percona.com>

---------

Co-authored-by: Alina Derkach <alina.derkach@percona.com>
2023-10-03 16:29:53 +03:00
Sveta Smirnova
f8c43118bd PT-2168 pt-osc shouldnt fail while unable to monitor a replica node (#676)
* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Proof of concept
- Fixed regular expression in lib/TableParser.pm mistakenly chaged in the tool's code

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Added basic test case for PT-2168
- Added more details for replica lag information
- Disconnecting replica if lag is not checked. This prevents "Too many
  connections" error

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Implemented option --wait-lost-replicas for pt-osc, added test case

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Added more tests for situations where connection to the replica can
  fail

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Removed extra checks for wait_no_die variable
- Added test cases for SQL queries that pt-osc sends to replicas

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Allow to reload dsns table while waiting for missed replica if
  --recursion-method is dsn
- Fixed logic in replica rediscovery, so it works with replicas on the
  same host but with different ports
- Renamed option wait-lost-replicas to fail-on-stopped-replication, so
  it is in line with pt-table-checksum
- Adjusted tests
- Removed debug code for PT-1760
- Added test case for PT-1760
- Added exception for variable Open_tables_with_triggers in
  lib/bash/collect.sh due to failed test in Percona Server 8.0.34+
- Updated pt-stalk

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Updated modules
- Fixed typo in t/pt-table-sync/bidirectional.t
- Removed trailing whitespaces in lib/MasterSlave.pm

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Help for option --fail-on-stopped-replication

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Added check for availability of the simple_rewrite_plugin in t/pt-online-schema-change/pt-2168.t

* PT-2168 - PT-OSC shouldn't fail while unable to monitor a replica node

- Added link to the simple_rewrite_plugin source code
- Removed tests for code that runs only in the beginning of pt-osc
  action, so should not be affected by the option fail-on-stopped-replication
2023-09-19 23:31:33 +03:00
Viktor Szépe
f2f7a7993d Add final newlines and remove trailing spaces (#672)
* Add final newlines and remove trailing spaces

* Replace TAB characters with spaces

* Fix indent
2023-09-15 17:31:40 +03:00
Viktor Szépe
2bd40d8c39 Remove trailing spaces (#665)
* Remove trailing spaces

* PR-665 -  Remove trailing spaces

- Updated not stable test t/pt-online-schema-change/preserve_triggers.t
- Updated utilities in bin directory

* PR-665 -  Remove trailing spaces

- Fixed typos

* PR-665 -  Remove trailing spaces

- Fixed typos

---------

Co-authored-by: Sveta Smirnova <sveta.smirnova@percona.com>
2023-09-06 01:15:12 +03:00
Viktor Szépe
c57441be8c War on typos Act 1 (#655)
* Fix typos in lib/ directory

* Update generated bin/ files

* PR 655 -  War on typos Act 1 #655

- Updated modules in tools that were not updated
- Fixed tests to reflect proposed changes

---------

Co-authored-by: Sveta Smirnova <sveta.smirnova@percona.com>
2023-08-22 16:18:38 +03:00
Sveta Smirnova
9755074cd8 PT-2156 Fix tests for lib, part2 (#650)
* PT-2156 - Fix tests for lib

Fixed tests, broken by putting fix for PT-1554 into the library code, mostly by updating checksums.
Added AutoCommit option for test lib/QueryReview.t
Reset SQL mode to empty to allow TableChunker to work with invalid and zero dates

* PT-2156 - Fix tests for lib

Adjusted t/lib/SchemaIterator.t to work with version 8.0

* PT-2156 - Fix tests for lib

Adjusted b/t/lib/Processlist.t, so it reflects fix for PT-981

* PT-2156 - Fix tests for lib

Adjusted t/lib/HTTP/Micro.t, so it works with different order of parameters, returned at v.percona.com/
Adjusted test for empty files in t/lib/bash/collect.t
Disabled mysqladmin debug test in t/lib/bash/collect.t, because of PT-2242

* PT-2156 - Fix tests for lib

Added LC_NUMERRIC=POSIX into t/lib/bash/report_system_info.sh, so reports in tests are not environment-dependent
Updated expected results in t/lib/bash/report_system_info.sh, so they reflect new information, collected by pt-summary

* PT-2156 - Fix tests for lib

- Improved fix for PT-76, so it handles inline comments
- Added test case for PT-76
- Improved fix for PT-1720, so it ignores unrecognizable option only if it comes from the toolkit-wide files and still errors out if wrong option was passed via command line or the tool-specific option file.

* PT-2156 - Fix tests for lib

- Improved fix for PT-2102, so it finds running instance configuration file using PID and also does not break t/lib/bash/report_mysql_info.t test
- Removed unnecessary copy-paste from t/pt-mysql-summary/pt-2102.t test
- Adjusted number of collected files in t/lib/bash/collect_mysql_info.sh

* PT-2156 - Fix tests for lib

- Fix for PT-1543 and MyRocks collection were originally put only into lib/bash/report_mysql_info.shthat broke the logic of collecting data first, then formatting report from this data. This, in its turn, broke test t/lib/bash/report_mysql_info.sh/t/lib/bash/report_mysql_info.t, because CMD_MYSQL is not defined in this library. I rewrote these fixes, so they follow original logic of the tool (pt-mysql-summary)
- Added tests for keyring plugin, encrypted tables, and MyRocks for t/lib/bash/report_mysql_info.sh and t/pt-mysql-summary/pt-mysql-summary_encryption.t

* PT-2156 - Fix tests for lib

- Added FLUSH TABLES to t/lib/bash/collect.sh, so it does not fail opentables tests if run when more than 1000 tables open in the sandbox environment
- Changed number of expected sample files to reflect keyring colletion file

* PT-2156 - Fix tests for lib

- Added skip to some tests in lib that file when run with PXC, because not supported to work with PXC
- Adjusted configuration files for PXC, so they allow LOAD DATA/SELECT INTO OUTFILE commands
- Adjusted data samples, so they do not depend on auto increment values
- Fixed lib tests, failing with PXC

* PT-2156 - Fix tests for lib

Updated tests for pt-online-schema-change, so they work with PXC and skipped if designed for semi-synchronous replication setup

* PT-2156 - Fix tests for lib

- Added cluster-specific samples for t/lib/SchemaIterator.t
- Removed extra debugging print from t/pt-table-checksum/pt-1728.t

* PT-2156 - Fix tests for lib

Evgeniy's review on July 20

* Update t/lib/TableSyncer.t

Removed comment left after debugging

* Update t/pt-mysql-summary/pt-mysql-summary_encryption.t

Removed extra debug output

* PT-2156 - Fix tests for lib

Tabs to spaces
2023-08-02 14:50:21 +03:00
Sveta Smirnova
205d7baf5b Release v3.5.4 (#647)
* Version updated

* PT-2232 Release Notes 3.5.4 (#633)

modified:   Changelog
	modified:   Makefile.PL
	modified:   config/sphinx-build/conf.py
	modified:   docs/release_notes.rst

* Updated version in the second place

---------

Co-authored-by: Alina Derkach <alina.derkach@percona.com>
2023-06-30 18:44:59 +03:00
Sveta Smirnova
b566350b64 Release v3.5.3 (#629)
* PT-2206 Release Notes 3.5.3 (#621)

* PT-2206 Release Notes 3.5.3
modified:   Changelog
	modified:   Makefile.PL
	modified:   config/sphinx-build/conf.py
	modified:   docs/release_notes.rst

* Update Changelog

* Update docs/release_notes.rst

---------

Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>

* RM-1222 - Percona Toolkit 3.5.3

Updated versions

* Add the release date for 3.5.3 release notes (#624)

modified:   Changelog
	modified:   docs/release_notes.rst

* ENG-7 fix toolkit build (#626)

* Release v3.5.3 (#627)

* ENG-7 fix toolkit build

* ENG-7 fix build

* Update the RNs date (#625)

modified:   Changelog
	modified:   docs/release_notes.rst

---------

Co-authored-by: Alina Derkach <alina.derkach@percona.com>
Co-authored-by: EvgeniyPatlan <evgeniy.patlan@percona.com>
2023-06-05 22:48:13 +03:00
Carlos Salguero
a85b7aa32c Release v3.5.2 (#609)
* Release v3.5.1 Go 1.20 and upgrade deps

* Release v3.5.2

* PT-2193 Release Notes 3.5.2 (#610)

modified:   Changelog
	modified:   Makefile.PL
	modified:   config/sphinx-build/conf.py
	modified:   docs/release_notes.rst
	new file:   docs/rn.3-5-2.txt

* Updated version properly (#611)

* Release v3.5.1 Go 1.20 and upgrade deps

* Release v3.5.2

* PT-2193 Release Notes 3.5.2 (#610)

modified:   Changelog
	modified:   Makefile.PL
	modified:   config/sphinx-build/conf.py
	modified:   docs/release_notes.rst
	new file:   docs/rn.3-5-2.txt

* Updated version properly

---------

Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>
Co-authored-by: Alina Derkach <alina.derkach@percona.com>

---------

Co-authored-by: Alina Derkach <alina.derkach@percona.com>
Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>
2023-03-28 17:33:37 +03:00
Sveta Smirnova
642253ce52 PT-2156 Fix tests for lib (#606)
* PT-2156 - Fix tests for lib

- Fixed tests broken for lib/TableParser.pm after fix for PT-1059
- Updated tests for lib/TableParser.pm that are broken due to SHOW CREATE TABLE output format change in 8.0
- Updated modules for all tools that use lib/TableParser.pm

* Revert "Fixed pt-archiver tests"

This reverts commit a3ab87b12e.

This commit wa needed, because removed code in sandbox/slave_channels.sql broked the test. Proper fix would be to do not remove channel names rather than chaging the test. So revertig it.

* PT-2156 - Fix tests for lib

- Updated test t/lib/TableChecksum.t so it reflects changes, introduced in the fix for PT-2016
- Updated test t/lib/RowChecksum.t so it reflects changes added to in the fix for PT-2138: UTF8 support
- Uncommented SQL in sandbox/slave_channels.sql that made t/lib/MasterSlave.t to fail
- Added check for undef into t/pt-archiver/archive_using_channels.t
- Updated lib/Cxn.pm so it uses $dbh->{Active} after issue with the ping() call, reported at https://github.com/perl5-dbi/DBD-mysql/issues/306

* PT-2156 - Fix tests for lib

- Impoved the fix for PT-2016, so it does not files with keys with USING keyword
- Added brackets to expression in lib/TableNibbler.pm, so it does not crap query wit many indexes with OR keyword
- Adjusted test t/lib/TableNibbler.t, so it reflects above chages
- Modified lib/Cxn.pm, so it has workaround for https://github.com/perl5-dbi/DBD-mysql/issues/306 , introduced in DBD::mysql 4.0.50
- Updated tests: added debugging code and cleanups
- Updated modules for tools
2023-03-27 14:18:26 +03:00
Jakob
79757865e3 Only rollback destination if it's defined
This change addresses https://jira.percona.com/browse/PT-2064 which causes PT-archiver to crash when hitting an error. We use pt-archiver to delete rows that shouldn't be moved to another destination, so `--dest` is not provided.

In most places, we seem to check  if `$dst` is actually defined before using it, so we might want to do the same in the `do_with_retries` function?
2023-03-09 15:24:46 +01:00
Sveta Smirnova
bfc00b93a8 PT-2160 fix tests for pt online schema change (#602)
* PT-2160 - Fix tests for pt-online-schema-change

Re-enabled tests after MySQL bug #89441 fix in 8.0.14 and later
Updated t/pt-online-schema-change/samples/basic_no_fks_innodb.sql, so its unique index is NOT NULL
Added synchonizations when tests are waiting for too long due to slave catch up
Added innodb_directories option for t/pt-online-schema-change/pt-244.t

* PT-2160 - Fix tests for pt-online-schema-change
PT-2048 - pt-osc spans excessive connections to the replica when executing in the source

t/pt-online-schema-change/slave_lag.t was failing due to PT-2048. I added code that reuses already created connections when checks for slave in the wait loop. It fixes slave_lag.t too.
I also added that prints more detailed error when fails to connect to the replica.

* PT-2160 - Fix tests for pt-online-schema-change

Modified t/pt-online-schema-change/slave_lag.t, so it runs and clean ups faster.

* PT-2160 - Fix tests for pt-online-schema-change

Modified t/pt-online-schema-change/pt-1455.t, so it runs and clean ups faster.

* PT-2160 - Fix tests for pt-online-schema-change

Fixed search for HASH and BTREE keys broken by fix for PT-2123
Updated modules for pt-archiver
Fixed die message for sandbox/start-sandbox
Added global_grants table to list of exceptions for the ok function in lib/Sandbox.pm
Added wait_for_slaves to bugs.t, so changes on the source are copied to replica before pt-osc starts working
Updated PXC tests
Added wait_for_slaves to pt-1455.t
Fixed regular expression in pt-229.t, so it works with both 5.x and 8.0 versions
Added innodb_directories option for pt-244.t, so it does not fail on 8.0
Modified slave_lag.t, so it is more stable and slow enough, so pt-osc can print message about delayed replicas
2023-02-27 01:19:11 +03:00