Commit Graph

2444 Commits

Author SHA1 Message Date
Sveta Smirnova
2f584c85db PT-2140 - Update modules in the main branch (#570)
* PT-2140 - Update modules in the main branch

Updated modules for:
- pt-align
- pt-archiver
- pt-config-diff
- pt-deadlock-logger
- pt-diskstats
- pt-duplicate-key-checker
- pt-fifo-split
- pt-find
- pt-fingerprint
- pt-fk-error-logger
- pt-heartbeat
- pt-index-usage
- pt-ioprofile
- pt-kill
- pt-mext
- pt-mysql-summary
- pt-online-schema-change
- pt-pmp
- pt-query-digest
- pt-show-grants
- pt-sift
- pt-slave-delay
- pt-slave-find
- pt-slave-restart
- pt-summary
- pt-table-sync
- pt-upgrade
- pt-variable-advisor

Added execute bit for pt-query-digest

* PT-2140 Update modules in the main branch

Fixed Daemon.pm plugin usage
Updated all tests, related to the Daemon plugin.
I intentionally did not fix failing tests, not related to Daemon plugin,
to avoid making this PR too big.

- bin/pt-archiver
	- PT-2141 - Fixed usage of Daemon.pm
	- PT-2141 - Updated t/pt-archiver/standard_options.t
- bin/pt-deadlock-logger
	- PT-2143 - Fixed usage of Daemon.pm
	- PT-2143 - Updated t/pt-deadlock-logger/standard_options.t
- bin/pt-fifo-split
	- PT-2144 - Fixed usage of Daemon.pm
	- PT-2144 - Updated t/pt-fifo-split/pt-fifo-split.t
- bin/pt-find
	- PT-2145 - Fixed usage of Daemon.pm
	- PT-2145 - Updated t/pt-find/pt-find.t
- bin/pt-fk-error-logger
	- PT-2146 - Fixed usage of Daemon.pm
	- PT-2146 - Updated t/pt-fk-error-logger/basics.t
- bin/pt-heartbeat
	- PT-2147 - Fixed usage of Daemon.pm
	- PT-2147 - Updated t/pt-heartbeat/standard_options.t, t/pt-heartbeat/basics.t
- bin/pt-kill
	- PT-2148 - Fixed usage of Daemon.pm
	- PT-2148 - Updated t/pt-kill/standard_options.t
- bin/pt-show-grants
	- PT-2152 - Fixed usage of Daemon.pm
	- PT-2152 - Updated t/pt-show-grants/standard_options.t
- bin/pt-slave-delay
	- Fixed usage of Daemon.pm
	- Updated t/pt-slave-delay/standard_options.t
- bin/pt-slave-find
	- PT-2153 - Fixed usage of Daemon.pm
	- PT-2153 -Updated t/pt-slave-find/pt-slave-find.t
- bin/pt-slave-restart
	- Fixed usage of Daemon.pm
	- Updated t/pt-slave-restart/pt-slave-restart.t
- bin/pt-table-sync
	- PT-2154 - Fixed usage of Daemon.pm
	- PT-2154 -Updated t/pt-table-sync/standard_options.t
2023-01-06 15:04:10 +03:00
Sveta Smirnova
a7efd8c94e PT-2138 - fix tests for pt table checksum (#568)
* PT-2138 - Fix tests for pt-table-checksum

- Updated t/pt-table-checksum/samples/default-results-8.0.txt and
t/pt-table-checksum/samples/static-chunk-size-results-8.0.txt
to support latest MySQL 8.0 version. Tests are now incompatible with elder 8.0 releases.
- Put fix for PT-136 into package RowChecksum
- Added execution bit for pt-online-schema-change

* PT-2138 - Fix tests for pt-table-checksum

- Fixed percona_test.load_data, so it really tests if LOAD DATA LOCAL INFILE is enabled
- Fixed option --[no]create-replicate-table, broken by commit c9836d5962

* PT-2138 - Fix tests for pt-table-checksum

- Enabled t/pt-table-checksum/error_handling.t for MySQL 8.0
- Fixed test t/pt-table-checksum/fnv_64.t and it's samples file
t/pt-table-checksum/samples/fnv64-sakila-city.txt
to reflect new function name convention and
changes after 62d84e5dba

* PT-2138 - Fix tests for pt-table-checksum

- Fixed t/pt-table-checksum/issue_1485195.t, so it checks only one table and
isn't get broken when we add tables into percona_test database
- Fixed typo in error output of bin/pt-table-checksum
- Skipped issue_47.t in 8.0 until https://jira.percona.com/browse/PT-1805 is fixed

* PT-2138 - Fix tests for pt-table-checksum

- Disabled pt-131.t for 8.0, because it does not have the QUERY_RESPONSE_TIME plugins
- Added SLOW_TESTS check to pt-1616.t
- Updated pt-226.t to include the fix for PT-1766
- For replication_filters.t: excluded false positive expression for tests 10 and 11 and added sys schema to the list of checked databases for 8.0
- Changed get_slaves in lib/MasterSlave.pm, so it returns slave's parent, required for wait_for_slaves in pt-table-checksum to work properly with chained slaves

* PT-2138 - Fix tests for pt-table-checksum

- Modified pt-204.t  to support 8.0 and diffs in system tables due to timestamps
Moved the fix for PT-1616 into the proper place: lib/NibbleIterator.pm

* PT-2138 - Fix tests for pt-table-checksum

- pxc.t -added mysql.proxies_priv into ignore list, because its timestamp is different on node
- pxc.t - removed FORK=pxc from statup options for slave (non-cluster) nodes
- pxc.t - disabled wsrep replication with help of the variable wsrep_on: sql_log_bin doesn't disable wsrep replication anymore. See https://jira.percona.com/browse/PXC-3464 for details
- Removed data.tar.gz from 5.7 sandbox configuration, because it has an outdated definition for Performance Schema
- Disabled pxc.t for version 8.0 until PT-1699 is fixed
- start-sandbox - removed the first line (ALTER USER) from the init file, because it was rewritten by the next echo command, and then repeated later.

* PT-2138 - Fix tests for pt-table-checksum

- Adopted issue_1485195.t and basics.t for MyRocks-enabled setup
- replication_filters.t - added sys schema to the list of expected schemas for 5.7 and 8.0
- issue_1485195.t - added checks for the existence of mysql.plugin, func, and proxies_priv tables
- added samples/pt-131-wipe.sql that uninstalls QRT plugin if it was earlier installed by this test
-adjusted return code in pt-204.t, because expected differences in mysql.proxies_priv

* Update lib/PerconaTest.pm

removed diagnostic code

Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>

Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>
2022-12-28 23:09:13 +03:00
Dario Minnucci
7e5c51d0fb Fix some spelling errors (#462)
Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>
2022-12-01 15:54:24 +03:00
Carlos Salguero
882d8787c7 Release 3.5.0 (#563)
* Changelog and version for Release 3.5.0

* PT-2076 add rhel9 dependency

* PT-2128 Updated version to 3.5.0

Co-authored-by: svetasmirnova <sveta.smirnova@percona.com>
Co-authored-by: EvgeniyPatlan <evgeniy.patlan@percona.com>
2022-12-01 14:31:46 +03:00
Kazuya Yokogawa
bc3056f29d PT-2110 - Fixed PTDEBUG output with --ignore-engines option (#555) 2022-11-30 14:51:49 +03:00
Sveta Smirnova
cf8f824d5e PT-2107 merge community prs 466 298 58 53 (#554)
* Correcting specified package type in error message (#58)

* Docs table regex (#53)

* Add more specific info about regex

* noop change to trigger checks

* Fix typo (#298)

* Updated readme

* Fix typo in comment

Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>

* Fix for PT-1799 (#466)

Co-authored-by: Daniel Hoherd <daniel.hoherd@gmail.com>
Co-authored-by: Daniël van Eeden <git@myname.nl>
Co-authored-by: Moritz Lenz <moritz.lenz@noris.de>
Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>
2022-11-16 20:48:17 +03:00
E. Souhrada
9bd0fef2c2 PT-1739 - fix unexpected QRT disabling on slaves (#416) 2022-09-28 20:55:20 +03:00
kokopellis
60b3750d96 PT-1631 fix ioprofile ubuntu18.04, ubuntu16.04 support (#431) 2022-09-28 02:58:05 +03:00
dm-2
26522d20fd PT-2080 Add JSON logging to pt-kill (#543)
* Add JSON output support to pt-kill

Introduces new flags `--log-json` and `--log-json-fields` to control JSON output:
- `--log-json`: when combined with `--print`, outputs kill information in JSON format instead of plain text, with the same fields as the `kill_log` table when using `--log-dsn`
- `--log-json-fields`: when combined with `--log-json`, adds user-defined key-value pairs to the JSON document that is output (such as cluster name or hostname); parameter value is in the format of `key1:value1,key2:value2`

* Convert --log-json to write to file, remove dependency on --print, update docs

The `--log-json` parameter now takes a file path value, and writes JSON output to this file.

The `--print` parameter is now independent and does not need to be specified to use `--log-json`.

Docs updated for clarity.

* Add chksm to JSON output, make JSON key names consistent

- Adds chksm outout to JSON output to make it easier to group similar queries
- Moves chksm code so that it is always executed, is accessible within the `log-json` scope, and isn't duplicated
- Change JSON key names to consistently use uppercase letters at start of words, change `ts` to `Timestamp`

* Update docs to match changed field names

* Rename JSON parameter, output to STDOUT, fix tests, add test for --json, bug fix for --query-id

- `--json` parameter must now be used in conjunction with `--print`; JSON output replaces normal `--print` output when `--json` is also specified
- `--json` output is now sent to STDOUT instead of a specified file, as its new behaviour is to change the output format of `--print` instead of logging to a file
- Renamed `--log-json` and `--log-json-fields` parameters to `--json` and `--json-fields` to better represent their new behaviour
- Refactored checksum code to prevent test failures when `$query->{'Info'}` is empty
- Added test for `--json` and `--json-fields` parameters
- Fixed bug where specifying `--query-id` would cause errors when `$query->{'Info'}` is empty

* Fix typo: Unkown -> Unknown
2022-09-27 23:41:17 +03:00
Christian Bartolomäus
db32ba584b Fix some typos in documentation of pt-query-digest (#553) 2022-09-12 17:46:35 +03:00
Sveta Smirnova
19aa46d1a5 PT-1052 include numa information in pt tool output (#547)
* PT-1052: fix for pt-summary

* Re-added fix from commit 71fae6d117 into proper place: library directory
Added tests for PT-1052 (pt-summary only)

* Fixed tests for pt-summary,
set locale for pt-summary tests, so they are not affected by user environment.
Finished tests for PT-1052

* Added fix for PT-1983 into the proper place: library file

* PT-1052: fix for pt-stalk which now includes numastat data
2022-08-15 15:45:24 +03:00
I
a950b223ff Typo: Intsance -> Instance (#546) 2022-08-01 09:35:44 -03:00
Sveta Smirnova
ac3843bcb7 Pt 1897 lock information in8.0 in pt stalk (#544)
* PT-1897 pt-stalk on MySQL 8 not collecting lock information

For version prior 8.0 pt-stalk continue using Information Schema INNODB_LOCKS and INNODB_LOCK_WAITS tables for collecting lock information.
For version 8.0 and higher pt-stalk uses tables data_locks and data_lock_waits in Performance Schema

* PT-1897 - Better MariaDB support

We cannot simply compare version number with 8.0 to identify if lock tables are in Performance Schema,
because MariaDB 10.x still have them in the Information Schema.
Therefore added additional flag, indicating which syntax we should use

* Tests for PT-1897
2022-07-25 10:01:14 -03:00
Carlos Salguero
896fdcede8 Release 3.4.0 (#539)
* PT-1979 Improved mdb summary information

Added the MongoDB instance command line options to the output.
Also fixed the spacing in the replicaset section.

* Added new template file

* PT-1978 Implemented duplicated indexes detection

* WIP

* PT-1978 WIP

* PT-1978 New tool pt-mongodb-check-index

* Upgraded version and dependencies

* Updated Go sources

* Removed unused file

* Added version command

* Added changelog

* Updated required params

* Added system.profile to the ingored dbs

* Updated Changelog

* Updated version in all programs

* Updated changelog

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* PT-7 fix 3.4.0 build

* Updated Makefile

* Updated changelog

* Updated Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>
Co-authored-by: Evgeniy Patlan <evgeniy.patlan@percona.com>
2022-07-11 10:29:37 -03:00
Sveta Smirnova
07ee5eb1bf PS-2033, proof of concept (#540) 2022-07-11 15:51:29 +03:00
Sveta Smirnova
22ed941238 PT-1218: pt-stalk ominous open_tables function (#535)
After discussion change only affects error message that is more user-friendly now.
I also added two tests that check how open_tables function works in both cases:
with number of open tables less tha 1000 and greater than 1000
2022-03-22 09:33:05 -03:00
svetasmirnova
3767e57e15 PT-2037 - added test for the situation when both --system-only and --mysql-only are specified 2022-01-27 19:21:04 +03:00
svetasmirnova
9427c16314 PT-2037 - regression fixes
Re-added fix for PT-1340 that was reverted by fix for PT-1398
2022-01-23 03:55:35 +03:00
svetasmirnova
623fdaec91 PT-2037 - option --system-only for pt-stalk
This commit:
1. Moves all changes to the proper place: lib/bash/collect.sh
2. Refactors pt-stalk so it is more modular
3. Places fix for PT-1734 into the right place: lib/bash/collect.sh
4. Fixes tests for PT-1336
2022-01-22 21:21:05 +03:00
svetasmirnova
1f6c95f524 PT-2037 - reviewer suggestions
Refactoring pt-stalk to make it easier to call system and MySQL data collections separately
2022-01-21 04:12:43 +03:00
svetasmirnova
5a46d3df77 PT-2037 Added option --system-only to pt-stalk 2022-01-11 01:20:10 +03:00
Sveta Smirnova
b2860eee31 PT-1336 pt-stalk removes user files from destination (#524)
* PT-1336 Added file name checks and -maxdepth 1 parameter

* PT-1336 pt-stalk removes user files from the destination directory

* Added check that files to be removed were created by pt-stalk

* Added option -maxdepth 1 to find command, so the tool does not search for files in the nested directories

* Added unit tests

* This should fix PT-1812 also
2021-12-10 09:18:18 -03:00
Lenz Grimmer
fde683fb5d PT-1887: Fixed pt-diskstats to work on 5.x kernels (#526)
The amount of fields returned for disk statistics
changed in newer Linux kernel versions to 20 and
caused `pt-diskstat` to simply not provide any output
on these systems. Updated the check in the diskstats
parsing code to proceed with 20 fields as well.

Signed-off-by: Lenz Grimmer <lenz.grimmer@percona.com>
2021-12-03 13:19:36 -03:00
Carlos Salguero
a019907e6e PT-2023 Set out mode to utf8 (#523) 2021-11-18 14:41:35 -03:00
Carlos Salguero
c5a22407a7 PT-2016 pt-table-sync and CRC32 columns (#521)
* PT-2016 CRC32 on key

* PT-2016 Added tests

* Removed commented out line
2021-11-03 13:53:30 -03:00
Carlos Salguero
951a333c81 PT-1398 Updated way of getting MySQL PID (#517) 2021-10-01 09:27:20 -03:00
Carlos Salguero
0d2c48456f PT-1627 Improve pt-mysql-summary jemalloc detection (#514)
* PT-1627 Changed jemalloc detection function

* Updated Tests
2021-10-01 09:19:25 -03:00
Carlos Salguero
9d6508da5f PT-1900 pt-query-digest not hiding parameter properly sometimes when parameter=binary (#510)
* PT-1900 WIP

* Fixed quoted strings regexes

* PT-1900 Fixed query rewriter to properly handle quoted text

* Fixed merge conflicts with 3.x branch
2021-09-27 08:23:23 -03:00
Carlos Salguero
d91ba9cadd PT-1959 Updated version number (#515)
Not updated in files that will be updated during the build like the
python script, Makefile.PL and some other files like documentation.
2021-09-20 10:27:29 -03:00
Sergey Kuzmichev
ad593c6de9 Fix for PT-1983: pt-summary has missing records for memory due to dmidecode changes (#513)
* Fix for PT-1983 pt-summary missing one DIMM

This is a fix for PT-1983. dmidecode now doesn't print "Speed: UKNOWN" for
missing DIMMs. pt-summary code relies on the Speed field to be there, and when
it's missing, rows get concatenated (or rather not broken down correctly).
Adjusted the relevant code to not rely on the Speed field to be present.

* Added test for the fix for PT-1983

Due to dmidecode changes in 3.2, pt-summary needs to be able to work
with memory-related info with some missing fields in the dmidecode
output. This commit adds a test with dmidecode output modified to
include new format.

In addition, fixing test that are failing due to commit c9ec74c251

* PT-1983: updated changelog to reflect the bug fixed
2021-09-08 07:32:30 -03:00
Carlos Salguero
e389d2181a PT-1800 Updated docs (#511) 2021-08-31 08:16:34 -03:00
Carlos Salguero
d14d18a1bf PT-1997 Added error logs (#506) 2021-08-03 09:07:25 -03:00
duxthefux
4a32ca7675 Added mysql version check for bug fix introduced in 8.0.14 (#474)
Thanks for the fix.
2021-07-30 19:41:53 -03:00
Carlos Salguero
ecf43e2a17 PT-1985 pt-deadlock db schema check (#502) 2021-07-29 12:15:53 -03:00
Carlos Salguero
543f30c073 PT-1986 Check schema exists upon start (#503) 2021-07-29 12:15:23 -03:00
I
8e334d9ed2 PT-1974: Support fingerprinting for --print in pt-kill (#495)
* PT-1974: Support fingerprinting for --print in pt-kill

This commit implements `--fingerprint` that modifies the `--print`
behaviour and forces the query fingerprint to be displayed instead of an
original query.

* Added changelog entry.
2021-06-24 11:40:32 -03:00
Sergey Kuzmichev
49b570ed0c Fix for PT-1965 mysql-only lacks mysqladmin output (#492)
* Fix for PT-1965 mysql-only lacks mysqladmin output

This is a fix for PT-1965. Moved mysqladmin execution out of an if block
that contains stuff executed only when --mysql-only is not specified.
Before this fix, mysqladmin was not executed with --mysql-only.

* PT-1965: updated changelog to reflect the bug fixed
2021-06-10 08:43:07 -03:00
Carlos Salguero
0b97e1f471 PT-1747 pt-online-schema-change was bringing the database into a broken state (#491)
* PT-1717 Updated behavior for FKs

* Fixed rename_columns.t

* Fixed pt-244.t

* fixed drop swap test

* Fixed test for MySQL 8.0

* Updated test for MySQL 8

* Updated test for MySQL 8
2021-06-03 12:00:43 -03:00
Paul Jacobs
be856c84b5 Update copyright year 2021-05-04 15:42:25 +03:00
Carlos Salguero
c47ac4d4df PT release 3.3.1 testing branch (#486)
* PT release 3.3.1 testing branch

* Update pkgs version

* Updated version from build script
2021-04-30 10:08:05 -03:00
Roma Novikov
d9b3280f07 Update pt-summary (#487) 2021-04-27 08:55:10 -03:00
Carlos Salguero
5a0cbd79fc PT-716 [WIP] Forced free to use wide mode (#485) 2021-04-06 10:13:21 -03:00
Carlos Salguero
e570cecbab PT-1914 Column data lost when 'Generated' is in the column comment (#482)
* PMM-1914 Fixed column parsing having generated

Fixed table parser code that errouneously considered a column as
generated when the default was empty DEFAULT '' and the COMMENT had
the word 'Generated'.

* PMM-1914 Updated TableParser in all programs

* PT-1914 Updated changelog

* PT-1914 Added test
2021-03-09 11:46:05 -03:00
Carlos Salguero
4f78d5450c PT-1919 drop-swap can drop triggers (#483)
* PT-1919 drop-swap can drop triggers

* PT-1919 Update changelog
2021-03-05 11:32:03 -03:00
PaulJacobs-percona
df16170eda Release 3.3.0 (#480)
* Updated Percona::Toolkit version

* Updated version in POD headers

* RM-880-RN-3.3.0

* Update makefile to use go dep

* Update year

* Theme is downloaded by Makefile

* Correct path to icon

* Updated pt-osc --reverse-triggers help

* Update reverse-triggers option

* PT-1905 Reverse triggers wording

Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>
2021-01-14 15:22:19 -03:00
Carlos Salguero
00c437bfe2 Fixed --reverse-triggers help 2021-01-13 09:44:37 -03:00
Carlos Salguero
9ef2b56f4d PT-1905 Reverse triggers (#477)
* PT-1905 Reverse triggers

* WIP

* PT-1905 Updated reverse triggers creation

* PT-1905 Fixed trigger creation
2021-01-12 13:58:34 -03:00
Carlos Salguero
71c9902532 PT-1898 pt archiver keeps transaction open (#479)
* PT-1898 WIP

* PT-1898 Added commit for open txns

* PT-1898 Added commit for open txns

* PT-1898 removed dev leftover
2021-01-12 13:40:51 -03:00
Carlos Salguero
e1b7a6a1fd PT-1857 heartbeat can't reconnect (#475)
* PT-1857 pt-heartbeat reconnect

* PT-1857 Updated changelog
2020-12-24 16:23:42 -03:00
Carlos Salguero
2233224ff6 PT-1528 pt-online-schema-change "Invalid utf8mb4 character string (#473)
* PT-1528 [WIP] Proposed solution

* PT-1528 Ingore UTF8 warning
2020-12-24 16:22:46 -03:00