Commit Graph

2640 Commits

Author SHA1 Message Date
kokopellis
60b3750d96 PT-1631 fix ioprofile ubuntu18.04, ubuntu16.04 support (#431) 2022-09-28 02:58:05 +03:00
dm-2
26522d20fd PT-2080 Add JSON logging to pt-kill (#543)
* Add JSON output support to pt-kill

Introduces new flags `--log-json` and `--log-json-fields` to control JSON output:
- `--log-json`: when combined with `--print`, outputs kill information in JSON format instead of plain text, with the same fields as the `kill_log` table when using `--log-dsn`
- `--log-json-fields`: when combined with `--log-json`, adds user-defined key-value pairs to the JSON document that is output (such as cluster name or hostname); parameter value is in the format of `key1:value1,key2:value2`

* Convert --log-json to write to file, remove dependency on --print, update docs

The `--log-json` parameter now takes a file path value, and writes JSON output to this file.

The `--print` parameter is now independent and does not need to be specified to use `--log-json`.

Docs updated for clarity.

* Add chksm to JSON output, make JSON key names consistent

- Adds chksm outout to JSON output to make it easier to group similar queries
- Moves chksm code so that it is always executed, is accessible within the `log-json` scope, and isn't duplicated
- Change JSON key names to consistently use uppercase letters at start of words, change `ts` to `Timestamp`

* Update docs to match changed field names

* Rename JSON parameter, output to STDOUT, fix tests, add test for --json, bug fix for --query-id

- `--json` parameter must now be used in conjunction with `--print`; JSON output replaces normal `--print` output when `--json` is also specified
- `--json` output is now sent to STDOUT instead of a specified file, as its new behaviour is to change the output format of `--print` instead of logging to a file
- Renamed `--log-json` and `--log-json-fields` parameters to `--json` and `--json-fields` to better represent their new behaviour
- Refactored checksum code to prevent test failures when `$query->{'Info'}` is empty
- Added test for `--json` and `--json-fields` parameters
- Fixed bug where specifying `--query-id` would cause errors when `$query->{'Info'}` is empty

* Fix typo: Unkown -> Unknown
2022-09-27 23:41:17 +03:00
Christian Bartolomäus
db32ba584b Fix some typos in documentation of pt-query-digest (#553) 2022-09-12 17:46:35 +03:00
Sveta Smirnova
19aa46d1a5 PT-1052 include numa information in pt tool output (#547)
* PT-1052: fix for pt-summary

* Re-added fix from commit 71fae6d117 into proper place: library directory
Added tests for PT-1052 (pt-summary only)

* Fixed tests for pt-summary,
set locale for pt-summary tests, so they are not affected by user environment.
Finished tests for PT-1052

* Added fix for PT-1983 into the proper place: library file

* PT-1052: fix for pt-stalk which now includes numastat data
2022-08-15 15:45:24 +03:00
I
a950b223ff Typo: Intsance -> Instance (#546) 2022-08-01 09:35:44 -03:00
Sveta Smirnova
ac3843bcb7 Pt 1897 lock information in8.0 in pt stalk (#544)
* PT-1897 pt-stalk on MySQL 8 not collecting lock information

For version prior 8.0 pt-stalk continue using Information Schema INNODB_LOCKS and INNODB_LOCK_WAITS tables for collecting lock information.
For version 8.0 and higher pt-stalk uses tables data_locks and data_lock_waits in Performance Schema

* PT-1897 - Better MariaDB support

We cannot simply compare version number with 8.0 to identify if lock tables are in Performance Schema,
because MariaDB 10.x still have them in the Information Schema.
Therefore added additional flag, indicating which syntax we should use

* Tests for PT-1897
2022-07-25 10:01:14 -03:00
Carlos Salguero
896fdcede8 Release 3.4.0 (#539)
* PT-1979 Improved mdb summary information

Added the MongoDB instance command line options to the output.
Also fixed the spacing in the replicaset section.

* Added new template file

* PT-1978 Implemented duplicated indexes detection

* WIP

* PT-1978 WIP

* PT-1978 New tool pt-mongodb-check-index

* Upgraded version and dependencies

* Updated Go sources

* Removed unused file

* Added version command

* Added changelog

* Updated required params

* Added system.profile to the ingored dbs

* Updated Changelog

* Updated version in all programs

* Updated changelog

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* PT-7 fix 3.4.0 build

* Updated Makefile

* Updated changelog

* Updated Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>
Co-authored-by: Evgeniy Patlan <evgeniy.patlan@percona.com>
2022-07-11 10:29:37 -03:00
Sveta Smirnova
07ee5eb1bf PS-2033, proof of concept (#540) 2022-07-11 15:51:29 +03:00
Sveta Smirnova
22ed941238 PT-1218: pt-stalk ominous open_tables function (#535)
After discussion change only affects error message that is more user-friendly now.
I also added two tests that check how open_tables function works in both cases:
with number of open tables less tha 1000 and greater than 1000
2022-03-22 09:33:05 -03:00
svetasmirnova
3767e57e15 PT-2037 - added test for the situation when both --system-only and --mysql-only are specified 2022-01-27 19:21:04 +03:00
svetasmirnova
9427c16314 PT-2037 - regression fixes
Re-added fix for PT-1340 that was reverted by fix for PT-1398
2022-01-23 03:55:35 +03:00
svetasmirnova
623fdaec91 PT-2037 - option --system-only for pt-stalk
This commit:
1. Moves all changes to the proper place: lib/bash/collect.sh
2. Refactors pt-stalk so it is more modular
3. Places fix for PT-1734 into the right place: lib/bash/collect.sh
4. Fixes tests for PT-1336
2022-01-22 21:21:05 +03:00
svetasmirnova
1f6c95f524 PT-2037 - reviewer suggestions
Refactoring pt-stalk to make it easier to call system and MySQL data collections separately
2022-01-21 04:12:43 +03:00
svetasmirnova
5a46d3df77 PT-2037 Added option --system-only to pt-stalk 2022-01-11 01:20:10 +03:00
Sveta Smirnova
b2860eee31 PT-1336 pt-stalk removes user files from destination (#524)
* PT-1336 Added file name checks and -maxdepth 1 parameter

* PT-1336 pt-stalk removes user files from the destination directory

* Added check that files to be removed were created by pt-stalk

* Added option -maxdepth 1 to find command, so the tool does not search for files in the nested directories

* Added unit tests

* This should fix PT-1812 also
2021-12-10 09:18:18 -03:00
Lenz Grimmer
fde683fb5d PT-1887: Fixed pt-diskstats to work on 5.x kernels (#526)
The amount of fields returned for disk statistics
changed in newer Linux kernel versions to 20 and
caused `pt-diskstat` to simply not provide any output
on these systems. Updated the check in the diskstats
parsing code to proceed with 20 fields as well.

Signed-off-by: Lenz Grimmer <lenz.grimmer@percona.com>
2021-12-03 13:19:36 -03:00
Carlos Salguero
a019907e6e PT-2023 Set out mode to utf8 (#523) 2021-11-18 14:41:35 -03:00
Carlos Salguero
c5a22407a7 PT-2016 pt-table-sync and CRC32 columns (#521)
* PT-2016 CRC32 on key

* PT-2016 Added tests

* Removed commented out line
2021-11-03 13:53:30 -03:00
Carlos Salguero
951a333c81 PT-1398 Updated way of getting MySQL PID (#517) 2021-10-01 09:27:20 -03:00
Carlos Salguero
0d2c48456f PT-1627 Improve pt-mysql-summary jemalloc detection (#514)
* PT-1627 Changed jemalloc detection function

* Updated Tests
2021-10-01 09:19:25 -03:00
Carlos Salguero
9d6508da5f PT-1900 pt-query-digest not hiding parameter properly sometimes when parameter=binary (#510)
* PT-1900 WIP

* Fixed quoted strings regexes

* PT-1900 Fixed query rewriter to properly handle quoted text

* Fixed merge conflicts with 3.x branch
2021-09-27 08:23:23 -03:00
Carlos Salguero
d91ba9cadd PT-1959 Updated version number (#515)
Not updated in files that will be updated during the build like the
python script, Makefile.PL and some other files like documentation.
2021-09-20 10:27:29 -03:00
Sergey Kuzmichev
ad593c6de9 Fix for PT-1983: pt-summary has missing records for memory due to dmidecode changes (#513)
* Fix for PT-1983 pt-summary missing one DIMM

This is a fix for PT-1983. dmidecode now doesn't print "Speed: UKNOWN" for
missing DIMMs. pt-summary code relies on the Speed field to be there, and when
it's missing, rows get concatenated (or rather not broken down correctly).
Adjusted the relevant code to not rely on the Speed field to be present.

* Added test for the fix for PT-1983

Due to dmidecode changes in 3.2, pt-summary needs to be able to work
with memory-related info with some missing fields in the dmidecode
output. This commit adds a test with dmidecode output modified to
include new format.

In addition, fixing test that are failing due to commit c9ec74c251

* PT-1983: updated changelog to reflect the bug fixed
2021-09-08 07:32:30 -03:00
Carlos Salguero
e389d2181a PT-1800 Updated docs (#511) 2021-08-31 08:16:34 -03:00
Carlos Salguero
d14d18a1bf PT-1997 Added error logs (#506) 2021-08-03 09:07:25 -03:00
duxthefux
4a32ca7675 Added mysql version check for bug fix introduced in 8.0.14 (#474)
Thanks for the fix.
2021-07-30 19:41:53 -03:00
Carlos Salguero
ecf43e2a17 PT-1985 pt-deadlock db schema check (#502) 2021-07-29 12:15:53 -03:00
Carlos Salguero
543f30c073 PT-1986 Check schema exists upon start (#503) 2021-07-29 12:15:23 -03:00
I
8e334d9ed2 PT-1974: Support fingerprinting for --print in pt-kill (#495)
* PT-1974: Support fingerprinting for --print in pt-kill

This commit implements `--fingerprint` that modifies the `--print`
behaviour and forces the query fingerprint to be displayed instead of an
original query.

* Added changelog entry.
2021-06-24 11:40:32 -03:00
Sergey Kuzmichev
49b570ed0c Fix for PT-1965 mysql-only lacks mysqladmin output (#492)
* Fix for PT-1965 mysql-only lacks mysqladmin output

This is a fix for PT-1965. Moved mysqladmin execution out of an if block
that contains stuff executed only when --mysql-only is not specified.
Before this fix, mysqladmin was not executed with --mysql-only.

* PT-1965: updated changelog to reflect the bug fixed
2021-06-10 08:43:07 -03:00
Carlos Salguero
0b97e1f471 PT-1747 pt-online-schema-change was bringing the database into a broken state (#491)
* PT-1717 Updated behavior for FKs

* Fixed rename_columns.t

* Fixed pt-244.t

* fixed drop swap test

* Fixed test for MySQL 8.0

* Updated test for MySQL 8

* Updated test for MySQL 8
2021-06-03 12:00:43 -03:00
Paul Jacobs
be856c84b5 Update copyright year 2021-05-04 15:42:25 +03:00
Carlos Salguero
c47ac4d4df PT release 3.3.1 testing branch (#486)
* PT release 3.3.1 testing branch

* Update pkgs version

* Updated version from build script
2021-04-30 10:08:05 -03:00
Roma Novikov
d9b3280f07 Update pt-summary (#487) 2021-04-27 08:55:10 -03:00
Carlos Salguero
5a0cbd79fc PT-716 [WIP] Forced free to use wide mode (#485) 2021-04-06 10:13:21 -03:00
Carlos Salguero
e570cecbab PT-1914 Column data lost when 'Generated' is in the column comment (#482)
* PMM-1914 Fixed column parsing having generated

Fixed table parser code that errouneously considered a column as
generated when the default was empty DEFAULT '' and the COMMENT had
the word 'Generated'.

* PMM-1914 Updated TableParser in all programs

* PT-1914 Updated changelog

* PT-1914 Added test
2021-03-09 11:46:05 -03:00
Carlos Salguero
4f78d5450c PT-1919 drop-swap can drop triggers (#483)
* PT-1919 drop-swap can drop triggers

* PT-1919 Update changelog
2021-03-05 11:32:03 -03:00
PaulJacobs-percona
df16170eda Release 3.3.0 (#480)
* Updated Percona::Toolkit version

* Updated version in POD headers

* RM-880-RN-3.3.0

* Update makefile to use go dep

* Update year

* Theme is downloaded by Makefile

* Correct path to icon

* Updated pt-osc --reverse-triggers help

* Update reverse-triggers option

* PT-1905 Reverse triggers wording

Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>
2021-01-14 15:22:19 -03:00
Carlos Salguero
00c437bfe2 Fixed --reverse-triggers help 2021-01-13 09:44:37 -03:00
Carlos Salguero
9ef2b56f4d PT-1905 Reverse triggers (#477)
* PT-1905 Reverse triggers

* WIP

* PT-1905 Updated reverse triggers creation

* PT-1905 Fixed trigger creation
2021-01-12 13:58:34 -03:00
Carlos Salguero
71c9902532 PT-1898 pt archiver keeps transaction open (#479)
* PT-1898 WIP

* PT-1898 Added commit for open txns

* PT-1898 Added commit for open txns

* PT-1898 removed dev leftover
2021-01-12 13:40:51 -03:00
Carlos Salguero
e1b7a6a1fd PT-1857 heartbeat can't reconnect (#475)
* PT-1857 pt-heartbeat reconnect

* PT-1857 Updated changelog
2020-12-24 16:23:42 -03:00
Carlos Salguero
2233224ff6 PT-1528 pt-online-schema-change "Invalid utf8mb4 character string (#473)
* PT-1528 [WIP] Proposed solution

* PT-1528 Ingore UTF8 warning
2020-12-24 16:22:46 -03:00
Carlos Salguero
b716d128c0 PT-1908 Increased query id column size in report (#472) 2020-11-17 12:09:19 -03:00
Ayush Goyal
903a7d61c5 Support adding explain output for slow query in JSON report as well 2020-11-05 14:08:51 +05:30
Nayuta Yanagisawa
442a099268 PT-1881: Fix missing argument error of pt-upgrade (#461)
* Fix missing argument error of pt-upgrade

* Use print rather than printf
2020-10-27 19:15:21 -03:00
Carlos Salguero
d7a7a1c7a5 PT-169 Handle drop-swap errors in pt-online-schema-change (#467)
* PT-169 Fixed drop-swap error handling

* PT-169 Fixed drop_swap error handling
2020-10-12 15:36:45 -03:00
Carlos Salguero
a8ea8dfe56 PT-1892 Fixed sshd process recognition (#465) 2020-09-30 11:04:34 -03:00
Carlos
750d2ef9d9 Updated version to 3.2.1 in all tools 2020-08-12 14:54:55 -03:00
Carlos
232263f176 Updated Percona::Toolkit version and changelog 2020-08-12 14:46:09 -03:00