Commit Graph

263 Commits

Author SHA1 Message Date
Sveta Smirnova
cd727bf9af PT-2248 - pt-k8s-debug-collector does not run pg_gather with K8SPG 2 (#654)
* PT-2248 - pt-k8s-debug-collector does not run pg_gather with K8SPG 2

- Added check for K8SPG 2, so can run pg_gather for it
- Added new allowed value for option --resource:
-- pgv2 for K8SPG 2
-- auto to auto-detect custom resource
- Option --resource has now default value "auto"
- Updated documentation
- Added test cases for new options

* PT-2248 - pt-k8s-debug-collector does not run pg_gather with K8SPG 2

- Implemented custom user and secrets handling (in case when no default
  user exists).
2023-09-13 15:11:04 +03:00
Viktor Szépe
2bd40d8c39 Remove trailing spaces (#665)
* Remove trailing spaces

* PR-665 -  Remove trailing spaces

- Updated not stable test t/pt-online-schema-change/preserve_triggers.t
- Updated utilities in bin directory

* PR-665 -  Remove trailing spaces

- Fixed typos

* PR-665 -  Remove trailing spaces

- Fixed typos

---------

Co-authored-by: Sveta Smirnova <sveta.smirnova@percona.com>
2023-09-06 01:15:12 +03:00
Viktor Szépe
3867d10ba5 Fix line ends (#664)
* Fix line ends of pt-mongodb-query-digest README

* Fix line ends of t/pt-diskstats/samples/small.txt
2023-08-23 16:27:18 +03:00
Viktor Szépe
ea67327ff7 Fix more typos (#658) 2023-08-22 19:34:24 +03:00
Sveta Smirnova
156372fb73 PT-2235 - pt-mongodb-index-check does not support option --version (#653)
* PT-2235 - pt-mongodb-index-check does not support option --version

Changed command version to flag version

* PT-2235 - pt-mongodb-index-check does not support option --version

Updated documentation
2023-08-01 17:56:17 +03:00
Sveta Smirnova
ab4bf1b1c6 PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard (#652)
* PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard for option --version

- Redirected UsageWriter to os.Stdout for kingpin, so --version output
  goes to STDOUT, not to STDERR
- Adjusted text, printed by the --version flag
- Added test cases to check how --version flag works
- Adjusted test cases, so they use TOOLNAME constant

* PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard for option --version

Run go mod tidy as requested by Artem Gavrilov

* PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard for option --version

Renamed const TOOLNAME to toolname to follow Go coding standards
2023-07-31 16:59:30 +03:00
Sveta Smirnova
e998bd5b55 PT-2101 pt-mongodb-query-digest does not work on standalone server (#630)
* PT-2101 - pt-mongodb-query-digest doesn't work on standalone server

Restoring test case, disabled for new sandbox that was never created.
Added debugging output to find out why the tool behaves not as expected.
Most of changes into main.go will be removed after the fix is done.

* PT-2101 - pt-mongodb-query-digest doesn't work on standalone server

- Changed code so it works with the standalone server
- Updated main_test.go so it works for MongoDB 5.0
- Removed eval.js and group.js, because these command are not supported since MongoDB 4.2

* PT-2101 - pt-mongodb-query-digest doesn't work on standalone server

Updated go.mod and go.sum from the 3.x branch
2023-06-14 15:04:57 +03:00
Michael Patrick
832c928020 Fix broken --output-file (#620)
Fixes the broken write to output file option, --output-file.
2023-05-19 13:05:16 +03:00
Sveta Smirnova
501e4cd568 PT-2169 - pt-k8s-debug-collector integration of pg_gather requires croping first line of the output file (#615)
* PT-2169 - pt-k8s-debug-collector integration of pg_gather requires croping first line of the output file

Modified pt-k8s-debug-collector so it redirects only STDOUT to summary.txt
STDERR is stored in the logs and recorded in summary.txt only if summary fails with the error
Modified Makefile, so it does not include closing bracket into the version string
Created test case for the fix

* PT-2169 - pt-k8s-debug-collector integration of pg_gather requires croping first line of the output

Removed else as requested by Ege
2023-04-20 03:12:52 +03:00
Sveta Smirnova
46a466efcc PT-2165 actual version, go version, and build for go tools (#603)
* PT-2165 - Actual Version, GoVersion, and Build for Go tools

Removed outdated numbers from main.go, because we update these numbers in the Makefile.

* PT-2165 - Actual Version, GoVersion, and Build for Go tools

Added template version into Makefile for the latest tag.
2023-02-27 01:19:26 +03:00
Sveta Smirnova
08493a6d71 PT-2164 version flag for pt-k8s-debug-collector (#576)
* PT-2164 - pt-k8s-debug-collector does not have version flag

Added flag
Updated README

* PT-2164 - pt-k8s-debug-collector does not have version flag

Added test case

* PT-2164 version flag for pt-k8s-debug-collector

go fmt for src/go/pt-k8s-debug-collector/main.go

* PT-2164 version flag for pt-k8s-debug-collector

Updated docs/pt-k8s-debug-collector.rst
2023-01-31 19:11:06 +03:00
Sveta Smirnova
c1696fd9f8 Release 3.5.1 (#574)
* RM-1153 - Percona Toolkit 3.5.1

Updated version for Perl files to 3.5.1
Updated Perl module Percona::Toolkit to version 3.5.1 and updated all scripts using this module
Manually updated Go programs
Moved ONLY_UPDATE_VERSION after definition of DATE and SERIES variables, so the script does not fail because of undefined variables

* RM-1153 - Percona Toolkit 3.5.1

Updated Changelog
Updated release date in docs/percona-toolkit.pod

* PT-2162 Release Notes 3.5.1 (#571)

modified:   Makefile.PL
	modified:   config/sphinx-build/conf.py
	modified:   docs/release_notes.rst
	new file:   docs/rn.3-5-1.txt

Co-authored-by: Alina Derkach <“alina.derkach@percona.com”>

* PT-7 Fix syntax (#573)

Co-authored-by: Alina Derkach <81975178+alina-derkach-oaza@users.noreply.github.com>
Co-authored-by: Alina Derkach <“alina.derkach@percona.com”>
Co-authored-by: EvgeniyPatlan <evgeniy.patlan@percona.com>
2023-01-23 17:51:17 +03:00
Sveta Smirnova
e8ebc855d0 PT-2134 - support for PostgreSQL and MySQL operators (#567)
* PT-2134 - Support for PostgreSQL and PS MySQL operators

For PS MySQL operator: added support of pt-mysql-summary
For PostgreSQL operator: added support for pg_gather
Options added:
  - kubeconfig - path to kubeconfig
  - forwardport - port to use when calling pt-*-summary tools ad pt_gather
Options changed:
  - resource - now default value is  (was pxc). New values added:
    psql - for PostgreSQL operator
    ps - for PS MySQL operator
    if default value () is used: only K8 information is collected
    Otherwise resource-specific summary is collected

* PT-2134_support_for_PostgreSQL_and_MySQL_operators

Fixed summary collection when connecting to PostgreSQL opertor and no resource specified

* PT-2134 - support for PostgreSQL and MySQL operators

Added test case for full supported set of values for option --resource,
added --kubeconfig and --forwardport options for test cases,
added support for environment variables KUBECONFIG_PXC, KUBECONFIG_PS, KUBECONFIG_PSMDB, KUBECONFIG_PSQL, FORWARDPORT for test cases

* PT-2134 - support for PostgreSQL and MySQL operators

updated docs/pt-k8s-debug-collector.rst,
replaced README.md in src/go/pt-k8s-debug-collector wih symbolic link to docs/pt-k8s-debug-collector.rst
to avoid having inconsistent documentation

* PT-2134 - support for PostgreSQL and MySQL operators

Removed curl STDERR from the command output

* PT-2134 - support for PostgreSQL and MySQL operators

typo in pt-k8s-debug-collector.rst

* PT-2134 - support for PostgreSQL and MySQL operators

Renamed --resource=psql to --resource=pg after review suggestion by Ege
and collecting feedback from potential users

Co-authored-by: EvgeniyPatlan <evgeniy.patlan@percona.com>
Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>
2022-12-19 18:48:51 +03:00
Carlos Salguero
882d8787c7 Release 3.5.0 (#563)
* Changelog and version for Release 3.5.0

* PT-2076 add rhel9 dependency

* PT-2128 Updated version to 3.5.0

Co-authored-by: svetasmirnova <sveta.smirnova@percona.com>
Co-authored-by: EvgeniyPatlan <evgeniy.patlan@percona.com>
2022-12-01 14:31:46 +03:00
Sveta Smirnova
03ab27228f PT 2105 collect individual logs (#558)
* PT-2105 - Collect individual log files for PXC

Combined log files are hard to read by humans. Since pt-k8-debug-collector is the tool that
accesses data from the running pods, it can copy raw log files, necessary for troubleshooting PXC issues.
So, in addition to collecting logs.txt, this adds method getIndividualFiles for the Dumper that
reads individual files from PXC pods and stores them in the resulting archive.

Additionally, this commit fixes invalid timestamps in the resulting archive.

* PT-2105 - added support for non-default namespaces

* PT-2105 Let pt-k8-debug-collector to collect individual logs in PXC pods

Added test case for this new collection.

* Update go.mod

Co-authored-by: Viacheslav Sarzhan <slava.sarzhan@percona.com>

Co-authored-by: Viacheslav Sarzhan <slava.sarzhan@percona.com>
2022-11-17 19:32:19 +03:00
Chetan Shivashankar
1828526319 Fixing Readme (#527) 2022-09-28 20:59:11 +03:00
Carlos Salguero
896fdcede8 Release 3.4.0 (#539)
* PT-1979 Improved mdb summary information

Added the MongoDB instance command line options to the output.
Also fixed the spacing in the replicaset section.

* Added new template file

* PT-1978 Implemented duplicated indexes detection

* WIP

* PT-1978 WIP

* PT-1978 New tool pt-mongodb-check-index

* Upgraded version and dependencies

* Updated Go sources

* Removed unused file

* Added version command

* Added changelog

* Updated required params

* Added system.profile to the ingored dbs

* Updated Changelog

* Updated version in all programs

* Updated changelog

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* Update Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>

* PT-7 fix 3.4.0 build

* Updated Makefile

* Updated changelog

* Updated Changelog

Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>
Co-authored-by: Evgeniy Patlan <evgeniy.patlan@percona.com>
2022-07-11 10:29:37 -03:00
Carlos Salguero
d881738e2f PT-1978 Implemented duplicated indexes detection (#537)
* PT-1978 Implemented duplicated indexes detection

* WIP

* PT-1978 WIP

* PT-1978 New tool pt-mongodb-check-index

* Removed unused file

* Added version command

* Updated required params

* Added system.profile to the ingored dbs

* Fixed for CR
2022-04-20 09:15:45 -03:00
Carlos Salguero
8519e103ad PT-1979 Improved mdb summary information (#536)
* PT-1979 Improved mdb summary information

Added the MongoDB instance command line options to the output.
Also fixed the spacing in the replicaset section.

* Added new template file

* Upgraded deps due to security issues
2022-04-07 18:12:45 -03:00
Robert Lankford
60ed28ab31 remove depedency on github.com/prometheus/common/log (#520)
* remove depedency on github.com/prometheus/common/log

* go mod tidy
2021-11-09 09:52:55 -03:00
Carlos Salguero
16716b3a7b PMM-7116 incorrect startup state (#519)
* Updated changelog for (WIP) 3.3.2

* Updated changelog

* PT-7116 Updated MyState results

* PT-7116 Updated MyState results

* PT-7116 Updated MyState results

* PMM-7116 Return the error in MyState

Don't manage the error, just bubble it up.
The client must know there was an error, even when the error is no
'real' like in Mongos instances (there is no error because there is no
replicaset for mongos or standalone)
2021-11-08 10:59:54 -03:00
Carlos Salguero
d91ba9cadd PT-1959 Updated version number (#515)
Not updated in files that will be updated during the build like the
python script, Makefile.PL and some other files like documentation.
2021-09-20 10:27:29 -03:00
Jelle van der Waa
b19654fa48 Make build reproducible (#499)
By setting SOURCE_DATE_EPOCH builds the build binaries can become
reproducible as a rebuild will take the old build date as input.

Motivation: https://reproducible-builds.org
2021-09-20 10:26:27 -03:00
Carlos Salguero
dd2b144004 PMM-8575 migrate toolkit to modules (#512)
* PMM-8575 Fixed test

Fixed sandbox used by go programs, updated all tests.

* PMM-8575 Fixed test

Fixed sandbox used by go programs, updated all tests.

* Removed Gopkg

* Switched to go 1.16

* Updated go.mod

* Upgraded deps

* Updated deps
2021-09-10 16:08:58 -03:00
Carlos Salguero
4b8ae0563f PMM-8489 Cluster ID for standalone server (#507)
* PMM-8489 Cluster ID for standalone server

* Updated error type

* Updated MyState to ignore errors on standalone servers

* Deleted dep files

* Revert "Deleted dep files"

This reverts commit 236303196b.

* Removed go.mod &^ updated lock

* Updated mongodb driver

* PMM-8489 Updated errors list
2021-08-18 11:22:47 -03:00
Andrii Dema
d84cb2496e PT-1930: pt-k8s-debug-collector should not attempt collecting PXC information when running against PSMDB (#498)
* PT-1930: pt-k8s-debug-collector should not attempt collecting PXC information when running against PSMDB

https://jira.percona.com/browse/PT-1930

Removed PXC resources to collect while running with `--resource psmdb`

* Add more `psmdb` resources

Also resolves https://jira.percona.com/browse/PT-1934

* Add `resourceType` function to collect summaries for PXC/PSMDB when option `--cluster` is specified
2021-07-28 09:01:20 -03:00
Andrii Dema
b0597554ad PT-1931: pt-k8s-debug-collector does not gather summaries for PSMDB (#500)
https://jira.percona.com/browse/PT-1931

When I tried to reproduce the bug, `pt-mongodb-summary` didn't launch only when option `--cluster` was specified. This PR fixes this behavior and also fixes https://jira.percona.com/browse/PT-1932
2021-07-27 13:49:58 -03:00
Andrii Dema
1114eb4dad PT-1926: pt-k8s-debug-collector should not collect passwords (#497)
https://jira.percona.com/browse/PT-1926

Save secrets only with configuration data
2021-07-21 10:38:20 -03:00
Dávid Mikuš
4dd72d96b1 Add some missing attributes for members (#490) 2021-05-04 10:49:48 -03:00
Carlos Salguero
c47ac4d4df PT release 3.3.1 testing branch (#486)
* PT release 3.3.1 testing branch

* Update pkgs version

* Updated version from build script
2021-04-30 10:08:05 -03:00
Carlos Salguero
d78acd961c PMM-7689 Updated data type for getLastErrorDefaults.w (#484)
This field is not being used by any tool but it is needed to decode
bigger structures having it. Since the value can be an int to indicate
the number 1-primary, etc, or 'majority', I've changed the field type
from int to interface{}
2021-04-05 14:45:46 -03:00
PaulJacobs-percona
df16170eda Release 3.3.0 (#480)
* Updated Percona::Toolkit version

* Updated version in POD headers

* RM-880-RN-3.3.0

* Update makefile to use go dep

* Update year

* Theme is downloaded by Makefile

* Correct path to icon

* Updated pt-osc --reverse-triggers help

* Update reverse-triggers option

* PT-1905 Reverse triggers wording

Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>
2021-01-14 15:22:19 -03:00
PaulJacobs-percona
77d2025a5a Update README.md
Formatting, spelling
2021-01-07 09:20:12 +03:00
Max Dudin
03d4943617 Merge branch '3.x' into CLOUD-535 2020-11-04 13:34:59 +02:00
Carlos Salguero
ff6b05b381 PT-1891 Fixed mongodb-summary connection with ssl (#469)
* PT-1891 Fixed mongodb-summary connection with ssl

- Added SSL connection options
- Fixed old tests
- Replaced gofmt by gofumpt in Makefile
- There are no ssl test for mongodb-summary because the current sandbox
doesnt support it

* PT-1891 Ran gofumports

* PMM-1891 Fixes for CR

* PT-1891 Decreased minimum TLS reqs for compatibility
2020-11-02 17:13:29 -03:00
Carlos Salguero
0aac7b4cfc PMM-6494 Topology labels (#463)
* PMM-6494 Topology labels

Added common functions needed for topology lables in mongo exporter and
pt-mongodb-summary

* PMM-5723 Fix reviewdog.

Co-authored-by: Nurlan Moldomurov <nurlan.moldomurov@percona.com>
2020-09-08 19:48:09 +03:00
Max Dudin
08c9621c14 PT-1865 fix typo 2020-08-05 18:00:30 +03:00
Max Dudin
61ba3c729e CLOUD-535 gopath added, changed suumary func name 2020-08-05 13:59:46 +03:00
Max Dudin
212d2032ed PT-1865 small changes 2020-08-04 00:00:54 +03:00
Max Dudin
9b90408451 PT-1865 Changed tool name 2020-08-03 20:39:12 +03:00
Carlos Salguero
002303fc2e PMM-6256 Updated ExplainCmd for MongoDB (#459)
* PMM-6256 Updated ExplainCmd for MongoDB

In order to make explain for MongoDB work, we need to remove the "$db"
field from the explain command generated in proto.system.profile since
it is a duplicated field that triggers a MongoDB error.

* PMM-6256 New test

* PMM-6256 Removed commented out code

Co-authored-by: Carlos <cfsalguero@gmail.com>
2020-07-29 08:39:07 -03:00
Max Dudin
0d2a2f8fdc PT-1865 Updated README 2020-07-17 14:29:17 +03:00
Max Dudin
309bb24f8a PT-1865 Added pt-mysql-summury and pt-mobgodb-summary 2020-07-17 14:28:13 +03:00
Sergey Kuzmichev
8b5e885173 PT-1859 ( PT-1868 ) and general pt-pg-summary improvements (#455)
* Fix for PT-1868 and general pt-pg-summary improvements

This is a rather large piece of changes to pt-pg-summary, which
includes:
* Corrected dependency for models in lib/pginfo
* Fixed existing testing infrastructure and implemented new tests:
** Test for New in pginfo
** Test for TestCollectGlobalInfo
** Test for TestCollectPerDatabaseInfo
* Fixed models to reflect PG12 changes (datid 0 with no name)
** Modified gen.sh to include PG12 containers and work with the same
docker-compoe that tests use
* Updated templates and helper functions
* Fixed standby detection and template output

With these changes, pt-pg-summary works correctly with PG12 hosts.

* Extra port in pt-pg-summary models gen.sh, removing unused
2020-07-15 09:24:13 -03:00
Carlos Salguero
c7eed08e33 MongoDB EXPLAIN JSON APIs (#454)
* WIP

* PMM-4192 Updated MongoDB fingerprint

SystemProfile has been changed to use the new bson.D from the official
MongoDB driver instead of the old BsonD. Updated the fingerprinter
module and all tests

* PMM-4192 Updated MongoDB explain tests

Updated test to use bson.D instead of BsonD

* PMM-4192 Code clean-up

* PMM-4192 Changes for CR

* PMM-4192 Changes for CR

* PMM-4192 Removed unused deps

Co-authored-by: Carlos <cfsalguero@gmail.com>
2020-07-08 10:25:12 -03:00
Max Dudin
57c7769c5b PT-1865 Update readme, rename tool 2020-07-07 12:06:01 +03:00
Max Dudin
dbb1982321 PT-1865 Clean-up 2020-07-06 12:19:32 +03:00
Max Dudin
36e9a2f07f PT-1865 add README file 2020-07-03 12:12:45 +03:00
Max Dudin
118110a671 PT-1865 small changes 2020-07-03 12:02:30 +03:00
Max Dudin
faffa70867 PT-1865 Remove archive package 2020-07-01 10:53:28 +03:00