Commit Graph

398 Commits

Author SHA1 Message Date
Sveta Smirnova
db5289138f Merge pull request #715 from szepeviktor/patch-3
Fix variable collision with imported package name
2023-11-22 21:31:43 +03:00
Sveta Smirnova
2ec78b3de4 Merge pull request #716 from szepeviktor/patch-4
Make OrderedBy not exported
2023-11-22 01:28:03 +03:00
Viktor Szépe
81882b67cc fix 2023-11-21 02:41:26 +00:00
Viktor Szépe
c6b6a4086d Update src/go/lib/config/config_test.go
Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>
2023-11-19 22:13:53 +01:00
Viktor Szépe
ed7f4b3e39 Update src/go/lib/config/config_test.go
Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>
2023-11-19 22:13:47 +01:00
Viktor Szépe
61021d9ce6 Update src/go/lib/config/config_test.go
Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>
2023-11-19 22:13:42 +01:00
Viktor Szépe
629ded3960 Make OrderedBy's return type exported 2023-11-17 20:47:54 +01:00
Viktor Szépe
1c5048546a Fix variable collision with imported package name 2023-11-17 20:38:08 +01:00
Viktor Szépe
2ea8f4036c Fix MongoDB Security 2023-11-17 20:34:35 +01:00
Viktor Szépe
65358b715f Fix MongoDB StorageEngine 2023-11-17 20:32:07 +01:00
Sveta Smirnova
b862945245 PT-2281 - provide container name for copying files in the dump
- Changed getIndividualFiles function and Dumper data structure, so we
  can specify container name for PXC and other operators which store
  logs in the separate container.
2023-11-14 16:55:56 +03:00
Sveta Smirnova
f9c43e9bae ARM64 support for macOS
- Added option darwin-arm64 into Go tools Makefile
- Changed --delimiter option to its short version that works on all
  platforms
- Changed "Mongo tools" comments to "Go tools", because now Go tools are
  not only for Mongo
2023-11-13 17:40:58 +03:00
Yoann La Cancellera
208708a58b Add: concurrent SSTs handling
It is a thing: 2 nodes joining at the same time, with 2 JOINERs and 2
DONORs cluster-wide
It can happen on operators with 2 garbd joining at the same time

Before, pt-galera-log-explainer was using SST metadata naively.
Basically if a node was DONOR and we found a "transfer completed"
message, we assumed the donor name we found is the correct one.
So for concurrent SSTs, donors were swapping names.

Now, it is handled by a map, indexed by a donor name. To know if a node
is actual donor or not, it now compare timestamps of events. It assumes
both "selected donor" and "shifting DONOR" messages should have happen
in less than 0.01 secs to avoid any conflict.

Regression tests coming in next commit with an operator logs having
concurrent SSTs. Another conflicts was sometimes breaking the test
depending on the order on which we read files, hence why it's not added
here yet
2023-11-07 18:08:24 +01:00
Yoann La Cancellera
117d683872 Change: simplify verbose mode 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
161c2af084 Fix: pointer dereference if votes was missing 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
12ea8033a9 Add: proper error handling for files 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
b0477bc37f Fix: don't cut node name if it's an ip
rare case, but it was not producing great results
regression tests will come for this
2023-11-07 18:08:24 +01:00
Yoann La Cancellera
94c89c2892 Add: requirements in README.rst 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
72c55a15ec Remove: extra space in --version out 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
3ad09d8418 Update README.rst 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
a7a4b4adca Add: --no-color regression test 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
6613880a9b Change: README to pt standard, regex-list --json 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
0d554e9515 Add: merge regression tests 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
75ce300d7c Fix: "not found" should not be an error
if it shows nothing, it can be understood as easily as a grep returning
nothing
adding an error log is too verbose
2023-11-07 18:08:24 +01:00
Yoann La Cancellera
fea9be26bd Add: regression tests 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
6d6f30372c Fix: error msg with uppercase, usage missed pt- 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
5638387713 Fix: --version formatting 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
bbba087a83 Fix: crash when timeline is empty
It was due to a silly regression when reformatting the main.go
The function iterating was doing too many things, and returning an error
when nothing was found, and a "continue" was done on the main
"timelineFromPaths" loop
It is now a simple foreach loop that does not return error so we have to
check if the localtimeline slice is empty
2023-11-07 18:08:24 +01:00
Yoann La Cancellera
7f8071ce3f Simplified the main loop 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
5bbb11df17 Add: merge by directory, Simplified: timeline merges 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
e03fe1801c Fix: forgot to inherit conflicts 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
b054429057 Improve: "not found" error handling 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
b4cad31e77 Add pt-galera-log-explainer 2023-11-07 18:08:19 +01:00
Viktor Szépe
d01c0cea32 Fix last typos (#678) 2023-09-18 15:53:57 +03:00
Viktor Szépe
f2f7a7993d Add final newlines and remove trailing spaces (#672)
* Add final newlines and remove trailing spaces

* Replace TAB characters with spaces

* Fix indent
2023-09-15 17:31:40 +03:00
Sveta Smirnova
cd727bf9af PT-2248 - pt-k8s-debug-collector does not run pg_gather with K8SPG 2 (#654)
* PT-2248 - pt-k8s-debug-collector does not run pg_gather with K8SPG 2

- Added check for K8SPG 2, so can run pg_gather for it
- Added new allowed value for option --resource:
-- pgv2 for K8SPG 2
-- auto to auto-detect custom resource
- Option --resource has now default value "auto"
- Updated documentation
- Added test cases for new options

* PT-2248 - pt-k8s-debug-collector does not run pg_gather with K8SPG 2

- Implemented custom user and secrets handling (in case when no default
  user exists).
2023-09-13 15:11:04 +03:00
Viktor Szépe
2bd40d8c39 Remove trailing spaces (#665)
* Remove trailing spaces

* PR-665 -  Remove trailing spaces

- Updated not stable test t/pt-online-schema-change/preserve_triggers.t
- Updated utilities in bin directory

* PR-665 -  Remove trailing spaces

- Fixed typos

* PR-665 -  Remove trailing spaces

- Fixed typos

---------

Co-authored-by: Sveta Smirnova <sveta.smirnova@percona.com>
2023-09-06 01:15:12 +03:00
Viktor Szépe
3867d10ba5 Fix line ends (#664)
* Fix line ends of pt-mongodb-query-digest README

* Fix line ends of t/pt-diskstats/samples/small.txt
2023-08-23 16:27:18 +03:00
Viktor Szépe
ea67327ff7 Fix more typos (#658) 2023-08-22 19:34:24 +03:00
Sveta Smirnova
156372fb73 PT-2235 - pt-mongodb-index-check does not support option --version (#653)
* PT-2235 - pt-mongodb-index-check does not support option --version

Changed command version to flag version

* PT-2235 - pt-mongodb-index-check does not support option --version

Updated documentation
2023-08-01 17:56:17 +03:00
Sveta Smirnova
ab4bf1b1c6 PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard (#652)
* PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard for option --version

- Redirected UsageWriter to os.Stdout for kingpin, so --version output
  goes to STDOUT, not to STDERR
- Adjusted text, printed by the --version flag
- Added test cases to check how --version flag works
- Adjusted test cases, so they use TOOLNAME constant

* PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard for option --version

Run go mod tidy as requested by Artem Gavrilov

* PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard for option --version

Renamed const TOOLNAME to toolname to follow Go coding standards
2023-07-31 16:59:30 +03:00
Sveta Smirnova
e998bd5b55 PT-2101 pt-mongodb-query-digest does not work on standalone server (#630)
* PT-2101 - pt-mongodb-query-digest doesn't work on standalone server

Restoring test case, disabled for new sandbox that was never created.
Added debugging output to find out why the tool behaves not as expected.
Most of changes into main.go will be removed after the fix is done.

* PT-2101 - pt-mongodb-query-digest doesn't work on standalone server

- Changed code so it works with the standalone server
- Updated main_test.go so it works for MongoDB 5.0
- Removed eval.js and group.js, because these command are not supported since MongoDB 4.2

* PT-2101 - pt-mongodb-query-digest doesn't work on standalone server

Updated go.mod and go.sum from the 3.x branch
2023-06-14 15:04:57 +03:00
Michael Patrick
832c928020 Fix broken --output-file (#620)
Fixes the broken write to output file option, --output-file.
2023-05-19 13:05:16 +03:00
Sveta Smirnova
501e4cd568 PT-2169 - pt-k8s-debug-collector integration of pg_gather requires croping first line of the output file (#615)
* PT-2169 - pt-k8s-debug-collector integration of pg_gather requires croping first line of the output file

Modified pt-k8s-debug-collector so it redirects only STDOUT to summary.txt
STDERR is stored in the logs and recorded in summary.txt only if summary fails with the error
Modified Makefile, so it does not include closing bracket into the version string
Created test case for the fix

* PT-2169 - pt-k8s-debug-collector integration of pg_gather requires croping first line of the output

Removed else as requested by Ege
2023-04-20 03:12:52 +03:00
Sveta Smirnova
46a466efcc PT-2165 actual version, go version, and build for go tools (#603)
* PT-2165 - Actual Version, GoVersion, and Build for Go tools

Removed outdated numbers from main.go, because we update these numbers in the Makefile.

* PT-2165 - Actual Version, GoVersion, and Build for Go tools

Added template version into Makefile for the latest tag.
2023-02-27 01:19:26 +03:00
Sveta Smirnova
08493a6d71 PT-2164 version flag for pt-k8s-debug-collector (#576)
* PT-2164 - pt-k8s-debug-collector does not have version flag

Added flag
Updated README

* PT-2164 - pt-k8s-debug-collector does not have version flag

Added test case

* PT-2164 version flag for pt-k8s-debug-collector

go fmt for src/go/pt-k8s-debug-collector/main.go

* PT-2164 version flag for pt-k8s-debug-collector

Updated docs/pt-k8s-debug-collector.rst
2023-01-31 19:11:06 +03:00
Sveta Smirnova
c1696fd9f8 Release 3.5.1 (#574)
* RM-1153 - Percona Toolkit 3.5.1

Updated version for Perl files to 3.5.1
Updated Perl module Percona::Toolkit to version 3.5.1 and updated all scripts using this module
Manually updated Go programs
Moved ONLY_UPDATE_VERSION after definition of DATE and SERIES variables, so the script does not fail because of undefined variables

* RM-1153 - Percona Toolkit 3.5.1

Updated Changelog
Updated release date in docs/percona-toolkit.pod

* PT-2162 Release Notes 3.5.1 (#571)

modified:   Makefile.PL
	modified:   config/sphinx-build/conf.py
	modified:   docs/release_notes.rst
	new file:   docs/rn.3-5-1.txt

Co-authored-by: Alina Derkach <“alina.derkach@percona.com”>

* PT-7 Fix syntax (#573)

Co-authored-by: Alina Derkach <81975178+alina-derkach-oaza@users.noreply.github.com>
Co-authored-by: Alina Derkach <“alina.derkach@percona.com”>
Co-authored-by: EvgeniyPatlan <evgeniy.patlan@percona.com>
2023-01-23 17:51:17 +03:00
Sveta Smirnova
e8ebc855d0 PT-2134 - support for PostgreSQL and MySQL operators (#567)
* PT-2134 - Support for PostgreSQL and PS MySQL operators

For PS MySQL operator: added support of pt-mysql-summary
For PostgreSQL operator: added support for pg_gather
Options added:
  - kubeconfig - path to kubeconfig
  - forwardport - port to use when calling pt-*-summary tools ad pt_gather
Options changed:
  - resource - now default value is  (was pxc). New values added:
    psql - for PostgreSQL operator
    ps - for PS MySQL operator
    if default value () is used: only K8 information is collected
    Otherwise resource-specific summary is collected

* PT-2134_support_for_PostgreSQL_and_MySQL_operators

Fixed summary collection when connecting to PostgreSQL opertor and no resource specified

* PT-2134 - support for PostgreSQL and MySQL operators

Added test case for full supported set of values for option --resource,
added --kubeconfig and --forwardport options for test cases,
added support for environment variables KUBECONFIG_PXC, KUBECONFIG_PS, KUBECONFIG_PSMDB, KUBECONFIG_PSQL, FORWARDPORT for test cases

* PT-2134 - support for PostgreSQL and MySQL operators

updated docs/pt-k8s-debug-collector.rst,
replaced README.md in src/go/pt-k8s-debug-collector wih symbolic link to docs/pt-k8s-debug-collector.rst
to avoid having inconsistent documentation

* PT-2134 - support for PostgreSQL and MySQL operators

Removed curl STDERR from the command output

* PT-2134 - support for PostgreSQL and MySQL operators

typo in pt-k8s-debug-collector.rst

* PT-2134 - support for PostgreSQL and MySQL operators

Renamed --resource=psql to --resource=pg after review suggestion by Ege
and collecting feedback from potential users

Co-authored-by: EvgeniyPatlan <evgeniy.patlan@percona.com>
Co-authored-by: Carlos Salguero <carlos.salguero@percona.com>
2022-12-19 18:48:51 +03:00
Carlos Salguero
882d8787c7 Release 3.5.0 (#563)
* Changelog and version for Release 3.5.0

* PT-2076 add rhel9 dependency

* PT-2128 Updated version to 3.5.0

Co-authored-by: svetasmirnova <sveta.smirnova@percona.com>
Co-authored-by: EvgeniyPatlan <evgeniy.patlan@percona.com>
2022-12-01 14:31:46 +03:00
Sveta Smirnova
03ab27228f PT 2105 collect individual logs (#558)
* PT-2105 - Collect individual log files for PXC

Combined log files are hard to read by humans. Since pt-k8-debug-collector is the tool that
accesses data from the running pods, it can copy raw log files, necessary for troubleshooting PXC issues.
So, in addition to collecting logs.txt, this adds method getIndividualFiles for the Dumper that
reads individual files from PXC pods and stores them in the resulting archive.

Additionally, this commit fixes invalid timestamps in the resulting archive.

* PT-2105 - added support for non-default namespaces

* PT-2105 Let pt-k8-debug-collector to collect individual logs in PXC pods

Added test case for this new collection.

* Update go.mod

Co-authored-by: Viacheslav Sarzhan <slava.sarzhan@percona.com>

Co-authored-by: Viacheslav Sarzhan <slava.sarzhan@percona.com>
2022-11-17 19:32:19 +03:00