Commit Graph

307 Commits

Author SHA1 Message Date
Yoann La Cancellera
68e1b25001 preallocate lastcontexts map 2023-12-29 17:19:25 +03:00
Yoann La Cancellera
5a0574c828 Move: chan closing in "main" func 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
5030769bc8 Rename ctx to logCtx, remove any mention of ctx 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
8ccd547042 Fix: missed errors, minor formatting issues 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
671706c0da Imp: use strings.builder for conflicts 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
1e132b25fd Add: parallel on 2 unit tests 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
da3acc8e57 Update src/go/pt-galera-log-explainer/README.rst
Co-authored-by: Sveta Smirnova <svetasmirnova@users.noreply.github.com>
2023-12-29 17:19:24 +03:00
Yoann La Cancellera
fd9d09d9ae Fix: README with -vvv, --grep-args, regex-list 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
d1f7c72a9f PR-669 - Add pt-galera-log-explainer percona#669
- Commented out whois command in README.rst
- Copied README.rst into docs directory, so user manual will include
  this tool
2023-12-29 17:19:24 +03:00
Yoann La Cancellera
a5cf09a79e Remove: --grep-args
Can be used to break the tool, and I actually never had an use-case
2023-12-29 17:19:24 +03:00
Yoann La Cancellera
6d9136890a Fix: typos 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
8b84adb1e0 Remove old comments, dead code 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
4f5b089a34 Add: translate tests 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
84dcf1ef9c Commenting out whois+sed, will be re-added later 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
c910c35d3e Refactoring: migrate translations to singleton
It was using maps in each context, which would be merged between
contexts, then injected each time we needed a message to display.

It had a limitation on complicated operator setups: historical
information would be overriden by newer associations.
(e.g, that IP was for node0 yesterday, now it's node1, so associations
have been overwritten and incorrect)

It also introduced complexity, such as forcing to define closures too
many times, merging maps, it would be harder to debug, and every files
were starting from empty translation maps.
Moreover, iterating on maps is guaranteed to be random so it could create
hard-to-debug output variations on complex cases.

Now it is a singleton in translate package, still using maps but now it
associates an array of "units" storing the timestamp with each piece of information.
It is protected by rwmutex, because map are not threadsafe. (there's no
parallel processing for now)

No regressions, and it passes "operator_ambiguous_ips_list_all_no_color"
where the old system failed.
It nows also can be used as an easy to read source of information in
itself
2023-12-29 17:19:24 +03:00
Yoann La Cancellera
ea262eff27 Add: inconsistent vote regex corner-case 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
580ccb7428 Add: shortuuid check, new date layout found 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
3eaf60bd88 Add: same ip/name limitation on README.rst 2023-12-29 17:19:24 +03:00
Yoann La Cancellera
7513e1ded7 Add: operator member assocations regex
It existed for non-operator setup, but was not working for operators due
to k8s logs not interpreting newlines and tabs
This operator version re-uses existing regular regex handlers directly
2023-12-29 17:19:24 +03:00
Yoann La Cancellera
d13292ba67 Improve: main_test
It must test multiple times to remove doubts.
As the tool is reading files and relying on maps, their access order are
random. It can impact some translations
2023-12-29 17:19:24 +03:00
Yoann La Cancellera
c67047e781 Remove: propagation of ip propagation to older hash, Add: operator regression test
When adding "ownip", it was also propagating the new IP to the old hash.
But with operators, when IP where changing hash will also change, so
linking the new IP to old hash is anachronic. It is not wrong, but
depending on the order of map merges, the newest information could have
been overriden depending on the order of events.

That situation was producing X(2*number of conflicts) versions of output for operators, with
different md5sum, which could produce false-positive regression tests

So currently some information are not linked anymore, so some IPs are
not translated even though they could, but it is a limitation of using
maps as source of truth, as they are not versioned
2023-12-29 17:19:01 +03:00
Yoann La Cancellera
208708a58b Add: concurrent SSTs handling
It is a thing: 2 nodes joining at the same time, with 2 JOINERs and 2
DONORs cluster-wide
It can happen on operators with 2 garbd joining at the same time

Before, pt-galera-log-explainer was using SST metadata naively.
Basically if a node was DONOR and we found a "transfer completed"
message, we assumed the donor name we found is the correct one.
So for concurrent SSTs, donors were swapping names.

Now, it is handled by a map, indexed by a donor name. To know if a node
is actual donor or not, it now compare timestamps of events. It assumes
both "selected donor" and "shifting DONOR" messages should have happen
in less than 0.01 secs to avoid any conflict.

Regression tests coming in next commit with an operator logs having
concurrent SSTs. Another conflicts was sometimes breaking the test
depending on the order on which we read files, hence why it's not added
here yet
2023-11-07 18:08:24 +01:00
Yoann La Cancellera
117d683872 Change: simplify verbose mode 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
161c2af084 Fix: pointer dereference if votes was missing 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
12ea8033a9 Add: proper error handling for files 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
b0477bc37f Fix: don't cut node name if it's an ip
rare case, but it was not producing great results
regression tests will come for this
2023-11-07 18:08:24 +01:00
Yoann La Cancellera
94c89c2892 Add: requirements in README.rst 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
72c55a15ec Remove: extra space in --version out 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
3ad09d8418 Update README.rst 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
a7a4b4adca Add: --no-color regression test 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
6613880a9b Change: README to pt standard, regex-list --json 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
0d554e9515 Add: merge regression tests 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
75ce300d7c Fix: "not found" should not be an error
if it shows nothing, it can be understood as easily as a grep returning
nothing
adding an error log is too verbose
2023-11-07 18:08:24 +01:00
Yoann La Cancellera
fea9be26bd Add: regression tests 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
6d6f30372c Fix: error msg with uppercase, usage missed pt- 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
5638387713 Fix: --version formatting 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
bbba087a83 Fix: crash when timeline is empty
It was due to a silly regression when reformatting the main.go
The function iterating was doing too many things, and returning an error
when nothing was found, and a "continue" was done on the main
"timelineFromPaths" loop
It is now a simple foreach loop that does not return error so we have to
check if the localtimeline slice is empty
2023-11-07 18:08:24 +01:00
Yoann La Cancellera
7f8071ce3f Simplified the main loop 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
5bbb11df17 Add: merge by directory, Simplified: timeline merges 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
e03fe1801c Fix: forgot to inherit conflicts 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
b054429057 Improve: "not found" error handling 2023-11-07 18:08:24 +01:00
Yoann La Cancellera
b4cad31e77 Add pt-galera-log-explainer 2023-11-07 18:08:19 +01:00
Viktor Szépe
d01c0cea32 Fix last typos (#678) 2023-09-18 15:53:57 +03:00
Viktor Szépe
f2f7a7993d Add final newlines and remove trailing spaces (#672)
* Add final newlines and remove trailing spaces

* Replace TAB characters with spaces

* Fix indent
2023-09-15 17:31:40 +03:00
Sveta Smirnova
cd727bf9af PT-2248 - pt-k8s-debug-collector does not run pg_gather with K8SPG 2 (#654)
* PT-2248 - pt-k8s-debug-collector does not run pg_gather with K8SPG 2

- Added check for K8SPG 2, so can run pg_gather for it
- Added new allowed value for option --resource:
-- pgv2 for K8SPG 2
-- auto to auto-detect custom resource
- Option --resource has now default value "auto"
- Updated documentation
- Added test cases for new options

* PT-2248 - pt-k8s-debug-collector does not run pg_gather with K8SPG 2

- Implemented custom user and secrets handling (in case when no default
  user exists).
2023-09-13 15:11:04 +03:00
Viktor Szépe
2bd40d8c39 Remove trailing spaces (#665)
* Remove trailing spaces

* PR-665 -  Remove trailing spaces

- Updated not stable test t/pt-online-schema-change/preserve_triggers.t
- Updated utilities in bin directory

* PR-665 -  Remove trailing spaces

- Fixed typos

* PR-665 -  Remove trailing spaces

- Fixed typos

---------

Co-authored-by: Sveta Smirnova <sveta.smirnova@percona.com>
2023-09-06 01:15:12 +03:00
Viktor Szépe
3867d10ba5 Fix line ends (#664)
* Fix line ends of pt-mongodb-query-digest README

* Fix line ends of t/pt-diskstats/samples/small.txt
2023-08-23 16:27:18 +03:00
Viktor Szépe
ea67327ff7 Fix more typos (#658) 2023-08-22 19:34:24 +03:00
Sveta Smirnova
156372fb73 PT-2235 - pt-mongodb-index-check does not support option --version (#653)
* PT-2235 - pt-mongodb-index-check does not support option --version

Changed command version to flag version

* PT-2235 - pt-mongodb-index-check does not support option --version

Updated documentation
2023-08-01 17:56:17 +03:00
Sveta Smirnova
ab4bf1b1c6 PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard (#652)
* PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard for option --version

- Redirected UsageWriter to os.Stdout for kingpin, so --version output
  goes to STDOUT, not to STDERR
- Adjusted text, printed by the --version flag
- Added test cases to check how --version flag works
- Adjusted test cases, so they use TOOLNAME constant

* PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard for option --version

Run go mod tidy as requested by Artem Gavrilov

* PT-2236 - pt-secure-collect, pt-pg-summary do not follow PT standard for option --version

Renamed const TOOLNAME to toolname to follow Go coding standards
2023-07-31 16:59:30 +03:00