percona/percona-toolkit
1
0
Fork 0
mirror of https://github.com/percona/percona-toolkit.git synced 2025-09-12 06:00:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

PT-2340 - Support MySQL 8.4

Browse Source 
- Updated documentation
This commit is contained in:
Sveta Smirnova
2024-08-30 18:45:28 +03:00
parent 2ca815650e
commit 94bcfd2b69
39 changed files with 565 additions and 1616 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
bin/pt-archiver
View File

@@ -7519,7 +7519,7 @@ sub do_with_retries {
# Formats a row the same way SELECT INTO OUTFILE does by default. This is
# described in the LOAD DATA INFILE section of the MySQL manual,
# http://dev.mysql.com/doc/refman/5.0/en/load-data.html
# https://dev.mysql.com/doc/refman/8.4/en/load-data-local-security.html
sub escape {
my ($row, $fields_separated_by, $optionally_enclosed_by) = @_;
$fields_separated_by ||= "\t";
@@ -7627,7 +7627,7 @@ server. Before using this tool, please:
=head1 DESCRIPTION
pt-archiver is the tool I use to archive tables as described in
pt-archiver is the tool to archive tables as described in
L<https://web.archive.org/web/20071014031743/http://www.xaprb.com/blog/2006/05/02/how-to-write-efficient-archiving-and-purging-jobs-in-sql>. The goal is a low-impact, forward-only
job to nibble old data out of the table without impacting OLTP queries much.
You can insert the data into another table, which need not be on the same
@@ -7687,7 +7687,7 @@ included. The tool does not check for this!
=item Non-cluster options
Certain options may or may not work. For example, if a cluster node
is not also a slave, then L<"--check-replica-lag"> does not work. And since PXC
is not also a replica, then L<"--check-replica-lag"> does not work. And since PXC
tables are usually InnoDB, but InnoDB doesn't support C<INSERT DELAYED>, then
L<"--delayed-insert"> does not work. Other options may also not work, but
the tool does not check them, therefore you should test archiving on a test
@@ -7846,12 +7846,12 @@ for the C<L> DSN option.
type: string
Channel name used when connected to a server using replication channels.
Suppose you have two masters, master_a at port 12345, master_b at port 1236 and
a slave connected to both masters using channels chan_master_a and chan_master_b.
If you want to run pt-archiver to synchronize the slave against master_a, pt-archiver
won't be able to determine what's the correct master since SHOW SLAVE STATUS
will return 2 rows. In this case, you can use --channel=chan_master_a to specify
the channel name to use in the SHOW SLAVE STATUS command.
Suppose you have two sources, source_a at port 12345, source_b at port 1236 and
a replica connected to both sources using channels chan_source_a and chan_source_b.
If you want to run pt-archiver to synchronize the replica against source_a, pt-archiver
won't be able to determine what's the correct source since SHOW REPLICA STATUS
will return 2 rows. In this case, you can use --channel=chan_source_a to specify
the channel name to use in the SHOW REPLICA STATUS command.
=item --charset
@@ -7896,15 +7896,15 @@ To disable this check, specify --no-check-columns.
type: time; default: 1s
If L<"--check-replica-lag"> is given, this defines how long the tool pauses each
time it discovers that a slave is lagging.
time it discovers that a replica is lagging.
This check is performed every 100 rows.
=item --check-replica-lag
type: string; repeatable: yes
Pause archiving until the specified DSN's slave lag is less than L<"--max-lag">.
This option can be specified multiple times for checking more than one slave.
Pause archiving until the specified DSN's replica lag is less than L<"--max-lag">.
This option can be specified multiple times for checking more than one replica.
=item --columns
@@ -8107,13 +8107,13 @@ This option is available for PXC versions 5.6 or higher.
type: time; default: 1s
Pause archiving if the slave given by L<"--check-replica-lag"> lags.
Pause archiving if the replica given by L<"--check-replica-lag"> lags.
This option causes pt-archiver to look at the slave every time it's about
to fetch another row. If the slave's lag is greater than the option's value,
or if the slave isn't running (so its lag is NULL), pt-archiver sleeps
This option causes pt-archiver to look at the replica every time it's about
to fetch another row. If the replica's lag is greater than the option's value,
or if the replica isn't running (so its lag is NULL), pt-archiver sleeps
for L<"--check-interval"> seconds and then looks at the lag again. It repeats
until the slave is caught up, then proceeds to fetch and archive the row.
until the replica is caught up, then proceeds to fetch and archive the row.
This option may eliminate the need for L<"--sleep"> or L<"--sleep-coef">.
@@ -8292,8 +8292,8 @@ server. However, pt-archiver implements simple distributed transactions in
code, so commits and rollbacks should happen as desired across the two
connections.
At this time I have not written any code to handle errors with transactional
storage engines other than InnoDB. Request that feature if you need it.
At this time the tool does not handle errors with transactional storage engines
other than InnoDB. Request that feature if you need it.
=item --run-time
@@ -8329,21 +8329,21 @@ stop archiving and exit. The default is /tmp/pt-archiver-sentinel. You
might find this handy to stop cron jobs gracefully if necessary. See also
L<"--stop">.
=item --slave-user
=item --replica-user
type: string
Sets the user to be used to connect to the slaves.
Sets the user to be used to connect to the replicas.
This parameter allows you to have a different user with less privileges on the
slaves but that user must exist on all slaves.
replicas but that user must exist on all replicas.
=item --slave-password
=item --replica-password
type: string
Sets the password to be used to connect to the slaves.
It can be used with --slave-user and the password for the user must be the same
on all slaves.
Sets the password to be used to connect to the replicas.
It can be used with --replica-user and the password for the user must be the same
on all replicas.
=item --set-vars
@@ -8441,12 +8441,12 @@ discussion of how they interact with ascending indexes.
The 'a' and 'b' options allow you to control how statements flow through the
binary log. If you specify the 'b' option, binary logging will be disabled on
the specified connection. If you specify the 'a' option, the connection will
C<USE> the specified database, which you can use to prevent slaves from
C<USE> the specified database, which you can use to prevent replicas from
executing the binary log events with C<--replicate-ignore-db> options. These
two options can be used as different methods to achieve the same goal: archive
data off the master, but leave it on the slave. For example, you can run a
purge job on the master and prevent it from happening on the slave using your
method of choice.
data off the replication source, but leave it on the replica. For example, you
can run a purge job on the replication source and prevent it from happening on the
replica using your method of choice.
B<WARNING>: Using a default options file (F) DSN option that defines a
socket for L<"--source"> causes pt-archiver to connect to L<"--dest"> using
@@ -8521,9 +8521,9 @@ server, which for example is doing heavy OLTP work, you need to choose a good
balance between transaction size and commit overhead. Larger transactions
create the possibility of more lock contention and deadlocks, but smaller
transactions cause more frequent commit overhead, which can be significant. To
give an idea, on a small test set I worked with while writing pt-archiver, a
give an idea, on a small test set worked with while writing pt-archiver, a
value of 500 caused archiving to take about 2 seconds per 1000 rows on an
otherwise quiet MySQL instance on my desktop machine, archiving to disk and to
otherwise quiet MySQL instance on a desktop machine, archiving to disk and to
another table. Disabling transactions with a value of zero, which turns on
autocommit, dropped performance to 38 seconds per thousand rows.
@@ -8669,7 +8669,7 @@ lead to weird situations, like the server allowing LOCAL INFILE, but
the client throwing exceptions if it's used.
However, as long as the server allows LOAD DATA, clients can easily
re-enable it; See L<https://dev.mysql.com/doc/refman/5.0/en/load-data-local.html>
re-enable it; See L<https://dev.mysql.com/doc/refman/en/load-data-local-security.html>
and L<http://search.cpan.org/~capttofu/DBD-mysql/lib/DBD/mysql.pm>.
This option does exactly that.

8
bin/pt-config-diff
View File

@@ -5692,7 +5692,7 @@ There is no output when there are no differences. When there are differences,
pt-config-diff prints a report to STDOUT that looks similar to the following:
2 config differences
Variable my.master.cnf my.slave.cnf
Variable my.soource.cnf my.replica.cnf
========================= =============== ===============
datadir /tmp/12345/data /tmp/12346/data
port 12345 12346
@@ -5935,6 +5935,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

10
bin/pt-deadlock-logger
View File

@@ -5383,8 +5383,8 @@ PROCESSLIST.
=item txn_id
The InnoDB transaction ID, which InnoDB expresses as two unsigned integers. I
have multiplied them out to be one number.
The InnoDB transaction ID, which InnoDB expresses as two unsigned integers. The tool
multiplies them out to be one number.
=item txn_time
@@ -5711,6 +5711,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

21
bin/pt-duplicate-key-checker
View File

@@ -584,7 +584,7 @@ sub get_keys {
ddl => $key_ddl,
};
if ( ($engine || '') =~ m/InnoDB/i && !$clustered_key ) {
if ( ($engine || '') =~ m/(InnoDB)|(TokuDB)|(RocksDB)/i && !$clustered_key ) {
my $this_key = $keys->{$name};
if ( $this_key->{name} eq 'PRIMARY' ) {
$clustered_key = 'PRIMARY';
@@ -2501,7 +2501,10 @@ sub get_duplicate_keys {
if ( $clustered_key
&& $args{clustered}
&& $args{tbl_info}->{engine}
&& $args{tbl_info}->{engine} =~ m/InnoDB/i )
&& ( $args{tbl_info}->{engine} =~ m/InnoDB/i
|| $args{tbl_info}->{engine} =~ m/TokuDB/i
|| $args{tbl_info}->{engine} =~ m/RocksDB/i )
)
{
PTDEBUG && _d('Removing UNIQUE dupes of clustered key');
push @dupes,
@@ -5460,7 +5463,7 @@ PK columns appended to secondary key is duplicate.
Detects when a suffix of a secondary key is a leftmost prefix of the primary
key, and treats it as a duplicate key. Only detects this condition on storage
engines whose primary keys are clustered (currently InnoDB and solidDB).
engines whose primary keys are clustered (currently InnoDB, TokuDB and MyRocks).
Clustered storage engines append the primary key columns to the leaf nodes of
all secondary keys anyway, so you might consider it redundant to have them
@@ -5727,6 +5730,18 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

41
bin/pt-find
View File

@@ -4481,9 +4481,9 @@ print their names:
pt-find --ctime +1 --engine MyISAM
Find InnoDB tables and convert them to MyISAM:
Find MyISAM tables and convert them to InnoDB:
pt-find --engine InnoDB --exec "ALTER TABLE %D.%N ENGINE=MyISAM"
pt-find --engine MyISAM --exec "ALTER TABLE %D.%N ENGINE=InnoDB"
Find tables created by a process that no longer exists, following the
name_sid_pid naming convention, and remove them.
@@ -4728,8 +4728,8 @@ time. It can take some time for MySQL to process all the SHOW queries, and
pt-find can't do anything about that. These measurements are as of the
time they're taken.
If you need some test that's not in this list, file a bug report and I'll
enhance pt-find for you. It's really easy.
If you need some test that's not in this list, file a feature request at
L<https://jira.percona.com/projects/PT>.
=over
@@ -4807,14 +4807,11 @@ with. And perhaps most importantly, if the connection no longer exists, you
can assume the connection died without cleaning up its tables, and this table
is a candidate for removal.
This is how I manage scratch tables, and that's why I included this test in
pt-find.
The argument I use to L<"--connection-id"> is "\D_(\d+)$". That finds tables
The argument to use for L<"--connection-id"> is "\D_(\d+)$". That finds tables
with a series of numbers at the end, preceded by an underscore and some
non-number character (the latter criterion prevents me from examining tables
non-number character (the latter criterion prevents from examining tables
with a date at the end, which people tend to do: baron_scratch_2007_05_07 for
example). It's better to keep the scratch tables separate of course.
example). It's better to keep the scratch tables separate.
If you do this, make sure the user pt-find runs as has the PROCESS privilege!
Otherwise it will only see connections from the same user, and might think some
@@ -4940,15 +4937,15 @@ Table name contains the server ID. If you create temporary tables with the
naming convention explained in L<"--connection-id">, but also add the server ID of the
server on which the tables are created, then you can use this pattern match to
ensure tables are dropped only on the server they're created on. This prevents
a table from being accidentally dropped on a slave while it's in use (provided
a table from being accidentally dropped on a replica while it's in use (provided
that your server IDs are all unique, which they should be for replication to
work).
For example, on the master (server ID 22) you create a table called
scratch_table_22_12345. If you see this table on the slave (server ID 23), you
For example, on the replication source (server ID 22) you create a table called
scratch_table_22_12345. If you see this table on the replica (server ID 23), you
might think it can be dropped safely if there's no such connection 12345. But
if you also force the name to match the server ID with C<--server-id '\D_(\d+)_\d+$'>,
the table won't be dropped on the slave.
the table won't be dropped on the replica.
=item --tablesize
@@ -4998,8 +4995,8 @@ CREATE VIEW matches this pattern.
=head2 ACTIONS
The L<"--exec-plus"> action happens after everything else, but otherwise actions
happen in an indeterminate order. If you need determinism, file a bug report
and I'll add this feature.
happen in an indeterminate order. If you need determinism, file a feature request
at L<https://jira.percona.com/projects/PT>.
=over
@@ -5031,8 +5028,7 @@ You might use this, for example, to drop all the tables you found:
DROP TABLE %s
This is sort of like GNU find's "-exec command {} +" syntax. Only it's not
totally cryptic. And it doesn't require me to write a command-line parser.
This is sort of like GNU find's "-exec command {} +" syntax.
=item --print
@@ -5049,8 +5045,7 @@ Print format on the standard output, interpreting '\' escapes and '%'
directives. Escapes are backslashed characters, like \n and \t. Perl
interprets these, so you can use any escapes Perl knows about. Directives are
replaced by %s, and as of this writing, you can't add any special formatting
instructions, like field widths or alignment (though I'm musing over ways to do
that).
instructions, like field widths or alignment.
Here is a list of the directives. Note that most of them simply come from
columns of SHOW TABLE STATUS. If the column is NULL or doesn't exist, you get
@@ -5142,6 +5137,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

2
bin/pt-fingerprint
View File

@@ -2075,7 +2075,7 @@ Query fingerprinting accommodates a great many special cases, which have proven
necessary in the real world. For example, an IN list with 5 literals is really
equivalent to one with 4 literals, so lists of literals are collapsed to a
single one. If you want to understand more about how and why all of these cases
are handled, please review the test cases in the Subversion repository. If you
are handled, please review the test cases in the Github repository. If you
find something that is not fingerprinted properly, please submit a bug report
with a reproducible test case. Here is a list of transformations during
fingerprinting, which might not be exhaustive:

6
bin/pt-fk-error-logger
View File

@@ -4699,6 +4699,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

166
bin/pt-heartbeat
View File

@@ -6933,22 +6933,22 @@ pt-heartbeat - Monitor MySQL replication delay.
Usage: pt-heartbeat [OPTIONS] [DSN] --update|--monitor|--check|--stop
pt-heartbeat measures replication lag on a MySQL or PostgreSQL server. You can
use it to update a master or monitor a replica. If possible, MySQL connection
options are read from your .my.cnf file.
use it to update a replication source or monitor a replica. If possible, MySQL
connection options are read from your .my.cnf file.
Start daemonized process to update test.heartbeat table on master:
Start daemonized process to update test.heartbeat table on replicaiton source:
pt-heartbeat -D test --update -h master-server --daemonize
pt-heartbeat -D test --update -h source-server --daemonize
Monitor replication lag on slave:
Monitor replication lag on replica:
pt-heartbeat -D test --monitor -h slave-server
pt-heartbeat -D test --monitor -h replica-server
pt-heartbeat -D test --monitor -h slave-server --dbi-driver Pg
pt-heartbeat -D test --monitor -h replica-server --dbi-driver Pg
Check slave lag once and exit (using optional DSN to specify slave host):
Check replica lag once and exit (using optional DSN to specify replica host):
pt-heartbeat -D test --check h=slave-server
pt-heartbeat -D test --check h=replica-server
=head1 RISKS
@@ -6973,23 +6973,23 @@ server. Before using this tool, please:
pt-heartbeat is a two-part MySQL and PostgreSQL replication delay monitoring
system that measures delay by looking at actual replicated data. This
avoids reliance on the replication mechanism itself, which is unreliable. (For
example, C<SHOW SLAVE STATUS> on MySQL).
example, C<SHOW REPLICA STATUS> on MySQL).
The first part is an L<"--update"> instance of pt-heartbeat that connects to
a master and updates a timestamp ("heartbeat record") every L<"--interval">
seconds. Since the heartbeat table may contain records from multiple
masters (see L<"MULTI-SLAVE HIERARCHY">), the server's ID (@@server_id) is
used to identify records.
a replicaiton source and updates a timestamp ("heartbeat record") every
L<"--interval"> seconds. Since the heartbeat table may contain records from
multiple replication sources (see L<"MULTI-REPLICA HIERARCHY">), the server's ID
(@@server_id) is used to identify records.
The second part is a L<"--monitor"> or L<"--check"> instance of pt-heartbeat
that connects to a slave, examines the replicated heartbeat record from its
immediate master or the specified L<"--source-server-id">, and computes the
difference from the current system time. If replication between the slave and
the master is delayed or broken, the computed difference will be greater than
that connects to a replica, examines the replicated heartbeat record from its
immediate source or the specified L<"--source-server-id">, and computes the
difference from the current system time. If replication between the replica and
the source is delayed or broken, the computed difference will be greater than
zero and potentially increase if L<"--monitor"> is specified.
You must either manually create the heartbeat table on the master or use
L<"--create-table">. See L<"--create-table"> for the proper heartbeat
You must either manually create the heartbeat table on the replication source or
use L<"--create-table">. See L<"--create-table"> for the proper heartbeat
table structure. The C<MEMORY> storage engine is suggested, but not
required of course, for MySQL.
@@ -6998,16 +6998,16 @@ row is inserted if it doesn't exist. This feature can be disabled with the
L<"--[no]insert-heartbeat-row"> option in case the database user does not
have INSERT privileges.
pt-heartbeat depends only on the heartbeat record being replicated to the slave,
pt-heartbeat depends only on the heartbeat record being replicated to the replica,
so it works regardless of the replication mechanism (built-in replication, a
system such as Continuent Tungsten, etc). It works at any depth in the
replication hierarchy; for example, it will reliably report how far a slave lags
its master's master's master. And if replication is stopped, it will continue
to work and report (accurately!) that the slave is falling further and further
behind the master.
replication hierarchy; for example, it will reliably report how far a replica lags
its source's source's source. And if replication is stopped, it will continue
to work and report (accurately!) that the replica is falling further and further
behind the source.
pt-heartbeat has a maximum resolution of 0.01 second. The clocks on the
master and slave servers must be closely synchronized via NTP. By default,
source and replica servers must be closely synchronized via NTP. By default,
L<"--update"> checks happen on the edge of the second (e.g. 00:01) and
L<"--monitor"> checks happen halfway between seconds (e.g. 00:01.5).
As long as the servers' clocks are closely synchronized and replication
@@ -7020,42 +7020,42 @@ not retry if it can't get a connection when it first starts.
The L<"--dbi-driver"> option lets you use pt-heartbeat to monitor PostgreSQL
as well. It is reported to work well with Slony-1 replication.
=head1 MULTI-SLAVE HIERARCHY
=head1 MULTI-REPLICA HIERARCHY
If the replication hierarchy has multiple slaves which are masters of
other slaves, like "master -> slave1 -> slave2", L<"--update"> instances
can be ran on the slaves as well as the master. The default heartbeat
If the replication hierarchy has multiple replicas which are sources of
other replicas, like "source -> replica1 -> replica2", L<"--update"> instances
can be ran on the replicas as well as the source. The default heartbeat
table (see L<"--create-table">) is keyed on the C<server_id> column, so
each server will update the row where C<server_id=@@server_id>.
For L<"--monitor"> and L<"--check">, if L<"--source-server-id"> is not
specified, the tool tries to discover and use the slave's immediate master.
If this fails, or if you want monitor lag from another master, then you can
specified, the tool tries to discover and use the replica's immediate source.
If this fails, or if you want monitor lag from another source, then you can
specify the L<"--source-server-id"> to use.
For example, if the replication hierarchy is "master -> slave1 -> slave2"
For example, if the replication hierarchy is "source -> replica1 -> replica2"
with corresponding server IDs 1, 2 and 3, you can:
pt-heartbeat --daemonize -D test --update -h master
pt-heartbeat --daemonize -D test --update -h slave1
pt-heartbeat --daemonize -D test --update -h source
pt-heartbeat --daemonize -D test --update -h replica1
Then check (or monitor) the replication delay from master to slave2:
Then check (or monitor) the replication delay from source to replica2:
pt-heartbeat -D test --source-server-id 1 --check slave2
pt-heartbeat -D test --source-server-id 1 --check replica2
Or check the replication delay from slave1 to slave2:
Or check the replication delay from replica1 to replica2:
pt-heartbeat -D test --source-server-id 2 --check slave2
pt-heartbeat -D test --source-server-id 2 --check replica2
Stopping the L<"--update"> instance one slave1 will not affect the instance
on master.
Stopping the L<"--update"> instance one replica1 will not affect the instance
on the source.
=head1 MASTER AND SLAVE STATUS
=head1 BINARY LOG (MASTER) AND REPLICA STATUS
The default heartbeat table (see L<"--create-table">) has columns for saving
information from C<SHOW MASTER STATUS> and C<SHOW SLAVE STATUS>. These
columns are optional. If any are present, their corresponding information
will be saved.
information from C<SHOW BINARY LOG STATUS> (C<SHOW MASTER STATUS> before MySQL 8.4)
and C<SHOW REPLICA STATUS>. These columns are optional. If any are present,
their corresponding information will be saved.
=head1 Percona XtraDB Cluster
@@ -7098,9 +7098,9 @@ utf8 layer, and runs SET NAMES after connecting to MySQL.
=item --check
Check slave delay once and exit. If you also specify L<"--recurse">, the
tool will try to discover slave's of the given slave and check and print
their lag, too. The hostname or IP and port for each slave is printed
Check replica delay once and exit. If you also specify L<"--recurse">, the
tool will try to discover replica's of the given replica and check and print
their lag, too. The hostname or IP and port for each replica is printed
before its delay. L<"--recurse"> only works with MySQL.
=item --check-read-only
@@ -7125,10 +7125,10 @@ be created with the following MAGIC_create_heartbeat table definition:
CREATE TABLE heartbeat (
ts varchar(26) NOT NULL,
server_id int unsigned NOT NULL PRIMARY KEY,
file varchar(255) DEFAULT NULL, -- SHOW MASTER STATUS
position bigint unsigned DEFAULT NULL, -- SHOW MASTER STATUS
relay_source_log_file varchar(255) DEFAULT NULL, -- SHOW SLAVE STATUS
exec_source_log_pos bigint unsigned DEFAULT NULL -- SHOW SLAVE STATUS
file varchar(255) DEFAULT NULL, -- SHOW BINARY LOG STATUS
position bigint unsigned DEFAULT NULL, -- SHOW BINARY LOG STATUS
relay_source_log_file varchar(255) DEFAULT NULL, -- SHOW REPLICA STATUS
exec_source_log_pos bigint unsigned DEFAULT NULL -- SHOW REPLICA STATUS
);
The heartbeat table requires at least one row. If you manually create the
@@ -7141,7 +7141,7 @@ or if using L<"--utc">:
INSERT INTO heartbeat (ts, server_id) VALUES (UTC_TIMESTAMP(), N);
where C<N> is the server's ID; do not use @@server_id because it will replicate
and slaves will insert their own server ID instead of the master's server ID.
and replicas will insert their own server ID instead of the source's server ID.
This is done automatically by L<"--create-table">.
@@ -7152,8 +7152,8 @@ A legacy version of the heartbeat table is still supported:
ts datetime NOT NULL
);
Legacy tables do not support L<"--update"> instances on each slave
of a multi-slave hierarchy like "master -> slave1 -> slave2".
Legacy tables do not support L<"--update"> instances on each replica
of a multi-replica hierarchy like "source -> replica1 -> replica2".
To manually insert the one required row into a legacy table:
INSERT INTO heartbeat (id, ts) VALUES (1, NOW());
@@ -7164,7 +7164,7 @@ or if using L<"--utc">:
The tool automatically detects if the heartbeat table is legacy.
See also L<"MULTI-SLAVE HIERARCHY">.
See also L<"MULTI-REPLICA HIERARCHY">.
=item --create-table-engine
@@ -7278,15 +7278,15 @@ Print all output to this file when daemonized.
type: string
Calculate delay from this master server ID for L<"--monitor"> or L<"--check">.
If not given, pt-heartbeat attempts to connect to the server's master and
Calculate delay from this source server ID for L<"--monitor"> or L<"--check">.
If not given, pt-heartbeat attempts to connect to the server's source and
determine its server id.
=item --monitor
Monitor slave delay continuously.
Monitor replica delay continuously.
Specifies that pt-heartbeat should check the slave's delay every second and
Specifies that pt-heartbeat should check the replica's delay every second and
report to STDOUT (or if L<"--file"> is given, to the file instead). The output
is the current delay followed by moving averages over the timeframe given in
L<"--frames">. For example,
@@ -7340,11 +7340,11 @@ server is found to be read-only. If unspecified, L<"--interval"> is used.
type: int
Check slaves recursively to this depth in L<"--check"> mode.
Check replicas recursively to this depth in L<"--check"> mode.
Try to discover slave servers recursively, to the specified depth. After
Try to discover replica servers recursively, to the specified depth. After
discovering servers, run the check on each one of them and print the hostname
(if possible), followed by the slave delay.
(if possible), followed by the replica delay.
This currently works only with MySQL. See L<"--recursion-method">.
@@ -7352,21 +7352,21 @@ This currently works only with MySQL. See L<"--recursion-method">.
type: array; default: processlist,hosts
Preferred recursion method used to find slaves.
Preferred recursion method used to find replicas.
Possible methods are:
METHOD USES
=========== ==================
processlist SHOW PROCESSLIST
hosts SHOW SLAVE HOSTS
none Do not find slaves
hosts SHOW REPLICA HOSTS
none Do not find replicas
The processlist method is preferred because SHOW SLAVE HOSTS is not reliable.
The processlist method is preferred because SHOW REPLICA HOSTS is not reliable.
However, the hosts method is required if the server uses a non-standard
port (not 3306). Usually pt-heartbeat does the right thing and finds
the slaves, but you may give a preferred method and it will be used first.
If it doesn't find any slaves, the other methods will be tried.
the replicas, but you may give a preferred method and it will be used first.
If it doesn't find any replicas, the other methods will be tried.
=item --replace
@@ -7389,21 +7389,21 @@ type: string; default: /tmp/pt-heartbeat-sentinel
Exit if this file exists.
=item --slave-user
=item --replica-user
type: string
Sets the user to be used to connect to the slaves.
Sets the user to be used to connect to the replicas.
This parameter allows you to have a different user with less privileges on the
slaves but that user must exist on all slaves.
replicas but that user must exist on all replicas.
=item --slave-password
=item --replica-password
type: string
Sets the password to be used to connect to the slaves.
It can be used with --slave-user and the password for the user must be the same
on all slaves.
Sets the password to be used to connect to the replicas.
It can be used with --replica-user and the password for the user must be the same
on all replicas.
=item --set-vars
@@ -7430,12 +7430,12 @@ type: float; default: 0.5
How long to delay checks.
The default is to delay checks one half second. Since the update happens as
soon as possible after the beginning of the second on the master, this allows
one half second of replication delay before reporting that the slave lags the
master by one second. If your clocks are not completely accurate or there is
some other reason you'd like to delay the slave more or less, you can tweak this
value. Try setting the C<PTDEBUG> environment variable to see the effect this
has.
soon as possible after the beginning of the second on the replication source,
this allows one half second of replication delay before reporting that the
replica lags the source by one second. If your clocks are not completely
accurate or there is some other reason you'd like to delay the replica more
or less, you can tweak this value. Try setting the C<PTDEBUG> environment
variable to see the effect this has.
=item --socket
@@ -7481,7 +7481,7 @@ See L<"--create-table">.
=item --update
Update a master's heartbeat.
Update a replication source's heartbeat.
=item --user

15
bin/pt-index-usage
View File

@@ -7170,12 +7170,7 @@ cases where it fails, submit a bug report and a reproducible test case.
Queries that cannot be EXPLAINed will cause all subsequent queries with the
same fingerprint to be blacklisted. This is to reduce the work they cause, and
prevent them from continuing to print error messages. However, at least in
this stage of the tool's development, it is my opinion that it's not a good
idea to preemptively silence these, or prevent them from being EXPLAINed at
all. I am looking for lots of feedback on how to improve things like the
query parsing. So please submit your test cases based on the errors the tool
prints!
prevent them from continuing to print error messages.
=head1 OUTPUT
@@ -7376,8 +7371,6 @@ should be careful if you use this feature on a production server. It might
increase load, or cause trouble if you don't want the server to be written to,
or so on.
This is a new feature. It may change in future releases.
After a run, you can query the usage tables to answer various questions about
index usage. The tables have the following CREATE TABLE definitions:
@@ -7661,6 +7654,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

37
bin/pt-kill
View File

@@ -7876,9 +7876,9 @@ pt-kill - Kill MySQL queries that match certain criteria.
Usage: pt-kill [OPTIONS] [DSN]
pt-kill kills MySQL connections. pt-kill connects to MySQL and gets queries
from SHOW PROCESSLIST if no FILE is given. Else, it reads queries from one
or more FILE which contains the output of SHOW PROCESSLIST. If FILE is -,
pt-kill reads from STDIN.
from SHOW PROCESSLIST unless option L<"--test-matching"> is given. Else, it
reads queries from one or more files which contains the output of SHOW PROCESSLIST.
If file is -, pt-kill reads from STDIN.
Kill queries running longer than 60s:
@@ -7988,8 +7988,8 @@ For example, using:
will kill all queries having busy-time > 114 C<OR> where the command is C<Query> or C<Execute>
If you want to kill only the queries where C<busy-time > 114> C<AND> the command is Query or
Execute, you need to use L<"--kill-busy-commands>:
If you want to kill only the queries where C<busy-time> is greater than 114 and the
command is Query or Execute, you need to use L<"--kill-busy-commands">:
--busy-time 114 --kill-busy-commands 'Query|Execute'
@@ -8132,7 +8132,7 @@ L<"--victims"> for an explanation of that term) requires that queries are
grouped by the C<arg> attribute. This creates classes of identical queries
(stripped of comments). So queries C<"SELECT c FROM t WHERE id=1"> and
C<"SELECT c FROM t WHERE id=1"> are grouped into the same class, but
query c<"SELECT c FROM t WHERE id=3"> is not identical to the first two
query C<"SELECT c FROM t WHERE id=3"> is not identical to the first two
queries so it is grouped into another class. Then when L<"--victims">
C<all-but-oldest> is specified, all but the oldest query in each class is
killed for each class of queries that matches the match criteria.
@@ -8320,21 +8320,21 @@ The presence of the file specified by L<"--sentinel"> will cause all
running instances of pt-kill to exit. You might find this handy to stop cron
jobs gracefully if necessary. See also L<"--stop">.
=item --slave-user
=item --replica-user
type: string
Sets the user to be used to connect to the slaves.
Sets the user to be used to connect to the replicas.
This parameter allows you to have a different user with less privileges on the
slaves but that user must exist on all slaves.
replicas but that user must exist on all replicas.
=item --slave-password
=item --replica-password
type: string
Sets the password to be used to connect to the slaves.
It can be used with --slave-user and the password for the user must be the same
on all slaves.
Sets the password to be used to connect to the replicas.
It can be used with --replica-user and the password for the user must be the same
on all replicas.
=item --set-vars
@@ -8585,7 +8585,7 @@ Common Command values are:
Reset stmt
Table Dump
See L<http://dev.mysql.com/doc/refman/5.1/en/thread-commands.html> for a full
See L<https://dev.mysql.com/doc/refman/en/general-thread-states.html> for a full
list and description of Command values.
=item --match-db
@@ -8650,7 +8650,8 @@ Allow matching and killing replication threads.
By default, matches do not apply to replication threads; i.e. replication
threads are completely ignored. Specifying this option allows matches to
match (and potentially kill) replication threads on masters and slaves.
match (and potentially kill) replication threads on replication sources and
replicas.
=item --test-matching
@@ -8865,6 +8866,12 @@ User for login if not current user.
Table to log actions in, if passed through --log-dsn.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

36
bin/pt-online-schema-change
View File

@@ -12764,7 +12764,7 @@ You must specify C<--alter "DROP FOREIGN KEY _fk_foo">.
=item *
The tool does not use C<LOCK IN SHARE MODE> with MySQL 5.0 because it can
cause a slave error which breaks replication:
cause a replica error which breaks replication:
Query caused different errors on master and slave. Error on master:
'Deadlock found when trying to get lock; try restarting transaction' (1213),
@@ -12896,12 +12896,10 @@ See L<"--history"> and L<"--resume">.
type: string
Channel name used when connected to a server using replication channels.
Suppose you have two masters, master_a at port 12345, master_b at port 1236 and
a slave connected to both masters using channels chan_master_a and chan_master_b.
If you want to run pt-table-sync to synchronize the slave against master_a, pt-table-sync
won't be able to determine what's the correct master since SHOW SLAVE STATUS
will return 2 rows. In this case, you can use --channel=chan_master_a to specify
the channel name to use in the SHOW SLAVE STATUS command.
Suppose you have two replication sources, source_a at port 12345, source_b at port 1236 and
a replica connected to both sources using channels chan_source_a and chan_source_b.
You can use --channel=chan_source_a to specify the channel name to use in the
SHOW REPLICA STATUS command that pt-online-schema-change uses to calculate replication lag.
=item --charset
@@ -12994,8 +12992,8 @@ server options that filter replication, such as binlog_ignore_db and
replicate_do_db. If it finds any such filters, it aborts with an error.
If the replicas are configured with any filtering options, you should be careful
not to modify any databases or tables that exist on the master and not the
replicas, because it could cause replication to fail. For more information on
not to modify any databases or tables that exist on the replication source and not
the replicas, because it could cause replication to fail. For more information on
replication rules, see L<http://dev.mysql.com/doc/en/replication-rules.html>.
=item --check-replica-lag
@@ -13297,7 +13295,7 @@ type: time; default: 1s
Pause the data copy until all replicas' lag is less than this value. After each
data-copy query (each chunk), the tool looks at the replication lag of
all replicas to which it connects, using Seconds_Behind_Master. If any replica
all replicas to which it connects, using Seconds_Behind_Source. If any replica
is lagging more than the value of this option, then the tool will sleep
for L<"--check-interval"> seconds, then check all replicas again. If you
specify L<"--check-replica-lag">, then the tool only examines that server for
@@ -13445,9 +13443,7 @@ type: string
Perl module file that defines a C<pt_online_schema_change_plugin> class.
A plugin allows you to write a Perl module that can hook into many parts
of pt-online-schema-change. This requires a good knowledge of Perl and
Percona Toolkit conventions, which are beyond this scope of this
documentation. Please contact Percona if you have questions or need help.
of pt-online-schema-change.
See L<"PLUGIN"> for more information.
@@ -13495,11 +13491,11 @@ Preferred recursion method for discovering replicas. Possible methods are:
METHOD USES
=========== ==================
processlist SHOW PROCESSLIST
hosts SHOW SLAVE HOSTS
hosts SHOW REPLICAS (SHOW SLAVE HOSTS before MySQL 8.1)
dsn=DSN DSNs from a table
none Do not find slaves
none Do not find replicas
The processlist method is the default, because SHOW SLAVE HOSTS is not
The processlist method is the default, because SHOW REPLICAS is not
reliable. However, the hosts method can work better if the server uses a
non-standard port (not 3306). The tool usually does the right thing and
finds all replicas, but you may give a preferred method and it will be used
@@ -13547,7 +13543,7 @@ Resume altering table from the last completed chunk. If the tool stops before it
copying rows, this option makes copying resume from the last chunk copied.
The option accepts ID of the failed job. This ID is printed when pt-online-schema-change
is running with option L<i"--history"> and stored in the L<"--history-table">.
is running with option L<"--history"> and stored in the L<"--history-table">.
Warning! To use this option previous, failed, run of pt-online-schema-change should use options
L<"--history">, L<"--nodrop-new-table">, and L<"--nodrop-triggers">. Otherwise,
@@ -13934,8 +13930,8 @@ Here's a plugin file template for all hooks:
1;
Notice that C<get_slave_lag> must return a function reference;
ideally one that returns actual slave lag, not simply zero like in the example.
Notice that C<get_replica_lag> must return a function reference;
ideally one that returns actual replica lag, not simply zero like in the example.
Here's an example that actually does something:
@@ -14091,7 +14087,7 @@ This tool works only on MySQL 5.0.2 and newer versions, because earlier versions
do not support triggers. Also a number of permissions should be set on MySQL
to make pt-online-schema-change operate as expected. PROCESS, SUPER, REPLICATION SLAVE
global privileges, as well as SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER,
and TRIGGER table privileges should be granted on server. Slave needs only
and TRIGGER table privileges should be granted on server. Replica needs only
REPLICATION SLAVE and REPLICATION CLIENT privileges.
=head1 BUGS

5
bin/pt-pmp
View File

@@ -856,6 +856,11 @@ tool to diagnose an unresponsive server, there is really no reason not to do
this. In addition to freezing the server, there is also some risk of the server
crashing or performing badly after GDB detaches from it.
Dumpers C<eu> and C<pteu> use eu-stack (L<https://sourceware.org/elfutils/>) instead of GDB.
eu-stack collects stacks with minimal impact and does not cause same issues as GDB
does. Dumpers C<eu> and C<pteu> are recommended to use instead of GDB and one of them
will be default dumper in future versions.
=head1 OPTIONS
=over

28
bin/pt-query-digest
View File

@@ -16064,7 +16064,7 @@ MAGIC_create_history_table
Rows_examined_stddev FLOAT,
Rows_examined_median FLOAT,
-- Percona extended slowlog attributes
-- http://www.percona.com/docs/wiki/patches:slow_extended
-- https://docs.percona.com/percona-server/8.0/slow-extended.html
Rows_affected_sum FLOAT,
Rows_affected_min FLOAT,
Rows_affected_max FLOAT,
@@ -16296,9 +16296,6 @@ The entire C<report> output can be disabled by specifying C<--no-report>
(see L<"--[no]report">), and its sections can be disabled or rearranged
by specifying L<"--report-format">.
C<json> output was introduced in 2.2.1 and is still in development,
so the data structure may change in future versions.
=item --password
short form: -p; type: string
@@ -16572,21 +16569,21 @@ queries. A complete example:
pt-query-digest --sample 2 --no-report --output slowlog slow.log
=item --slave-user
=item --replica-user
type: string
Sets the user to be used to connect to the slaves.
Sets the user to be used to connect to the replicas.
This parameter allows you to have a different user with less privileges on the
slaves but that user must exist on all slaves.
replicas but that user must exist on all replicas.
=item --slave-password
=item --replica-password
type: string
Sets the password to be used to connect to the slaves.
It can be used with --slave-user and the password for the user must be the same
on all slaves.
Sets the password to be used to connect to the replicas.
It can be used with --replica-user and the password for the user must be the same
on all replicas.
=item --set-vars
@@ -16755,7 +16752,8 @@ traffic without dropping packets on busy servers. Dropped packets cause
pt-query-digest to miss the response to a request, then see the response to a
later request and assign the wrong execution time to the query. You can change
the filter to something like the following to help capture a subset of the
queries. (See L<http://www.mysqlperformanceblog.com/?p=6092> for details.)
queries. (See L<https://www.percona.com/blog/how-to-use-tcpdump-on-very-busy-hosts/>
for details.)
tcpdump -i any -s 65535 -x -n -q -tttt \
'port 3306 and tcp[1] & 7 == 2 and tcp[3] & 7 == 2'
@@ -16961,6 +16959,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

46
bin/pt-replica-find
View File

@@ -4347,17 +4347,17 @@ if ( !caller ) { exit main(@ARGV); }
=head1 NAME
pt-replica-find - Find and print replication hierarchy tree of MySQL slaves.
pt-replica-find - Find and print replication hierarchy tree of MySQL replicas.
=head1 SYNOPSIS
Usage: pt-replica-find [OPTIONS] [DSN]
pt-replica-find finds and prints a hierarchy tree of MySQL slaves.
pt-replica-find finds and prints a hierarchy tree of MySQL replicas.
Examples:
pt-replica-find --host master-host
pt-replica-find --host source-host
=head1 RISKS
@@ -4379,16 +4379,16 @@ server. Before using this tool, please:
=head1 DESCRIPTION
pt-replica-find connects to a MySQL replication master and finds its slaves.
pt-replica-find connects to a MySQL replication source and finds its replicas.
Currently the only thing it can do is print a tree-like view of the replication
hierarchy.
The master host can be specified using one of two methods. The first method is
The source host can be specified using one of two methods. The first method is
to use the standard connection-related command line options:
L<"--defaults-file">, L<"--password">, L<"--host">, L<"--port">, L<"--socket">
or L<"--user">.
The second method to specify the master host is a DSN. A DSN is a special
The second method to specify the source host is a DSN. A DSN is a special
syntax that can be either just a hostname (like C<server.domain.com> or
C<1.2.3.4>), or a C<key=value,key=value> string. Keys are a single letter:
@@ -4496,34 +4496,34 @@ See L<"--recursion-method">.
type: array; default: processlist,hosts
Preferred recursion method used to find slaves.
Preferred recursion method used to find replicas.
Possible methods are:
METHOD USES
=========== ==================
processlist SHOW PROCESSLIST
hosts SHOW SLAVE HOSTS
none Do not find slaves
hosts SHOW REPLICAS (SHOW SLAVE HOSTS before MySQL 8.1)
none Do not find replicas
The processlist method is preferred because SHOW SLAVE HOSTS is not reliable.
The processlist method is preferred because SHOW REPLICAS is not reliable.
However, the hosts method is required if the server uses a non-standard
port (not 3306). Usually pt-replica-find does the right thing and finds
the slaves, but you may give a preferred method and it will be used first.
If it doesn't find any slaves, the other methods will be tried.
the replicas, but you may give a preferred method and it will be used first.
If it doesn't find any replicas, the other methods will be tried.
=item --report-format
type: string; default: summary
Set what information about the slaves is printed. The report format can be
Set what information about the replicas is printed. The report format can be
one of the following:
=over
=item * hostname
Print just the hostname name of the slaves. It looks like:
Print just the hostname name of the replicas. It looks like:
127.0.0.1:12345
+- 127.0.0.1:12346
@@ -4531,8 +4531,8 @@ Print just the hostname name of the slaves. It looks like:
=item * summary
Print a summary of each slave's settings. This report shows more information
about each slave, like:
Print a summary of each replica's settings. This report shows more information
about each replica, like:
127.0.0.1:12345
Version 5.1.34-log
@@ -4567,21 +4567,21 @@ Example:
Might delay runtime a few seconds.
=item --slave-user
=item --replica-user
type: string
Sets the user to be used to connect to the slaves.
Sets the user to be used to connect to the replicas.
This parameter allows you to have a different user with less privileges on the
slaves but that user must exist on all slaves.
replicas but that user must exist on all replicas.
=item --slave-password
=item --replica-password
type: string
Sets the password to be used to connect to the slaves.
It can be used with --slave-user and the password for the user must be the same
on all slaves.
Sets the password to be used to connect to the replicas.
It can be used with --replica-user and the password for the user must be the same
on all replicas.
=item --set-vars

138
bin/pt-replica-restart
View File

@@ -5240,9 +5240,9 @@ sub main {
$o->set('verbose', 0) if $o->get('quiet');
if ( !$o->get('help') ) {
if ( $o->get('until-master') ) {
if ( $o->get('until-master') !~ m/^[.\w-]+,\d+$/ ) {
$o->save_error("Invalid --until-master argument, must be file,pos");
if ( $o->get('until-source') ) {
if ( $o->get('until-source') !~ m/^[.\w-]+,\d+$/ ) {
$o->save_error("Invalid --until-source argument, must be file,pos");
}
}
if ( $o->get('until-relay') ) {
@@ -5442,8 +5442,8 @@ sub watch_server {
}
my $start_sql = $vp >= '4.0.5' ? "START ${replica_name}" : 'SLAVE START';
if ( $o->get('until-master') ) {
my ( $file, $pos ) = split(',', $o->get('until-master'));
if ( $o->get('until-source') ) {
my ( $file, $pos ) = split(',', $o->get('until-source'));
$start_sql .= " UNTIL ${source_name}_LOG_FILE = '$file', ${source_name}_LOG_POS = $pos";
}
elsif ( $o->get('until-relay') ) {
@@ -5614,8 +5614,8 @@ sub watch_server {
$stat->{last_error} ||= '';
$stat->{last_errno} ||= 0;
if ( $o->get('until-master') && pos_ge($stat, $source_name, $source_name) ) {
die "Replica has advanced past " . $o->get('until-master')
if ( $o->get('until-source') && pos_ge($stat, $source_name, $source_name) ) {
die "Replica has advanced past " . $o->get('until-source')
. " on master.\n";
}
elsif ( $o->get('until-relay') && pos_ge($stat, 'relay', $source_name) ) {
@@ -5788,7 +5788,7 @@ pt-replica-restart - Watch and restart MySQL replication after errors.
Usage: pt-replica-restart [OPTIONS] [DSN]
pt-replica-restart watches one or more MySQL replication slaves for
pt-replica-restart watches one or more MySQL replication replicas for
errors, and tries to restart replication if it stops.
=head1 RISKS
@@ -5811,18 +5811,18 @@ server. Before using this tool, please:
=head1 DESCRIPTION
pt-replica-restart watches one or more MySQL replication slaves and tries to skip
statements that cause errors. It polls slaves intelligently with an
pt-replica-restart watches one or more MySQL replication replicas and tries to skip
statements that cause errors. It polls replicas intelligently with an
exponentially varying sleep time. You can specify errors to skip and run the
slaves until a certain binlog position.
replicas until a certain binlog position.
Although this tool can help a slave advance past errors, you should not
rely on it to "fix" replication. If slave errors occur frequently or
Although this tool can help a replica advance past errors, you should not
rely on it to "fix" replication. If replica errors occur frequently or
unexpectedly, you should identify and fix the root cause.
=head1 OUTPUT
pt-replica-restart prints a line every time it sees the slave has an error.
pt-replica-restart prints a line every time it sees the replica has an error.
By default this line is: a timestamp, connection information, relay_log_file,
relay_log_pos, and last_errno.
You can add more information using the L<"--verbose"> option.
@@ -5830,7 +5830,7 @@ You can suppress all output using the L<"--quiet"> option.
=head1 SLEEP
pt-replica-restart sleeps intelligently between polling the slave. The current
pt-replica-restart sleeps intelligently between polling the replica. The current
sleep time varies.
=over
@@ -5870,15 +5870,15 @@ introduced in MySQL 5.6.5. It's important to keep in mind that:
=item *
pt-replica-restart will not skip transactions when multiple replication threads
are being used (slave_parallel_workers > 0). pt-replica-restart does not know
what the GTID event is of the failed transaction of a specific slave thread.
are being used (replica_parallel_workers > 0). pt-replica-restart does not know
what the GTID event is of the failed transaction of a specific replica thread.
=item *
The default behavior is to skip the next transaction from the slave's master.
The default behavior is to skip the next transaction from the replica's source.
Writes can originate on different servers, each with their own UUID.
See L<"--master-uuid">.
See L<"--source-uuid">.
=back
@@ -5892,7 +5892,7 @@ servers to monitor.
=head1 COMPATIBILITY
pt-replica-restart should work on many versions of MySQL. Lettercase of many
output columns from SHOW SLAVE STATUS has changed over time, so it treats them
output columns from SHOW REPLICA STATUS has changed over time, so it treats them
all as lowercase.
=head1 OPTIONS
@@ -5904,8 +5904,8 @@ L<"SYNOPSIS"> and usage information for details.
=item --always
Start slaves even when there is no error. With this option enabled,
pt-replica-restart will not let you stop the slave manually if you want to!
Start replicas even when there is no error. With this option enabled,
pt-replica-restart will not let you stop the replica manually if you want to!
=item --ask-pass
@@ -5925,14 +5925,14 @@ connecting to MySQL.
default: yes
Check the last relay log file and position before checking for slave errors.
Check the last relay log file and position before checking for replica errors.
By default pt-replica-restart will not doing anything (it will just sleep)
if neither the relay log file nor the relay log position have changed since
the last check. This prevents infinite loops (i.e. restarting the same
error in the same relay log file at the same relay log position).
For certain slave errors, however, this check needs to be disabled by
For certain replica errors, however, this check needs to be disabled by
specifying C<--no-check-relay-log>. Do not do this unless you know what
you are doing!
@@ -5977,7 +5977,7 @@ Only restart this comma-separated list of errors. Makes pt-replica-restart only
try to restart if the error number is in this comma-separated list of errors.
If it sees an error not in the list, it will exit.
The error number is in the C<last_errno> column of C<SHOW SLAVE STATUS>.
The error number is in the C<last_errno> column of C<SHOW REPLICA STATUS>.
=item --error-text
@@ -5985,10 +5985,10 @@ type: string
Only restart errors that match this pattern. A Perl regular expression against
which the error text, if any, is matched. If the error text exists and matches,
pt-replica-restart will try to restart the slave. If it exists but doesn't match,
pt-replica-restart will try to restart the replica. If it exists but doesn't match,
pt-replica-restart will exit.
The error text is in the C<last_error> column of C<SHOW SLAVE STATUS>.
The error text is in the C<last_error> column of C<SHOW REPLICA STATUS>.
=item --help
@@ -6012,7 +6012,7 @@ type: float; default: 64
Maximum sleep seconds.
The maximum time pt-replica-restart will sleep before polling the slave again.
The maximum time pt-replica-restart will sleep before polling the replica again.
This is also the time that pt-replica-restart will wait for all other running
instances to quit if both L<"--stop"> and L<"--monitor"> are specified.
@@ -6022,12 +6022,12 @@ See L<"SLEEP">.
type: float; default: 0.015625
The minimum time pt-replica-restart will sleep before polling the slave again.
The minimum time pt-replica-restart will sleep before polling the replica again.
See L<"SLEEP">.
=item --monitor
Whether to monitor the slave (default). Unless you specify --monitor
Whether to monitor the replica (default). Unless you specify --monitor
explicitly, L<"--stop"> will disable it.
=item --password
@@ -6063,19 +6063,19 @@ Suppresses normal output (disables L<"--verbose">).
type: int; default: 0
Watch slaves of the specified server, up to the specified number of servers deep
in the hierarchy. The default depth of 0 means "just watch the slave
Watch replicas of the specified server, up to the specified number of servers deep
in the hierarchy. The default depth of 0 means "just watch the replica
specified."
pt-replica-restart examines C<SHOW PROCESSLIST> and tries to determine which
connections are from slaves, then connect to them. See L<"--recursion-method">.
connections are from replicas, then connect to them. See L<"--recursion-method">.
Recursion works by finding all slaves when the program starts, then watching
them. If there is more than one slave, C<pt-replica-restart> uses C<fork()> to
Recursion works by finding all replicas when the program starts, then watching
them. If there is more than one replica, C<pt-replica-restart> uses C<fork()> to
monitor them.
This also works if you have configured your slaves to show up in C<SHOW SLAVE
HOSTS>. The minimal configuration for this is the C<report_host> parameter, but
This also works if you have configured your replicas to show up in C<SHOW REPLICAS>.
The minimal configuration for this is the C<report_host> parameter, but
there are other "report" parameters as well for the port, username, and
password.
@@ -6083,21 +6083,21 @@ password.
type: array; default: processlist,hosts
Preferred recursion method used to find slaves.
Preferred recursion method used to find replicas.
Possible methods are:
METHOD USES
=========== ==================
processlist SHOW PROCESSLIST
hosts SHOW SLAVE HOSTS
none Do not find slaves
hosts SHOW REPLICAS (SHOW SLAVE HOSTS before MySQL 8.1)
none Do not find replicas
The processlist method is preferred because SHOW SLAVE HOSTS is not reliable.
The processlist method is preferred because SHOW REPLCIAS is not reliable.
However, the hosts method is required if the server uses a non-standard
port (not 3306). Usually pt-replica-restart does the right thing and finds
the slaves, but you may give a preferred method and it will be used first.
If it doesn't find any slaves, the other methods will be tried.
the replicas, but you may give a preferred method and it will be used first.
If it doesn't find any replicas, the other methods will be tried.
=item --run-time
@@ -6113,21 +6113,21 @@ type: string; default: /tmp/pt-replica-restart-sentinel
Exit if this file exists.
=item --slave-user
=item --replica-user
type: string
Sets the user to be used to connect to the slaves.
Sets the user to be used to connect to the replicas.
This parameter allows you to have a different user with less privileges on the
slaves but that user must exist on all slaves.
replicas but that user must exist on all replicas.
=item --slave-password
=item --replica-password
type: string
Sets the password to be used to connect to the slaves.
It can be used with --slave-user and the password for the user must be the same
on all slaves.
Sets the password to be used to connect to the replicas.
It can be used with --replica-user and the password for the user must be the same
on all replicas.
=item --set-vars
@@ -6151,7 +6151,7 @@ The tool prints a warning and continues if a variable cannot be set.
type: int; default: 1
Number of statements to skip when restarting the slave.
Number of statements to skip when restarting the replica.
=item --source-uuid
@@ -6161,15 +6161,15 @@ When using GTID, an empty transaction should be created in order to skip it.
If writes are coming from different nodes in the replication tree above, it is
not possible to know which event from which UUID to skip.
By default, transactions from the slave's master (C<'Master_UUID'> from
C<SHOW SLAVE STATUS>) are skipped.
By default, transactions from the replica's source (C<'Source_UUID'> from
C<SHOW REPLICA STATUS>) are skipped.
For example, with
master1 -> slave1 -> slave2
source1 -> replica1 -> replica2
When skipping events on slave2 that were written to master1, you must specify
the UUID of master1, else the tool will use the UUID of slave1 by default.
When skipping events on replica2 that were written to source1, you must specify
the UUID of source1, else the tool will use the UUID of replica1 by default.
See L<"GLOBAL TRANSACTION IDS">.
@@ -6177,7 +6177,7 @@ See L<"GLOBAL TRANSACTION IDS">.
type: int; default: 1
Initial sleep seconds between checking the slave.
Initial sleep seconds between checking the replica.
See L<"SLEEP">.
@@ -6212,27 +6212,27 @@ same C<cron> job).
See also L<"--sentinel">.
=item --until-master
=item --until-source
type: string
Run until this master log file and position. Start the slave, and retry if it
fails, until it reaches the given replication coordinates. The coordinates are
the logfile and position on the master, given by relay_master_log_file,
exec_master_log_pos. The argument must be in the format "file,pos". Separate
Run until this source binary log file and position. Start the replica, and retry
if it fails, until it reaches the given replication coordinates. The coordinates
are the logfile and position on the source, given by relay_source_log_file,
exec_source_log_pos. The argument must be in the format "file,pos". Separate
the filename and position with a single comma and no space.
This will also cause an UNTIL clause to be given to START SLAVE.
This will also cause an UNTIL clause to be given to START REPLICA.
After reaching this point, the slave should be stopped and pt-replica-restart
After reaching this point, the replica should be stopped and pt-replica-restart
will exit.
=item --until-relay
type: string
Run until this relay log file and position. Like L<"--until-master">, but in
the slave's relay logs instead. The coordinates are given by relay_log_file,
Run until this relay log file and position. Like L<"--until-source">, but in
the replica's relay logs instead. The coordinates are given by relay_log_file,
relay_log_pos.
=item --user
@@ -6346,6 +6346,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

6
bin/pt-show-grants
View File

@@ -2621,6 +2621,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

2
bin/pt-slave-delay
View File

@@ -4858,7 +4858,7 @@ pt-slave-delay - Make a MySQL slave server lag behind its master.
This tool is deprecated and will be removed in future releases. It does not support MySQL 8.1 or newer.
Use built-in Delayed Replication instead: https://dev.mysql.com/doc/refman/8.4/en/replication-delayed.html
Use built-in Delayed Replication instead: https://dev.mysql.com/doc/refman/en/replication-delayed.html
=head1 SYNOPSIS

185
bin/pt-table-checksum
View File

@@ -12835,12 +12835,12 @@ pt-table-checksum - Verify MySQL replication integrity.
Usage: pt-table-checksum [OPTIONS] [DSN]
pt-table-checksum performs an online replication consistency check by executing
checksum queries on the master, which produces different results on replicas
that are inconsistent with the master. The optional DSN specifies the master
host. The tool's L<"EXIT STATUS"> is non-zero if any differences are found,
checksum queries on the replication source, which produces different results on
replicas that are inconsistent with the source. The optional DSN specifies the
source host. The tool's L<"EXIT STATUS"> is non-zero if any differences are found,
or if any warnings or errors occur.
The following command will connect to the replication master on localhost,
The following command will connect to the replication source on localhost,
checksum every table, and report the results on every detected replica:
pt-table-checksum
@@ -12932,8 +12932,8 @@ than the desired number of rows. You can configure the sensitivity of this
safeguard with the L<"--chunk-size-limit"> option. If a table will be
checksummed in a single chunk because it has a small number of rows, then
pt-table-checksum additionally verifies that the table isn't oversized on
replicas. This avoids the following scenario: a table is empty on the master
but is very large on a replica, and is checksummed in a single large query,
replicas. This avoids the following scenario: a table is empty on the replication
source but is very large on a replica, and is checksummed in a single large query,
which causes a very long delay in replication.
There are several other safeguards. For example, pt-table-checksum sets its
@@ -12965,27 +12965,27 @@ exit. You can resume it as usual afterwards.
After pt-table-checksum finishes checksumming all of the chunks in a table, it
pauses and waits for all detected replicas to finish executing the checksum
queries. Once that is finished, it checks all of the replicas to see if they
have the same data as the master, and then prints a line of output with the
have the same data as the source, and then prints a line of output with the
results. You can see a sample of its output later in this documentation.
The tool prints progress indicators during time-consuming operations. It prints
a progress indicator as each table is checksummed. The progress is computed by
the estimated number of rows in the table. It will also print a progress report
when it pauses to wait for replication to catch up, and when it is waiting to
check replicas for differences from the master. You can make the output less
check replicas for differences from the source. You can make the output less
verbose with the L<"--quiet"> option.
If you wish, you can query the checksum tables manually to get a report of which
tables and chunks have differences from the master. The following query will
tables and chunks have differences from the source. The following query will
report every database and table with differences, along with a summary of the
number of chunks and rows possibly affected:
SELECT db, tbl, SUM(this_cnt) AS total_rows, COUNT(*) AS chunks
FROM percona.checksums
WHERE (
master_cnt <> this_cnt
OR master_crc <> this_crc
OR ISNULL(master_crc) <> ISNULL(this_crc))
source_cnt <> this_cnt
OR source_crc <> this_crc
OR ISNULL(source_crc) <> ISNULL(this_crc))
GROUP BY db, tbl;
The table referenced in that query is the checksum table, where the checksums
@@ -13010,7 +13010,7 @@ collisions. On the other hand, C<CRC32> algorithm is faster and less CPU-intensi
than C<MD5> and C<SHA1>.
Related reading material:
Percona Toolkit UDFs: L<https://www.percona.com/doc/percona-server/LATEST/management/udf_percona_toolkit.html>
Percona Toolkit UDFs: L<https://docs.percona.com/percona-server/8.0/udf-percona-toolkit.html>
How to avoid hash collisions when using MySQL’s CRC32 function: L<https://www.percona.com/blog/2014/10/13/how-to-avoid-hash-collisions-when-using-mysqls-crc32-function/>
=head1 LIMITATIONS
@@ -13020,9 +13020,9 @@ How to avoid hash collisions when using MySQL’s CRC32 function: L<https://www.
=item Replicas using row-based replication
pt-table-checksum requires statement-based replication, and it sets
C<binlog_format=STATEMENT> on the master, but due to a MySQL limitation
C<binlog_format=STATEMENT> on the source, but due to a MySQL limitation
replicas do not honor this change. Therefore, checksums will not replicate
past any replicas using row-based replication that are masters for
past any replicas using row-based replication that are sources for
further replicas.
The tool automatically checks the C<binlog_format> on all servers.
@@ -13032,10 +13032,10 @@ See L<"--[no]check-binlog-format"> .
=item Schema and table differences
The tool presumes that schemas and tables are identical on the master and
The tool presumes that schemas and tables are identical on the source and
all replicas. Replication will break if, for example, a replica does not
have a schema that exists on the master (and that schema is checksummed),
or if the structure of a table on a replica is different than on the master.
have a schema that exists on the source (and that schema is checksummed),
or if the structure of a table on a replica is different than on the source.
=back
@@ -13045,7 +13045,7 @@ pt-table-checksum works with Percona XtraDB Cluster (PXC) 5.5.28-23.7 and newer.
The number of possible Percona XtraDB Cluster setups is large given that
it can be used with regular replication as well. Therefore, only the setups
listed below are supported and known to work. Other setups, like cluster
to cluster, are not support and probably don't work.
to cluster, are not supported and probably don't work.
Except where noted, all of the following supported setups require that you
use the C<dsn> method for L<"--recursion-method"> to specify cluster nodes.
@@ -13068,9 +13068,9 @@ supported. This applies to all supported setups.
=item Single cluster with replicas
Cluster nodes can also be regular masters and replicate to regular replicas.
However, the tool can only detect diffs on a replica if ran on the replica's
"master node". For example, if the cluster setup is,
Cluster nodes can also be regular replication sources and replicate to regular
replicas. However, the tool can only detect diffs on a replica if ran on the
replica's "source node". For example, if the cluster setup is,
node1 <-> node2 <-> node3
| |
@@ -13086,23 +13086,23 @@ cannot be checked (e.g. replica2 when running on node3).
Replicas in this setup are still subject to L<"--[no]check-binlog-format">.
=item Master to single cluster
=item Replication source to single cluster
It is possible for a regular master to replicate to a cluster, as if the
cluster were one logical slave, like:
It is possible for a regular source to replicate to a cluster, as if the
cluster were one logical replica, like:
master -> node1 <-> node2 <-> node3
source -> node1 <-> node2 <-> node3
The tool supports this setup but only if ran on the master and if all nodes
The tool supports this setup but only if ran on the source and if all nodes
in the cluster are consistent with the "direct replica" (node1 in this example)
of the master. For example, if all nodes have value "foo" for row 1 but
the master has value "bar" for the same row, this diff will be detected.
of the source. For example, if all nodes have value "foo" for row 1 but
the source has value "bar" for the same row, this diff will be detected.
Or if only node1 has this diff, it will also be detected. But if only node2
or node3 has this diff, it will not be detected. Therefore, this setup is
used to check that the master and the cluster as a whole are consistent.
used to check that the source and the cluster as a whole are consistent.
In this setup, the tool can automatically detect the "direct replica" (node1)
when ran on the master, so you do not have to use the C<dsn> method for
when ran on the source, so you do not have to use the C<dsn> method for
L<"--recursion-method"> because node1 will represent the entire cluster,
which is why all other nodes must be consistent with it.
@@ -13148,7 +13148,7 @@ progress.
=item DIFFS
The number of chunks that differ from the master on one or more replicas.
The number of chunks that differ from the source on one or more replicas.
If C<--no-replicate-check> is specified, this column will always have zeros.
If L<"--replicate-check-only"> is specified, then only tables with differences
are printed.
@@ -13211,21 +13211,21 @@ The columns are as follows:
=item TABLE
The database and table that differs from the master.
The database and table that differs from the source.
=item CHUNK
The chunk number of the table that differs from the master.
The chunk number of the table that differs from the source.
=item CNT_DIFF
The number of chunk rows on the replica minus the number of chunk rows
on the master.
on the source.
=item CRC_DIFF
1 if the CRC of the chunk on the replica is different than the CRC of the
chunk on the master, else 0.
chunk on the source, else 0.
=item CHUNK_INDEX
@@ -13260,7 +13260,7 @@ with these flags:
ERROR 1 A non-fatal error occurred
ALREADY_RUNNING 2 --pid file exists and the PID is running
CAUGHT_SIGNAL 4 Caught SIGHUP, SIGINT, SIGPIPE, or SIGTERM
NO_SLAVES_FOUND 8 No replicas or cluster nodes were found
NO_REPLICAS_FOUND 8 No replicas or cluster nodes were found
TABLE_DIFF 16 At least one diff was found
SKIP_CHUNK 32 At least one chunk was skipped
SKIP_TABLE 64 At least one table was skipped
@@ -13292,12 +13292,13 @@ Prompt for a password when connecting to MySQL.
type: string
Channel name used when connected to a server using replication channels.
Suppose you have two masters, master_a at port 12345, master_b at port 1236 and
a slave connected to both masters using channels chan_master_a and chan_master_b.
If you want to run pt-table-sync to synchronize the slave against master_a, pt-table-sync
won't be able to determine what's the correct master since SHOW SLAVE STATUS
will return 2 rows. In this case, you can use --channel=chan_master_a to specify
the channel name to use in the SHOW SLAVE STATUS command.
Suppose you have two replication sources, source_a at port 12345, source_b
at port 1236 and a replica connected to both sources using channels chan_source_a
and chan_source_b. If you want to run pt-table-sync to synchronize the replica
against source_a, pt-table-checksum won't be able to determine what's the correct
source since SHOW REPLICA STATUS will return 2 rows. In this case, you can use
--channel=chan_source_a to specify the channel name to use in the SHOW REPLICA STATUS
command.
=item --[no]check-binlog-format
@@ -13365,8 +13366,8 @@ binlog_ignore_db and replicate_do_db. If it finds any such filters,
it aborts with an error.
If the replicas are configured with any filtering options, you should be careful
not to checksum any databases or tables that exist on the master and not the
replicas. Changes to such tables might normally be skipped on the replicas
not to checksum any databases or tables that exist on the replication source and
not the replicas. Changes to such tables might normally be skipped on the replicas
because of the filtering options, but the checksum queries modify the contents
of the table that stores the checksums, not the tables whose data you are
checksumming. Therefore, these queries will be executed on the replica, and if
@@ -13386,11 +13387,11 @@ See also L<"REPLICA CHECKS">.
type: string; group: Throttle
Pause checksumming until this replica's lag is less than L<"--max-lag">. The
value is a DSN that inherits properties from the master host and the connection
value is a DSN that inherits properties from the source host and the connection
options (L<"--port">, L<"--user">, etc.). By default, pt-table-checksum
monitors lag on all connected replicas, but this option limits lag monitoring
to the specified replica. This is useful if certain replicas are intentionally
lagged (with L<pt-slave-delay> for example), in which case you can specify
lagged (with L<https://dev.mysql.com/doc/refman/en/replication-delayed.html> for example), in which case you can specify
a normal replica to monitor.
See also L<"REPLICA CHECKS">.
@@ -13399,11 +13400,11 @@ See also L<"REPLICA CHECKS">.
default: yes; group: Safety
Checks that tables on slaves exist and have all the checksum L<"--columns">.
Tables missing on slaves or not having all the checksum L<"--columns"> can
Checks that tables on replicas exist and have all the checksum L<"--columns">.
Tables missing on replicas or not having all the checksum L<"--columns"> can
cause the tool to break replication when it tries to check for differences.
Only disable this check if you are aware of the risks and are sure that all
tables on all slaves exist and are identical to the master.
tables on all replicas exist and are identical to the source.
=item --chunk-index
@@ -13570,6 +13571,8 @@ the table from which the tool resumes won't be emptied.
To empty the entire replicate table, you must manually execute C<TRUNCATE TABLE>
before running the tool.
See also -L<"--truncate-replicate-table">.
=item --engines
short form: -e; type: hash; group: Filter
@@ -13682,10 +13685,10 @@ type: time; default: 1s; group: Throttle
Pause checksumming until all replicas' lag is less than this value. After each
checksum query (each chunk), pt-table-checksum looks at the replication lag of
all replicas to which it connects, using Seconds_Behind_Master. If any replica
all replicas to which it connects, using Seconds_Behind_Source. If any replica
is lagging more than the value of this option, then pt-table-checksum will sleep
for L<"--check-interval"> seconds, then check all replicas again. If you
specify L<"--check-slave-lag">, then the tool only examines that server for
specify L<"--check-replica-lag">, then the tool only examines that server for
lag, not all servers.
The tool waits forever for replicas to stop lagging. If any replica is
@@ -13754,9 +13757,7 @@ type: string
Perl module file that defines a C<pt_table_checksum_plugin> class.
A plugin allows you to write a Perl module that can hook into many parts
of pt-table-checksum. This requires a good knowledge of Perl and
Percona Toolkit conventions, which are beyond this scope of this
documentation. Please contact Percona if you have questions or need help.
of pt-table-checksum.
See L<"PLUGIN"> for more information.
@@ -13805,7 +13806,7 @@ Preferred recursion method for discovering replicas. pt-table-checksum
performs several L<"REPLICA CHECKS"> before and while running.
Although replicas are not required to run pt-table-checksum, the tool
cannot detect diffs on slaves that it cannot discover. Therefore,
cannot detect diffs on replicas that it cannot discover. Therefore,
a warning is printed and the L<"EXIT STATUS"> is non-zero if no replicas
are found and the method is not C<none>. If this happens, try a different
recursion method, or use the C<dsn> method to specify the replicas to check.
@@ -13813,14 +13814,14 @@ recursion method, or use the C<dsn> method to specify the replicas to check.
Possible methods are:
METHOD USES
=========== =============================================
=========== =================================================
processlist SHOW PROCESSLIST
hosts SHOW SLAVE HOSTS
hosts SHOW REPLICAS (SHOW SLAVE HOSTS before MySQL 8.1)
cluster SHOW STATUS LIKE 'wsrep\_incoming\_addresses'
dsn=DSN DSNs from a table
none Do not find slaves
none Do not find replicas
The C<processlist> method is the default, because C<SHOW SLAVE HOSTS> is not
The C<processlist> method is the default, because C<SHOW REPLICAS> is not
reliable. However, if the server uses a non-standard port (not 3306), then
the C<hosts> method becomes the default because it works better in this case.
@@ -13837,7 +13838,7 @@ but this functionality is experimental.
The C<dsn> method is special: rather than automatically discovering replicas,
this method specifies a table with replica DSNs. The tool will only connect
to these replicas. This method works best when replicas do not use the same
MySQL username or password as the master, or when you want to prevent the tool
MySQL username or password as the source, or when you want to prevent the tool
from connecting to certain replicas. The C<dsn> method is specified like:
C<--recursion-method dsn=h=host,D=percona,t=dsns>. The specified DSN must
have D and t parts, or just a database-qualified t part, which specify the
@@ -13854,10 +13855,10 @@ DSNs are ordered by C<id>, but C<id> and C<parent_id> are otherwise ignored.
The C<dsn> column contains a replica DSN like it would be given on the command
line, for example: C<"h=replica_host,u=repl_user,p=repl_pass">.
The C<none> method makes the tool ignore all slaves and cluster nodes. This
The C<none> method makes the tool ignore all replicas and cluster nodes. This
method is not recommended because it effectively disables the
L<"REPLICA CHECKS"> and no differences can be found. It is useful, however, if
you only need to write checksums on the master or a single cluster node. The
you only need to write checksums on the source or a single cluster node. The
safer alternative is C<--no-replicate-check>: the tool finds replicas and
cluster nodes, performs the L<"REPLICA CHECKS">, but does not check for
differences. See L<"--[no]replicate-check">.
@@ -13897,8 +13898,8 @@ are checksumming InnoDB tables, and you use MyISAM for this table, a deadlock
will break replication, because the mixture of transactional and
non-transactional tables in the checksum statements will cause it to be written
to the binlog even though it had an error. It will then replay without a
deadlock on the replicas, and break replication with "different error on master
and slave." This is not a problem with pt-table-checksum; it's a problem with
deadlock on the replicas, and break replication with "different error on source
and replica." This is not a problem with pt-table-checksum; it's a problem with
MySQL replication, and you can read more about it in the MySQL manual.
The replicate table is never checksummed (the tool automatically adds this
@@ -13910,7 +13911,7 @@ default: yes
Check replicas for data differences after finishing each table. The tool finds
differences by executing a simple SELECT statement on all detected replicas.
The query compares the replica's checksum results to the master's checksum
The query compares the replica's checksum results to the source's checksum
results. It reports differences in the DIFFS column of the output.
=item --replicate-check-only
@@ -13929,7 +13930,7 @@ type: int; default: 1
Retry checksum comparison this many times when a difference is encountered.
Only when a difference persists after this number of checks is it considered valid.
Using this option with a value of 2 or more alleviates spurious differences that
arise when using the --resume option.
arise when using the L<"--resume"> option.
=item --replicate-database
@@ -13972,31 +13973,31 @@ time it is ran.
type: string; default: #
The separator character used for CONCAT_WS(). This character is used to join
The separator character used for C<CONCAT_WS()>. This character is used to join
the values of columns when checksumming.
=item --skip-check-replica-lag
type: DSN; repeatable: yes
DSN to skip when checking slave lag. It can be used multiple times.
Example: --skip-check-slave-lag h=127.1,P=12345 --skip-check-slave-lag h=127.1,P=12346
DSN to skip when checking replica lag. It can be used multiple times.
Example: C<--skip-check-replica-lag h=127.1,P=12345 --skip-check-replica-lag h=127.1,P=12346>
=item --replica-user
type: string
Sets the user to be used to connect to the slaves.
Sets the user to be used to connect to the replicas.
This parameter allows you to have a different user with less privileges on the
slaves but that user must exist on all slaves.
replicas but that user must exist on all replicas.
=item --replica-password
type: string
Sets the password to be used to connect to the slaves.
It can be used with --replica-user and the password for the user must be the same
on all slaves.
Sets the password to be used to connect to the replicas.
It can be used with L<"--replica-user"> and the password for the user must be the same
on all replicas.
=item --set-vars
@@ -14027,12 +14028,12 @@ Socket file to use for connection.
type: float; default: 1.0
When a master table is marked to be checksummed in only one chunk but a slave
When a source table is marked to be checksummed in only one chunk but a replica
table exceeds the maximum accepted size for this, the table is skipped.
Since number of rows are often rough estimates, many times tables are skipped
needlessly for very small differences.
This option provides a max row excess tolerance to prevent this.
For example a value of 1.2 will tolerate slave tables with up to 20% excess rows.
For example a value of 1.2 will tolerate replica tables with up to 20% excess rows.
=item --tables
@@ -14049,18 +14050,18 @@ Checksum only tables whose names match this Perl regex.
=item --trim
Add TRIM() to VARCHAR columns (helps when comparing 4.1 to >= 5.0).
Add C<TRIM()> to C<VARCHAR> columns (helps when comparing 4.1 to >= 5.0).
This is useful when you don't care about the trailing space differences between
MySQL versions that vary in their handling of trailing spaces. MySQL 5.0 and
later all retain trailing spaces in VARCHAR, while previous versions would
later all retain trailing spaces in C<VARCHAR>, while previous versions would
remove them. These differences will cause false checksum differences.
=item --truncate-replicate-table
Truncate the replicate table before starting the checksum.
This parameter differs from L<--empty-replicate-table> which only deletes the rows
This parameter differs from L<"--[no]empty-replicate-table"> which only deletes the rows
for the table being checksummed when starting the checksum for that table, while
L<--truncate-replicate-table> will truncate the replicate table at the beginning of the
L<"--truncate-replicate-table"> will truncate the replicate table at the beginning of the
process and thus, all previous checksum information will be losti, even if the process
stops due to an error.
@@ -14121,8 +14122,8 @@ keyword. You might need to quote the value. Here is an example:
=head1 REPLICA CHECKS
By default, pt-table-checksum attempts to find and connect to all replicas
connected to the master host. This automated process is called
"slave recursion" and is controlled by the L<"--recursion-method"> and
connected to the source host. This automated process is called
"replica recursion" and is controlled by the L<"--recursion-method"> and
L<"--recurse"> options. The tool performs these checks on all replicas:
=over
@@ -14138,17 +14139,17 @@ disabled by specifying C<--no-check-replication-filters>.
pt-table-checksum checks that the L<"--replicate"> table exists on all
replicas, else checksumming can break replication when updates to the table
on the master replicate to a replica that doesn't have the table. This
on the source replicate to a replica that doesn't have the table. This
check cannot be disabled, and the tool waits forever until the table
exists on all replicas, printing L<"--progress"> messages while it waits.
=item 3. Single chunk size
If a table can be checksummed in a single chunk on the master,
If a table can be checksummed in a single chunk on the source,
pt-table-checksum will check that the table size on all replicas is less than
L<"--chunk-size"> * L<"--chunk-size-limit">. This prevents a rare problem
where the table on the master is empty or small, but on a replica it is much
larger. In this case, the single chunk checksum on the master would overload
where the table on the source is empty or small, but on a replica it is much
larger. In this case, the single chunk checksum on the source would overload
the replica.
Another rare problem occurs when the table size on a replica is close to
@@ -14165,10 +14166,10 @@ This check cannot be disabled.
=item 4. Lag
After each chunk, pt-table-checksum checks the lag on all replicas, or only
the replica specified by L<"--check-slave-lag">. This helps the tool
the replica specified by L<"--check-replica-lag">. This helps the tool
not to overload the replicas with checksum data. There is no way to
disable this check, but you can specify a single replica to check with
L<"--check-slave-lag">, and if that replica is the fastest, it will help
L<"--check-replica-lag">, and if that replica is the fastest, it will help
prevent the tool from waiting too long for replica lag to abate.
=item 5. Checksum chunks
@@ -14206,9 +14207,9 @@ to a hook, search for the hook's name in the tool's source code, like:
# --plugin hook
if ( $plugin && $plugin->can('init') ) {
$plugin->init(
slaves => $slaves,
slave_lag_cxns => $slave_lag_cxns,
repl_table => $repl_table,
replicas => $replicas,
replica_lag_cxns => $replica_lag_cxns,
repl_table => $repl_table,
);
}

20
bin/pt-table-sync
View File

@@ -12296,21 +12296,11 @@ Selects the entire table in one big stream and compares all columns. Selects
all columns. Much less efficient than the other algorithms, but works when
there is no suitable index for them to use.
=item Future Plans
Possibilities for future algorithms are TempTable (what I originally called
bottom-up in earlier versions of this tool), DrillDown (what I originally
called top-down), and GroupByPrefix (similar to how SqlYOG Job Agent works).
Each algorithm has strengths and weaknesses. If you'd like to implement your
favorite technique for finding differences between two sources of data on
possibly different servers, I'm willing to help. The algorithms adhere to a
simple interface that makes it pretty easy to write your own.
=back
=head1 BIDIRECTIONAL SYNCING
Bidirectional syncing is a new, experimental feature. To make it work
Bidirectional syncing is an experimental feature. To make it work
reliably there are a number of strict limitations:
* only works when syncing one server to other independent servers
@@ -12376,7 +12366,7 @@ These options are:
* --conflict-comparison
* --conflict-value
* --conflict-threshold
* --conflict-error"> (optional)
* --conflict-error (optional)
Use L<"--print"> to test this option before L<"--execute">. The printed
SQL statements will have comments saying on which host the statement
@@ -12922,7 +12912,7 @@ Possible methods are:
METHOD USES
=========== ==================
processlist SHOW PROCESSLIST
hosts SHOW REPLICAS
hosts SHOW REPLICAS (SHOW SLAVE HOSTS before MySQL 8.1)
none Do not find replicas
The processlist method is preferred because SHOW REPLICAS is not reliable.
@@ -12951,7 +12941,7 @@ L<pt-table-checksum>. That is, it contains records of which tables (and ranges
of values) differ between the source and replica.
For each table and range of values that shows differences between the source and
replica, C<pt-table-checksum> will sync that table, with the appropriate C<WHERE>
replica, pt-table-sync will sync that table, with the appropriate C<WHERE>
clause, to its source.
This automatically sets L<"--wait"> to 60 and causes changes to be made on the
@@ -13141,7 +13131,7 @@ the tables. The value is the number of seconds to wait before timing out (see
also L<"--timeout-ok">). Sets L<"--lock"> to 1 and L<"--[no]transaction"> to 0
by default. If you see an error such as the following,
MASTER_POS_WAIT returned -1
SOURCE_POS_WAIT returned -1
It means the timeout was exceeded and you need to increase it.

6
bin/pt-table-usage
View File

@@ -8420,6 +8420,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

14
bin/pt-upgrade
View File

@@ -10835,7 +10835,7 @@ while pt-upgrade is running, the report could contain more false-positives
than legitimate differences. B<pt-upgrade assumes that both MySQL servers
are static, unchanging (except for any changes made by the tool if ran
with C<--no-read-only>).> A read-only workload shouldn't affect the tool,
except maybe query times, so read-only slaves could be used.
except maybe query times, so read-only replicas could be used.
=head2 COMPARED TO
@@ -10874,8 +10874,8 @@ pt-upgrade runs with C<autocommit=1> by default.
=head2 THROTTLING
pt-upgrade has no throttling options because the tool should only be ran
on dedicated testing or development servers. B<Do not run pt-upgrade
on production servers!> Consequently, the tool is CPU, memory, disk, and
on dedicated testing or development servers. B<Do not run> pt-upgrade
B<on production servers!> Consequently, the tool is CPU, memory, disk, and
network intensive. It executes queries as fast as possible.
=head1 QUERY DIFFERENCES
@@ -11418,7 +11418,7 @@ lead to weird situations, like the server allowing LOCAL INFILE, but
the client throwing exceptions if it's used.
However, as long as the server allows LOAD DATA, clients can easily
re-enable it; See L<https://dev.mysql.com/doc/refman/5.0/en/load-data-local.html>
re-enable it; See L<https://dev.mysql.com/doc/refman/en/load-data-local-security.html>
and L<http://search.cpan.org/~capttofu/DBD-mysql/lib/DBD/mysql.pm>.
This option does exactly that.
@@ -11451,6 +11451,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

45
bin/pt-variable-advisor
View File

@@ -3863,7 +3863,12 @@ sub get_rules {
return 1 if ($mysql_version == '3' && $mysql_version < '3.23' )
|| ($mysql_version == '4' && $mysql_version < '4.1.20')
|| ($mysql_version == '5.0' && $mysql_version < '5.0.37')
|| ($mysql_version == '5.1' && $mysql_version < '5.1.30');
|| ($mysql_version == '5.1' && $mysql_version < '5.1.30')
|| ($mysql_version == '5.5' && $mysql_version < '5.5.8')
|| ($mysql_version == '5.6' && $mysql_version < '5.6.10')
|| ($mysql_version == '5.7' && $mysql_version < '5.7.9')
|| ($mysql_version == '8.0' && $mysql_version < '8.0.11')
;
return 0;
},
},
@@ -3873,7 +3878,7 @@ sub get_rules {
my ( %args ) = @_;
my $mysql_version = $args{mysql_version};
return 0 unless $mysql_version;
return $mysql_version < '5.1' ? 1 : 0; # 5.1.x
return $mysql_version < '8.0' ? 1 : 0; # 8.x
},
},
};
@@ -5530,10 +5535,6 @@ pt-variable-advisor examines C<SHOW VARIABLES> for bad values and settings
according to the L<"RULES"> described below. It reports on variables that
match the rules, so you can find bad settings in your MySQL server.
At the time of this release, pt-variable-advisor only examines
C<SHOW VARIABLES>, but other input sources are planned like C<SHOW STATUS>
and C<SHOW SLAVE STATUS>.
=head1 RULES
These are the rules that pt-variable-advisor will apply to SHOW VARIABLES.
@@ -5875,28 +5876,28 @@ necessary to change it.
severity: warn
Setting relay_log_space_limit can cause replicas to stop fetching binary logs
from their master immediately. This could increase the risk that your data will
be lost if the master crashes. If the replicas have encountered a limit on relay
log space, then it is possible that the latest transactions exist only on the
master and no replica has retrieved them.
from their source immediately. This could increase the risk that your data will
be lost if the replication source crashes. If the replicas have encountered a limit
on relay log space, then it is possible that the latest transactions exist only
on the source and no replica has retrieved them.
=item slave_net_timeout
severity: warn
This variable is set too high. This is too long to wait before
noticing that the connection to the master has failed and retrying. This
should probably be set to 60 seconds or less. It is also a good idea to
noticing that the connection to the replication source has failed and retrying.
This should probably be set to 60 seconds or less. It is also a good idea to
use pt-heartbeat to ensure that the connection does not appear to time out
when the master is simply idle.
when the source is simply idle.
=item replica_net_timeout
severity: warn
This variable is set too high. This is too long to wait before
noticing that the connection to the source has failed and retrying. This
should probably be set to 60 seconds or less. It is also a good idea to
noticing that the connection to the replication source has failed and retrying.
This should probably be set to 60 seconds or less. It is also a good idea to
use pt-heartbeat to ensure that the connection does not appear to time out
when the source is simply idle.
@@ -5906,7 +5907,7 @@ severity: crit
You should not set this option. If replication is having errors,
you need to find and resolve the cause of that; it is likely that your
slave's data is different from the master. You can find out with
replica's data is different from the source. You can find out with
pt-table-checksum.
=item replica_skip_errors
@@ -6060,13 +6061,15 @@ as tmp_table_size.
severity: warn
These are the recommended minimum version for each major release: 3.23, 4.1.20, 5.0.37, 5.1.30.
These are the recommended minimum version for each major release: 3.23, 4.1.20,
5.0.37, 5.1.30, 5.5.8, 5.6.10, 5.7.9, 8.0.11. This optiion does not complain
about Innovation releases.
=item end-of-life mysql version
severity: note
Every release older than 5.1 is now officially end-of-life.
Every release older than 8.0 is now officially end-of-life.
=back
@@ -6295,6 +6298,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

65
bin/pt-visual-explain
View File

@@ -2656,9 +2656,7 @@ bookmark lookup into the film_actor table.
=back
For more information on how to read EXPLAIN output, please see
L<http://dev.mysql.com/doc/en/explain.html>, and this talk titled
"MySQL query optimizer internals and upcoming features in v. 5.2":
from Timour Katchaounov, one of the MySQL developers: L<http://goo.gl/VIWvo>
L<http://dev.mysql.com/doc/en/explain.html>.
=head1 MODULES
@@ -2693,13 +2691,11 @@ trying to figure out how this works, but otherwise this section will probably
not make your life richer.
The tree can be built by examining the id, select_type, and table columns of
each row. Here's what I know about them:
each row.
The id column is the sequential number of the select. This does not indicate
nesting; it just comes from counting SELECT from the left of the SQL statement.
It's like capturing parentheses in a regular expression. A UNION RESULT row
doesn't have an id, because it isn't a SELECT. The source code actually refers
to UNIONs as a fake_lex, as I recall.
It's like capturing parentheses in a regular expression.
If two adjacent rows have the same id value, they are joined with the standard
single-sweep multi-join method.
@@ -2749,7 +2745,7 @@ access to the temporary table that holds the result of the subquery whose id is
N. If it says <unionN,..N> it's the same thing, but it refers to the results it
UNIONs together.
Finally, order matters. If a row's id is less than the one before it, I think
Finally, order matters. If a row's id is less than the one before it,
that means it is dependent on something other than the one before it. For
example,
@@ -2765,7 +2761,7 @@ example,
| 3 | SUBQUERY | film_actor |
| 2 | SUBQUERY | film |
If the results were in order 2-3-4, I think that would mean 3 is a subquery of
If the results were in order 2-3-4, that would mean 3 is a subquery of
2, 4 is a subquery of 3. As it is, this means 4 is a subquery of the nearest
previous recent row with a smaller id, which is 1. Likewise for 3 and 2.
@@ -2802,8 +2798,8 @@ following:
| NULL | UNION RESULT | <union5,6> |
| NULL | UNION RESULT | <union1,4> |
This would be a lot easier to work with if it looked like this (I've
bracketed the id on rows I moved):
This would be a lot easier to work with if it looked like this (we've
bracketed the id on rows we moved):
| id | select_type | table |
+------+--------------+------------+
@@ -2819,7 +2815,7 @@ bracketed the id on rows I moved):
In fact, why not re-number all the ids, so the PRIMARY row becomes 2, and so on?
That would make it even easier to read. Unfortunately that would also have the
effect of destroying the meaning of the id column, which I think is important to
effect of destroying the meaning of the id column, which is important to
preserve in the final tree. Also, though it makes it easier to read, it doesn't
make it easier to manipulate programmatically; so it's fine to leave them
numbered as they are.
@@ -2831,7 +2827,7 @@ the slice of rows that should be child nodes in the tree: you just look for the
first row whose ID is the same as the first number in the table.
The next question is how to find the last row that should be a child node of a
UNION or DERIVED. I'll start with DERIVED, because the solution makes UNION
UNION or DERIVED. We'll start with DERIVED, because the solution makes UNION
easy.
Consider how MySQL numbers the SELECTs sequentially according to their position
@@ -2871,14 +2867,14 @@ Here's the output of EXPLAIN:
| NULL | UNION RESULT | <union3,5> |
| 2 | DERIVED | film |
The siblings all have id 1, and the middle one I care about is derived3.
(Notice MySQL doesn't execute them in the order I defined them, which is fine).
Now notice that MySQL prints out the rows in the opposite order I defined the
subqueries: 6, 3, 2. It always seems to do this, and there might be other
The siblings all have id 1, and the middle one we care about is derived3.
(Notice MySQL doesn't execute them in the order they are defined, which is fine).
Now notice that MySQL prints out the rows in the opposite order the subqueries
were defined: 6, 3, 2. It always seems to do this, and there might be other
methods of finding the scope boundaries including looking for the lower boundary
of the next largest sibling, but this is a good enough heuristic. I am forced
to rely on it for non-DERIVED subqueries, so I rely on it here too. Therefore,
I decide that everything greater than or equal to 3 belongs to the DERIVED
of the next largest sibling, but this is a good enough heuristic. We are forced
to rely on it for non-DERIVED subqueries, so we rely on it here too. Therefore,
we decide that everything greater than or equal to 3 belongs to the DERIVED
scope.
The rule for UNION is simple: they consume the entire enclosing scope, and to
@@ -2894,8 +2890,8 @@ the enclosing scope's boundaries, and the rest is easy. Notice in the example
above, the UNION is over <union3,5>, which includes the row with id 4 -- it
includes every row between 3 and 5.
Finally, there are non-derived subqueries to deal with as well. In this case I
can't look at siblings to find the end of the scope as I did for DERIVED. I
Finally, there are non-derived subqueries to deal with as well. In this case we
can't look at siblings to find the end of the scope as we did for DERIVED. We
have to trust that MySQL executes depth-first. Here's an example:
explain
@@ -2950,16 +2946,14 @@ select 4 came first. In other words, if EXPLAIN looked like this,
| 4 | DEPENDENT SUBQUERY | actor |
I would be forced to assume upon seeing select 3 that select 4 is a subquery
of it, rather than just being the next sibling in the enclosing scope. If this
is ever wrong, then the algorithm is wrong, and I don't see what could be done
about it.
of it, rather than just being the next sibling in the enclosing scope.
UNION is a little more complicated than just "the entire scope is a UNION,"
because the UNION might itself be inside an enclosing scope that's only
indicated by the first item inside the UNION. There are only three kinds of
enclosing scopes: UNION, DERIVED, and SUBQUERY. A UNION can't enclose a UNION,
and a DERIVED has its own "scope markers," but a SUBQUERY can wholly enclose a
UNION, like this strange example on the empty table t1:
and a DERIVED has its own "scope markers," but a SUBQUERY in earlier versions
could wholly enclose a UNION, like this strange example on the empty table t1:
explain select * from t1 where not exists(
(select t11.i from t1 t11) union (select t12.i from t1 t12));
@@ -2974,12 +2968,8 @@ UNION, like this strange example on the empty table t1:
The UNION's backward references might make it look like the UNION encloses the
subquery, but studying the query makes it clear this isn't the case. So when a
UNION's first row says SUBQUERY, it is this special case.
By the way, I don't fully understand this query plan; there are 4 numbered
SELECT in the plan, but only 3 in the query. The parens around the UNIONs are
meaningful. Removing them will make the EXPLAIN different. Please tell me how
and why this works if you know.
UNION's first row says SUBQUERY, it is this special case that we cannot repeat
starting from MySQL 5.7.
Armed with this knowledge, it's possible to use recursion to turn the
parent-child relationship between all the rows into a tree representing the
@@ -3011,8 +3001,7 @@ The execution plan looks like a depth-first traversal of this tree:
The JOIN might not be a JOIN. It might be a subquery, for example. This comes
from the type column of EXPLAIN. The documentation says this is a "join type,"
but I think "access type" is more accurate, because it's "how MySQL accesses
rows."
but "access type" may be more accurate, because it's "how MySQL accesses rows."
pt-visual-explain decorates the tree significantly more than just turning
rows into nodes. Each node may get a series of transformations that turn it
@@ -3217,6 +3206,12 @@ dsn: user; copy: yes
User for login if not current user.
=item * s
dsn: mysql_ssl; copy: yes
Create SSL connection
=back
=head1 ENVIRONMENT

2
config/sphinx-build/_static/_templates/theme/relbar2.html
View File

@@ -2,7 +2,7 @@
<div class="md-content">
<article class="md-content__inner md-typeset" role="main">
<h4>Get Expert Help </h4>
<p>If you need assistance, visit the <a class="reference external" href="https://forums.percona.com/c/mongodb/percona-server-for-mongodb/17?utm_campaign=Doc%20pages" target="_blank">community forum</a> for comprehensive and free database knowledge.</p>
<p>If you need assistance, visit the <a class="reference external" href="https://forums.percona.com/?utm_campaign=Doc%20pages" target="_blank">community forum</a> for comprehensive and free database knowledge.</p>
<p>Contact our <a class="reference external" href="https://www.percona.com/about/contact">Percona Database Experts</a> for professional support and services.</p>
<hr>
{# Add the last updated timestamp from git commits using gitstamp extension #}

12
docs/percona-toolkit.pod
View File

@@ -406,7 +406,7 @@ databases while running.
=item F
Defaults file for the MySQL client library (the C client library used by
DBD::mysql, I<not Percona Toolkit itself>). All tools all read the
DBD::mysql, I<not Percona Toolkit itself>). All tools read the
C<[client]> section within the defaults file. If you omit this, the standard
defaults files will be read in the usual order. "Standard" varies from system
to system, because the filenames to read are compiled into the client library.
@@ -441,7 +441,7 @@ lead to weird situations, like the server allowing LOCAL INFILE, but
the client throwing exceptions if it's used.
However, as long as the server allows LOAD DATA, clients can easily
re-enable it; see L<https://dev.mysql.com/doc/refman/5.0/en/load-data-local.html>
re-enable it; see L<https://dev.mysql.com/doc/refman/en/load-data-local-security.html>
and L<http://search.cpan.org/~capttofu/DBD-mysql/lib/DBD/mysql.pm>.
This option does exactly that.
@@ -464,6 +464,10 @@ MySQL socket file to use for the connection (on Unix systems).
MySQL username to use when connecting, if not current system user.
=item s
Create SSL connection
=back
=head2 BAREWORD
@@ -539,7 +543,7 @@ Include the following information in your bug report:
=item * Tool C<--version>
=item * MySQL version of all servers involved
=item * MySQL, MongoDB, or PostgreSQL version of all servers involved
=item * Output from the tool including STDERR
@@ -603,6 +607,6 @@ Place, Suite 330, Boston, MA 02111-1307 USA.
=head1 VERSION
Percona Toolkit v3.6.0 released 2024-05-25
Percona Toolkit v3.6.0 released 2024-06-12
=cut

246
docs/pt-galera-log-explainer.rst
View File

@@ -1,246 +0,0 @@
.. _pt-galera-log-explainer:
==================================
:program:`pt-galera-log-explainer`
==================================
Filter, aggregate and summarize multiple galera logs together.
This is a toolbox to help navigating Galera logs.
Usage
=====
.. code-block:: bash
pt-galera-log-explainer [--since=] [--until=] [-vv] [--merge-by-directory] [--pxc-operator] <command> <paths ...>
Commands available
==================
list
~~~~
.. code-block:: bash
pt-galera-log-explainer [flags] list { --all | [--states] [--views] [--events] [--sst] [--applicative] } <paths ...>
List key events in chronological order from any number of nodes (sst, view changes, general errors, maintenance operations)
It will aggregates logs together by identifying them using node names, IPs and internal Galera identifiers.
It can be from a single node:
.. code-block:: bash
pt-galera-log-explainer list --all --since 2023-01-05T03:24:26.000000Z /var/log/mysql/*.log
or from multiple nodes.
.. code-block:: bash
pt-galera-log-explainer list --all *.log
You can filter by type of events
.. code-block:: bash
pt-galera-log-explainer list --sst --views *.log
whois
~~~~~
Find out information about nodes, using any type of information
.. code-block:: bash
pt-galera-log-explainer [flags] whois [--json] [--type { nodename | ip | uuid | auto }] <information to search> <paths ...>
.. code-block:: bash
pt-galera-log-explainer whois '218469b2' mysql.log
pt-galera-log-explainer whois '172.17.0.3' mysql.log
pt-galera-log-explainer whois 'galera-node2' mysql.log
conflicts
~~~~~~~~~
List every replication failure votes (Galera 4)
.. code-block:: bash
pt-galera-log-explainer conflicts [--json|--yaml] *.log
ctx
~~~
Get the tool crafted context for a single log.
It will contain everything the tool extracted from the log file: version, sst information, known uuid-ip-nodename mappings, ...
.. code-block:: bash
pt-galera-log-explainer ctx mysql.log
regex-list
~~~~~~~~~~
Will print every implemented regexes:
* regex: the regex that will be used against the log files
* internalRegex: the golang regex that will be used to extract piece of information
* type: the regex group it belong to
* verbosity: the required level of verbosity to which it will be printed
.. code-block:: bash
pt-galera-log-explainer regex-list
Available flags
~~~~~~~~~~~~~~~
``-h``, ``--help``
Show help and exit.
``--no-color``
Remove every color special characters
``--since``
Only list events after this date. It will affect the regex applied to the logs.
Format: 2023-01-23T03:53:40Z (RFC3339)
``--until``
Only list events before this date. This is only implemented in the tool loop, it does not alter regexes.
Format: 2023-01-23T03:53:40Z (RFC3339)
``--merge-by-directory``
Instead of relying on extracted information, logs will be merged by their base directory
It is useful when logs are very sparse and already organized by nodes.
``--skip-merge``
Disable the ability to merge log files together. Can be used when every nodes have the same ``wsrep_node_name``
``-v``, ``--verbosity``
``-v``: display in the timeline every mysql info the tool used
``-vv``: internal tool debug
``--pxc-operator``
Analyze logs from Percona PXC operator. Operator logs should be automatically detected (see ``--skip-operator-detection``).
It will prevent logs from being merged together, add operator specific regexes, and fine-tune regexes for logs taken from ``pt-k8s-debug-collector``.
Off by default because it negatively impacts performance for non-k8s setups.
``--skip-operator-detection``
Disable automatic detection of PXC operator logs. When detected, a message will be shown.
Detection is done using a prefix regex.
``--exclude-regexes``
Remove regexes from analysis. Use ``pt-galera-log-explainer regex-list | jq .`` to have the list
``--grep-cmd``
grep v3 binary command path. For Darwin systems, it could need to be set to ``ggrep``
Default: ``grep``
``--version``
Show version and exit.
``--custom-regexes``
Add custom regexes, printed in magenta. Format: (golang regex string)=[optional static message to display].
If the static message is left empty, the captured string will be printed instead. Custom regexes are separated using semi-colon.
Example: ``--custom-regexes="Page cleaner took [0-9]*ms to flush [0-9]* pages=;doesn't recommend.*pxc_strict_mode=unsafe query used"``
Example outputs
===============
.. code-block:: bash
$ pt-galera-log-explainer list --all --no-color --since=2023-03-12T19:41:28.493046Z --until=2023-03-12T19:44:59.855491Z tests/logs/upgrade/*
identifier 172.17.0.2 node2 tests/logs/upgrade/node3.log
current path tests/logs/upgrade/node1.log tests/logs/upgrade/node2.log tests/logs/upgrade/node3.log
last known ip 172.17.0.2
last known name node2
mysql version 8.0.28
2023-03-12T19:41:28.493046Z starting(8.0.28) | |
2023-03-12T19:41:28.500789Z started(cluster) | |
2023-03-12T19:43:17.630191Z | node3 joined |
2023-03-12T19:43:17.630208Z node3 joined | |
2023-03-12T19:43:17.630221Z node2 joined | |
2023-03-12T19:43:17.630243Z | node1 joined |
2023-03-12T19:43:17.634138Z | | node2 joined
2023-03-12T19:43:17.634229Z | | node1 joined
2023-03-12T19:43:17.643210Z | PRIMARY(n=3) |
2023-03-12T19:43:17.648163Z | | PRIMARY(n=3)
2023-03-12T19:43:18.130088Z CLOSED -> OPEN | |
2023-03-12T19:43:18.130230Z PRIMARY(n=3) | |
2023-03-12T19:43:18.130916Z OPEN -> PRIMARY | |
2023-03-12T19:43:18.904410Z will receive IST(seqno:178226792) | |
2023-03-12T19:43:18.913328Z | | node1 cannot find donor
2023-03-12T19:43:18.913429Z node1 cannot find donor | |
2023-03-12T19:43:18.913565Z | node1 cannot find donor |
2023-03-12T19:43:19.914122Z | | node1 cannot find donor
2023-03-12T19:43:19.914259Z node1 cannot find donor | |
2023-03-12T19:43:19.914362Z | node1 cannot find donor |
2023-03-12T19:43:20.914957Z | | (repeated x97)node1 cannot find donor
2023-03-12T19:43:20.915143Z (repeated x97)node1 cannot find donor | |
2023-03-12T19:43:20.915262Z | (repeated x97)node1 cannot find donor |
2023-03-12T19:44:58.999603Z | | node1 cannot find donor
2023-03-12T19:44:58.999791Z node1 cannot find donor | |
2023-03-12T19:44:58.999891Z | node1 cannot find donor |
2023-03-12T19:44:59.817822Z timeout from donor in gtid/keyring stage | |
2023-03-12T19:44:59.839692Z SST error | |
2023-03-12T19:44:59.840669Z | | node2 joined
2023-03-12T19:44:59.840745Z | | node1 left
2023-03-12T19:44:59.840933Z | node3 joined |
2023-03-12T19:44:59.841034Z | node1 left |
2023-03-12T19:44:59.841189Z NON-PRIMARY(n=1) | |
2023-03-12T19:44:59.841292Z PRIMARY -> OPEN | |
2023-03-12T19:44:59.841352Z OPEN -> CLOSED | |
2023-03-12T19:44:59.841515Z terminated | |
2023-03-12T19:44:59.841529Z former SST cancelled | |
2023-03-12T19:44:59.848349Z | | node1 left
2023-03-12T19:44:59.848409Z | | PRIMARY(n=2)
2023-03-12T19:44:59.855443Z | node1 left |
2023-03-12T19:44:59.855491Z | PRIMARY(n=2) |
$ pt-galera-log-explainer whois 172.17.0.2 --no-color tests/logs/upgrade/*
ip:
└── 172.17.0.2
├── nodename:
│ └── node1 (2023-03-12 19:35:07.644683 +0000 UTC)
│
└── uuid:
├── 1d3ea8f5 (2023-03-12 07:24:13.789261 +0000 UTC)
├── 54ab931e (2023-03-12 07:43:08.563339 +0000 UTC)
├── fecde235 (2023-03-12 08:46:48.963504 +0000 UTC)
├── a07872e1 (2023-03-12 08:49:41.206124 +0000 UTC)
├── 60da0bf9-aa9c (2023-03-12 12:29:48.873397 +0000 UTC)
├── 35b62086-902c (2023-03-12 13:04:23.979636 +0000 UTC)
├── ca2c2a5f-a82a (2023-03-12 19:35:05.878879 +0000 UTC)
└── eefb9c8a-b69a (2023-03-12 19:43:17.133756 +0000 UTC)
Requirements
============
grep, version 3
On Darwin based OS, grep is only version 2 due to license limitations. --grep-cmd can be used to point the correct grep binary, usually ggrep
Compatibility
=============
* Percona XtraDB Cluster: 5.5 to 8.0
* MariaDB Galera Cluster: 10.0 to 10.6
* logs from PXC operator pods (error.log, recovery.log, post.processing.log)
Known issues
============
* Nodes sharing the same ip, or nodes with identical names are not supported
* Sparse files identification can be missed, resulting in many columns displayed. ``--merge-by-directory`` can be used, but files need to be organized already in separate directories
This is mainly when the log file does not contain enough information.
* Some information will seems missed. Depending on the case, it may be simply unimplemented yet, or it was disabled later because it was found to be unreliable (node index numbers are not reliable for example)
* Columns width are sometimes too large to be easily readable. This usually happens when printing SST events with long node names
* When some display corner-cases seems broken (events not deduplicated, ...), it is because of extra hidden internal events.

1
docs/pt-galera-log-explainer.rst Symbolic link
View File

@@ -0,0 +1 @@
../src/go/pt-galera-log-explainer/README.rst

191
docs/pt-k8s-debug-collector.rst
View File

@@ -1,191 +0,0 @@
.. _pt-k8s-debug-collector:
==================================
:program:`pt-k8s-debug-collector`
==================================
Collects debug data (logs, resource statuses etc.) from a k8s/OpenShift cluster. Data is packed into the ``cluster-dump.tar.gz`` archive in the current working directory.
Data that will be collected
===========================
.. code-block:: bash
"pods",
"replicasets",
"deployments",
"statefulsets",
"replicationcontrollers",
"events",
"configmaps",
"cronjobs",
"jobs",
"podsecuritypolicies",
"poddisruptionbudgets",
"perconaxtradbbackups",
"perconaxtradbclusterbackups",
"perconaxtradbclusterrestores",
"perconaxtradbclusters",
"clusterrolebindings",
"clusterroles",
"rolebindings",
"roles",
"storageclasses",
"persistentvolumeclaims",
"persistentvolumes",
"modes",
"your-custom-resource" (depends on 'resource' flag)
Data, collected for PXC
~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: bash
"perconaxtradbbackups",
"perconaxtradbclusterbackups",
"perconaxtradbclusterrestores",
"perconaxtradbclusters"
Summary, collected for PXC (available in file summary.txt)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: bash
"pt-mysql-summary"
Individual files, collected for PXC
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: bash
"var/lib/mysql/mysqld-error.log",
"var/lib/mysql/innobackup.backup.log",
"var/lib/mysql/innobackup.move.log",
"var/lib/mysql/innobackup.prepare.log",
"var/lib/mysql/grastate.dat",
"var/lib/mysql/gvwstate.dat",
"var/lib/mysql/mysqld.post.processing.log",
"var/lib/mysql/auto.cnf"
Data, collected for MySQL
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: bash
"perconaservermysqlbackups",
"perconaservermysqlrestores",
"perconaservermysqls"
Summary, collected for MySQL (available in file summary.txt)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: bash
"pt-mysql-summary"
Data, collected for MongoDB
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: bash
"perconaservermongodbbackups",
"perconaservermongodbrestores",
"perconaservermongodbs"
Summary, collected for MongoDB (available in file summary.txt)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: bash
"pt-mongodb-summary"
Data, collected for PostgreSQL
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: bash
"perconapgclusters",
"pgclusters",
"pgpolicies",
"pgreplicas",
"pgtasks"
Summary, collected for PostgreSQL (available in file summary.txt)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: bash
"pg_gather"
Usage
=====
``pt-k8s-debug-collector <flags>``
Supported Flags
================
``--resource``
Targeted custom resource name. Supported values:
* ``pxc`` - PXC
* ``psmdb`` - MongoDB
* ``pg`` - PostgreSQL Operator v1 (deprecated)
* ``pgv2`` - PostgreSQL Operator v2
* ``ps`` - MySQL
* ``none`` - Collect only general Kubernetes data, do not collect anything specific to the particular operator).
* ``auto`` - Auto-detect custom resource
Default: ``auto``
``--namespace``
Targeted namespace. By default data will be collected from all namespaces
``--cluster``
Targeted cluster. By default data from all available clusters to be collected
``--kubeconfig``
Path to kubeconfig. Default configuration be used if none specified
``--forwardport``
Port to use when collecting database-specific summaries. By default, 3306 will be used for PXC and MySQL, 27017 for MongoDB, and 5432 for PostgreSQL
``--version``
Print version info
Requirements
============
- Installed, configured, and available in PATH ``kubectl``
- Installed, configured, and available in PATH ``pt-mysql-summary`` for PXC and MySQL
- Installed, configured, and available in PATH ``mysql`` for PXC and MySQL
- Installed, configured, and available in PATH ``pt-mongodb-summary`` for MongoDB
- Installed, configured, and available in PATH ``psql`` for PostgreSQL
Known Issues
============
On Kubernetes 1.21 - 1.24 warning is printed:
.. code-block:: bash
2022/12/15 17:43:16 Error: get resource podsecuritypolicies in namespace default: error: <nil>, stderr: Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
, stdout: apiVersion: v1
items: []
kind: List
metadata:
resourceVersion: ""
This warning is harmless and does not affect data collection. We will remove podsecuritypolicies once everyone upgrade to Kubernetes 1.25 or newer. Before that we advise to ignore this warning.

1
docs/pt-k8s-debug-collector.rst Symbolic link
View File

@@ -0,0 +1 @@
../src/go/pt-k8s-debug-collector/README.rst

72
docs/pt-mongodb-index-check.rst
View File

@@ -1,72 +0,0 @@
.. _pt-mongodb-index-check:
=================================
:program:`pt-mongodb-index-check`
=================================
Performs checks on MongoDB indexes.
Checks available
================
Duplicated indexes
~~~~~~~~~~~~~~~~~~
Check for indexes that are the prefix of other indexes. For example if we have these 2 indexes
.. code-block:: javascript
db.getSiblingDB("testdb").test_col.createIndex({"f1": 1, "f2": -1, "f3": 1, "f4": 1}, {"name": "idx_01"});
db.getSiblingDB("testdb").test_col.createIndex({"f1": 1, "f2": -1, "f3": 1}, {"name": "idx_02"});
The index ``idx_02`` is the prefix of ``idx_01`` because it has the same
keys in the same order so, ``idx_02`` can be dropped.
Unused indexes.
~~~~~~~~~~~~~~~
This check gets the ``$indexstats`` for all indexes and reports those
having ``accesses.ops`` = 0.
Usage
=====
Run the program as ``pt-mongodb-index-check <command> [flags]``
Available commands
~~~~~~~~~~~~~~~~~~
================ ==================================
Command Description
================ ==================================
check-duplicated Run checks for duplicated indexes.
check-unused Run check for unused indexes.
check-all Run all checks
================ ==================================
Available flags
~~~~~~~~~~~~~~~
+----------------------------+----------------------------------------+
| Flag | Description |
+============================+========================================+
| –all-databases | Check in all databases excluding |
| | system dbs. |
+----------------------------+----------------------------------------+
| –databases=DATABASES,… | Comma separated list of databases to |
| | check. |
+----------------------------+----------------------------------------+
| –all-collections | Check in all collections in the |
| | selected databases. |
+----------------------------+----------------------------------------+
| –collections=COLLECTIONS,… | Comma separated list of collections to |
| | check. |
+----------------------------+----------------------------------------+
| –mongodb.uri= | Connection URI |
+----------------------------+----------------------------------------+
| –json | Show output as JSON |
+----------------------------+----------------------------------------+
| –version | Show version information |
+----------------------------+----------------------------------------+

1
docs/pt-mongodb-index-check.rst Symbolic link
View File

@@ -0,0 +1 @@
../src/go/pt-mongodb-index-check/README.rst

168
docs/pt-mongodb-summary.rst
View File

@@ -1,168 +0,0 @@
.. pt-mongodb-summary:
=============================
:program:`pt-mongodb-summary`
=============================
``pt-mongodb-summary`` collects information about a MongoDB cluster.
It collects information from several sources
to provide an overview of the cluster.
Usage
=====
.. code-block:: bash
pt-mongodb-summary [OPTIONS] [HOST[:PORT]]
By default, if you run ``pt-mongodb-summary`` without any parameters,
it will try to connect to ``localhost`` on port ``27017``.
The program collects information about MongoDB instances
by running administration commands and formatting the output.
.. note:: ``pt-mongodb-summary`` requires to be run by user
with the ``clusterAdmin`` or ``root`` built-in roles.
.. note:: ``pt-mongodb-summary`` cannot collect statistics
from MongoDB instances that require connection via SSL.
Support for SSL will be added in the future.
Options
-------
``-a``, ``--auth-db``
Specifies the database used to establish credentials and privileges
with a MongoDB server.
By default, the ``admin`` database is used.
``-f``, ``--output-format``
Specifies the report output format. Valid options are: ``text``, ``json``.
The default value is ``text``.
``-p``, ``--password``
Specifies the password to use when connecting to a server
with authentication enabled.
Do not add a space between the option and its value: ``-p<password>``.
If you specify the option without any value,
``pt-mongodb-summary`` will ask for password interactively.
``-u``, ``--user``
Specifies the user name for connecting to a server
with authentication enabled.
Output Example
==============
.. code-block:: none
# Instances ####################################################################################
ID Host Type ReplSet
0 localhost:17001 PRIMARY r1
1 localhost:17002 SECONDARY r1
2 localhost:17003 SECONDARY r1
0 localhost:18001 PRIMARY r2
1 localhost:18002 SECONDARY r2
2 localhost:18003 SECONDARY r2
# This host
# Mongo Executable #############################################################################
Path to executable | /home/karl/tmp/MongoDB32Labs/3.0/bin/mongos
# Report On 0 ########################################
User | karl
PID Owner | mongos
Time | 2016-10-30 00:18:49 -0300 ART
Hostname | karl-HP-ENVY
Version | 3.0.11
Built On | Linux x86_64
Started | 2016-10-30 00:18:49 -0300 ART
Datadir | /data/db
Process Type | mongos
# Running Ops ##################################################################################
Type Min Max Avg
Insert 0 0 0/5s
Query 0 0 0/5s
Update 0 0 0/5s
Delete 0 0 0/5s
GetMore 0 0 0/5s
Command 0 22 16/5s
# Security #####################################################################################
Users 0
Roles 0
Auth disabled
SSL disabled
# Oplog ########################################################################################
Oplog Size 18660 Mb
Oplog Used 55 Mb
Oplog Length 0.91 hours
Last Election 2016-10-30 00:18:44 -0300 ART
# Cluster wide #################################################################################
Databases: 3
Collections: 17
Sharded Collections: 1
Unsharded Collections: 16
Sharded Data Size: 68 GB
Unsharded Data Size: 0 KB
# Balancer (per day)
Success: 6
Failed: 0
Splits: 0
Drops: 0
Sections
--------
Output is separated into the following sections:
* **Instances**
This section lists all hosts connected to the current MongoDB instance.
For this, ``pt-mongodb-summary`` runs the ``listShards`` command
and then the ``replSetGetStatus`` on every instance
to collect its ID, type, and replica set.
* **This host**
This section provides an overview of the current MongoDB instance
and the underlying OS.
For this, ``pt-mongodb-summary`` groups information
collected from ``hostInfo``, ``getCmdLineOpts``, ``serverStatus``,
and the OS process (by process ID).
* **Running Ops**
This section provides minimum, maximum, and average operation counters
for ``insert``, ``query``, ``update``, ``delete``, ``getMore``,
and ``command`` operations.
For this, ``pt-mongodb-summary`` runs the ``serverStatus`` command
5 times at regular intervals (every second).
* **Security**
This section provides information about the security settings.
For this, ``pt-mongodb-summary``, parses ``getCmdLineOpts`` output
and queries the ``admin.system.users``
and ``admin.system.roles`` collections.
* **Oplog**
This section contains details about the MongoDB operations log (oplog).
For this, ``pt-mongodb-summary`` collects statistics
from the oplog on every host in the cluster,
and returns those with the smallest ``TimeDiffHours`` value.
* **Cluster wide**
This section provides information about the number of sharded and
unsharded databases, collections, and their size.
For this, ``pt-mongodb-summary`` runs the ``listDatabases`` command
and then runs ``collStats`` for every collection in every database.

1
docs/pt-mongodb-summary.rst Symbolic link
View File

@@ -0,0 +1 @@
../src/go/pt-mongodb-summary/README.rst

475
docs/pt-pg-summary.rst
View File

@@ -1,475 +0,0 @@
========================
:program:`pt-pg-summary`
========================
**pt-pg-summary** collects information about a PostgreSQL cluster.
Usage
-----
``pt-pg-summary [options] [host:[port]]``
Binaries
--------
Please check the `releases <https://github.com/percona/toolkit-go/releases>`_ tab to download the binaries.
Parameters
^^^^^^^^^^
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| Short | Long | Default | Description |
+========+=================================+=========+============================================================================+
| | ``--help`` | | Show context-sensitive help (also try ``--help-long`` and ``--help-man``). |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| | ``--version`` | | Show application version. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| | ``--databases=DATABASES`` | | Summarize this comma-separated list of databases. All if not specified. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| ``-h`` | ``--host=HOST`` | | Host to connect to or local Unix socket for connection |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| ``-W`` | ``--password=PASSWORD`` | | Password to use when connecting. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| ``-p`` | ``--port=PORT`` | | Port number to use for connection. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| | ``--sleep=SLEEP`` | 10 | Seconds to sleep when gathering status counters. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| ``-U`` | ``--username=USERNAME`` | | User for login if not current user. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| | ``--disable-ssl`` | true | Disable SSL for the connection. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| | ``--verbose`` | false | Show verbose log. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| | ``--debug`` | false | Show debug information in the logs. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
.. Currently hidden
.. --list-encrypted-tables Include a list of the encrypted tables in all databases
.. --ask-pass Prompt for a password when connecting to PostgreSQL
.. --config Config file
.. --defaults-file Only read PostgreSQL options from the given file
.. --read-samples Create a report from the files found in this directory
.. --save-samples Save the data files used to generate the summary in this directory
Output
^^^^^^
The output is grouped into these categories:
AllDatabases
Selects ``datname`` from ``pg_database`` where ``datistemplate`` is false.
ClusterInfo
Selects cluster information from ``pg_stat_activity``.
ConnectedClients
Counts the connected clients by selecting from ``pg_stat_activity``.
Connections
Selects ``state`` from ``pg_stat_activity`` and counts them.
Counters
Selects various counter values from ``pg_stat_database``.
DatabaseWaitEvents
Shows database wait events from ``pg_locks``, ``pg_stat_database``, ``pg_class``, and ``pg_stat_activity``.
Databases
Shows the name and size of databases from ``pg_stat_database``.
GlobalWaitEvents
Shows global wait evens from ``pg_stat_activity``.
IndexCacheHitRatio
Shows index hit ratios from ``pg_statio_user_indexes``.
PortAndDatadir
Shows port and data directory name from ``pg_settings``.
ServerVersion
Shows the value of ``server_version_num``.
Setting
Selects ``name`` and ``setting`` from ``pg_settings``.
SlaveHosts10
Selects information for PostgreSQL version 10.
SlaveHosts96
Selects information for PostgreSQL version 9.6.
TableAccess
Shows table access information by selecting from ``pg_locks``, ``pg_stat_database`` and ``pg_class``.
TableCacheHitRatio
Shows table cache hit ratio information from ``pg_statio_user_tables``.
Tablespaces
Show owner and location from ``pg_catalog.pg_tablespace``.
Output example
""""""""""""""
.. code-block:: html
##### --- Database Port and Data_Directory --- ####
+----------------------+----------------------------------------------------+
| Name | Setting |
+----------------------+----------------------------------------------------+
| data_directory | /var/lib/postgresql/data |
+----------------------+----------------------------------------------------+
##### --- List of Tablespaces ---- ######
+----------------------+----------------------+----------------------------------------------------+
| Name | Owner | Location |
+----------------------+----------------------+----------------------------------------------------+
| pg_default | postgres | |
| pg_global | postgres | |
+----------------------+----------------------+----------------------------------------------------+
##### --- Cluster Information --- ####
+------------------------------------------------------------------------------------------------------+
Usename : postgres
Time : 2020-04-21 13:38:22.770077 +0000 UTC
Client Address : 172.19.0.1
Client Hostname:
Version : PostgreSQL 9.6.17 on x86_64-pc-linux-gnu (Debian 9.6.17-2.pgdg90+1), compiled by
Started : 2020-04-21 13:36:59.909175 +0000 UTC
Is Slave : false
+------------------------------------------------------------------------------------------------------+
##### --- Databases --- ####
+----------------------+------------+
| Dat Name | Size |
+----------------------+------------+
| postgres | 7071 kB |
| template1 | 6961 kB |
| template0 | 6961 kB |
+----------------------+------------+
##### --- Index Cache Hit Ratios --- ####
Database: postgres
+----------------------+------------+
| Index Name | Ratio |
+----------------------+------------+
| index hit rate | 0.00 |
+----------------------+------------+
##### --- Table Cache Hit Ratios --- ####
Database: postgres
+----------------------+------------+
| Index Name | Ratio |
+----------------------+------------+
| cache hit rate | 0.00 |
+----------------------+------------+
##### --- List of Wait_events for the entire Cluster - all-databases --- ####
No stats available
##### --- List of users and client_addr or client_hostname connected to --all-databases --- ####
+----------------------+------------+---------+----------------------+---------+
| Wait Event Type | Client | State | Count |
+----------------------+------------+---------+----------------------+---------+
| postgres | 172.19.0.1/32 | active | 1 |
+----------------------+------------+---------+----------------------+---------+
##### --- Counters diff after 10 seconds --- ####
+----------------------+-------------+------------+--------------+-------------+------------+-------------+------------+-------------+------------+------------+-----------+-----------+-----------+------------+
| Database | Numbackends | XactCommit | XactRollback | BlksRead | BlksHit | TupReturned | TupFetched | TupInserted | TupUpdated | TupDeleted | Conflicts | TempFiles | TempBytes | Deadlocks |
+----------------------+-------------+------------+--------------+-------------+------------+-------------+------------+-------------+------------+------------+-----------+-----------+-----------+------------+
| postgres | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| template0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| template1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+----------------------+-------------+------------+--------------+-------------+------------+-------------+------------+-------------+------------+------------+-----------+-----------+-----------+------------+
##### --- Table access per database --- ####
Database: postgres
+----------------------------------------------------+------+--------------------------------+---------+
| Relname | Kind | Datname | Count |
+----------------------------------------------------+------+--------------------------------+---------+
| pg_class | r | postgres | 1 |
| pg_stat_database | v | postgres | 1 |
| pg_locks | v | postgres | 1 |
| pg_class_tblspc_relfilenode_index | i | postgres | 1 |
| pg_class_relname_nsp_index | i | postgres | 1 |
| pg_class_oid_index | i | postgres | 1 |
+----------------------------------------------------+------+--------------------------------+---------+
##### --- Instance settings --- ####
Setting Value
allow_system_table_mods : off
application_name :
archive_command : (disabled)
archive_mode : off
archive_timeout : 0
array_nulls : on
authentication_timeout : 60
autovacuum : on
autovacuum_analyze_scale_factor : 0.1
autovacuum_analyze_threshold : 50
autovacuum_freeze_max_age : 200000000
autovacuum_max_workers : 3
autovacuum_multixact_freeze_max_age : 400000000
autovacuum_naptime : 60
autovacuum_vacuum_cost_delay : 20
autovacuum_vacuum_cost_limit : -1
autovacuum_vacuum_scale_factor : 0.2
autovacuum_vacuum_threshold : 50
autovacuum_work_mem : -1
backend_flush_after : 0
backslash_quote : safe_encoding
bgwriter_delay : 200
bgwriter_flush_after : 64
bgwriter_lru_maxpages : 100
bgwriter_lru_multiplier : 2
block_size : 8192
bonjour : off
bonjour_name :
bytea_output : hex
check_function_bodies : on
checkpoint_completion_target : 0.5
checkpoint_flush_after : 32
checkpoint_timeout : 300
checkpoint_warning : 30
client_encoding : UTF8
client_min_messages : notice
cluster_name :
commit_delay : 0
commit_siblings : 5
config_file : /var/lib/postgresql/data/postgresql.conf
constraint_exclusion : partition
cpu_index_tuple_cost : 0.005
cpu_operator_cost : 0.0025
cpu_tuple_cost : 0.01
cursor_tuple_fraction : 0.1
data_checksums : off
data_directory : /var/lib/postgresql/data
data_sync_retry : off
DateStyle : ISO, MDY
db_user_namespace : off
deadlock_timeout : 1000
debug_assertions : off
debug_pretty_print : on
debug_print_parse : off
debug_print_plan : off
debug_print_rewritten : off
default_statistics_target : 100
default_tablespace :
default_text_search_config : pg_catalog.english
default_transaction_deferrable : off
default_transaction_isolation : read committed
default_transaction_read_only : off
default_with_oids : off
dynamic_library_path : $libdir
dynamic_shared_memory_type : posix
effective_cache_size : 524288
effective_io_concurrency : 1
enable_bitmapscan : on
enable_hashagg : on
enable_hashjoin : on
enable_indexonlyscan : on
enable_indexscan : on
enable_material : on
enable_mergejoin : on
enable_nestloop : on
enable_seqscan : on
enable_sort : on
enable_tidscan : on
escape_string_warning : on
event_source : PostgreSQL
exit_on_error : off
external_pid_file :
extra_float_digits : 2
force_parallel_mode : off
from_collapse_limit : 8
fsync : on
full_page_writes : on
geqo : on
geqo_effort : 5
geqo_generations : 0
geqo_pool_size : 0
geqo_seed : 0
geqo_selection_bias : 2
geqo_threshold : 12
gin_fuzzy_search_limit : 0
gin_pending_list_limit : 4096
hba_file : /var/lib/postgresql/data/pg_hba.conf
hot_standby : off
hot_standby_feedback : off
huge_pages : try
ident_file : /var/lib/postgresql/data/pg_ident.conf
idle_in_transaction_session_timeout : 0
ignore_checksum_failure : off
ignore_system_indexes : off
integer_datetimes : on
IntervalStyle : postgres
join_collapse_limit : 8
krb_caseins_users : off
krb_server_keyfile : FILE:/etc/postgresql-common/krb5.keytab
lc_collate : en_US.utf8
lc_ctype : en_US.utf8
lc_messages : en_US.utf8
lc_monetary : en_US.utf8
lc_numeric : en_US.utf8
lc_time : en_US.utf8
listen_addresses : *
lo_compat_privileges : off
local_preload_libraries :
lock_timeout : 0
log_autovacuum_min_duration : -1
log_checkpoints : off
log_connections : off
log_destination : stderr
log_directory : pg_log
log_disconnections : off
log_duration : off
log_error_verbosity : default
log_executor_stats : off
log_file_mode : 0600
log_filename : postgresql-%Y-%m-%d_%H%M%S.log
log_hostname : off
log_line_prefix :
log_lock_waits : off
log_min_duration_statement : -1
log_min_error_statement : error
log_min_messages : warning
log_parser_stats : off
log_planner_stats : off
log_replication_commands : off
log_rotation_age : 1440
log_rotation_size : 10240
log_statement : none
log_statement_stats : off
log_temp_files : -1
log_timezone : Etc/UTC
log_truncate_on_rotation : off
logging_collector : off
maintenance_work_mem : 65536
max_connections : 100
max_files_per_process : 1000
max_function_args : 100
max_identifier_length : 63
max_index_keys : 32
max_locks_per_transaction : 64
max_parallel_workers_per_gather : 0
max_pred_locks_per_transaction : 64
max_prepared_transactions : 0
max_replication_slots : 0
max_stack_depth : 2048
max_standby_archive_delay : 30000
max_standby_streaming_delay : 30000
max_wal_senders : 0
max_wal_size : 64
max_worker_processes : 8
min_parallel_relation_size : 1024
min_wal_size : 5
old_snapshot_threshold : -1
operator_precedence_warning : off
parallel_setup_cost : 1000
parallel_tuple_cost : 0.1
password_encryption : on
port : 5432
post_auth_delay : 0
pre_auth_delay : 0
quote_all_identifiers : off
random_page_cost : 4
replacement_sort_tuples : 150000
restart_after_crash : on
row_security : on
search_path : "$user", public
segment_size : 131072
seq_page_cost : 1
server_encoding : UTF8
server_version : 9.6.17
server_version_num : 90617
session_preload_libraries :
session_replication_role : origin
shared_buffers : 16384
shared_preload_libraries :
sql_inheritance : on
ssl : off
ssl_ca_file :
ssl_cert_file : server.crt
ssl_ciphers : HIGH:MEDIUM:+3DES:!aNULL
ssl_crl_file :
ssl_ecdh_curve : prime256v1
ssl_key_file : server.key
ssl_prefer_server_ciphers : on
standard_conforming_strings : on
statement_timeout : 0
stats_temp_directory : pg_stat_tmp
superuser_reserved_connections : 3
synchronize_seqscans : on
synchronous_commit : on
synchronous_standby_names :
syslog_facility : local0
syslog_ident : postgres
syslog_sequence_numbers : on
syslog_split_messages : on
tcp_keepalives_count : 9
tcp_keepalives_idle : 7200
tcp_keepalives_interval : 75
temp_buffers : 1024
temp_file_limit : -1
temp_tablespaces :
TimeZone : Etc/UTC
timezone_abbreviations : Default
trace_notify : off
trace_recovery_messages : log
trace_sort : off
track_activities : on
track_activity_query_size : 1024
track_commit_timestamp : off
track_counts : on
track_functions : none
track_io_timing : off
transaction_deferrable : off
transaction_isolation : read committed
transaction_read_only : off
transform_null_equals : off
unix_socket_directories : /var/run/postgresql
unix_socket_group :
unix_socket_permissions : 0777
update_process_title : on
vacuum_cost_delay : 0
vacuum_cost_limit : 200
vacuum_cost_page_dirty : 20
vacuum_cost_page_hit : 1
vacuum_cost_page_miss : 10
vacuum_defer_cleanup_age : 0
vacuum_freeze_min_age : 50000000
vacuum_freeze_table_age : 150000000
vacuum_multixact_freeze_min_age : 5000000
vacuum_multixact_freeze_table_age : 150000000
wal_block_size : 8192
wal_buffers : 512
wal_compression : off
wal_keep_segments : 0
wal_level : minimal
wal_log_hints : off
wal_receiver_status_interval : 10
wal_receiver_timeout : 60000
wal_retrieve_retry_interval : 5000
wal_segment_size : 2048
wal_sender_timeout : 60000
wal_sync_method : fdatasync
wal_writer_delay : 200
wal_writer_flush_after : 128
work_mem : 4096
xmlbinary : base64
xmloption : content
zero_damaged_pages : off
##### --- Processes start up command --- ####
No postgres process found
Minimum auth role
^^^^^^^^^^^^^^^^^
This program needs to run some commands like ``getShardMap`` and to be able to run those commands
it needs to run under a user with the ``clusterAdmin`` or ``root`` built-in roles.

1
docs/pt-pg-summary.rst Symbolic link
View File

@@ -0,0 +1 @@
../src/go/pt-pg-summary/README.rst

2
docs/release_notes.rst
View File

@@ -1,4 +1,4 @@
Percona Toolkit
Release Notes
***************
v3.6.0 released 2024-06-12

2
docs/version-check.rst
View File

@@ -80,7 +80,7 @@ in case of |pt| `this can be done <https://www.percona.com/doc/percona-toolkit/L
# Disable Version Check for all tools:
no-version-check
In case of |pxb| this can be done `in its configuration file <https://www.percona.com/doc/percona-xtrabackup/2.4/using_xtrabackup/configuring.htm>`_ in a similar way::
In case of |pxb| this can be done `in its configuration file <https://docs.percona.com/percona-xtrabackup/2.4/using_xtrabackup/configuring.html>`_ in a similar way::
[xtrabackup]
no-version-check

5
lib/DuplicateKeyFinder.pm
View File

@@ -154,7 +154,10 @@ sub get_duplicate_keys {
if ( $clustered_key
&& $args{clustered}
&& $args{tbl_info}->{engine}
&& $args{tbl_info}->{engine} =~ m/InnoDB/i )
&& ( $args{tbl_info}->{engine} =~ m/InnoDB/i
|| $args{tbl_info}->{engine} =~ m/TokuDB/i
|| $args{tbl_info}->{engine} =~ m/RocksDB/i )
)
{
PTDEBUG && _d('Removing UNIQUE dupes of clustered key');
push @dupes,

2
lib/TableParser.pm
View File

@@ -457,7 +457,7 @@ sub get_keys {
};
# Find clustered key (issue 295).
if ( ($engine || '') =~ m/InnoDB/i && !$clustered_key ) {
if ( ($engine || '') =~ m/(InnoDB)|(TokuDB)|(RocksDB)/i && !$clustered_key ) {
my $this_key = $keys->{$name};
if ( $this_key->{name} eq 'PRIMARY' ) {
$clustered_key = 'PRIMARY';

5
src/go/pt-galera-log-explainer/README.rst
View File

@@ -244,3 +244,8 @@ Known issues
* Some information will seems missed. Depending on the case, it may be simply unimplemented yet, or it was disabled later because it was found to be unreliable (node index numbers are not reliable for example)
* Columns width are sometimes too large to be easily readable. This usually happens when printing SST events with long node names
* When some display corner-cases seems broken (events not deduplicated, ...), it is because of extra hidden internal events.
Authors
=======
Yoann La Cancellera

5
src/go/pt-k8s-debug-collector/README.rst
View File

@@ -189,3 +189,8 @@ On Kubernetes 1.21 - 1.24 warning is printed:
resourceVersion: ""
This warning is harmless and does not affect data collection. We will remove podsecuritypolicies once everyone upgrade to Kubernetes 1.25 or newer. Before that we advise to ignore this warning.
Authors
=======
Max Dudin, Andrii Dema, Carlos Salguero, Sveta Smirnova

4
src/go/pt-mongodb-index-check/README.rst
View File

@@ -70,3 +70,7 @@ Available flags
| –version | Show version information |
+----------------------------+----------------------------------------+
Authors
=======
Carlos Salguero

4
src/go/pt-mongodb-summary/README.rst
View File

@@ -96,3 +96,7 @@ Minimum auth role
This program needs to run some commands like ``getShardMap`` and to be able to run those commands
it needs to run under a user with the ``clusterAdmin`` or ``root`` built-in roles.
Authors
=======
Carlos Salguero

13
src/go/pt-pg-summary/README.rst
View File

@@ -1,5 +1,7 @@
pt-pg-summary
=============
========================
:program:`pt-pg-summary`
========================
**pt-pg-summary** collects information about a PostgreSQL cluster.
Usage
@@ -23,7 +25,7 @@ Parameters
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| | ``--databases=DATABASES`` | | Summarize this comma-separated list of databases. All if not specified. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| ``-h`` | ``--host=HOST`` | | Host to connect to. |
| ``-h`` | ``--host=HOST`` | | Host to connect to or local Unix socket for connection |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
| ``-W`` | ``--password=PASSWORD`` | | Password to use when connecting. |
+--------+---------------------------------+---------+----------------------------------------------------------------------------+
@@ -471,3 +473,8 @@ Minimum auth role
This program needs to run some commands like ``getShardMap`` and to be able to run those commands
it needs to run under a user with the ``clusterAdmin`` or ``root`` built-in roles.
Authors
=======
Carlos Salguero