percona/percona-toolkit
1
0
Fork 0
mirror of https://github.com/percona/percona-toolkit.git synced 2025-10-18 08:24:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

Remove pt-align and pt-rel. Update/clarify docu in some tools.

Browse Source
This commit is contained in:
Daniel Nichter
2011-08-03 11:53:12 -06:00
parent 1a720f7c67
commit 5ca914b932
12 changed files with 45 additions and 543 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
MANIFEST
View File

@@ -4,7 +4,6 @@ INSTALL
MANIFEST
Makefile.PL
README
bin/pt-align
bin/pt-archiver
bin/pt-checksum-filter
bin/pt-collect
@@ -27,7 +26,6 @@ bin/pt-profile-compact
bin/pt-query-advisor
bin/pt-query-digest
bin/pt-query-profiler
bin/pt-rel
bin/pt-show-grants
bin/pt-sift
bin/pt-slave-delay

194
bin/pt-align
View File

@@ -1,194 +0,0 @@
#!/usr/bin/env perl
# This program is part of Percona Toolkit: http://www.percona.com/software/
# See "COPYRIGHT, LICENSE, AND WARRANTY" at the end of this file for legal
# notices and disclaimers.
use strict;
use warnings FATAL => 'all';
# Finds the max element in the list
sub max {
my $i = shift @_;
foreach my $n ( @_ ) {
$i = $n if $n > $i;
}
return $i;
}
# Read all lines
my @lines;
my %word_count;
while ( <> ) {
my $line = $_;
my @words = $line =~ m/(\S+)/g;
push @lines, \@words;
$word_count{ scalar @words }++;
}
# Find max number of words per line
my @wc = reverse sort { $word_count{$a}<=>$word_count{$b} } keys %word_count;
my $m_words = $wc[0];
# Filter out non-conformists
@lines = grep { scalar @$_ == $m_words } @lines;
die "I need at least 2 lines" unless @lines > 1;
# Find the widths and alignments of each column
my @fmt;
foreach my $i ( 0 .. $m_words-1 ) {
my $m_len = max(map { length($_->[$i]) } @lines);
my $code = $lines[1]->[$i] =~ m/[^0-9.-]/ ? "%-${m_len}s" : "%${m_len}s";
push @fmt, $code;
}
my $fmt = join(' ', @fmt) . "\n";
# Print!
foreach my $l ( @lines ) {
printf $fmt, @$l;
}
exit 0;
# ############################################################################
# Documentation
# ############################################################################
=pod
=head1 NAME
pt-align - Read lines and split them into words.
=head1 SYNOPSIS
Usage: pt-align [FILES]
pt-align reads lines in files and splits them into words. This is useful for
things like aligning the output of vmstat or iostat so it is easier to read.
=head1 RISKS
The following section is included to inform users about the potential risks,
whether known or unknown, of using this tool. The two main categories of risks
are those created by the nature of the tool (e.g. read-only tools vs. read-write
tools) and those created by bugs.
pt-align is a read-only tool. It should be very low-risk.
At the time of this release, we know of no bugs that could cause serious harm
to users.
The authoritative source for updated information is always the online issue
tracking system. Issues that affect this tool will be marked as such. You can
see a list of such issues at the following URL:
L<http://www.percona.com/bugs/pt-align>.
See also L<"BUGS"> for more information on filing bugs and getting help.
=head1 DESCRIPTION
pt-align counts how many words each line has, and if there is one number that
predominates, it assumes this is the number of words in each line. Then it
discards all lines that don't have that many words, and looks at the 2nd line
that does. It assumes this is the first non-header line. Based on whether
each word looks numeric or not, it decides on column alignment. Finally, it
goes through and decides how wide each column should be, and then prints them
out.
The tool's behavior has some important consequences. Reading the entire input
before formatting means that you can't use it for aligning data as it is
generated incrementally, and you probably don't want to use this tool on very
large files. Discarding lines with the wrong number of words means that some
lines won't be printed.
=head1 OPTIONS
This tool does not have any command-line options.
=head1 ENVIRONMENT
This tool does not use any environment variables.
=head1 SYSTEM REQUIREMENTS
This tool requires Perl v5.8 or newer built with core modules.
=head1 BUGS
For a list of known bugs, see L<http://www.percona.com/bugs/pt-align>.
Please report bugs at L<https://bugs.launchpad.net/percona-toolkit>.
Include the following information in your bug report:
=over
=item * Complete command-line used to run the tool
=item * Tool L<"--version">
=item * MySQL version of all servers involved
=item * Output from the tool including STDERR
=item * Input files (log/dump/config files, etc.)
=back
If possible, include debugging output by running the tool with C<PTDEBUG>;
see L<"ENVIRONMENT">.
=head1 DOWNLOADING
Visit L<http://www.percona.com/software/percona-toolkit/> to download the
latest release of Percona Toolkit. Or, get the latest release from the
command line:
wget percona.com/get/percona-toolkit.tar.gz
wget percona.com/get/percona-toolkit.rpm
wget percona.com/get/percona-toolkit.deb
You can also get individual tools from the latest release:
wget percona.com/get/TOOL
Replace C<TOOL> with the name of any tool.
=head1 AUTHORS
Baron Schwartz
=head1 ABOUT PERCONA TOOLKIT
This tool is part of Percona Toolkit, a collection of advanced command-line
tools developed by Percona for MySQL support and consulting. Percona Toolkit
was forked from two projects in June, 2011: Maatkit and Aspersa. Those
projects were created by Baron Schwartz and developed primarily by him and
Daniel Nichter, both of whom are employed by Percona. Visit
L<http://www.percona.com/software/> for more software developed by Percona.
=head1 COPYRIGHT, LICENSE, AND WARRANTY
This program is copyright 2010 Baron Schwartz, 2011 Percona Inc.
Feedback and improvements are welcome.
THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED
WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, version 2; OR the Perl Artistic License. On UNIX and similar
systems, you can issue `man perlgpl' or `man perlartistic' to read these
licenses.
You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc., 59 Temple
Place, Suite 330, Boston, MA 02111-1307 USA.
=head1 VERSION
Percona Toolkit v1.0.0 released 2011-08-01
=cut

2
bin/pt-mext
View File

@@ -160,7 +160,7 @@ This tool does not use any environment variables.
=head1 SYSTEM REQUIREMENTS
This tool requires the Bourne shell (F</bin/sh>).
This tool requires the Bourne shell (F</bin/sh>) and the seq program.
=head1 BUGS

11
bin/pt-query-advisor
View File

@@ -6665,15 +6665,18 @@ Usage: pt-query-advisor [OPTION...] [FILE]
pt-query-advisor analyzes queries and advises on possible problems.
Queries are given either by specifying slowlog files, --query, or --review.
# Analyzer all queries in the given slowlog
Analyze all queries in a slow log:
pt-query-advisor /path/to/slow-query.log
# Get queries from tcpdump using pt-query-digest
pt-query-digest --type tcpdump.txt --print --no-report | pt-query-advisor
Analyze all queires in a general log:
# Get queries from a general log
pt-query-advisor --type genlog mysql.log
Get queries from tcpdump using pt-query-digest:
pt-query-digest --type tcpdump.txt --print --no-report | pt-query-advisor
=head1 RISKS
The following section is included to inform users about the potential risks,

208
bin/pt-rel
View File

@@ -1,208 +0,0 @@
#!/usr/bin/env perl
# This program is part of Percona Toolkit: http://www.percona.com/software/
# See "COPYRIGHT, LICENSE, AND WARRANTY" at the end of this file for legal
# notices and disclaimers.
use strict;
use warnings FATAL => 'all';
use English qw(-no_match_vars);
use constant MKDEBUG => $ENV{MKDEBUG};
my $file = shift @ARGV || '-';
my $fh;
if ( $file eq '-' ) {
$fh = *STDIN;
}
else {
open $fh, "<", $file or die "Cannot open $file: $OS_ERROR";
}
my @line_patterns;
my %last_match;
my %printf_for;
while ( my $line = <$fh> ) {
MKDEBUG && print "read: $line\n";
my $matched = 0;
foreach my $pat ( @line_patterns ) {
MKDEBUG && print "trying to match: $pat\n";
if ( my @vals = $line =~ m/$pat/ ) {
MKDEBUG && print "matches\n";
$matched = 1;
my @last_vals = @{ $last_match{$pat} };
my @offset_vals;
for my $i ( 0..$#vals ) {
$offset_vals[$i] = $vals[$i] - $last_vals[$i];
}
$last_match{$pat} = \@vals;
printf $printf_for{$pat}, @offset_vals;
last;
}
}
if ( !$matched ) {
my $new_pat = $line;
$new_pat =~ s/\d+/(\\d+)/g;
my @vals = $line =~ m/$new_pat/;
MKDEBUG && print "new pattern: $new_pat: @vals\n";
push @line_patterns, $new_pat;
(my $printf_pat = $line ) =~ s/\d+/%d/g;
$printf_for{$new_pat} = $printf_pat;
$last_match{$new_pat} = \@vals;
print $line;
}
}
close $fh or warn "Cannot close $file: $OS_ERROR";
exit;
=pod
=head1 NAME
pt-rel - Relativize values to previous matching lines.
=head1 SYNOPSIS
Usage: pt-rel [FILES]
pt-rel matches lines and subtracts the value of the previous line's values
from the current line's values. The lines must be text with numeric values
that repeat, varying only the values.
=head1 RISKS
The following section is included to inform users about the potential risks,
whether known or unknown, of using this tool. The two main categories of risks
are those created by the nature of the tool (e.g. read-only tools vs. read-write
tools) and those created by bugs.
pt-rel is a read-only tool. It should be very low-risk.
At the time of this release, we know of no bugs that could cause serious harm
to users.
The authoritative source for updated information is always the online issue
tracking system. Issues that affect this tool will be marked as such. You can
see a list of such issues at the following URL:
L<http://www.percona.com/bugs/pt-rel>.
See also L<"BUGS"> for more information on filing bugs and getting help.
=head1 DESCRIPTION
For example, if the text is this:
Mutex spin waits 0, rounds 99584819933, OS waits 437663963
RW-shared spins 834337527, OS waits 20258150; RW-excl spins 1769749834
Mutex spin waits 0, rounds 99591465498, OS waits 437698122
RW-shared spins 834352175, OS waits 20259032; RW-excl spins 1769762980
Then the output will be:
Mutex spin waits 0, rounds 99584819933, OS waits 437663963
RW-shared spins 834337527, OS waits 20258150; RW-excl spins 1769749834
Mutex spin waits 0, rounds 6645565, OS waits 34159
RW-shared spins 14648, OS waits 882; RW-excl spins 13146
The first values (line 1) for "Mutex spin waits", "rounds", and "OS waits"
were subtracted from the second values (line 3); the same happened for values
from lines 2 and 4.
=head1 OPTIONS
This tool does not have any command-line options.
=head1 ENVIRONMENT
The environment variable C<PTDEBUG> enables verbose debugging output to STDERR.
To enable debugging and capture all output to a file, run the tool like:
PTDEBUG=1 pt-rel ... > FILE 2>&1
Be careful: debugging output is voluminous and can generate several megabytes
of output.
=head1 SYSTEM REQUIREMENTS
This tool requires Perl v5.8 or newer.
=head1 BUGS
For a list of known bugs, see L<http://www.percona.com/bugs/pt-rel>.
Please report bugs at L<https://bugs.launchpad.net/percona-toolkit>.
Include the following information in your bug report:
=over
=item * Complete command-line used to run the tool
=item * Tool L<"--version">
=item * MySQL version of all servers involved
=item * Output from the tool including STDERR
=item * Input files (log/dump/config files, etc.)
=back
If possible, include debugging output by running the tool with C<PTDEBUG>;
see L<"ENVIRONMENT">.
=head1 DOWNLOADING
Visit L<http://www.percona.com/software/percona-toolkit/> to download the
latest release of Percona Toolkit. Or, get the latest release from the
command line:
wget percona.com/get/percona-toolkit.tar.gz
wget percona.com/get/percona-toolkit.rpm
wget percona.com/get/percona-toolkit.deb
You can also get individual tools from the latest release:
wget percona.com/get/TOOL
Replace C<TOOL> with the name of any tool.
=head1 AUTHORS
Baron Schwartz
=head1 ABOUT PERCONA TOOLKIT
This tool is part of Percona Toolkit, a collection of advanced command-line
tools developed by Percona for MySQL support and consulting. Percona Toolkit
was forked from two projects in June, 2011: Maatkit and Aspersa. Those
projects were created by Baron Schwartz and developed primarily by him and
Daniel Nichter, both of whom are employed by Percona. Visit
L<http://www.percona.com/software/> for more software developed by Percona.
=head1 COPYRIGHT, LICENSE, AND WARRANTY
This program is copyright 2010 Baron Schwartz, 2011 Percona Inc.
Feedback and improvements are welcome.
THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED
WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, version 2; OR the Perl Artistic License. On UNIX and similar
systems, you can issue `man perlgpl' or `man perlartistic' to read these
licenses.
You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc., 59 Temple
Place, Suite 330, Boston, MA 02111-1307 USA.
=head1 VERSION
Percona Toolkit v1.0.0 released 2011-08-01
=cut

88
bin/pt-table-checksum
View File

@@ -7105,21 +7105,26 @@ Each host is specified as a DSN and missing values are inherited from the
first host. If you specify multiple hosts, the first is assumed to be the
master.
STOP! Are you checksumming a slave(s) against its master? Then be sure to learn
B<STOP!> Are you checksumming slaves against a master? Then be sure to learn
what L<"--replicate"> does. It is probably the option you want to use.
pt-table-checksum --replicate=mydb.checksum master-host
... time passses, replication catches up ...
pt-table-checksum --replicate=mydb.checksum --replicate-check 2 \
master-host
Checksum all slaves against the master:
Or,
pt-table-checksum \
h=master-host \
--replicate mydb.checksums
pt-table-checksum h=host1,u=user,p=password h=host2 ...
# Wait for first command to complete and replication to catchup
# on all slaves, then...
Or,
pt-table-checksum \
h=master-host \
--replicat mydb.checksums \
--replicate-check 2
pt-table-checksum host1 host2 ... hostN | pt-checksum-filter
Checksum all databases and tables on two servers and print the differences:
pt-table-checksum h=host1,u=user h=host2 | pt-checksum-filter
See L<"SPECIFYING HOSTS"> for more on the syntax of the host arguments.
@@ -7181,50 +7186,32 @@ the checksums to be different.
=head1 SPECIFYING HOSTS
pt-table-checksum connects to a theoretically unlimited number of MySQL
servers. You specify a list of one or more host definitions on the command
line, such as "host1 host2". Each host definition can be just a hostname, or it
can be a complex string that specifies connection options as well. You can
specify connection options two ways:
Each host is specified on the command line as a DSN. A DSN is a comma-separted
list of C<option=value> pairs. The most basic DSN is C<h=host> to specify
the hostname of the server and use default for everything else (port, etc.).
See L<"DSN OPTIONS"> for more information.
=over
DSN options that are listed as C<copy: yes> are copied from the first DSN
to subsequent DSNs that do not specify the DSN option. For example,
C<h=host1,P=12345 h=host2> is equivalent to C<h=host1,P=12345 h=host2,P=12345>.
This allows you to avoid repeating DSN options that have the same value
for all DSNs.
=item *
Connection-related command-line options like L<"--user"> and L<"--password">
provide default DSN values for the corresponding DSN options indicated by
the short form of each option. For example, the short form of L<"--user">
is C<-u> which corresponds to the C<u> DSN option, so C<--user bob h=host>
is equivalent to C<h=host,u=bob>. These defaults apply to all DSNs that
do not specify the DSN option.
Format a host definition in a key=value,key=value form. If an argument on the
command line contains the letter '=', pt-table-checksum will parse it into
its component parts. Examine the L<"--help"> output for details on the allowed
keys.
The DSN option value precedence from higest to lowest is:
Specifying a list of simple host definitions "host1 host2" is equivalent to the
more complicated "h=host1 h=host2" format.
=item *
With the command-line options such as L<"--user"> and L<"--password">. These
options, if given, apply globally to all host definitions.
=back
In addition to specifying connection options this way, pt-table-checksum
allows shortcuts. Any options specified for the first host definition on the
command line fill in missing values in subsequent ones. Any options that are
still missing after this are filled in from the command-line options if
possible.
In other words, the places you specify connection options have precedence:
highest precedence is the option specified directly in the host definition, next
is the option specified in the first host definition, and lowest is the
command-line option.
You can mix simple and complex host definitions and/or command-line arguments.
For example, if all your servers except one of your slaves uses a non-standard
port number:
pt-table-checksum --port 4500 master h=slave1,P=3306 slave2 slave3
* explicit values in each DSN on the command-line
* copied values from the first DSN
* default values from connection-related command-line options
If you are confused about how pt-table-checksum will connect to your servers,
give the L<"--explain-hosts"> option and it will tell you.
use the L<"--explain-hosts"> option and it will tell you.
=head1 HOW FAST IS IT?
@@ -7878,10 +7865,9 @@ Show, but do not execute, checksum queries (disables L<"--empty-replicate-table"
group: Help
Print connection information and exit.
Print out a list of hosts to which pt-table-checksum will connect, with all
the various connection options, and exit. See L<"SPECIFYING HOSTS">.
Print full DSNs for each host and exit. This option allows you to see how
pt-table-checksum parses DSNs from the command-line and how it will connect
to those hosts. See L<"SPECIFYING HOSTS">.
=item --float-precision

35
t/pt-align/pt-align.t
View File

@@ -1,35 +0,0 @@
#!/usr/bin/env perl
BEGIN {
die "The PERCONA_TOOLKIT_BRANCH environment variable is not set.\n"
unless $ENV{PERCONA_TOOLKIT_BRANCH} && -d $ENV{PERCONA_TOOLKIT_BRANCH};
unshift @INC, "$ENV{PERCONA_TOOLKIT_BRANCH}/lib";
};
use strict;
use warnings FATAL => 'all';
use English qw(-no_match_vars);
use Test::More tests => 2;
use PerconaTest;
my $sample = "$trunk/t/pt-align/samples/";
like(
`perl -c $trunk/bin/pt-align 2>&1`,
qr/syntax OK/,
'syntax OK'
);
ok(
no_diff(
"$trunk/bin/pt-align $sample/vmstat-01.in",
"t/pt-align/samples/vmstat-01.out",
),
"vmstat-01"
);
# #############################################################################
# Done.
# #############################################################################
exit;

7
t/pt-align/samples/vmstat-01.in
View File

@@ -1,7 +0,0 @@
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 6736 117044 39936 147128 0 2 109 159 56 152 3 6 91 0
0 0 6736 117044 39936 147132 0 0 0 0 41 173 1 4 95 0
0 0 6736 117044 39936 147132 0 0 0 0 37 155 0 5 95 0
0 0 6736 117044 39936 147132 0 0 0 0 31 148 0 2 98 0
0 0 6736 117004 39944 147136 0 0 0 68 37 125 0 4 96 0

6
t/pt-align/samples/vmstat-01.out
View File

@@ -1,6 +0,0 @@
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 6736 117044 39936 147128 0 2 109 159 56 152 3 6 91 0
0 0 6736 117044 39936 147132 0 0 0 0 41 173 1 4 95 0
0 0 6736 117044 39936 147132 0 0 0 0 37 155 0 5 95 0
0 0 6736 117044 39936 147132 0 0 0 0 31 148 0 2 98 0
0 0 6736 117004 39944 147136 0 0 0 68 37 125 0 4 96 0

27
t/pt-rel/pt-rel.t
View File

@@ -1,27 +0,0 @@
#!/usr/bin/env perl
BEGIN {
die "The PERCONA_TOOLKIT_BRANCH environment variable is not set.\n"
unless $ENV{PERCONA_TOOLKIT_BRANCH} && -d $ENV{PERCONA_TOOLKIT_BRANCH};
unshift @INC, "$ENV{PERCONA_TOOLKIT_BRANCH}/lib";
};
use strict;
use warnings FATAL => 'all';
use English qw(-no_match_vars);
use Test::More tests => 1;
use PerconaTest;
ok(
no_diff(
"cat $trunk/t/pt-rel/samples/samp01.in | $trunk/bin/pt-rel",
"t/pt-rel/samples/samp01.out",
),
"samp01"
);
# #############################################################################
# Done.
# #############################################################################
exit;

4
t/pt-rel/samples/samp01.in
View File

@@ -1,4 +0,0 @@
Mutex spin waits 0, rounds 99584819933, OS waits 437663963
RW-shared spins 834337527, OS waits 20258150; RW-excl spins 1769749834
Mutex spin waits 0, rounds 99591465498, OS waits 437698122
RW-shared spins 834352175, OS waits 20259032; RW-excl spins 1769762980

4
t/pt-rel/samples/samp01.out
View File

@@ -1,4 +0,0 @@
Mutex spin waits 0, rounds 99584819933, OS waits 437663963
RW-shared spins 834337527, OS waits 20258150; RW-excl spins 1769749834
Mutex spin waits 0, rounds 6645565, OS waits 34159
RW-shared spins 14648, OS waits 882; RW-excl spins 13146