percona/percona-toolkit
1
0
Fork 0
mirror of https://github.com/percona/percona-toolkit.git synced 2025-09-10 13:11:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

finish docs

Browse Source
This commit is contained in:
baron@percona.com
2012-01-21 11:19:50 -05:00
parent 63ea85e755 acbc6b4622
commit b13ff30bb7
4 changed files with 216 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files

115
bin/pt-stalk
View File

@@ -1092,9 +1092,27 @@ Threads_running usually is. Your job, as the tool's user, is to define an
appropriate trigger condition for the tool. Choose carefully, because the
quality of your results will depend on the trigger you choose.
You can define the trigger with the L<"--function">, L<"--variable">, and
L<"--threshold"> options, among others. Please read the documentation for
--function to learn how to do this.
The pt-stalk tool, by default, simply watches MySQL repeatedly until the trigger
becomes true. It then gathers diagnostics for a while, and sleeps afterwards for
some time to prevent repeatedly gathering data if the condition remains true.
In crude pseudocode, omitting some subtleties,
while true; do
if --variable from --function is greater than --threshold; then
observations++
if observations is greater than --cycles; then
capture diagnostics for --run-time seconds
exit if --iterations is exceeded
sleep for --sleep seconds
done
done
clean up data that's older than --retention-time
sleep for --interval seconds
done
The diagnostic data is written to files whose names begin with a timestamp, so
you can distinguish samples from each other in case the tool collects data
@@ -1203,49 +1221,55 @@ will not collect any data unless both margins are satisfied.
type: string; default: status
Built-in function name or plugin file name which returns the value of C<VARIABLE>.
Possible values are:
Specifies what to watch for a diagnostic trigger. The default value watches
SHOW GLOBAL STATUS, but you can also watch SHOW PROCESSLIST or supply a plugin
file with your own custom code. This function supplies the value of
L<"--variable">, which is then compared against L<"--threshold"> to see if the
trigger condition is met. Additional options may be required as well; see
below. Possible values:
=over
=item * status
Grep the value of C<VARIABLE> from C<mysqladmin extended-status>.
This value specifies that the source of data for the diagnostic trigger is SHOW
GLOBAL STATUS. The value of L<"--variable"> then defines which status counter
is the trigger.
=item * processlist
Count the number of processes in C<mysqladmin processlist> whose
C<VARIABLE> column matches C<MATCH>. For example:
This value specifies that the data for the diagnostic trigger comes from SHOW
FULL PROCESSLIST. The trigger value is the count of processes whose
L<"--variable"> column matches the L<"--match"> option. For example, to trigger
when more than 10 processes are in the "statistics" state, use the following
options:
TRIGGER_FUNCTION="processlist" \
VARIABLE="State" \
MATCH="statistics" \
THRESHOLD="10"
--trigger processlist --variable State --match statistics --threshold 10
The above triggers when more than 10 processes are in the "statistics" state.
C<MATCH> must be specified for this trigger function.
=back
=item * magic
In addition, you can specify a file that contains your custom trigger function,
written in Unix shell script. This can be a wrapper that executes anything you
wish. If the argument to --function is a file, then it takes precedence over
builtin functions, so if there is a file in the working directory named "status"
or "processlist" then the tool will use that file as a plugin, even though those
are otherwise recognized as reserved words for this option.
TODO
=item * plugin file name
A plugin file allows you to specify a custom trigger function. The plugin
file must contain a function called C<trg_plugin>. For example:
The plugin file works by providing a function called C<trg_plugin>, and the tool
simply sources the file and executes the function. For example, the function
might look like the following:
trg_plugin() {
# Do some stuff.
echo "$value"
mysql $EXT_ARGV -e "SHOW ENGINE INNODB STATUS" | grep -c "has waited at"
}
The last output if the function (its "return value") must be a number.
This number is compared to C<THRESHOLD>. All L<"ENVIRONMENT"> variables
are available to the function.
This snippet will count the number of mutex waits inside of InnoDB. It
illustrates the general principle: the function must output a number, which is
then compared to the threshold as usual. The $EXT_ARGV variable contains the
MySQL options mentioned in the L<"SYNOPSIS"> above.
Do not alter the tool's existing global variables. Prefix any plugin-specific
global variables with "PLUGIN_".
The plugin should not alter the tool's existing global variables. Prefix any
plugin-specific global variables with "PLUGIN_" or make them local.
=back
@@ -1257,14 +1281,15 @@ Print help and exit.
type: int; default: 1
Interval between checks.
Interval between checks for the diagnostic trigger.
=item --iterations
type: int
Exit after triggering C<pt-collect> this many times. By default, the tool
will collect as many times as it's triggered.
Exit after collecting diagnostics this many times. By default, the tool
will continue to watch the server forever, but this is useful for scenarios
where you want to capture once and then exit, for example.
=item --log
@@ -1276,13 +1301,14 @@ Print all output to this file when daemonized.
type: string
Match pattern for C<processlist> L<"--function">.
The pattern to use when watching SHOW PROCESSLIST. See the documentation for
L<"--function"> for details.
=item --notify-by-email
type: string
Send mail to this list of addresses when C<pt-collect> triggers.
Send mail to this list of addresses when data is collected.
=item --pid
@@ -1294,42 +1320,47 @@ Create a PID file when daemonized.
type: string
Collect file prefix.
If not specified, the current local time is used like C<2011_12_06_14_02_02>,
which is December 6, 2011 at 14:02:02.
The filename prefix for diagnostic samples. By default, samples have a timestamp
prefix based on the current local time, such as 2011_12_06_14_02_02, which is
December 6, 2011 at 14:02:02.
=item --retention-time
type: int; default: 30
Remove samples after this many days.
Number of days to retain collected samples. Any samples that are older will be
purged.
=item --run-time
type: int; default: 30
How long to collect statistics data for?
Make sure that this isn't longer than SLEEP.
How long the tool will collect data when it triggers. This should not be longer
than L<"--sleep">. It is usually not necessary to change this; if the default 30
seconds hasn't gathered enough diagnostic data, running longer is not likely to
do so. In fact, in many cases a shorter collection period is appropriate.
=item --sleep
type: int; default: 300
How long to sleep after collecting?
How long to sleep after collecting data. This prevents the tool from triggering
continuously, which might be a problem if the collection process is intrusive.
It also prevents filling up the disk or gathering too much data to analyze
reasonably.
=item --threshold
type: int; default: 25
Max number of C<N> to tolerate.
The threshold at which the diagnostic trigger should fire. See L<"--function">
for details.
=item --variable
type: string; default: Threads_running
This is the thing to check for.
The variable to compare against the threshold. See L<"--function"> for details.
=item --version

163
lib/bash/parse_options.sh
View File

@@ -22,15 +22,20 @@
# parse_options parses Perl POD options from Bash tools and creates
# global variables for each option.
# ***********************************************************
# GLOBAL $TMPDIR AND $TOOL MUST BE SET BEFORE USING THIS LIB!
# ***********************************************************
set -u
# Global variables. These must be global because declare inside a
# sub will be scoped locally.
ARGV="" # Non-option args (probably input files)
EXT_ARGV="" # Everything after -- (args for an external command)
OPT_ERRS=0 # How many command line option errors
OPT_VERSION="no" # If --version was specified
OPT_HELP="no" # If --help was specified
ARGV="" # Non-option args (probably input files)
EXT_ARGV="" # Everything after -- (args for an external command)
OPT_ERRS=0 # How many command line option errors
OPT_VERSION="no" # If --version was specified
OPT_HELP="no" # If --help was specified
PO_DIR="$TMPDIR/po" # Directory with program option spec files
# Sub: usage
# Print usage (--help) and list the program's options.
@@ -48,9 +53,9 @@ usage() {
local file="$1"
local usage=$(grep '^Usage: ' "$file")
echo $usage >&2
echo >&2
echo "For more information, 'man $TOOL' or 'perldoc $file'." >&2
echo $usage
echo
echo "For more information, 'man $TOOL' or 'perldoc $file'."
}
usage_or_errors() {
@@ -64,20 +69,20 @@ usage_or_errors() {
if [ "$OPT_HELP" = "yes" ]; then
usage "$file"
echo >&2
echo "Command line options:" >&2
echo >&2
echo
echo "Command line options:"
echo
for opt in $(ls $TMPDIR/po/); do
local desc=$(cat $TMPDIR/po/$opt | grep '^desc:' | sed -e 's/^desc://')
echo "--$opt" >&2
echo " $desc" >&2
echo >&2
echo "--$opt"
echo " $desc"
echo
done
return 1
fi
if [ $OPT_ERRS -gt 0 ]; then
echo >&2
echo
usage "$file"
return 1
fi
@@ -103,6 +108,33 @@ parse_options() {
local file="$1"
shift
# Change --op=val to --op val because _parse_command_line() needs
# a space-separated list of "op val op val" etc.
local opts=$(echo "$@" | perl -ne 's/--(\S+)=/--$1 /g, print')
if [ ! -d "$PO_DIR" ]; then
mkdir "$PO_DIR"
if [ $? -ne 0 ]; then
echo "Cannot mkdir $PO_DIR" >&2
exit 1
fi
fi
rm -rf "$PO_DIR"/*
if [ $? -ne 0 ]; then
echo "Cannot rm -rf $PO_DIR/*" >&2
exit 1
fi
_parse_pod "$file"
_eval_po
_parse_config_files
_parse_command_line $opts # do NOT quote, we want "--op" "val" not "--op val"
}
_parse_pod() {
local file="$1"
# Parse the program options (po) from the POD. Each option has
# a spec file like:
# $ cat po/string-opt2
@@ -111,57 +143,42 @@ parse_options() {
# default=foo
# That's the spec for --string-opt2. Each line is a key:value pair
# from the option's POD line like "type: string; default: foo".
if [ ! -d "$TMPDIR/po/" ]; then
mkdir "$TMPDIR/po/"
if [ $? -ne 0 ]; then
echo "Cannot mkdir $TMPDIR/po/" >&2
exit 1
fi
fi
rm -rf "$TMPDIR"/po/*
if [ $? -ne 0 ]; then
echo "Cannot rm -rf $TMPDIR/po/*" >&2
exit 1
fi
(
export PO_DIR="$TMPDIR/po"
cat "$file" | perl -ne '
BEGIN { $/ = ""; }
next unless $_ =~ m/^=head1 OPTIONS/;
while ( defined(my $para = <>) ) {
last if $para =~ m/^=head1/;
cat "$file" | PO_DIR="$PO_DIR" perl -ne '
BEGIN { $/ = ""; }
next unless $_ =~ m/^=head1 OPTIONS/;
while ( defined(my $para = <>) ) {
last if $para =~ m/^=head1/;
chomp;
if ( $para =~ m/^=item --(\S+)/ ) {
my $opt = $1;
my $file = "$ENV{PO_DIR}/$opt";
open my $opt_fh, ">", $file or die "Cannot open $file: $!";
print $opt_fh "long:$opt\n";
$para = <>;
chomp;
if ( $para =~ m/^=item --(\S+)/ ) {
my $opt = $1;
my $file = "$ENV{PO_DIR}/$opt";
open my $opt_fh, ">", $file or die "Cannot open $file: $!";
print $opt_fh "long:$opt\n";
if ( $para =~ m/^[a-z ]+:/ ) {
map {
chomp;
my ($attrib, $val) = split(/: /, $_);
print $opt_fh "$attrib:$val\n";
} split(/; /, $para);
$para = <>;
chomp;
if ( $para =~ m/^[a-z ]+:/ ) {
map {
chomp;
my ($attrib, $val) = split(/: /, $_);
print $opt_fh "$attrib:$val\n";
} split(/; /, $para);
$para = <>;
chomp;
}
my ($desc) = $para =~ m/^([^?.]+)/;
print $opt_fh "desc:$desc.\n";
close $opt_fh;
}
my ($desc) = $para =~ m/^([^?.]+)/;
print $opt_fh "desc:$desc.\n";
close $opt_fh;
}
last;
'
)
}
last;
'
}
_eval_po() {
# Evaluate the program options into existence as global variables
# transformed like --my-op == $OPT_MY_OP. If an option has a default
# value, it's assigned that value. Else, it's value is an empty string.
for opt_spec in $(ls "$TMPDIR/po/"); do
for opt_spec in $(ls "$PO_DIR"); do
local opt=""
local default_val=""
local neg=0
@@ -187,13 +204,13 @@ parse_options() {
fi
;;
*)
echo "Invalid attribute in $TMPDIR/po/$opt_spec: $line" >&2
echo "Invalid attribute in $PO_DIR/$opt_spec: $line" >&2
exit 1
esac
done < "$TMPDIR/po/$opt_spec"
done < "$PO_DIR/$opt_spec"
if [ -z "$opt" ]; then
echo "No long attribute in option spec $TMPDIR/po/$opt_spec" >&2
echo "No long attribute in option spec $PO_DIR/$opt_spec" >&2
exit 1
fi
@@ -204,9 +221,29 @@ parse_options() {
fi
fi
# Eval the option into existence as a global variable.
eval "OPT_${opt}"="$default_val"
done
}
_parse_config_files() {
local config_files="/etc/percona-toolkit/percona-toolkit.conf /etc/percona-toolkit/$TOOL.conf $HOME/.percona-toolkit.conf $HOME/.$TOOL.conf"
for config_file in $config_files; do
test -f "$config_file" || continue
# The config file syntax is just like a command line except there
# is one option per line. In Bash, --foo --bar is the same as
# --foo
# --bar
# So we can simply cat the config file into/as the command line.
# The Perl changes --foo=bar to --foo bar because _parse_command_line()
# needs a space-separated list of "opt val opt val" etc.
_parse_command_line \
$(cat "$config_file" | perl -ne 's/--(\S+)=/--$1 /g, print')
done
}
_parse_command_line() {
# Parse the command line options. Anything after -- is put into
# EXT_ARGV. Options must begin with one or two hyphens (--help or -h),
# else the item is put into ARGV (it's probably a filename, directory,
@@ -217,7 +254,7 @@ parse_options() {
# a default value 100, then $OPT_FOO=100 already, but if --foo=500 is
# specified on the command line, then we re-eval $OPT_FOO=500 to update
# $OPT_FOO.
for opt; do
for opt in "$@"; do
if [ $# -eq 0 ]; then
break # no more opts
fi
@@ -267,7 +304,7 @@ parse_options() {
# says it has a type, then it requires a value and that value should
# be the next item ($1). Else, typeless options (like --version) are
# either "yes" if specified, else "no" if negatable and --no-opt.
local required_arg=$(cat $spec | awk -F: '/^type:/{print $2}')
local required_arg=$(cat "$spec" | awk -F: '/^type:/{print $2}')
if [ -n "$required_arg" ]; then
if [ $# -eq 0 ]; then
OPT_ERRS=$(($OPT_ERRS + 1))
@@ -286,7 +323,7 @@ parse_options() {
fi
# Get and transform the opt's long form. E.g.: -q == --quiet == QUIET.
opt=$(cat $spec | grep '^long:' | cut -d':' -f2 | sed 's/-/_/g' | tr [:lower:] [:upper:])
opt=$(cat "$spec" | grep '^long:' | cut -d':' -f2 | sed 's/-/_/g' | tr [:lower:] [:upper:])
# Re-eval the option to update its global variable value.
eval "OPT_$opt"="$val"

43
t/lib/bash/parse_options.sh
View File

@@ -1,8 +1,10 @@
#!/usr/bin/env bash
TESTS=27
TESTS=37
TMPFILE="$TEST_TMPDIR/parse-opts-output"
TOOL="pt-stalk"
TMPDIR="$TEST_TMPDIR"
source "$LIB_DIR/log_warn_die.sh"
source "$LIB_DIR/parse_options.sh"
@@ -11,8 +13,6 @@ source "$LIB_DIR/parse_options.sh"
# Parse options from POD using all default values.
# ############################################################################
TOOL="pt-stalk"
TMPDIR="$TEST_TMPDIR"
parse_options "$T_LIB_DIR/samples/bash/po001.sh" "" 2>$TMPFILE
is "`cat $TMPFILE`" "" "No warnings or errors"
@@ -20,7 +20,7 @@ is "`cat $TMPFILE`" "" "No warnings or errors"
is "$OPT_STRING_OPT" "" "Default string option"
is "$OPT_STRING_OPT2" "foo" "Default string option with default"
is "$OPT_TYPELESS_OPTION" "" "Default typeless option"
is "$OPT_NOPTION" "yes" "Defailt neg option"
is "$OPT_NOPTION" "yes" "Default neg option"
is "$OPT_INT_OPT" "" "Default int option"
is "$OPT_INT_OPT2" "42" "Default int option with default"
is "$OPT_VERSION" "" "--version"
@@ -39,6 +39,14 @@ is "$OPT_INT_OPT" "50" "Specified int option (spec)"
is "$OPT_INT_OPT2" "42" "Default int option with default (spec)"
is "$OPT_VERSION" "" "--version (spec)"
# ############################################################################
# --option=value should work like --option value.
# ############################################################################
parse_options "$T_LIB_DIR/samples/bash/po001.sh" --int-opt=42
is "$OPT_INT_OPT" "42" "Specified int option (--option=value)"
# ############################################################################
# Negate an option like --no-option.
# ############################################################################
@@ -46,7 +54,7 @@ is "$OPT_VERSION" "" "--version (spec)"
parse_options "$T_LIB_DIR/samples/bash/po001.sh" --no-noption
is "$OPT_STRING_OPT" "" "Default string option (neg)"
is "$OPT_STRING_OPT2" "foo" "Default string option with default (net)"
is "$OPT_STRING_OPT2" "foo" "Default string option with default (neg)"
is "$OPT_TYPELESS_OPTION" "" "Default typeless option (neg)"
is "$OPT_NOPTION" "no" "Negated option (neg)"
is "$OPT_INT_OPT" "" "Default int option (neg)"
@@ -89,6 +97,31 @@ cmd_ok \
"grep -q 'Exit if the disk is less than this %full.' $TMPFILE" \
"Don't interpolate --help descriptions"
# ###########################################################################
# Config files.
# ###########################################################################
TOOL="pt-test"
cp "$T_LIB_DIR/samples/bash/config001.conf" "$HOME/.$TOOL.conf"
parse_options "$T_LIB_DIR/samples/bash/po001.sh" ""
is "$OPT_STRING_OPT" "abc" "Default string option (conf)"
is "$OPT_STRING_OPT2" "foo" "Default string option with default (conf)"
is "$OPT_TYPELESS_OPTION" "yes" "Default typeless option (conf)"
is "$OPT_NOPTION" "yes" "Default neg option (conf)"
is "$OPT_INT_OPT" "" "Default int option (conf)"
is "$OPT_INT_OPT2" "42" "Default int option with default (conf)"
is "$OPT_VERSION" "" "--version (conf)"
is "$EXT_ARGV" "--host 127.1 --user daniel" "External ARGV (conf)"
# Command line should override config file.
parse_options "$T_LIB_DIR/samples/bash/po001.sh" --string-opt zzz
is "$OPT_STRING_OPT" "zzz" "Command line overrides config file"
rm "$HOME/.$TOOL.conf"
TOOL="pt-stalk"
# ############################################################################
# Done
# ############################################################################

5
t/lib/samples/bash/config001.conf Normal file
View File

@@ -0,0 +1,5 @@
--string-opt=abc
--typeless-option
--
--host=127.1
--user=daniel