Compare commits

...

10 Commits

Author SHA1 Message Date
dependabot[bot]
ea8fc17dbc build(deps): bump k8s.io/api from 0.34.0 to 0.34.1
Bumps [k8s.io/api](https://github.com/kubernetes/api) from 0.34.0 to 0.34.1.
- [Commits](https://github.com/kubernetes/api/compare/v0.34.0...v0.34.1)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-version: 0.34.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-15 09:50:34 +00:00
Sveta Smirnova
2dbdb8642b Merge pull request #1006 from percona/PT-1151_LP_1225577_pt-online-schema-change_can_silently_drop_rows
PT-1151 - LP #1225577: pt-online-schema-change can silently drop rows
2025-09-12 14:25:42 +03:00
Sveta Smirnova
cd6183dbcc Merge pull request #1014 from percona/PT-2346_t/pt-table-sync/specify_column_or_index.t_fails_with_5.7
PT-2346 - t/pt-table-sync/specify_column_or_index.t fails with 5.7
2025-09-12 14:22:44 +03:00
Sveta Smirnova
2ac8385d2d Merge pull request #1012 from nilnandan/PT-2014-pt-config-diff_does_not_honor_case_insensitivity_flag
PT-2014 - Updated MySQLConfigComparer.pm so now it can honor case insensitivity…
2025-09-10 18:16:33 +03:00
Sveta Smirnova
128dc2938e Merge pull request #1011 from percona/PT-2470_pt-table-sync_supports_recursion_methods_dsn_and_cluster_but_documentation_does_not_mention_it
PT-2470 - pt-table-sync supports recursion methods dsn and cluster bu…
2025-09-10 18:15:47 +03:00
Sveta Smirnova
f550652dbf PT-2014 - pt-config-diff does not honor case insensitivity flag
- Fixed the proposed fix
- Fixed proposed configuration files
- Added test case
2025-09-09 19:25:13 +03:00
Sveta Smirnova
fef02540fe PT-2346 - t/pt-table-sync/specify_column_or_index.t fails with 5.7
- Adjusted regular expression, so it is independent from randomness of the query plan calculation by EXPLAIN: we test if specified index or column used, not the range of values.
2025-09-09 16:10:45 +03:00
nilnandan
4e190a34ef Updated MySQLConfigComparer.pm so now it can honor case insensitivity. cnf1 and cnf2 are test files. 2025-09-08 14:12:12 +00:00
Sveta Smirnova
21fd4f7b90 PT-2470 - pt-table-sync supports recursion methods dsn and cluster but documentation does not mention it
- Updated documentation for the option --recursion-method
- I did not mention cluster, because pt-table-sync does not make sense with the cluster
- Test case for the --recursion-method=dsn is in ssl.t
2025-09-08 15:27:20 +03:00
Sveta Smirnova
20b3196cda PT-1151 - LP #1225577: pt-online-schema-change can silently drop rows
- Found case when --check-unique-index-change does not catch error:
  UNIQUE without KEY or INDEX keyword and fixed regular expression
2025-09-08 02:26:30 +03:00
11 changed files with 145 additions and 35 deletions

View File

@@ -3515,12 +3515,12 @@ use English qw(-no_match_vars);
use constant PTDEBUG => $ENV{PTDEBUG} || 0;
my %alt_val_for = (
ON => 1,
YES => 1,
TRUE => 1,
OFF => 0,
NO => 0,
FALSE => 0,
ON => 1,
YES => 1,
TRUE => 1,
OFF => 0,
NO => 0,
FALSE => 0,
);
sub new {
@@ -3637,7 +3637,8 @@ sub diff {
next CONFIG if $val0 == $valN;
}
else {
next CONFIG if $ignore_case
next CONFIG if $ignore_case
? lc($val0) eq lc($valN)
: $val0 eq $valN;
@@ -3699,7 +3700,7 @@ sub _normalize_value {
my ($val, $is_dir, $base_path) = @args{qw(value is_directory base_path)};
$val = defined $val ? $val : '';
$val = $alt_val_for{$val} if exists $alt_val_for{$val};
$val = $alt_val_for{uc($val)} if exists $alt_val_for{uc($val)};
if ( $val =~ m/,/ && !$is_dir && !$base_path) {
$val = join(',', sort(split(',', $val)));

View File

@@ -11211,7 +11211,7 @@ sub get_unique_index_fields {
$clean .= $suffix;
my $fields = [];
my $fields_re = qr/\s(?:PRIMARY|UNIQUE)\s+(?:INDEX|KEY|)\s*(?:.*?)\s*\((.*?)\)/i;
my $fields_re = qr/\s(?:(?:(?:PRIMARY|UNIQUE)\s+(?:INDEX|KEY|))|UNIQUE)\s*(?:.*?)\s*\((.*?)\)/i;
while($clean =~ /$fields_re/g) {
push @$fields, [ split /\s*,\s*/, $1 ];

View File

@@ -12965,14 +12965,43 @@ Possible methods are:
=========== ==================
processlist SHOW PROCESSLIST
hosts SHOW REPLICAS (SHOW SLAVE HOSTS before MySQL 8.1)
dsn=DSN DSNs from a table
none Do not find replicas
The processlist method is preferred because SHOW REPLICAS is not reliable.
However, the hosts method is required if the server uses a non-standard
port (not 3306). Usually pt-table-sync does the right thing and finds
the replicas, but you may give a preferred method and it will be used first.
If it doesn't find any replicas, the other methods will be tried.
The C<processlist> method is the default, because C<SHOW REPLICAS> is not
reliable. However, if the server uses a non-standard port (not 3306), then
the C<hosts> method becomes the default because it works better in this case.
The C<hosts> method requires replicas to be configured with C<report_host>,
C<report_port>, etc.
The C<dsn> method is special: rather than automatically discovering replicas,
this method specifies a table with replica DSNs. The tool will only connect
to these replicas. This method works best when replicas do not use the same
MySQL username or password as the source, or when you want to prevent the tool
from connecting to certain replicas. The C<dsn> method is specified like:
C<--recursion-method dsn=h=host,D=percona,t=dsns>. The specified DSN must
have D and t parts, or just a database-qualified t part, which specify the
DSN table. The DSN table must have the following structure:
CREATE TABLE `dsns` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`parent_id` int(11) DEFAULT NULL,
`dsn` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
);
DSNs are ordered by C<id>, but C<id> and C<parent_id> are otherwise ignored.
The C<dsn> column contains a replica DSN like it would be given on the command
line, for example: C<"h=replica_host,u=repl_user,p=repl_pass">.
The C<none> method makes the tool ignore all replicas and cluster nodes. This
method is not recommended because it effectively disables the
L<"REPLICA CHECKS"> and no differences can be found. It is useful, however, if
you only need to write checksums on the source or a single cluster node. The
safer alternative is C<--no-replicate-check>: the tool finds replicas and
cluster nodes, performs the L<"REPLICA CHECKS">, but does not check for
differences. See L<"--[no]replicate-check">.
=item --replace

4
go.mod
View File

@@ -33,7 +33,7 @@ require (
golang.org/x/exp v0.0.0-20230321023759-10a507213a29
gopkg.in/mgo.v2 v2.0.0-20190816093944-a6b53ec6cb22
gopkg.in/yaml.v2 v2.4.0
k8s.io/api v0.34.0
k8s.io/api v0.34.1
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397
)
@@ -69,7 +69,7 @@ require (
golang.org/x/text v0.28.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apimachinery v0.34.0 // indirect
k8s.io/apimachinery v0.34.1 // indirect
k8s.io/klog/v2 v2.130.1 // indirect
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 // indirect
sigs.k8s.io/randfill v1.0.0 // indirect

8
go.sum
View File

@@ -205,10 +205,10 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
k8s.io/api v0.34.0 h1:L+JtP2wDbEYPUeNGbeSa/5GwFtIA662EmT2YSLOkAVE=
k8s.io/api v0.34.0/go.mod h1:YzgkIzOOlhl9uwWCZNqpw6RJy9L2FK4dlJeayUoydug=
k8s.io/apimachinery v0.34.0 h1:eR1WO5fo0HyoQZt1wdISpFDffnWOvFLOOeJ7MgIv4z0=
k8s.io/apimachinery v0.34.0/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw=
k8s.io/api v0.34.1 h1:jC+153630BMdlFukegoEL8E/yT7aLyQkIVuwhmwDgJM=
k8s.io/api v0.34.1/go.mod h1:SB80FxFtXn5/gwzCoN6QCtPD7Vbu5w2n1S0J5gFfTYk=
k8s.io/apimachinery v0.34.1 h1:dTlxFls/eikpJxmAC7MVE8oOeP1zryV7iRyIjB0gky4=
k8s.io/apimachinery v0.34.1/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 h1:hwvWFiBzdWw1FhfY1FooPn3kzWuJ8tmbZBHi4zVsl1Y=

View File

@@ -31,12 +31,12 @@ use constant PTDEBUG => $ENV{PTDEBUG} || 0;
# in SHOW VARS as var=TRUE. I.e. there's several synonyms for basic
# true (1) and false (0), so we normalize them to make comparisons easier.
my %alt_val_for = (
ON => 1,
YES => 1,
TRUE => 1,
OFF => 0,
NO => 0,
FALSE => 0,
ON => 1,
YES => 1,
TRUE => 1,
OFF => 0,
NO => 0,
FALSE => 0,
);
# Sub: new
@@ -202,7 +202,8 @@ sub diff {
next CONFIG if $val0 == $valN;
}
else {
next CONFIG if $ignore_case
next CONFIG if $ignore_case
? lc($val0) eq lc($valN)
: $val0 eq $valN;
@@ -293,7 +294,7 @@ sub _normalize_value {
my ($val, $is_dir, $base_path) = @args{qw(value is_directory base_path)};
$val = defined $val ? $val : '';
$val = $alt_val_for{$val} if exists $alt_val_for{$val};
$val = $alt_val_for{uc($val)} if exists $alt_val_for{uc($val)};
if ( $val =~ m/,/ && !$is_dir && !$base_path) {
$val = join(',', sort(split(',', $val)));

View File

@@ -449,6 +449,30 @@ is_deeply(
"..but can be turned off"
);
# ############################################################################
# https://perconadev.atlassian.net/browse/PT-2014
# pt-config-diff does not honor case insensitivity flag for boolean values
# ############################################################################
$c1 = new MySQLConfig(
file => "$trunk/$sample/pt-2014-1.txt",
TextResultSetParser => $trp,
);
$c2 = new MySQLConfig(
file => "$trunk/$sample/pt-2014-2.txt",
TextResultSetParser => $trp,
);
{
my $diff = $cc->diff(
configs => [$c1, $c2],
);
is_deeply(
$diff,
undef,
"Boolean values are the same regardless of the case"
) or diag(Dumper($diff));
}
# #############################################################################
# Done.
# #############################################################################

View File

@@ -0,0 +1,7 @@
[mysqld]
log_bin=1
gtid_mode=on
read_only=off
autocommit=no
binlog_format=mixed
skip_name_resolve=true

View File

@@ -0,0 +1,7 @@
[mysqld]
log_bin=on
gtid_mode=True
read_only=FALSE
autocommit=0
binlog_format=MIXED
skip_name_resolve=TRUE

View File

@@ -19,7 +19,7 @@ use Sandbox;
use SqlModes;
use File::Temp qw/ tempdir /;
plan tests => 6;
plan tests => 10;
require "$trunk/bin/pt-online-schema-change";
@@ -88,6 +88,47 @@ like(
"PT-153 Adding multiple unique indexes -> multime example queries.",
);
# UNIQUE is possible without INDEX or KEY, we need to check this as well.
($output, $exit_status) = full_output(
sub { pt_online_schema_change::main(@args, "$source_dsn,D=test,t=t1",
'--execute',
'--alter', "ADD UNIQUE c1 (f2, f3)",
),
},
);
isnt(
$exit_status,
0,
"PT-153 Adding unique index without index/key keyword exit status != 0.",
);
like(
$output,
qr/You are trying to add an unique key. This can result in data loss if the data is not unique/s,
"PT-153 Adding unique index without index/key keyword warning message.",
);
($output, $exit_status) = full_output(
sub { pt_online_schema_change::main(@args, "$source_dsn,D=test,t=t1",
'--execute',
'--alter', "ADD UNIQUE(f2, f3)",
),
},
);
isnt(
$exit_status,
0,
"PT-153 Adding unique index without index/key keyword and index name exit status != 0.",
);
like(
$output,
qr/You are trying to add an unique key. This can result in data loss if the data is not unique/s,
"PT-153 Adding unique index without index/key keyword and index name warning message.",
);
$source_dbh->do("DROP DATABASE IF EXISTS test");
# #############################################################################

View File

@@ -44,16 +44,16 @@ $output = `$trunk/bin/pt-table-sync --sync-to-source h=127.1,P=12346,u=msandbox,
# and EXPLAIN does not guarantee accuracy of results.
like(
$output,
qr/FROM `issue_375`.`t` FORCE INDEX \(`updated_at`\) WHERE \(`updated_at` > 0 AND `updated_at` < '2009-09-05 02:\d\d:\d\d'/,
qr/FROM `issue_375`.`t` FORCE INDEX \(`updated_at`\) WHERE \(`updated_at` > 0 AND `updated_at` < '2009-09-05 \d\d:\d\d:\d\d'/,
'--chunk-index',
);
) or diag($output);
$output = `$trunk/bin/pt-table-sync --sync-to-source h=127.1,P=12346,u=msandbox,p=msandbox -d issue_375 --print -v -v --chunk-size 50 --chunk-column updated_at`;
like(
$output,
qr/FROM `issue_375`.`t` FORCE INDEX \(`updated_at`\) WHERE \(`updated_at` > 0 AND `updated_at` < '2009-09-05 02:\d\d:\d\d'/,
qr/FROM `issue_375`.`t` FORCE INDEX \(`updated_at`\) WHERE \(`updated_at` > 0 AND `updated_at` < '2009-09-05 \d\d:\d\d:\d\d'/,
'--chunk-column',
);
) or diag($output);
# #############################################################################
# Done.